Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings
Discussion options

jnewman67
Oct 24, 2024

have to migrate a user with 79,000 emails. when there are thousands of emails in a folder, it can take a LONG time to parse them and get them ready to copy. it would be nice if, during the "parsing headers" phase, when there is no visible action, there were a spinner added that showed it was actively doing something. something like when it writes a character, backs up and writes the next, and such, giving the illusion that something is spinning.
print - then backup and print / then backup and print | then backup and print | (repeat)
have it print one of the characters, in order, every so often (after processing 5/10/20 headers) to give an indication its not stuck or locked up.
You must be logged in to vote

Replies: 1 comment · 4 replies

Comment options

gilleslamiral
Oct 24, 2024
Maintainer

I just updated
https://imapsync.lamiral.info/FAQ.d/FAQ.Big_Mailbox.txt

....
Q. How to know if imapsync is stalling or working?

Sometimes with big folders, imapsync seems to stall when sizing the
folders or maybe before syncing a huge folder. So you wonder whether
imapsync is doing something or just being frozen. 

A1. A solution to increase the verbosity is to add the option
--debugimap. Option --debugimap shows what is currently done, it's
quite a big output but it helps to wait, saying to ourselves "ok it's
long but it's working, let's wait a little more".  The real purpose of
--debugimap is to show genuine IMAP commands used and their
responses. The problem with --debugimap is that it is very verbose and
so with a big mailbox it generates a very BIG logfile.

A2. Another solution is to monitor the imapsync process with standard
Linux tools like strace, iftop, nethogs etc.


  strace -f -p 12345 # print all the system calls of process PID 12345

  iftop -f 'port imap or port imaps' -B # then press t p > 

  nethogs # see all processes
  
  nethogs -P 12345 # Only the process with PID 12345

You can have the imapsync PID with the command:

  head -1 $HOME/tmp/imapsync.pid 

You can also set the pidfile with the imapsync option --pidfile

  imapsync ... --pidfile whatever.txt

and use it in the previous commands:

  strace -f -p `head -1 whatever.txt`
  nethogs   -P `head -1 whatever.txt`
You must be logged in to vote
4 replies
@jnewman67
Comment options

jnewman67 Oct 24, 2024
Author

first, I didn't find/see that Q/A originally, so sorry for the duplication and maybe wasting your time.
second, thank you VERY much for the response - full of information and details, and without any tone of disapproval in my oversight.
I'll try all of them, and see which yields the most useful. I was really hoping for "processed another one" without all the output or guessing that CPU usage means "it's working", but I'll take that if only to know that it's not sitting there waiting for some input or an error timeout.
Thank you.
@jnewman67
Comment options

jnewman67 Oct 24, 2024
Author

just a note: nethogs requires a capital P for that PID option. lowercase p indicates promiscuous mode :)

nethogs also seems to be a pretty good indicator to use, because it's watching traffic to from the hosts, and that seems to be the point where not much verbose activity is taking place for large email accounts.

for any nethog developers lurking, promiscuous is also misspelled in the help message :)
@gilleslamiral
Comment options

gilleslamiral Oct 24, 2024
Maintainer

first, I didn't find/see that Q/A originally, so sorry for the duplication and maybe wasting your time.

It made me discover nethogs :-)
It's my job to support users, no matter what, that's been my ditto for a long time

Second, thank you VERY much for the response - full of information and details, and without any tone of disapproval in my oversight. I'll try all of them, and see which yields the most useful. I was really hoping for "processed another one" without all the output or guessing that CPU usage means "it's working", but I'll take that if only to know that it's not sitting there waiting for some input or an error timeout. Thank you.

Technically, I could do what you described in imapsync, a "still working" thing.
Technically, it's grabbing the --debugimap thing and making it a dot every 1 KB or so.
But I'm lazy and the request, you only, is small for now.
@gilleslamiral
Comment options

gilleslamiral Oct 24, 2024
Maintainer

just a note: nethogs requires a capital P for that PID option. lowercase p indicates promiscuous mode :)

Ok. Fixed upstream and above.

nethogs also seems to be a pretty good indicator to use, because it's watching traffic to from the hosts, and that seems to be the point where not much verbose activity is taking place for large email accounts.

Yes. Imapsync spends most of its time networking, on one side or the other.
No surprise, it's its main job, the rest is sugar printed.

for any nethog developers lurking, promiscuous is also misspelled in the help message :)

Don't bet on it :-)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
💡
Ideas
Labels
Speed
2 participants
@jnewman67 @gilleslamiral
Morty Proxy This is a proxified and sanitized view of the page, visit original site.