Hassath wanted to download a couple of gigabytes worth of mail from her
GMail account for offline storage, but was frustrated by the IMAP server
closing the connection during large transfers, which made Thunderbird go
into a sulk, and forced her to restart the download by hand every time.
Using POP was not an option, since it had no way to preserve the labels
she had assigned to the messages (which show up as mailboxes via IMAP).
I tried some other programs, but they didn't work the way I wanted. Some
of them provided little or no progress information. Some would fetch all
of the messages into memory before writing anything to disk. Some would
fetch messages one by one, with separate command-response cycles. Most
importantly, none of them coped well with the frequent disconnections.
Having exhausted my patience, and being in need of a weekend hack, I set
out to write a program to download the mail. I tried the IMAP modules on
CPAN, but found them lacking in one way or another. IMAP::Client parses
IMAP responses in a fundamentally broken way (scanning literal data for
things which look like OK responses). Net::IMAP::Client doesn't provide
debugging information or allow MSN fetches. Net::IMAP doesn't know about
IMAP+SSL. Most importantly, none of the modules would allow me to stream
messages to disk sensibly. So I wrote one from scratch.
My program connects to imap.gmail.com and downloads messages from each
mailbox (except special ones like "All Mail" and "Spam") into local
Maildir-format directories. It doesn't waste bandwidth, and saves data
to disk as it is received, so that it can use partial fetches to resume
after an interrupted download. It is safe to stop or restart it at any
time. It never deletes any mail. When it completes, you will have an
offline mirror of all your GMail messages.
Hassath felt I should make the program available, especially since a
friend of ours with an even more flaky network connection also wants
to use it.
Installation and usage instructions
Download
gmail-mirror
(from github.com/amenonsen/gmail-mirror)
and place it somewhere in your PATH. (Use, modification, and
distribution of this code is allowed without any limitations.
There is no warranty, express or implied.)
You will need to have the IO::Socket::SSL module installed
(Ubuntu users can apt-get install libio-socket-ssl-perl).
There are no other module dependencies.
Run "gmail-mirror.pl username" to download all your mail, or
specify a list of mailboxes explicitly: "gmail-mirror.pl username
INBOX work personal". (Note: GMail doesn't really have mailboxes.
An IMAP mailbox named x is the set of messages labelled x.) In either
case, mail will be stored locally in a Maildir-format directory named
"gmail/mailboxname/somenumber" in your current directory. You can run
"gmail-mirror.pl -v username" for more verbose progress
information. It is safe to interrupt (^C) and restart the
program at any time.
Questions are welcome
Write to me if you need help or have any suggestions. If you find this
program useful, I'd love to hear about it.