Backing Up Gmail/Google Apps to a Dovecot Server

I have been trying to find a way to copy everything from a Gmail account to a Dovecot mail server. The way I have ended up doing it so far is simply by using Apple Mail (if you regularly read this blog, you’d know that I use a Mac). The steps are as follows:

  1. Create 2 accounts in Apple Mail: Gmail and the Dovecot account
  2. Sync the Gmail account to your local computer
  3. Copy everything to the Dovecot server

This works, but I have to use a slow connection (my home connection) and I have a lot of accounts to do this for, so I would much prefer to script this. The problem is that I have been trying to get this to work with either imapsync or imapcopy. Neither seem to work properly.

According to the documentation, I have the proper version and it should work with GMail. I tried all the supported –authmech1 possibilities. I found an article saying that Google doesn’t accept the word AUTHENTICATE when authenticating, but then imapsync wouldn’t work and it claims it does. So, when trying imapsync, this is what it looked like (some of the security issues have been blocked to protect the innocent):

[root@smtp4 imapsync-1.286]# ./imapsync --host1 imap.gmail.com --ssl1 --user1 user1@gmail.com --password1 MASKED --host2 mail.server.com --user2 user1@server.com --password2 MASKED
$RCSfile: imapsync,v $ $Revision: 1.286 $ $Date: 2009/07/24 15:53:04 $
Here is a [linux] system (Linux mail.server.com 2.6.18-92.1.18.el5xen #1 SMP Wed Nov 12 09:48:10 EST 2008 x86_64)
with perl 5.8.8
Mail::IMAPClient  3.21
IO::Socket        1.31
IO::Socket::SSL  
Digest::MD5       2.36
Digest::HMAC_MD5  
Term::ReadKey    
Date::Manip      
 and the module Mail::IMAPClient version used here is 3.21
Command line used:
./imapsync --host1 imap.gmail.com --ssl1 --user1 user1@gmail.com --password1 MASKED --host2 mail.server.com --user2 user1@server.com --password2 MASKED --useheader Message-Id --skipsize --regextrans2 s/\[Gmail\]/Gmail/

My attempt at imapcopy was closer to successful, but still not what I needed. I ran into some Dovecot errors (at least I think that’s what it is) that I am not sure what to do with.

[root@smtp4 imapCopy]# time sh imapCopy.sh imaps://user1%40gmail.com:MASKED@imap.gmail.com imap://user1%40server.com:MASKED@mail.server.com
2009-12-02 18:13:28,811  INFO ImapCopier - Starting
2009-12-02 18:13:28,812 DEBUG ImapCopier - Parameters length:2
2009-12-02 18:13:28,813 DEBUG ImapCopier - opening conections
2009-12-02 18:13:28,816 DEBUG ImapCopier - opening imaps conection to imap.gmail.com
2009-12-02 18:13:32,633 DEBUG ImapCopier - opening imap conection to mail.server.com
2009-12-02 18:13:34,666 DEBUG ImapCopier - Creating target Folder: Adbrite
Exception in thread "main" javax.mail.MessagingException: Unsupported type;
  nested exception is:
        com.sun.mail.iap.ProtocolException: Unsupported type
        at com.sun.mail.imap.IMAPFolder.doCommandIgnoreFailure(IMAPFolder.java:2618)
        at com.sun.mail.imap.IMAPFolder.create(IMAPFolder.java:725)
        at com.fisbein.joan.model.ImapCopier.copyFolderAndMessages(ImapCopier.java:190)
        at com.fisbein.joan.model.ImapCopier.copy(ImapCopier.java:152)
        at com.fisbein.joan.model.ImapCopier.main(ImapCopier.java:35)
Caused by: com.sun.mail.iap.ProtocolException: Unsupported type
        at com.sun.mail.imap.IMAPFolder$6.doCommand(IMAPFolder.java:745)
        at com.sun.mail.imap.IMAPFolder.doProtocolCommand(IMAPFolder.java:2639)
        at com.sun.mail.imap.IMAPFolder.doCommandIgnoreFailure(IMAPFolder.java:2611)
        ... 4 more

So if you have any idea what to do here in a more intelligent manner than I am stuck with, please let me know in the comments. Thanks.

Posted in Mail. Tags: , , . 8 Comments »
  • Pingback: Backing Up Gmail/Google Apps to a Dovecot Server | Eric's Tech Blog | Gmail HOT news

  • http://sol.anferny.me.uk/wordpress Nevyn

    Can Fetchmail (fetchmail.berlios.de) download from Google Mail?

  • http://eric.lubow.org Eric Lubow

    It can. Ideally, I'd like to mimic the folder structure that exists for the accounts as some of the GMail accounts have over 50 folders/sub-folders. I'm not sure if Fetchmail will just download all folders. I will give it a try.

  • http://blogs.law.harvard.edu/hoanga Al Hoang

    Hi Eric,

    I ran into a somewhat similar situation so wrote a ruby script (http://gist.github.com/166350) to deal with it. But here are some things to remember:

    * It only pulls a limited # of emails at a time to prevent from getting kicked off by Gmail
    * It only handles one mailbox at a time
    * It copies into a Maildir type directory so you might need something like maildirmake to create the directory structure that you want
    * Just point it into the new/ subfolder in a Maildir structure
    * I have tested that Dovecot will read and index these mails properly after downloading.

    Good luck!

  • http://blogs.law.harvard.edu/hoanga Al Hoang

    Hi Eric,

    I ran into a somewhat similar situation so wrote a ruby script (http://gist.github.com/166350) to deal with it. But here are some things to remember:

    * It only pulls a limited # of emails at a time to prevent from getting kicked off by Gmail
    * It only handles one mailbox at a time
    * It copies into a Maildir type directory so you might need something like maildirmake to create the directory structure that you want
    * Just point it into the new/ subfolder in a Maildir structure
    * I have tested that Dovecot will read and index these mails properly after downloading.

    Good luck!

  • http://eric.lubow.org/ eric

    Thanks. This is roughly what I am looking for (except I want it in mbox format). I am going to give this a shot (but change the format to mbox rather than Maildir). I’ll let you know how this works out for me.

  • Pingback: Transferring Email From Gmail/Google Apps to Dovecot With Larch | TuxWire : The Linux Blog

  • Pingback: » Fleeing from Gmail Hynek Schlawack