« Back to blog

Complete GMail redundancy in 7 easy steps

If there is anything that the recent GMail outages taught us, it's that losing access to your email for a few hours should not mean that the end of the world has arrived. I think it also served as a reminder that if you rely 100% on the cloud for access to your data, then be prepared to be disconnected from it every once in a while. Fortunately, it's quite easy to take advantage of free email services from the likes of Google while also making sure that should the worst happen, you can still access your data. With more and more communication, both professional and personal, being conducted through email, it's no wonder that people do get nervous when they suddenly lose control over several years worth of contacts and emails. This short article will show you how easy it is to back up your GMail automatically to a remote server which you can then access in the event that GMail goes down or you lose access to your account. We'll be using nothing but free open-source software. If you already have a net-connected machine with Ubuntu installed then you're set to go. Otherwise, consider recycling an old pentium-or-better-cpu machine with a network card—the processing demands are relatively low and the most important resource is storage space for all the emails and attachments. Although these instructions are for Ubuntu, they should also work on Debian or any other Debian-based distro. To set this up you obviously need a GMail account, although any email account which can be accessed using the POP protocol can be backed up using the same method. We'll be using the excellent Fetchmail to retrieve messages from the GMail account while leaving them untouched in your GMail inbox, Procmail to deliver and filter them into a local Maildir-format mailbox, and Dovecot to access the emails remotely, all on the latest release of Ubuntu, Hardy Heron (8.04 LTS).

1. Setup GMail

  • Log in to your GMail account
  • Click 'Settings>Imap/Pop' and enable POP access
  • You can chose whether to only download messages which arrive from now on or to download all existing messages. The latter is useful if you have already been using your GMail account for a while and would like to back all existing messages to a secure alternative location.

2. Install packages

  • Fire up a terminal window and run the following command: sudo apt-get install dovecot-imapd procmail fetchmail mutt
  • This should automatically download and install everything you need. The mutt package is optional and is a useful tool for checking if your IMAP server is working properly and that your emails are being downloaded (not to mention a nice command-line email client!).

3. Configure Fetchmail

  • We need to define our GMail accounts in fetchmail's configuration file. Open the file /etc/fetchmailrc for editing as root. The file won't exist yet so we'll have to create it.
  • Copy & paste the following into the file:
    set daemon 60
    set syslog
    set postmaster "localuser"
    poll pop.gmail.com
    protocol pop3
    username "username@gmail.com" there with password "yourpassword" is localuser here
    keep ssl
    mda "/usr/bin/procmail -d localuser"
  • Change localuser to your local username on the linux box and the email and password to match your gmail account. If you want, you can add as many other accounts as you want here. For each account, just copy the text starting from 'poll...' and change the login details.
  • The first line tells fetchmail to check for new emails every 60 seconds. Edit to match your preferred update frequency.
  • Edit /etc/default/fetchmail as root and change the line START_DAEMON=no to START_DAEMON=yes

4. Configure Procmail (~/.procmailrc)

  • We're using procmail to deliver email from fetchmail to a mailbox in Maildir format. This is necessary due to the lack of Maildir support in Fetchmail, which supports the mbox format. The main difference between the formats is that Maildir stores each email in a single file, making it very efficient when you have a lot of emails. In contrast, mbox format stores all emails in a single file.
  • Create ~/.procmailrc and put the following text in it:
    MAILDIR=$HOME/Mail/
    DEFAULT=$MAILDIR
    VERBOSE=on
    LOGFILE=$MAILDIR/procmail.log
    :0
    $MAILDIR
  • Set permissions on the file with: chmod 640 ~/.procmailrc
  • This file simply tells procmail where to store the emails (in ~/Mail) and where to store it's logs. Procmail is capable of a lot more than this, and it supports a powerful filtering language. You can specify rules in your ~/.procmailrc file to filter emails based on sender, subject, etc.

5. Configure Dovecot

  • We're almost there! We can retrieve emails from the GMail servers and deliver them locally. Now we need a way to access the email remotely. For this we're using Dovecot, and we need to tweak the configure a little.
  • As root, open up /etc/dovecot/dovecot.conf and update the following lines:
    protocols = imaps
    mail_location = maildir:~/Mail

6. Setup maildir

  • We need to create the local mailbox with: maildirmake.dovecot ~/Mail

7. Start services

Okay, now everything is set up we need to start up the services: sudo /etc/init.d/fetchmail start sudo /etc/init.d/dovecot start

8. Enjoy!

That's it! Run tail -f /var/log/syslog to check fetchmail is receiving messages and also check ~/Mail/procmail.log to see that procmail is not throwing up any errors. The final thing you need to do is make sure you can access your email server remotely. Dovecot is running over SSL on port 993, so just configure your router to enable external access to that port. Then just configure Outlook , Thunderbird, etc. to access IMAP over SSL and you have full access to all your emails on your own server. Note: if you chose to setup GMail POP access to allow all existing messages to be downloaded and you have lots of messages in your account, it will take a few runs of fetchmail to collect them all. Leave things running overnight and check back in the morning!

Optional step: Setup mutt

  • Create the file ~/.muttrc and put the following in it:
    set mbox_type=Maildir
    set folder="~/Mail"
    set mask="!^\\.[^.]"
    set mbox="~/Mail"
    set record="+.Sent"
    set postponed="+.Drafts"
    set spoolfile="~/Mail"
  • Now just run mutt and read your email!

Related Links

| Viewed
times | Favorited 0 times
Filed under:                

11 Comments

Apr 08, 2009
Chappel said...
Nice post. This is my first time I�m seeing your blog and I�ve read some of your earlier posts too. Amusing!
Apr 16, 2009
Brad said...
Gr8 blog! Ill definitely be coming back
Apr 16, 2009
Antique Ring said...
I usually do not comment on blog posts but I found this quite interesting, so here goes. Thanks! Regards, P.
Apr 24, 2009
Mitsue Agle said...
VRy interesting to read it :P :D
Jul 03, 2009
Fiona said...
Thoughtful post and well written. Please write more on this if you have time.
Jul 28, 2009
Mig21 said...
Hi Pádraig , I found this post very interesting. But I do something bad, because I configure one account and fetchmail or procmail stores all in one big file , but in /var/mail/myaccount instead of /home/myaccount/Mail

Im doing something bad , but i dont know what ! Sorry for my english and my ignorance!

Mig21

Sep 02, 2009
Edwin Olson said...
A few updates for Ubuntu 9.04 that I needed:

apt-get install dovecot-common dovecot-imapd

and in the dovecont.conf:

mail_location = maildir:~/Mail

Nov 08, 2009
Karl Camilleri said...
Hi, quick question: With the above setup, how does one log in to localuser's account? I have it set to root but it wouldn't accept my password! Otherwise eventhing else is working 100% - there are messages in the "~/Mail/new" folder...Otherwise great howto :)

Regards,
Karl

Feb 07, 2010
pc tech said...
Will come back again, taking your feeds also, Thanks.
May 03, 2010
Dima said...
This looks good, except that's not quite clear how to change this for a multiuser setup.
Does one need to use ~/.fetchmailrc ?
May 04, 2010

Leave a comment...