File Under: Programming

Set Up IMAP on Your Mail Server

My friend remarked the other day that if a life can be said to have a killer app, peer-to-peer file sharing is his. Well, e-mail is mine. It is the key conduit for my work, entertainment, hobbies, relations with family and friends, and all sorts of other activities. But as I come to rely on it more and more, its shortcomings are magnified, and my patience for them diminished.

I am endlessly irritated by the inelegances of POP e-mail. When I’m forced to use other people’s computers, or just when I switch from one laptop to another, I want all my e-mail, fifteen years’ worth, received as well as sent, to be right there at my fingertips, neatly categorized. If I have to stop in at a public web terminal and send an urgent message to my date for the evening, I want the message I send to be archived with the rest of my sent mail.

A year or so ago, I decided I had had enough. The old method of checking e-mail with a POP client at home, and stopping in to use Mail2Web or another public, on-the-road interface just wasn’t doing it for me. I wanted a central repository for all my e-mail, accessible easily from anywhere. After a perusal of the services that were available, I reached the conclusion that the amount of storage space and the functionality I desired would incur too high monthly cost. I took matters into my own hands.

Using an old Pentium box I had lying around, I tried out a few different configurations and possibilities, and eventually wound up with the system I wanted. Now, with that machine (which I named Potto) as a mail drop, I can access my e-mail — all my e-mail through history — from anywhere, using any mail client I like. There is also a handy web interface for checking e-mail from public terminals like the Apple store. And I even hacked together a hybrid interface so I can call up and have a Stephen-Hawking-a-like read me my e-mail over the phone! But that’s another story, and shall be told another time.

What follows is the story of how I changed my e-mail life, and the e-mail lives of several friends, by hosting my own e-mail, and theirs, at home. In a broom closet! And how you can do the same.

Contents

  1. The Basics of Mail Hosting
  2. Getting the Mail
  3. POP Versus IMAP
    1. List of IMAP Server Software
  4. Installing UW IMAP
  5. Set Up a Web Interface

The Basics of Mail Hosting

You don’t need a particularly powerful machine to handle e-mail. Unless you are planning to support hundreds of users — or you yourself get hundreds of users’ worth of e-mail — an old Pentium should be fine, speedwise. You’ll want plenty of hard drive space, but that’s cheap these days; RAM is very nice to have as well. Since you are going to be leaving this machine running constantly, you will want it to have powerful yet quiet fans installed. I learned that the hard way.

A basic install of Linux and a web server is a good place to start. You may already have one. If not, whatever are you waiting for?

When you’re setting it up, make sure to allot plenty of hard drive space for the /var partition, and plenty more for the /home partition. This is where all that e-mail is going to live.

What you will be doing is running a machine at home (or at another location of your choice) that is connected to the internet, presumably over your DSL or other high-speed connection. Unless you want to restrict your user base to people on your home network and never talk to the outside world, people will be accessing this machine remotely. This raises a few issues.

One: To resolve a domain name to your box, the box will need an IP address. You can either get a static IP address, which some ISPs charge a bit extra for, but which makes life easy; or you can use so-called dynamic DNS. This involves running a little program on your server that detects what its current IP address is, and, every time the address changes, notifying the DNS server. Technopagan has an excellent explanation of dynamic DNS, with listings and ratings of providers.

Two: Technically speaking, you are “running a server,” which some ISPs frown upon. However, since you are limiting access to a handful of friends and relatives, it’s not really a big drain on their bandwidth, so unless you have a truly draconian ISP and you flaunt the fact that you are occasionally connecting to ports 143 and 80 from outside, you shouldn’t anticipate any problems.

Three: Security is an issue. Lock down your box. In brief, don’t open any ports you don’t have to, don’t run any daemons you don’t have to, and keep everything up to date. Linux Security is a good security tutorial site.

If these obstacles seem large to you, you always have the option of paying someone else to host a machine remotely. That’s an excellent option if you are willing to trade money for the absence of headaches.

Anyway. Once you’ve got the machine up, running, and connected, with that lovely Apache test page, you are ready to proceed. Read any of the excellent guides to Linux out there if you are feeling lost.

Getting the Mail

Now then, a proper mail server has a relatively high-profile net presence:It runs Sendmail or Qmail or Postfix, has a proper DNS MX entry, and receives mail that is sent to its domain. That’s more than most people want to shoulder, in terms of configuration, maintenance, and server horsepower. I already have an ISP that receives my incoming e-mail and stores it briefly. Doubtless you do too. I also receive mail at a couple of other POP-accessible accounts. I wanted to set up my machine Potto to get the mail from the ISP and the other accounts and store it all locally. My ISP has a lot more man- and server power than I can muster, and they do what they do very well. I take the e-mail off their hands and make up for their front-end shortcomings with my flexible, customized system. (Another advantage to this setup is that if Potto ever goes down, all the new incoming mail will just stay on the ISP’s server, instead of bouncing or getting lost in the ether.)

The mail is downloaded from the ISP with Fetchmail. This is an elegant little piece of software that comes with every Linux distribution and does just what its name implies. Every couple of minutes, Fetchmail polls the various ISPs where the new mail comes in. It retrieves that mail and removes it from the server. For each user on Potto, there is a line in my /etc/fetchmailrc config file that looks like this:

poll mail.server.net with proto auto

 user "username" there with password "p@55w3rd" is username here

This is an instruction that tells Fetchmail, every time it runs, to poll the specified server, automatically detecting the protocol, and get all mail for “username”. That mail is then placed in the local mail spool for the specified user on Potto. I also include the line set daemon 120, which tells Fetchmail to run every 120 seconds.

OK, so now all this mail’s sitting on the mail drop machine. What next?

Well, in the simplest case, users can connect to the server using SSH or telnet, and access their mail via a terminal with Pine or Mutt or another such console-based client. These clients have very clean interfaces and powerful, streamlined features. And indeed they are popular with a certain geeky cross-section of Potto users. But even Mutt’s free keybinding and regex support weren’t enough to convince my dad to switch from Eudora. I needed an easier way to access the mail. Fortunately, what I required was invented many years ago.

POP Versus IMAP

Mail clients like Eudora communicate with a mail host via one of two protocols:POP and IMAP. Therefore, to support such mail clients, a mail host has to be able to speak at least one of these.

POP, the Post Office Protocol, is the most widespread protocol by which mail clients retrieve e-mail from a server. It uses the so-called offline operation model — it makes a quick connection to the server, downloads the messages that are waiting, and typically removes them from the server, then disconnects. The messages are then on the client for good.

This method is what gives rise to all the things I was complaining about in the introduction. When you offload e-mail from the server to your local machine, you lose flexibility. I wanted all the e-mail to live on Potto, and be readable from anywhere at any time. IMAP was the answer.

IMAP, the Internet Message Access Protocol, was designed to solve some of the issues that POP leaves unaddressed. Although IMAP allows for offline-style operation, its specialty is accessing messages that live permanently on a server. It supports multiple server folders, flags reflecting the New/Read/Answered, etc. status of messages, and a host of other features that place it far above POP, in terms of pretty much everything except popularity. Although IMAP has been around since 1986, POP is the de facto standard, and enjoys much more widespread use than IMAP. Rock-solid proof — as if we needed more — that the majority choice is not always the best choice. The differences between IMAP and POP — from an admittedly pro-IMAP standpoint, though I don’t know of anyone espousing the other view — are enumerated at imap.org.

List of IMAP Server Software

So yeah, IMAP. There’s a bit of a choice when it comes to which free UNIX IMAP server to use:

  • The University of Washington folks who brought you Pine distribute the UW IMAP server
  • Carnegie Mellon distributes the Cyrus server
  • And I’m assured that Dovecot and DKIMAP4 are excellent in their own right, though my testing was limited.

I chose UW IMAP to use on Potto, because it supports Unix mbox-style folders, which is the format in which my decade-plus of saved mail happens to be archived; and because it is pretty fast, stable, and secure. Cyrus and Courier may have their advantages when it comes to speed, configuration, and authentication, but both servers prefer alternate ways of storing mail. You can choose the server that best meets your needs — and I may yet switch — but for now UW IMAP is doing a fine job.

Installing UW IMAP

To install it, I downloaded the source code and unzipped it. The software requires, somewhat inflexibly, that configuration settings be set before compiling, but in the vast majority of cases the default settings are fine and nothing needs to be touched. So, following the onboard instructions, I simply typed “make lnp” and then copied the resulting file, imapd, into the /usr/sbin directory where it would run.

After installing the IMAP server software, it is necessary to configure inetd or xinetd, which is the daemon that handles internet traffic for services (like IMAP servers) that need it. In the version of Linux that Potto runs, xinetd is the order of the day, and its services are configured in the /etc/xinetd.d directory. In a config file called imap in this directory, I specified the following:

service imap { socket_type = stream wait = no user = root disable = no server = /usr/sbin/imapd }

disable = no turns on the service, and server = /usr/sbin/imapd indicates where the server program lives. Plenty of other configuration options are available here, involving security, logging, and other behavior, which you may customize for your needs.

Add the following line to /etc/hosts.allow (this can be modified later to restrict access):

imapd :ALL :ALLOW

Now restart inetd.

After restarting your inetd, you should find that the IMAP server is listening on port 143. (If you have a firewall blocking this port, unblock it.) Running telnet localhost 143 on the server machine should put you in contact with your IMAP server:

 $ telnet localhost 143

 Trying 127.0.0.1...

 Connected to localhost.localdomain (127.0.0.1).

 Escape character is '^]'.

 * OK [CAPABILITY IMAP4REV1 LOGIN-REFERRALS STARTTLS AUTH=LOGIN] localhost.localdomain IMAP4rev1

If this doesn’t work, make sure that the inetd settings are correct, and restart inetd.

You can use /etc/hosts.allow and /etc/hosts.deny to control access to your glistening new IMAP server.

Users can now configure their mail clients to talk to the IMAP server. They just need to specify the server name. Via the client, they can create new folders on the server, sort mail, and so forth.

Tip: UW IMAP keeps each user’s INBOX in their mail spool file — typically /var/spool/mail/username. This is where incoming mail goes by default. In addition to this main INBOX, users can create any number of other mail folders in their home directories, for sorting mail into categories. Procmail is extremely useful for automatic mail sorting.)

Tip: Mulberry is a terrific mail client that’s designed to work with IMAP.

But the grand plan isn’t complete yet. Mulberry and Eudora are all well and good when you’re at home or at work, but what about when you’re on the road? A web interface is a must.

Set Up a Web Interface

If you forgot to print out the directions to that party, just pull over at a public web terminal and look up the old e-mail!

There are several different ways to implement a web mail interface, but the easiest I’ve found is SquirrelMail, a PHP-based package that is fairly flexible but requires little setup. It works off an IMAP backend, which you have, and needs only Apache and PHP to run. Apache comes pre-packaged with most Linux distributions. PHP does too.

When you’ve got the requirements squared away, download the SquirrelMail package. Uncompress it in your web server directory. On Potto, this is /var/www/html. This will create a new subdirectory called squirrelmail-version, where “version” is the version number of SquirrelMail. Clear?

Create the attachments directory (which for some reason isn’t there by default):

mkdir squirrelmail-version/attachments

.

Fix the ownership and permissions, for security’s sake:

 chown -R root.apache squirrelmail-version

 chown -R apache.apache data

 chown apache.apache attachments

 chmod 730 attachments

 chmod 730 data



Did I mention that you need Perl as well? Well, you don’t strictly need it to run SquirrelMail, but the configuration program runs in Perl. Type

 cd squirrelmail-version ./configure

This will launch the configuration program. If you really don’t have Perl on your system, it’s easy enough to edit the config file by hand:it’s located at squirrelmail-version/config/config.php.

What you need to set to get the program running is the following:In Server Settings:

 1. Domain :''your domain name''

 2. IMAP Server :localhost

 3. IMAP Port :143



If you have an ISP through which you typically send your e-mail, it reduces overhead to use them as your SMTP server:

 4. Use Sendmail/SMTP :SMTP

 6. SMTP Server :mail.yourisp.net

 7. SMTP Port :25



 10. Server :uw

Next, in General Options:

 2. Data Directory :../data/

 3. Attachment Directory :../attachments/

Those are the essentials — the rest is customization. You can even use your own CSS files to get it looking exactly the way you want.

Now try it out! Browse to http://yourdomainname.net/squirrelmail-version/ and you should see the login screen. Even better, you should be able to log in successfully and see your e-mail folders!

Congratulations. You are free of POP tyranny. You’ve never been so hands-on before. Your friends will love your hosting service, and — yes! — that’s your e-mail purring in the next room.