Henk van Lingen
Department of Computer Science
Utrecht University
July 2002, April 2003, June 2007
"I don't even have an e-mail address. I have reached an age where my main purpose is not to receive messages."
1. IntroductionEarly 1998 our traditional email environment was replaced by a more modern one. Besides the introduction of the by that time brand new sendmail replacement Postfix, the most important change was the introduction of the Imap protocol. This environment has been used until this summer and is described on the page Mail Architecture 1998-2002. Now it is time for another change. The combination of the old POP protocol with an elderly server created a sense of urgency. POP can be a hassle when users have lots of megabytes in a mailbox which is in fact one file on a disk. So now we want to shut down POP completely and we also introduce in new format for mailboxes: the so called Maildir format. But there is more. SSL, a method of implementing secure internet connections, has grown up. With SSL we can safely provide external access to our imap server and thereby eliminate the need for our APOP solution. Finally we want to extend our ways of fighting SPAM and to provide a webmail solution, also by means of SSL of course. Webmail seems to be very popular these days.
2. Why IMAP-SSL?In recent years SSL, which is a method of implementing secure internet connections, has become a mature solution for various services, one of them being email. That means it's besides desirable also possible to use SSL to connect with your mailserver while still supporting a range of common mailclients. It is desirable to only connect with SSL to the imap server because that way passwords are no longer travelling through our network in plain text as was the case with our former pop and imap daemons and, most important, we can now facilitate our users on extern locations with an imap solution. Therefore there is no longer any need to support POP and users can access their central mailstore from different locations in a uniform way.It is possible to host a secure IMAP server because IMAP-SSL is around a while and most major mailclients have incorporated this way of doing things. We can and will support the following mailclients: Mutt, Pine, Eudora (>= 5.1), Netscape and Mozilla. We never supported Microsoft Outlook, and we still don't, but rumors have it that this client also supports SSL. We did already support IMAP but now there's nothing else but IMAP. IMAP stands for Internet Message Access Protocol. It is a method of accessing electronic mail or bulletin board messages that are kept on a (possibly shared) mail server. In other words, it permits a "client" email program to access remote message stores as if they were local. Let's repeat some of the nice things about IMAP for those who missed that before:
The IMAP server we selected for our mailserver is the Courier-IMAP daemon. One of the reasons was it was written for the specific purpose of providing IMAP access to Maildirs instead of the traditional mailbox files. 3. Why Maildir?Maildir is a structure for directories of incoming mail messages. The traditional mailbox format (mbox) stores mail in one big file. That brings along pitfalls like file locking, access speed and always parsing your entire mailbox. Also a single failure or corruption can result in corruption of your entire mailbox. To overcome these problems, the maildir format uses directories and multiple files. Every single message is stores in a separate file on the mailserver. We won't explain the details here (see the references), after all you are only talking to the mailserver through IMAP so how mail is stored doesn't really matter that much to the client. But with Maildir, a mailclient can read and delete messages while new mail is being delivered: each message is stored in a separate file with a unique name, so it isn't affected by operations on other messages. An mailclient doesn't have to worry about partially delivered mail: each message is safely written to disk in the 'tmp' subdirectory before it is moved to 'new'. The maildir format is reliable even over NFS (note however that we don't export our mailstore at the department so this point is pointless :-)).4. Authenticated mailAuthenticated mail (SMTP-AUTH, in sysadmin-speak), extends SMTP (the Internet e-mail transmission protocol) to include an authentication step through which the client effectively logs in to the mail server during the process of sending mail. This mechanism can be used to allow legitimate users to send mail from non-local networks or relay mail while denying relay service to unauthorized users, such as spammers.To be clear: we are talking about using a username and password to log in to the smtp daemon, the thing which sends our mail. Besides this, we also use a username and password to log in to the imap daemon, the thing which lets us retrieve our mail from the mailstore. The authentication of those two daemons have to be configured separately in our mailclients. A nice thing about being able to connect to our smtp daemon with your mailclient from non-local networks, is the fact that you can configure for instance your laptop in one way and use it all over the world to send mail without having to worry about your provider settings. You can use authenticated mail both on the standard port (25) as on the so called delivery port (587). Sending AUTH data (username/password) over an unencrypted channel poses a security risk. Therefore we require TLS layer encryption (the thing formerly known as SSL) for authenticated mail. This does not mean that our smtp server listens to TLS connections (it would be hard to use for other mail servers which want to send us mail) It means that when a smtp client (your mailclient) wants to authenticate, it can only do this using TLS. Using TLS also introduces yet another server certificate. In our case at the moment this is a self-signed certificate, so your mailclient should probably be told to accept this certificate. 5. WebmailWebmail seems to be a very popular email feature these days, looking at the vast amount of hotmail adresses being used. People want to handle their mail from anywhere in the world, using any browser. They don't want or can not install a specific emailclient and they certainly don't want to login with ssh on our local network to use one of the most advanced emailclients in the world (mutt).This is why we installed SquirrelMail, webmail for nuts. SquirrelMail is a standards-based webmail package written in PHP4. It includes built-in pure PHP support for the IMAP and SMTP protocols, and all pages render in pure HTML 4.0 (with no Javascript) for maximum compatibility across browsers. SquirrelMail has all the functionality you would want from an email client, including strong MIME support, address books, and folder manipulation. You can find our SquirrelMail installation at https://mail.cs.uu.nl/webmail/ or https://mail.students.cs.uu.nl/webmail/. Note that this webserver is using SSL (https) using a certificate signed by the Utrecht University Certification Authority (CA). Your browser will probably warn you about this. You can simply ignore the browserwarnings and accept the certificate or you can add the needed root certificates to your browser, as is explained on our UU SSL certificate installation page. 6. About SPAMWe now also have SpamAssassin in place, for which we also changed to 'maildrop' as the default local delivery agent. See this newsposting for some info about our SpamAssassin setup and how to benefit from it.Of course the methods we already used to block SPAM are still in place. See our SPAM-policy page. 7. Changes for usersThis migration is one of those rare events every user will notice. Probably everyone has to change some settings in their emailclients otherwise reading mail WILL NO LONGER WORK. Things you should check in your emailclient:
The gory details such as mail services naming scheme,
configuration hints for various clients, filtering using procmail
and e-mail tips may be found in our
Using E-mail doc page (work-in-progress). 8. References |