If your web browser formats this page incorrectly, try viewing the
page source. In netscape 4, click the view menu, then click page source.

parts of this were copied from Guylhem Aznar <guylhem at metalab.unc.edu>'s
Linux Electronic Mail Administrator HOWTO v3.2, January 2000

BACKGROUND INFORMATION

A email client is a program for reading and writing email. Some people call
email clients mail user agents, or MUAs; or mail readers. Some common email
clients are mail, elm, pine, mh, and mutt.

An email server can be either a program for transferring email, or a
computer which runs an email server program. To avoid confusion, I will try
to say email server program or email server computer. Some people call email
server programs mail transfer agents, or MTAs. The most common email server
program is sendmail, also called BSD sendmail; some other email server
programs are qmail, smail, and exim.

If you read the sendmail documentation, you will see that much of the
documentation is about email routing. Sendmail and smail were originally
created for UUCP networks, where email routing is important. However, with
TCP/IP networks like the internet, the networking software is responsible
for routing, so the email server program does not need to do any routing,
unless you need to route mail around your local network. Thus you can
probably ignore everything in your email server program documentation about
routing email.

Most email clients assume the email server program is sendmail; therefore
most other email server programs pretend to be sendmail by making
/usr/bin/sendmail a link to the real email server program; remember that
when you or an email client runs sendmail, you may actually be running some
other email server program.

Usually a computer has only one email server program, chosen by the system
administrator; the one email server program serves all users; all users have
to use the same email server program. But a computer often has more than one
email client; different users can choose to use different email clients.

Guylhem Aznar <guylhem at metalab.unc.edu>'s Linux Electronic Mail
Administrator HOWTO recommended qmail (http://pobox.com/~djb/qmail.html,
www.qmail.org) as an email server program because qmail is easier to
configure than sendmail, and because qmail is more secure than sendmail.
However, it is probably easier to use sendmail because your linux
distribution probably already includes sendmail, and sendmail is probably
configured already.

Your linux distribution probably already includes many email clients.

SIMPLE EXPLANATION OF HOW EMAIL WORKS

Email works like this: You run an email client. You use the email client to
create and send a message. The email client gives the message to the email
server program. The email server program gives the message to the email
server program on the recipient's computer. The email server program on the
recipient's computer puts the message in the recipient's mailbox. The
recipient's email checker sees the message and tells the recipient 'you have
new mail'. The recipient runs an email client. The recipient's email client
gets the message from the recipient's mailbox. The recipient uses the email
client to read the message.

DETAILED EXPLANATION OF HOW EMAIL WORKS

A message is a block of text data; first the message header, then a blank
line, then the body of the message. The header contains information about
who the message is from, who the message is to, etc. The body of the message
is everything else. The standard format of a message is described in RFC822
(ftp://ftp.isi.edu/in-notes/rfc822.txt).

When you tell your email client to send a message, your email client runs
'sendmail' and writes the message to the standard input of the sendmail
process. If you email server program is something other than sendmail, then
usr/bin/sendmail should be a link to your real email server program. If you
like doing things the hard way, you can skip your email client; you can run
sendmail, type in the message header, type a blank line, type the message
body, and press control-c when done. Or you could create a message in a file
with a text editor, then redirect the file to sendmail with a command like
'sendmail < message_file'. Some older email clients may run rmail instead of
sendmail to send a message. If this is a problem, reconfigure or recompile
the email client to use sendmail instead of rmail; or make /usr/bin/rmail a
link to sendmail.

Redhat 5.0 from about year 1996 has sendmail in /usr/bin, and it seemed to
work when I did 'sendmail < message_file'. Redhat 7.2 from year 2001 has
sendmail in /usr/sbin, not in /usr/bin, and 'sendmail < message_file' does
not work but 'sendmail -t < message_file' does work.

The email server program reads the 'To:' line in the message header and
determines if the message is local or remote. A message to another user on
the same computer is a local message; a message to a user on a different
computer is a remote message. If the message is local, the email server
program puts the message in the user's mailbox.

Usually the email server program delivers the message by changing
to the UID of the user whom the message is to, and then running procmail
as that user, and piping the message to the standard input of procmail.
Then procmail may reformat the message, puts the message in a directory or
file, or forwards the message, depending on what that user's procmailrc
file says. The email server program needs to be able to change to any UID,
so the email server program needs to run as root. If the email
server program is not started from the startup scripts, then the email
server program needs to be setuid root.

If the user has a $HOME/.forward file, most email servers will forward the
message to the address in $HOME/.forward instead of delivering the
message. Or is it procmail which forwards the message?

If the message is a remote message, the email server program determines
which computer to send the message to, and looks up the IP number of that
computer. If the email server program cannot determine the IP number, the
email server program displays an error message saying that the message
cannot be delivered, and the email server program abandons the message. If
the email server program was able to determine the IP number of the other
computer, then the email server program makes an SMTP connection to the
other computer and sends the message. An SMTP connection is a telnet
connection, but made to the SMTP port instead of the telnet port, and using
SMTP commands. If you like doing things the hard way, you could send a
message without using an email client or an email server program by running
a command like 'telnet example.com 25', and then type in the SMTP commands
and the message. You might want to do that if you are having problems with
SMTP connections, so you can see which SMTP commands are failing. SMTP
connections are described in RFC821 (ftp://ftp.isi.edu/in-notes/rfc821.txt).
The following is a sample of an SMTP connection; lines labeled 'S:' are sent
by the computer which is trying to send the message; lines labeled 'R:' are
sent by the computer which is receiving the message:
    S: MAIL FROM:<alice@wonderland.com>
    R: 250 OK
    S: RCPT TO:<bob@dobbs.com>
    R: 250 OK
    S: DATA
    R: 354 Start mail input; end with <CRLF>.<CRLF>
    S: From: "Alice" <alice@wonderland.com>
    S: Message-Id: <199711131704.MAA18447@wonderland.com>
    S: Subject: Have you seen my white rabbit?
    S: To: bob@dobbs.org (Bob)
    S: Date: Thu, 13 Nov 1997 12:04:05 -0500 (EST)
    S: Content-Type: text
    S:
    S: I'm most concerned.  I fear he may have fallen down a hole.
    S: --
    S:                                                 >>alice>>
    S: .
    R: 250 OK

Sometimes the remote computer does not respond to the SMTP commands. Usually
this means the remote computer is too busy to receive a message, or it might
mean the remote computer has crashed. If the remote computer does not
confirm that the message was successfully transferred within a reasonable
period of time, then the email server program on your computer puts the
message into the email queue. Your email server program should be configured
to check the email queue at regular intervals, usually about thirty minutes,
and attempt to send every message in the queue, and delete messages from the
queue if the messages were successfully sent. Your email server program
should also be configured to return messages which have been in the queue
for a long time and still have not been successfully sent, usually after
about five days. The email server program returns the message by changing
the message header so that the message is to the user who sent the message,
and the email server adds a few lines of text to the message explaining that
the message could not be sent, and then the email server program sends the
message to the original sender. The email queue for sendmail is usually a
directory named /var/spool/mqueue.

The email server program on the recipient's computer puts the message in the
mailbox of the recipient. The email server program, the email checker, and
the email client all interact with the mailbox; there will be problems if
the email server program, the email checker, and the email client do not
agree on what the mailbox is. Unfortunately, there are several kinds of
mailboxes. Fortunately, in most linux distributions the various programs are
already configured to expect the same kind of mailbox; so you only need to
worry about mailbox formats if you are trying to install some email program
which was not specially packaged for your linux distribution. I think some
email clients are able to detect the mailbox format and act accordingly.

The mailbox is usually a file in /var/spool/mail. There is one mailbox file
for each user. Each mailbox file has the same name as the user whose email
is stored in the mailbox. If the mailbox is empty, the file does not exist.
The first line of a message header begins with 'From '; thus every line
which begins with 'From ' is the beginning of a message. If a message
includes any lines which begin with 'From ' and are not the beginning of a
message, '>' is added to the beginning of the line, so that the line begins
with '>From ' instead of 'From '. If you want to see some examples, send
some messages to yourself, display your mailbox file in /var/spool/mail,
then read your email. This kind of mailbox was first used with Unix Version
7, so this kind of mailbox is sometimes called a 'V7 mailbox'; it is also
sometimes called 'mbox'.

The other formats to know about (and beware of!) are BABYL, MMDF, MH, and
qmail maildir.  Of these, MMDF is the simplest; it uses a delimiter line
consisting four control-As (ASCII 001) characters followed by CR-LF.  MMDF
was an early and rather crude Internet mail transport; a descendant is still
in use on SCO systems.

BABYL is from an early email system at MIT. It is used by Emacs's
mail-reader mode. If you have an normal mbox mailbox, and you use emacs to
read your email, emacs will automatically convert your mailbox to BABYL
format, which will confuse other programs which interact with your mailbox.
This is not a problem if you delete all messages before any other program
tries to do anything with your mailbox, but how can you be sure you will not
be receiving any new messages while you are reading your email? Using emacs
to send email is not a problem because your mailbox is not involved in
sending email.

MH and qmail maildir are `mailbox' formats which actually burst each mailbox
into a directory of files, one per message.  Running grep on such a
`mailbox' will get you nowhere, since all grep will see are the directory
bits.

Microsoft Outlook Express .mbx mailboxes can be converted to RFC822 format
with mbx2mbox app.

Mail checkers which tell you when you have new email are built into shells
like bash and tcsh; these email checkers are controlled by environment
variables. See the man page for your shell for more information. There are
also email checkers for X windows, which can be configured to beep or blink
or display a message box when you have new email. The most common X windows
email checker is xbiff.

If the message is addressed to a user who does not exist, then the email
server program will add some text saying that the user does not exist or
that the message cannot be delivered, and the email server program will
return the message to the sender, so that the sender will know that the
message could not be delivered. This is called bouncing a message. However,
some people configure their email server programs to ignore undeliverable
messages; in this case the original sender will not know that the message
was not delivered.

Once the message is in the user's mailbox, the user can use any email client
to read the message.

CONFIGURING MAIL ON A COMPUTER WITH CONTINUOUS INTERNET ACCESS

So what do you have to do to configure your linux system to send and receive
email? Nothing! If you have a typical linux distribution; and if you have
tcp/ip networking configured and running; and if you have continuous
internet access like t1, isdn, dsl, or cable modem; and if you have a
registered domain name; and if your domain name is registered as the IP
assigned to you by your ISP; and if you have all your email sent or
forwarded to your username on your computer; and if you leave your computer
running all the time; then the default email configuration which came with
your linux distribution is all you need; email should be working already.

If you do not want to leave your computer running all the time, maybe you
could buy a used notebook computer with a weak battery cheap, and leave the
notebook computer running all the time, and network the notebook computer to
your regular computer. A notebook computer takes up less space, uses less
electricity, and is quieter than a regular computer. You do not need a good
battery because the notebook computer will only use the battery when the
electricity fails. You do not need a powerful notebook computer because the
notebook computer will only be running your email server program; and a
wimpy computer is less likely to be attacked by crackers because a wimpy
computer is not very useful for attacking other computers. And the notebook
computer could serve as a firewall for your regular computer.

If you have email messages sent to your computer, but you do not leave your
computer on all the time, then the other computers will not be able to send
messages to your computer. However, the other computer will keep trying to
send the messages for several days. So if you have your computer on for an
hour or two twice a week, you should get all your messages. However, if you
do this you are wasting the cpu time and bandwidth of the other computers;
if the other system administrators find out you are doing it deliberately,
then they will be VERY ANGRY at you! And they may find some diabolical ways
of making your life miserable. It is ok if your computer is off accidently,
but do not do it on purpose. If you do not want to leave your computer on
all the time, have your email sent to a remote mailbox instead of to your
computer.

If you have a local network, and if you want to send some of your incoming
messages to one computer on your network, and send other incoming messages
to other computers on your network; then you need to configure your email
server program to forward or route the email. This is complicated. See your
email server program documentation for more information. It is probably
easier to reconfigure your network so that all mailboxes are on one
computer, so that you do not have to do any email forwarding or routing.

CONFIGURING EMAIL ON A COMPUTER WITH INTERMITTENT INTERNET ACCESS

Most linux distributions include a default email configuration for
continuous internet access, but do not include a configuration for
intermittant internet access like a dialup PPP connection through an ISP. So
if you do not have continuous internet access, then you are going to have to
reconfigure your email programs.

First you have to decide if you want manual intermittant internet access or
automatic intermittant internet access. Manual intermittant internet access
is where you run a command or script to connect to the internet; and then
after you are finished using the internet, you run another command or script
to disconnect from the internet. Automatic intermittant internet access is
where your computer connects to the internet automatically when you start
using the internet; and your computer automatically disconnects from the
internet when you have not used the internet for some amount of time, like
five minutes; some people call this dial-on-demand. Automatic intermittant
internet access is obviously better, but if you want the best method, then
you should get continuous internet access instead of automatic intermittant
internet access. Manual intermittant internet access gives you more control
over when you connect and disconnect from the internet and is less likely to
accidently connect to the internet; if you pay for internet access by the
minute, then manual intermittant internet access will use fewer minutes and
thus be less expensive than automatic intermittant internet access; manual
intermittant internet access uses less cpu time and memory when not
connected to the internet. To achieve automatic intermittant internet
access, you can either configure pppd to connect and disconnect
automatically, and start pppd from your startup scripts and leave pppd
running all the time; or you can use diald. See the pppd and diald howtos
and man pages for more information.

SIMPLE BUT INEFFICIENT EMAIL CONFIGURATION WITH INTERMITTANT INTERNET ACCESS

If you have automatic intermittant internet access, then sending email is
the same as if you have continuous internet access (except automatic
intermittant internet access is slower than continuous internet access). You
should be able to use the default email server program configuration for
automatic intermittant internet access. However, receiving email is a
problem. Your computer knows when it is trying to send email, so it can
connect to the internet to send an email. But your computer has no way of
knowing when another computer will want to send email to your computer, so
your computer cannot connect to the internet whenever some other computer
wants to send email. So other computers cannot send email to your computer.

So each user of your computer needs one or more remote mailboxes. Your
normal mailbox is on your computer; this is your local mailbox. A remote
mailbox is on a different computer. Your email client cannot access your
remote mailbox directly because your remote mailbox is not on your computer.
Your email client must use a remote email protocol like POP or IMAP to
request your email from the remote computer. POP is more common; IMAP is
better. Both work by telneting to a specific port, sending text commands,
and waiting for text responses, similar to SMTP. For more information about
POP, see ftp://ftp.isi.edu/in-notes/rfc1939.txt. For more information about
IMAP, see ftp://ftp.isi.edu/in-notes/rfc2060.txt. POP is sometimes called
POP3 or popmail. Some email clients are not capable of working with remote
mailboxes; you need to find an email client which is capable of working with
remote mailboxes. Each user must configure the email client for each of that
user's remote mailboxes. The email client needs to know the name of the
computer which hosts the remote mailbox (the name of the POP or IMAP
server), the username of the remote mailbox, the password of the remote
mailbox, and what protocol to use. If the email client has no way to
configure the protocol, then the email client probably assumes POP and
cannot use IMAP. Since you never access remote mailboxes directly, you do
not have to worry about the mailbox format; the remote email server program
has to worry about the mailbox format, but that is not your problem, and
there is nothing you can do about it.

When your email server program tries to send a message, it will take a long
time for the other computer to respond, because your computer must connect
to the internet before the request can be sent to the other computer. Your
email server program might not want to wait that long. So you should
probably increase the timeout in the configuration of your email server
program, so your email server program will be willing to wait longer. For
sendmail, edit /etc/sendmail.cf and look for a line like '#O DialDelay=10s'
and change it to something like 'O DialDelay=10s'.

So to receive email through a remote mailbox, tell your friends to send
email to the address of the remote mailbox. There are no mail checkers to
tell you when you have new mail in a remote mailbox. The only way to find
out if you have any email is to try to read your email. Your automatic
intermittant internet access should connect automatically when your email
client downloads your email; and it should disconnect automatically when
done.

Your ISP probably offers remote mailboxes. There are internet mail services
which offer remote mailboxes. Institutions like universities and companies
sometimes offer remote mailboxes to their people. If someone says they have
a popmail server, that means they have remote mailboxes.

If you cannot get enough remote mailboxes, it may be possible for more than
one user to share a remote mailbox. It used to be possible to have email
addresses like user1%mailbox@computer.com, user2%mailbox@computer.com, etc;
where user1 and user2 share mailbox@computer.com. I think it is still
possible, but you should not do it because some people say it will not be
allowed in the future, and because not all email programs support it. You
could also have multiple users share the exact same email address, and the
subject line, the first line of the message, or an X... line in the header
should say which user the message is for. However, this does not work very
will because whenever anyone wants to send a message to one of the shared
mailbox users, the sender must remember to format the message in the special
manner. If one of the users sends a message to someone else and that person
tries to reply to the message, that person will probably tell the email
client to generate a reply, and the email client will address the reply
incorrectly because the email client does not understand the special
addressing scheme.

Most linux distributions run sendmail or some other SMTP email server
program from the startup scripts so that it can receive SMTP email at any
time. With intermittant internet access, you never receive SMTP email (you
send SMTP email, but all email you receive is POP or IMAP). Therefore you
might want to edit your startup scripts and comment out the command to start
sendmail, to save cpu time and memory. However, the sendmail process which
runs all the time is also responsible for resending failed emails from the
email queue. So if you don't have sendmail running all the time, then you
will not be able to automatically resend failed emails. You will have to
manually resend failed emails. It is probably easier to leave sendmail
running all the time.

So configuring email on a computer with automatic intermittant internet
access is not difficult. The most difficult part is getting a remote
mailbox and configuring the email client to use the remote mailbox; and you
can tell your users to do that themselves. So the system administrator does
not need to do much. However, this is not an efficient configuration.

The problem is that you waste time connecting and disconnecting from the
internet all the time. You connect each time a user sends an email. You
connect each time a user reads email. If you pay by the minute for your
internet access, you probably want a more efficient mail configuration
because a more efficient configuration will use fewer minutes and therefore
will be less expensive. 

EFFICIENT BUT COMPLICATED EMAIL CONFIGURATION WITH INTERMITTANT INTERNET ACCESS

It would be more efficient to save up outgoing messages in an email queue;
then exchange messages by connecting to the internet, sending all outgoing
messages, getting all incoming messages from all remote mailboxes of all
users, putting the incoming messages into the local mailboxes, and
disconnecting from the internet. We should have an email exchange script
which does all of there things, and then we could set cron to run the email
exchange script at regular intervals, like every four hours or once a day;
or we could run the email exchange manually whenever we want.

The easy, messy way to save outgoing messages in an email queue is to do
nothing. When you send a message, your email server program will try to send
the message to the other computer. If you are not connected to the internet
and do not have automatic intermittant internet access, then your email
server program will fail to send the message. Your email server program will
probably display or email you a nasty error message, and then your email
server program will put the message in the email queue.

You can save yourself a lot of error messages and some cpu time by
configuring your email server program to put messages in the email queue
immediately without trying to send the messages. For sendmail, edit
/etc/sendmail.cf and change the line which says 'O DeliveryMode=background'
to 'O DeliveryMode=defer'.

To send all messages in the email queue with sendmail, run the command
'sendmail -q'.

So sending many email messages at once is easy. But how can we get email
messages from many remote mailboxes for many users at the same time? How is
the system administrator supposed to keep track of all the remote mailboxes
used by all the users? Ideally we should tell each user to put information
about that user's remote mailboxes in a configuration file in the user's
home directory, perhaps $HOME/.remote_mailboxes, and then we should have a
program which checks all the home directories for these configuration files,
and then gets all messages from all remote mailboxes and puts the messages
in the local mailboxes. Unfortunately, I do not know of any program which
can do this.

Email exchanges would be even more efficient if we could configure our email
so that if someone connected to the internet for some other reason, like to
run a web browser, then the computer would run an email exchange
automatically, even if the email exchange was not scheduled for this time.
And while the computer is connected to the internet, outgoing emails should
be sent immediately instead of going into the queue, and checks of the
remote mailboxes should occur much more frequently, like every fifteen
minutes, for as long as the computer is connected to the internet. Then when
the computer is disconnected from the internet, the email exchanger should
revert to the other configuration. To make this work, we would need two
configurations for the email server program, and we would need to make some
scripts which shut down the email server program, change configuration
files, and restart the email server program.

Email exchanges would be even more efficient if we bypassed PPP and the
standard email protocols. When we send data to our ISP using a PPP
connection, the data is organized into packets, headers and footers are
added to the packets, and many eight bit bytes are converted to pairs of
seven bit bytes. The total number of bytes transferred over the modem may be
twice as large as the number of bytes of real data actually being sent. The
standard email transfer protocols also send more bytes than neccessary, and
spends a lot of time waiting for responses. So if you really want to
transfer your email while being connected to the internet for the shortest
possible time, you need to minimize the number of bytes transferred, and you
need to minimize the wait times; you need to do the following: First gather
together all outgoing messages. Pack the outgoing messages into a file, and
compress the file. Gather together all the information about remote
mailboxes. Now use a modem telecommunications program like minicomm, pcomm,
or xc to call your ISP and log into your shell account. Transfer the data
about remote mailboxes and start a program on the remote computer to
retrieve messages from the remote mailboxes; this program should compress
the messages into a pipe, and this program should run in the background.
While that program is running, zmodem upload your file of outgoing messages.
Zmodem the file directly into a pipe to a program which uncompresses the
stream of data, extracts the messages, and sends the messages. By now there
should be several incoming messages in the first pipe; connect the incoming
message pipe to zmodem and download to a file. When zmodem is done, log out
of your shell account at your ISP and hang up the modem. Uncompress the file
you downloaded, extract the messages, and put the messages into the local
mailboxes. Note the use of pipes; your ISP's computer is probably a very
fast computer with very fast internet access; thus your ISP's computer
should have plenty of time to uncompress and send each message while it
waits for the slow modem to transfer the next message. You could speed up
the transfer more by filtering spam at your ISP's computer. This is
hideously complicated, but it is very fast if you can make it work. I do not
know of any programs which can do this.