|
Archive-name: mail/setup/unix/part1
Last-modified: Mon Feb 21 09:57:01 EST 2000
UNIX EMail Software - a Survey
Chris Lewis
clewis@ferret.ocunix.on.ca
[and a host of others - thanks]
Copyright 1991-1998, Chris Lewis
Redistribution for profit, or in altered content/format
prohibited without permission of the author.
Redistribution via printed book or CDROM expressly
prohibited without consent of the author. Any other
redistribution must include this copyright notice and
attribution.
Note to the patient readers who noticed that nothing has changed in this
FAQ since 1996...
Email systems have changed radically over the past few years, and I'm
beginning the daunting task of bringing this FAQ into the new world.
I'm planning a lot of changes:
- Adding POP/IMAP discussions, and common implementations
- Extensive coverage of anti-spam measures resources, and
packages.
- Updating recommendations to include things like the phase out
of UUCP, predominance of POP/SMTP/MIME etc., S/Mime, PGP
etc.
- Other suggestions?
I've started off the ball by mostly changes in the second and third parts:
- updated sendmail
- dropped IDA sendmail references.
- dropped EASE references
- begun the deprecation of obsolete solutions (UUCP, UUCP maps etc)
- added exim.
- added qmail.
| - updated MMDF
It would help a lot if anyone wanting to add a section on their favourite
email topic (UNIX please!) could write it and send a copy to me. I'll also
be dredging through my archives to find previous comments that haven't yet
been added.
Changes are marked with a preceding "|". You can skip to them
by typing g^| in (most) newsreaders.
Note: this FAQ has been formatted as a digest. Many newsreaders
can skip to each of the major subsections by pressing ^G.
Please direct comments or questions to mailfaq@ferret.ocunix.on.ca -
note Reply-to: line - automatic if you reply to this article.
Many changes made in the second and third parts.
Subject: Introduction
Configuring electronic mail systems can be quite a complicated
subject. Often far more complicated than, say, setting up
a Usenet news feed. This is because, unlike news, email is
expected to traverse multiple types of networks using their own
protocol, whereas, Usenet news tends to be a single protocol
supported by hook or by crook on different networks.
This document is intended for system administrators who need to
know how to set up their UNIX systems for email communication with
the outside world. It is intended for the email-naive SA
who gets more than a little confused by the acronyms, RFC's and
plethora of software.
This is intended to be a general survey of the software available,
so I won't spend too much time on some of the details. Most of
the available software comes with documentation that can
explain things much better than I can.
Additional detail can be obtained from several sources, such as:
Quarterman, John S.: "The Matrix -- Computer Networks
and Conferencing Systems Worldwide", Digital Press 1990,
(Order No. EY-C176E-DP), ISBN 1-55558-033-5.
Adams, Rick and Frey, Donnalyn: !%@:: A Directory of Mail
Addressing and Networks, 3rd Ed., O'Reilly & Associates 1993,
Provides a good reference for people seeking information
on how to access the various email networks.
ISBN 1-56592-031-7.
Kehoe, Brendan P.: Zen and the Art of the Internet: A
Beginner's Guide, Second Edition, Prentice Hall 1992,
ISBN 0-13-010778-6. Edition 1 is available via FTP on
cs.widener.edu in the tar file zen-1.0.tar.Z. [I think]
Krol, Ed: The Whole Internet: User's Guide & Catalog.
First edition, O'Reilly & Associates Sept. 1992.
ISBN: 1-56592-025-2. Very good introduction to
the Internet, history, facilities, uses, services,
etc. I learned a lot.
Albitz, Paul & Liu, Cricket: DNS and BIND, First edition,
O'Reilly & Associates, October 1992. ISBN: 0-56592-010-4.
Describes in great detail everything from what a domain
is, to how to install and configure BIND. A *MUST* for
people setting up large networks, or connecting
machines to the Internet. It has become mandatory reading
for network administrators in a large corporation for
good reason.
Costales, Bryan and Allman, Eric and Rickert, Neil: Sendmail.
O'Reilly & Associates, Nov (?) 1993. ISBN 1-56592-056-2
(ISBN from galley proof, which I've had a preview of).
An absolute necessity for anyone diving into the configuration
of sendmail. The material is presented in a very clear
form, and is quite exhaustive in its coverage. Perhaps a bit
too wordy and overlong, but that's a more than welcome contrast
to previous documentation (or lack thereof) on sendmail.
Further, this is primarily oriented towards UNIX email systems.
This is unfortunate, because it would be nice to have a general
document covering email in all of its forms. However, each
operating system tends to have radically different email mechanisms,
so it would be difficult to do justice to any other environment.
It seems more useful to cover one environment well here, and have
companion documents for other environments. Speaking of which,
why hasn't anybody else stepped in to do FAQs on other environments?
Like DOS, Mac etc.
And finally, this document is not intended to be pedantically
correct. Knowledgeable readers will know that I'm glossing
over a lot of detail, and absolute precision has been balanced
against readability and effectiveness in helping people get
going.
Subject: Layout
This FAQ is laid out in the following sections:
+ An overview of how mail systems go together.
+ A glossary of the important terms to know.
+ A list of general do's and don'ts of mail systems.
+ Configuration Issues
+ Several suggested mail configurations.
+ General overviews of specific software.
Subject: Electronic mail - A General Overview of Structure
Electronic mail generally consists of three basic pieces:
1) The link level transport - which could be
UUCP, TCP/IP, or a host of others. We'll call
this the "transport medium" (TM)
2) the "Mail Transport Agent" (MTA) which is responsible for
transporting mail from source to destination, possibly
transforming protocols, addresses, and routing the mail.
The MTA often has several components:
- Routing mechanisms
- Local delivery agent (LDA)
- Remote delivery agent
Many MTA's have all of these components, but some
do not. In other cases, it is possible to replace
certain components for increased functionality.
3) The "User Agent" (UA) is the user interface -
the software that the user uses to read his mail,
sort things around in folders, and send mail.
Sometimes called "Mail User Agent" (MUA).
Subject: Glossary
Rather than alphabetic, this glossary tends to group terms
referring to similar functionality together.
Transport Medium:
UUCP (Unix to Unix Copy Program):
Back in the mists of time, UNIX systems communicated only
over RS232 serial lines, usually over modems. UUCP is a
suite of programs developed back in the early 70's to
provide this communications link. All that UUCP does is
transfer files from one system to another. There is an
additional mechanism where one system can direct the
destination system to run a file through a specific program.
Electronic mail in UUCP is simply requesting the destination
machine to run "mail" on a data file.
UUCP communicates by means of "protocols", the most common
being "g", a method for transmission of data over telephone
lines and ensuring that the data is not corrupted. There
are several other protocols, none universally available,
and most oriented towards communication media other than
telephone voice lines (such as dialup X.25, PAD X.25, or
LAN connects).
UUCP operates over fixed system-to-system links, so sending
mail from one system to another often has to traverse
other intermediate systems.
| If you like source, Taylor UUCP is an excellent full-featured
| implementation of UUCP, with many enhancements to deal with higher
| modem speeds. It is FreeWARE.
| UUCP mail protocols (bang paths) are now being deprecated, because
| DNS and MX etc., are making it wholly unnecessary.
TCP/IP (Transmission Control Protocol/Internet Protocol):
TCP/IP is a protocol that allows any system on a network to
talk "directly" to any other, by passing packets of
information back and forth. TCP/IP (and its later relative
OSI) is usually used over networks built on top of Ethernet,
Token-Ring, Starlan and other LANS.
SMTP:
Or, "Simple Mail Transfer Protocol", is the communications
protocol used most commonly over TCP/IP links in UNIX
environments for mail. SMTP usually operates directly between
the source and destination machines, so intermediate machines
don't get involved (except for gateways, see below). SMTP
is usually part of the MTA.
SLIP (Serial Line Internet Protocol):
SLIP is an implementation of TCP/IP designed for use over
RS232 serial lines (ie: modems). The other difference is
that some SLIP implementations have the ability to "dial the
phone" to make a connection for a specific transfer, whereas
LAN TCP/IP is physically continuously connected. You'd also
need TCP/IP to run a SMTP mail connection.
PPP (Point-to-Point Protocol):
A successor to SLIP.
X.25/X.29:
X.25 is a packet switched data network which is usually
half-duplex. In this context, it's really an alternative
to dialup over voice telephone lines with modems. X.25
is available in several "flavours", either direct X.25
trunk connects over leased lines, through "PAD" interfaces,
or by ordinary dialup modem access to X.25 "ports".
To be useable in the context of mail transfers, you also
have to use a file transfer protocol/mechanism of some
sort on top of X.25. The most common being UUCP "f" protocol
(through PADS or dialup), or "x" with direct X.25 connects.
Whether you use X.25 or phones plus modems depends on a number
of factors - usually the determining factor is cost. In North
America, high speed modems (eg: 9600 baud and above) over telephone
lines tends to be less expensive. However, Europe's really
wierd phone system structure usually makes X.25 more cost-effective,
and therefore, X.25 use in UNIX mail systems is much more common
in Europe than North America.
X.29 is the command set used to configure and establish
X.25 connections when you're using asynchronous connections
to a PAD.
Networks:
Internet:
An "internet" is a network comprised of computers that talk
to each other using TCP/IP, and usually SMTP for mail.
The "Internet" is a vast network of hundreds of thousands of
machines using SMTP protocol mail, communicating with
each other over relatively high speed lines. But not all
"internets" are connected to *the* Internet.
The Internet grew out of a US government funded project in
inter-computer communications that grew into an enormous network
of systems.
| One of the principal characteristics of this network is that
machines are addressed by domain names which identify the
destination, rather than addresses that are constructed out
of the route from machine-to-machine-to-machine.
UUCP Network:
The UUCP network is that set of machines that talk to each other
via UUCP. Sending mail through this network requires that the sender
know the network topology of UUCP links, and specify a path from one
machine to the next. (There are, of course, ways around this.
See the section on "do's and don'ts".)
Mail addresses:
Addresses:
An email address is a method of specifying a given person on
a specific machine. There are scads of conventions, usually
determined by the presence of "@"'s, "!"'s and other special
characters in the name. An address usually consists of
two parts: a userid/name and a machine specification.
A Domain address usually looks like:
userid@domain-address
Whereas a UUCP address usually looks like:
siteA!siteB!siteC!userid
Domain Addresses:
Domains are a way of uniquely specifying a destination.
Much like a postal address, a domain specifies a set of
progressively more restrictive "domains" of the potential
address space. It would perhaps be illustrative to give an
example:
clewis@ferret.marketing.fooinc.com
You read these things right to left: "com" means the
commercial domain. "fooinc" is the name of an organization
within the commercial domain. "Marketing" is the name of a
suborganization within fooinc, and ferret gives the name of
a machine (usually). Domains can have any number of levels.
The top level domain (com in the above example) has many
possible values. In the United States, "com", "mil", "edu",
and "gov" are fairly standard. Elsewhere, the top level
domain tends to be a country code, the second level tends to
be a province or state, OR a classification like "edu" or "ac"
for academic (such as ac.jp, go.jp, ac.uk, edu.au, etc)
and the third an organization. But, for example, there are
many .com and .edu sites in Canada and other countries.
FQDN
A fully-qualified-domain-name (FQDN) has a entry for each
level of the domain, from individual machine to top-level
domain. In many cases, an organization has implemented an
organizational "gateway" at a higher level of domain, so
that people from outside don't have to specify FQDN's to get
to a specific person. In the above example, for instance,
"fooinc.com" may be sufficient to get to anyone inside
fooinc, and "ferret.marketing" may not be necessary.
On the other hand, people sometimes leave out the higher
levels of the address, as in "ferret.marketing".
This is a bad idea - because if the mail is cc'd out of the
organization, chances are the external recipient cannot reply,
because "ferret.marketing" is incomplete. So use addresses
that are specified sufficiently for external users to use.
(fooinc.com if a organizational gateway is used, the whole
ferret.marketing.fooinc.com if not)
NIC
Internet TOP-LEVEL domains (edu, com, gov, mil) are controlled
by a single organization, the NIC (internic.net). An organization
"gets a piece" of the namespace by registering with the NIC, and
then they are free to administer their own namespace (everything
under fooinc.com) as they choose. The same is true for foreign
countries; Once they have their top-level domain (usually the
two-letter ISO country code) registered with the NIC, they do
the rest, and divide it as they see fit.
In contrast, on UUCPnet, all machine names everywhere share a
single flat namespace. So it is important to choose a name
that has not been used before. (See do's and don'ts). This is
why FQDN's help. We can tell the difference between
ferret.fooinc.com and ferret.blah.edu by their full names.
(Instead of UUCP paths which may turn out to be wrong, and
autorouting will probably send the mail to the wrong machine)
MX record:
A non-SMTP/Internet site that wishes to register on the Internet
will need to get a "nearby" Internet site to set up a MX
record for them. An MX record is essentially a domain-server
database record that (effectively) registers your domain name
on the Internet, and indicates that the Internet site knows
how to forward mail to you. Usually via some non-SMTP/Internet
route, such as UUCP. You can get an MX record for one site, or
a "wildcard" MX record so that you can have your own subdomains.
Bang-Paths:
With UUCP mail, the MTA has to specify a route to get from one
machine to another. "A!B!C!userid" means go to machine A,
then B, then C, then user "userid" on C. You should strive,
however, for a MUA that allows you to use domain addressing,
and let the MTA figure out the bang routing as appropriate.
Miscellaneous:
Gateways:
There are several meanings of this term, only three are relevant
here.
The first is a mechanism for getting from one network to another
network that uses different protocols.
The second is a mechanism for getting from one logical (often
organizational) network to another using the same protocol.
Often for example, there will be a LAN in one department of
an organization, and one machine in the LAN has the connection
to another LAN in another department. This means that mail from
one LAN to the other has to pass thru the gateway machine.
Another form, which we'll mention later is that of mail to
news gatewaying.
Routers:
There are several definitions, but the most important is that
part of the TA that figures out how to send a message to
a given machine. This often uses a database that provides
routes from one machine to the other machines on the network.
Smarthost:
In many cases, your machine won't know how to get to a specific
destination. You can usually set up your mail system to send mail,
that it doesn't know how to deliver, to a machine that is more
likely to.
RFC's:
A set of documents that include formal descriptions of mail
formats used on the Internet, and are adhered to by many
non-Internet systems. More specifically, in the "worldnet"
of Usenet, Internet and UUCP, the RFC's set the standards
for mail exchange. RFC822, 1123 and 976 are the most important
for Internet/UUCP mail.
It should be pointed out, however, that there are some
regions where the RFC's are not entirely respected. For example,
the British academic email networks (JANET) uses domains, but
they're specified backwards (they drive on the wrong side of
the road too ;-).
MIME:
Mime is the official proposed standard format for multimedia Internet
mail encapsulated inside standard Internet RFC 822 messages. Facilities
include sending multiple objects in a single message, character sets
other than US-Ascii, multi-font text messages, non-textual material
such as images and audio fragments, and other extensions. For an
overview of Mime, see ftp.uu.net:networking/mail/metamail/MIME-overview.txt.Z.
The defining document is Internet RFC 1341: N Borenstein & N Freed,
``Mime (Multipurpose Internet Mail Extensions) mechanisms for specifying
and describing the format of Internet message bodies'' (June 1992).
Also see RFC 1344: N Borenstein, ``Implications of Mime for Internet
mail gateways'' (June 1992).
RFC1341 and 1342 have since been superceded by RFC 1521 and 1522.
Mime covers only message bodies, not message headers; to see how to
represent non-Ascii characters in message headers, see Internet
RFC 1342: K Moore, ``Representation of non-Ascii text in Internet
message headers'' (June 1992).
X.400:
A CCITT standard for email formats, more or less an alternative
to RFC 822/976/1123. This format will probably start taking over
from RFC 822/976/1123 mail. It is likely to (already has?) become an
ISO/IEEE standard along with OSI etc.
"The Maps":
A set of files describing machine-to-machine links distributed
over Usenet in the group comp.mail.maps. These are usually posted
on a monthly schedule, and can be automatically received and
transformed into a routing database that describes the "optimal"
route to each machine. These are operated by the "UUCP Mapping
Project". See the README posted along with the maps for
more details.
Aliases:
Aliases are a mechanism by which you can specify the destination
for mail on your machine. Through the use of aliases you can
redirect mail to "virtual userids". For example, you should
have a mail destination on your machine called "postmaster", which
is aliased to send the mail to the System Administrator (ie: you
probably). Aliasing often also permits you to send mail to groups
of users (not necessarily on the same machine as you) pipelines of
commands or to specific files.
Mailing lists:
Are similar to Usenet newsgroups. They are usually aliases
pointing to groups of users, and allow mail to be sent to the
whole group at once. Mailing lists are set up to carry certain
subjects. The difference between a mailing list and a Usenet
newsgroup is that the messages are sent by mail, probably as
a copy to each recipient, rather than broadcast.
Subject: Do's and Don'ts:
1) Register a domain name. Even on UUCP, where <machine>.UUCP is often
used as a kludge, it is MUCH preferred that you obtain a real
domain address. If you are directly connecting to the Internet,
you will get one as part of your registration with the NIC.
If you aren't connecting directly to the Internet, obtaining a
registration will usually require you finding a nearby friendly
Internet site willing to act as a mail forwarder to you from
the Internet - the site that will set up a "MX record" for you.
Many sites will do this for you for free, and several of the
commercial email services (eg: uunet) will do it for you for a
nominal charge (without requiring you buy the rest of their
services).
There are occasions where you can join what is called a "domain
park". These are most often small regional groups of systems that
have gotten one of their number properly registered as a domain,
and provides forwarding services out to other systems. For
example, in my address "ferret.ocunix.on.ca", "ocunix.on.ca"
is a domain park made up of the Ottawa-Carleton UNIX User's Group,
one of the other machines in the group provides a gateway between
our systems and the Internet.
2) If your machine is going to "speak" UUCP to the outside world,
choose a unique UUCP name. You can find out whether a name you
want is taken by consulting the UUCP maps. Or by asking someone
else who's using them.
3) Register your machine with the UUCP Mapping Project if you're going
to use UUCP. Information on how to do this is included in the
monthly maps postings in the file "README". This is usually only
required when your machine talks UUCP to the outside world, or when
other machines have to address you by your UUCP name. If you don't
do this, somone else may choose the same name, and gross confusion
will arise when smart routers won't be able to tell whether to send
a piece of mail to you, or your doppelganger[s]. If you register
with the UUCP Mapping Project, you have prior use, and people who
choose the same name afterwards will be told to get a new one.
If you're "behind" an organizational gateway, don't do this.
(Your organizational gateway is the thing that needs to be
registered)
If you do fill in a map, please take the time to fill it in
carefully, giving contact people and phone numbers. Just in
case your machine goes crazy and starts doing something nasty.
Note expecially the latitude and longitude. Get it right,
or omit it. Brian Reid gets really annoyed with sites that
are half a world away from where they really are.
4) If you're going to be setting up multiple machines, have only
one or two connections to the outside world.
5) Install a mail system that understands domain addressing, even
if you aren't registered. (In fact, all of the suggested
configurations in this FAQ do)
6) *Never* use UUCP bang-routing with the MUA if you can possibly
avoid it - each of the suggested mail configurations provide
mechanisms where you, the user, do not have to specify routes
to the MUA - you can specify domains, and the TA will do the
routing (possibly bang-routing) for you.
Important: many mailers that understand UUCP attempt to be
pedantically "UUCPish" in the construction of headers, such
as generating "bang routes" in From:/To: etc. lines. Which,
given that the whole "mail network" is generally converging on
more Internet-like standards, and that even UUCP sites are
using fully domain-capable mailers, is a big mistake. RFC976
attempts to codify a "meta standard" that allows the coexistance
of RFC822 (Internet mail) with UUCP-based networks. What
this means is, essentially, that headers are formed in the
SMTP form, even if the transport will be via UUCP. Unfortunately,
however, many mailers insist on "UUCP-izing" perfectly useable
Internet/domain headers. "Fixing" them to prevent this is sometimes
difficult. Sendmail is almost always a problem in this regard.
7) Find a friendly neighboring SA to help. A SA who has already
operating mail in your area will help smooth over the regional
"gotchas" that are bound to crop-up. And advise you on the
right software to use, where to obtain it, and how to install it.
8) Do NOT use "any old" Map unpacking program. Most available
map unpacking programs automatically run the shell (or shar)
to unpack map articles. Since it is trivially easy to forge
map articles, using this type of unpacking program can
easily let very destructive trojan horse or virus programs
into your machine.
The two specific map unpackers described in this FAQ are known
to be secure from such attacks. Do not run any other unpacker
unless you are aware of the issues and can inspect the code for
such vulnerabilities. [If you know of other "secure" map
unpackers that are generally available, please let me know]
9) If the people on your site, or small network, receive mailing
lists, it's often a good idea to gateway them to news:
Netnews often performs many of the same services as email.
The primary difference is that messages are centrally stored,
rather than delivered to individual's mailboxes, and that
distribution looks more like a broadcast then a set of point-to-point
communications. This means usually means that news can handle more
volume, more efficiently, then email can.
Because of the differences (and also the similarities) people often
want to tie news and mail together. This is known as "gatewaying."
For example, a small software development site might subscribe to the
X Windows mailing list. Rather than have (say) eight copies of each
mail message sent to their host, they would rather have it stored as a
local newsgroup that everyone in the company can read, and which can
be centrally archived. This is a typical use of a "mail to news"
gateway. When a user makes a posting to this local group the article
should be sent back out to the mailing list; this is a typical use of
a "news to mail" gateway.
On a larger scale, the "inet" groups are bi-directional gateways of
Internet mailing lists. Within mainstream Usenet, many popular
groups such as comp.windows.x, comp.protocols.tcp-ip, comp.unix.wizards,
and so on, are gatewayed to mailing lists and back.
Many subtle issues often come up when gatewaying mail and news, so
unless you are experienced you should use one of the already-available
packages for your local organization. For example, you probably do not
want to write a brand-new Perl script and create a new "inet" newsgroup.
The C News distribution includes some basic gateway tools in the
contrib/nntpmail directory. Many people use Rich $alz's "newsgate"
package that appeared in comp.sources.unix Volume 24; it includes
discussion of some of the more subtle issues that come up.
Before starting a mailing list gateway, apart from the technical aspect
of the job you should also be aware of one important point: mailing-lists
are considered private, whereas newsgroups are public.
One can know who gets a list, but not who reads the group. It is always
wise to get the authorization of the mailing-list manager and of the readers
before creating a mail/news gateway.
10) If you're connecting to the Internet, or are setting up a large local
internet, you really should get a copy of the DNS and BIND book mentioned
in the bibliography.
|
Comment about this article, ask questions, or add new information about this topic: