Search the FAQ Archives

3 - A - B - C - D - E - F - G - H - I - J - K - L - M
N - O - P - Q - R - S - T - U - V - W - X - Y - Z - Internet FAQ Archives

Mailing list management software FAQ
Section - 2.08 Loop detection and elimination

( Single Page )
[ Usenet FAQs | Web FAQs | Documents | RFC Index | Houses ]

Top Document: Mailing list management software FAQ
Previous Document: 2.07 Features and usability for users
Next Document: 3.00 MLM's in general
See reader questions & answers on this topic! - Help others by sharing your knowledge
One of the worst things that can happen to your list is to have a mailing
loop develop.  Especially when they involve big lists, loops are a huge waste
of time and also terribly embarrassing for you as an administrator.  By
definition, they happen when two servers start sending mail to each other,
neither realizing the other is a machine, and while that's bad enough, when
it's on a mailing list a whole bunch of "innocents" get to listen in on the
exchange.  The most typical offenders in mailing loops are mail-to-news
gateways, mail hosts that send error messages helter skelter, vacation
programs that reply "I'm away" to *everything* that arrives, and other MLM's.

If everything is set up "right," mailing loops never should happen.  "Right"
means that the headers of list mail are generated correctly (precedence
"bulk", for example, stops many vacation programs from replying), that the
errors-to address for list mail points somewhere besides the list submission
address, and several other points that are better covered elsewhere.  No
matter how well *your* MLM sets things up, though, some misconfigured system
elsewhere can always return mail to you in a way that provokes a loop.  In
this situation, detecting and stopping the loop *immediately* is paramount.

Mail-to-news gateways and linked mailing lists are a special case in terms of
the looping problems they generate.  The concept isn't difficult, but
gateways open up whole new possibilities for bad days -- you can get systems
that don't respond with the correct error messages, or local mail-to-news
gateways whose territories overlap, or two systems each stripping the markers
the other had put in for loop detection, or, or, or ...  If you plan to gate
your mailing list to USENET, look *very hard* at the facilities each MLM
offers for detecting and eliminating mailing loops.  ListProc and LISTSERV
are currently the best at this function, and SmartList offers good facilities
as well; it could be added to any of the source-provided packages, and you'd
be doing us all a service :-).

The basic strategy used by both ListProc and LISTSERV is to maintain a cache
of recent messages they've delivered (say, the past several hundred or
thousand messages), in which they store both the Message-ID field and a
checksum created from the message body excluding white space.  They then will
refuse later messages that generate the same checksum or have the same
Message-ID header.  SmartList also runs a cache, but keeps in it only the
Message-ID header.  (A ListProc FAQ which I will briefly answer here is:
ListProc 6.0c calculates its checksums using the Unix "sum" program, which
often results in non-unique numbers, and thus false positives on the test.  A
patch is available for the server so that it uses the much more effective MD5
checksum system, and you should apply that patch if you use ListProc 6.0c.)
All three MLM's add headers of their own to each message, then check for such
headers when a new message arrives (if they're found, that's good evidence of
a loop).  They also check subject lines and senders to avoid common goofs:
i.e.  they refuse postings from any user named "listproc" or "listserv"
because that's almost certainly a server, and they also recognize that
subject lines starting with "Delivery delayed:" are worth suspicion.
ListProc and LISTSERV go a few steps farther, scanning the message body for
common signs of re-posting (a second From: or Message-ID: line, for example),
and doing other tricks the authors consider proprietary and won't reveal :-)

When a duplicate message is found, ListProc and SmartList forward it to the
list owner.  CREN ListProc, as it does so, also calculates a checksum on the
error report it's forwarding (excluding the attached message text, if any),
so that if the list owner's MTA bounces the message a secondary loop can be
avoided.  LISTSERV returns the duplicate to the sender, with a note on why
it's being returned and how to change the text if it is in fact legitimate.
With LISTSERV's strategy, which was designed for the list owner's
convenience, new loops can (and do) develop between it and other servers.
However, they're invisible to the list and they're always limited: LISTSERV
"serves off" (ignores) any address that generates ten error mails in a row,
thus breaking the loop on its tenth iteration.

When all other strategies fail, the final damage-control step in both
ListProc and LISTSERV is to count and limit the messages that can be
distributed per list, per day.  Most list owners set the limit around 50 on
LISTSERV, so 50's the maximum number of bogus messages that can go out to
your list (!).  If you get a bad loop, and they are rare, the *final* step
for you to take is: post a copy of the loop's result to the relevant MLM's
support list, with a message saying "what happened?"  You will either find
out that there is something you can do to avoid the loop in the future, or
you will see that the next version of the software has been enhanced to
detect it :-) With ListProc, if the problem was that the server didn't
recognize a mailer address or returned-message subject, you can also edit its
configuration file to add items to the list of checks, thus protecting
yourself until the new and improved version arrives.

User Contributions:

Comment about this article, ask questions, or add new information about this topic:

Top Document: Mailing list management software FAQ
Previous Document: 2.07 Features and usability for users
Next Document: 3.00 MLM's in general

Single Page

[ Usenet FAQs | Web FAQs | Documents | RFC Index ]

Send corrections/additions to the FAQ Maintainer:

Last Update March 27 2014 @ 02:11 PM