Revision History | ||
---|---|---|
Revision 2.1 | 2002-08-20 | Revised by: sm |
New sections on Security and Software History, lots of other small additions and cleanup | ||
Revision 2.0 | 2002-07-30 | Revised by: sm |
Rewritten by new authors at Starcom Software | ||
Revision 1.4 | 1995-11-29 | Revised by: vs |
Original document; authored by Vince Skahan. |
Here we discuss the basic concepts behind the operation of a Usenet news system.
Xref: news.starcomsoftware.com starcom.tech.misc:211 starcom.tech.security:452 Newsgroups: starcom.tech.misc,starcom.tech.security Path: news.starcomsoftware.com!purva!shuvam From: Shuvam <shuvam@starcomsoftware.com> Subject: "You just throw up your hands and reboot" (fwd) Content-Type: TEXT/PLAIN; charset=US-ASCII Distribution: starcom Organization: Starcom Software Pvt Ltd, India Message-ID: <Pine.LNX.4.31.0107022153490.30462-100000@starcomsoftware.com> Mime-Version: 1.0 Date: Mon, 2 Jul 2001 16:27:57 GMT Interesting quote, and interesting article. Incidentally, comp.risks may be an interesting newsgroup to follow. We must be receiving the feed for this group on our server, since we receive all groups under comp.*, unless specifically cancelled. Check it out sometime. comp.risks tracks risks in the use of computer technology, including issues in protecting ourselves from failures of such stuff. Shuvam > Date: Thu, 14 Jun 2001 08:11:00 -0400 > From: "Chris Norloff" <cnorloff@norloff.com> > Subject: NYSE: "Throw up your hands and reboot" > > When the New York Stock Exchange computer systems crashed for 85 > minutes (8 Jun 2001), Andrew Brooks, chief of equity trading at > Baltimore mutual fund giant T. Rowe Price, was quoted as saying "Hey, > we're all subject to the vagaries of technology. It happens on your > own PC at home. You just throw up your hands and reboot." > > http://www.washingtonpost.com/ac3/ContentServer?articleid=A42885-2001Jun8&pagename=article > > Chris Norloff > > > This is from -- > > From: risko@csl.sri.com (RISKS List Owner) > Newsgroups: comp.risks > Subject: Risks Digest 21.48 > Date: Mon, 18 Jun 2001 19:14:57 +0000 (UTC) > Organization: University of California, Berkeley > > RISKS-LIST: Risks-Forum Digest Monday 19 June 2001 > Volume 21 : Issue 48 > > FORUM ON RISKS TO THE PUBLIC IN COMPUTERS AND RELATED SYSTEMS (comp.risks) > ACM Committee on Computers and Public Policy, > Peter G. Neumann, moderator > > This issue is archived at <URL:http://catless.ncl.ac.uk/Risks/21.48.html> > and by anonymous ftp at ftp.sri.com, cd risks . > |
NNTP is the de facto mechanism of choice for moving queued newsfeeds for carrier-class Usenet servers on the Internet, and unfortunately, for a lot of other Usenet servers as well. The reason why we find this choice unfortunate is discussed in Section 12.1> below. But in NNTP feeds, an intermediate step of building batches out of queue files can be eliminated --- this is both its strength and its weakness.
In the case of queued NNTP feeds, articles get added to queue files as described above. An NNTP transmit process periodically wakes up, picks up a queue file, and makes an NNTP connection to the downstream server. It then begins a processing loop where, for each queued article, it uses the NNTP IHAVE command to inform the downstream server of the article's message~ID. The downstream server checks its local repository to see whether it already has the message. If not, it responds with a SENDME response. The transmitting server then pumps out the article contents in plaintext form. When all articles in the queue have been thus processed, the sending server closes the connection. If the NNTP connection breaks in between due to any reason, the sending server truncates the queue file and retains only those articles which are yet to be transmitted, thus minimising repeat transmissions.
> A queued NNTP feed works with the sending server making an NNTP connection to the receiving server. This implies that the receiving server must have an IP address which is known to the sending server or can be looked up in the DNS. If the receiving server connects to the Internet periodically using a dialup connection and works with a dynamically assigned IP address, this can get tricky. UUCP feeds suffer no such problems because the sending server for the newsfeed can be the UUCP server, i.e. passive. The receiving server for the feed can be the UUCP master, i.e. the active party. So the receiving server can then initiate the UUCP connection and connect to the sending server. Thus, if even one of the two parties has a static IP address, UUCP queued feeds can work fine.
Thus, NNTP feeds can be sent out a little faster than the batched transmission processes used for UUCP and other older methods, because no batches need to be constructed. However, NNTP is often used in newsfeeds where it is not necessary and it results in colossal waste of bandwidth. Before we study efficiency issues of NNTP versus batched feeds, we will cover another way feeds can be organised using NNTP: the pull feeds.
Xref: news.starcomsoftware.com control:814217 Path: news.starcomsoftware.com!linux594.dn.net!news.dn.hoopoo.com! feed-out.newsfeeds.com!newsfeeds.com!feed.newsfeeds.com! newsfeeds.com!news-spur1.maxwell.syr.edu!news.maxwell.syr.edu! newsfeed.icl.net!newsfeed.skycache.com!Cidera!newsfeed.gamma.ru! Gamma.RU!carrier.kiev.ua!goblin.nadrabank.kiev.ua!not-for-mail From: tale@uunet.uu.net (David C Lawrence) Newsgroups: news.groups,humanities.hipcrime Subject: cmsg newgroup humanities.hipcrime Control: newgroup humanities.hipcrime Date: Sun, 18 Feb 2001 11:50:28 GMT Organization: The Cabal Lines: 20 Approved: tale@uunet.uu.net Message-ID: <3afWYZTIR.G5YOC2@uunet.uu.net> NNTP-Posting-Host: 203.145.147.67 X-Trace: goblin.nadrabank.kiev.ua 982528840 21455 203.145.147.67 (18 Feb 2001 20:40:40 GMT) X-Complaints-To: usenet@nadrabank.kiev.ua NNTP-Posting-Date: 18 Feb 2001 20:40:40 GMT X-No-Archive: Yes humanities.hipcrime is an unmoderated newsgroup which passed its vote for creation by 326:10 as reported in news.announce.newgroups on 18 Feb 2001. For your newsgroups file: humanities.hipcrime HipCrime for Humanity - you committed one now! Anyone can create a newsgroup in the alt, biz, comp, earth, humanities, misc, news, meow, rec, sci, soc, talk, us, or any other Usenet hierarchy. New newsgroup proposals may be optionally discussed in news.groups. Please be sure that your /usr/lib/news/control.ctl is configured correctly: ## NEWGROUP MESSAGES ## honor them all and log in \${LOG}/newgroup.log newgroup:*:alt.*|biz.*|comp.*|earth.*|humanities.*|misc.*|news.*|\ meaw.*|rec.*|sci.*|soc.*|talk.*|us.*:doit=newgroup ## RMGROUP MESSAGES ## drop them all and don't log rmgroup:*:*:drop Meow! David C Lawrence |
Towards the end of this HOWTO, we have added some information about the history of Usenet server software by quoting sections from an earlier Usenet Periodic Posting. We consider this historical perspective, and the Usenix papers and other documents referred to in it, essential reading for any Usenet server administrator. Please see the section titled "Usenet software: a historical perspective>".
INN addresses a Usenet news world which revolves around NNTP, though it has support for UUCP batches --- a fact that not many INN administrators seem to talk about. INN works faster than the CNews-NNTPd combination when processing multiple parallel incoming NNTP feeds. For multiple readers reading and posting news over NNTP, there is no difference between the efficiency of INN and NNTPd. Section 5.7> discusses the efficiency issues of INN over the earlier C-News architecture, based on Rich Salz' paper and our analyses of usage patterns.
INN's architecture has inspired a lot of high-performance Usenet news software, including a lot of commercial systems which address the ``carrier class'' market. That is the market for which the INN architecture has clear advantages over C-News.
nntp 119/tcp \# Network News Transfer Protocol |
nntp stream tcp nowait news path-to-tcpd path-to-nntpd |
Configuring control files: There are plenty of control files in $NEWSCTL that will need to be configured before you can start using the news system. The files mentioned here are also discussed in the first section of the section titled "Components of a running system>". These control files are dealt in detail in the following below.
sys: One line per system/NDN listing all the newsgroup hierarchies each system subscribes to. Each line is prefixed with the system name and the one beginning with
ME: |
ME:comp,news,misc,netscape |
server/server.starcomsoftware.com:all,!general/all:f |
explist: This file has entries indicating which articles expire and when and whether they have to be archived. The order in which the newsgroups are listed is important. An example follows:
comp.lang.java.3d x 60 /var/spool/news/Archive |
batchparms: sendbatches is a program that administers batched transmission of news articles to other sites. To do this it consults the batchparms file. Each line in the file specifies the behaviour for each of your NDN mentioned in the sys file. There are five fields for each site to be specified.
server u 100000 100 batcher | gzip -9 | viauux -d gunzip |
The first field is the site name which matches the entry in the sys file and has a corresponding directory in $NEWSARTS/out.going by that name.
The second field is the class of the site,u for UUCP and n for NNTP feeds. A "!" in this field means that batching for this site has been disabled.
The third field is the size of batches to be prepared in bytes.
The fourth field is the maximum length of the output queue for transmission to that site.
The fifth field is the command line to be used to build, compress and transmit batches to that site. The contents of the togo file are made available on standard input.
controlperm: This file controls how the news system responds to control messages. Each line consists of 4-5 fields separated by white space. Control messages has been discussed in "Section 2.4>".
comp,sci tale@uunet.uu.net nrc pv news.announce.newsgroups |
The first field is a newsgroup pattern to which the line applies.
The second field is either the keyword "any" or an e-mail address. The latter specifies that the line applies to control messages from only that author.
The third field is a set of opcode letters indicating what control operations need to be performed on messages emanating from the e-mail address mentioned in the second field. n stands for creating a newgroup, r stands for deleting a newsgroup and c stands for checkgroup.
The fourth field is a set of flag letters indicating how to respond to a control message that meets all the applicability tests:
y Do it. n Don't do it. v Report it and include the entire control message in the report. q Don't report it. p Do it iff the control message carries a valid PGP signature. |
The fifth field, which is optional, will be used if the fourth field contains a p. It must contain the PGP key ID of the public key to be used for signature verification.
mailpaths: This file describes how to reach the moderators of various hierarchies of newsgroups by mail. Each line consists of two fields: a news group pattern and an e-mail address. The first line whose group pattern matches the newsgroup is used. As an example:
comp.lang.java.3d somebody@mydomain.com all %s@moderators.uu.net |
Miscellaneous files: The other files to be modified are:
mailname: Contains the Internet domain name of the news system. Consider getting one if you don't have it.
organization: Contains the default value for the Organization: header for postings originating locally.
whoami: Contains the name of the news system. This is the site name used in the Path: headers and hence should concur with the names your neighbours use in their sys files.
active file: This file specifies one line for each newsgroup (not just the hierarchy) to be found on your news system. You will have to get the most recent copy of the active file from ftp://ftp.isc.org/usenet/CONFIG/active and prune it to delete newsgroups that you have not subscribed to. Run the script addgroup for each newsgroup in this file which will create relevant directories in the $NEWSARTS area. The addgroup script takes two paramters: the newsgroup name being created and a flag. The flag can be any one of the following:
y local postings are allowed n no local postings, only remote ones m postings to this group must be approved by the moderator j articles in this group are only passed and not kept x posting to this newsgroup is disallowed =foo.bar articles are locally filed in "foo.bar" group |
comp.lang.java.3d 0000003716 01346 m |
newsgroups file: This contains a one-line description of each newsgroup to be found in the active file. You will have to get the most recent file from ftp://ftp.isc.org/usenet/CONFIG/newsgroups and prune it to remove unwanted information. As an example:
comp.lang.java.3d 3D Graphics APIs for the Java language |
Aliases: These aliases are required for trouble reporting. Once the system is in place and scripts are run, anomalies/problems are reported to addresses in the /etc/aliases file. These entries include email addresses for newsmaster, newscrisis, news, usenet, newsmap. They should ideally point to an email address that will be accessed at regularly. Arrange the emails for newsmap to be discarded to minimize the effect of sendsys bombing by practical jokers.
Cron jobs: Certain scripts like newsrun that picks up incoming batches and maintenance scripts, should run through news-database owner's cron which is news. The cron entries ideally will be for the following: A more detailed report can be found in "Section 9.4>"
newsrun: This script processes incoming batches of article. Run this as frequently as you want them to get digested.
sendbatches: This script transmit batches to the NDNs. Set the frequency according to your requirements.
newsdaily: This should be run ideally once a day since it reports errors and anomalies in the news system.
newswatch: This looks for errors/anomalies at a more detailed level and hence should be run atleast once every hour
doexpire: This script expires old articles as determined by the explist file. Run this once a day.
newslog: Make an entry in the system's syslog.conf file for logging messages spewed out by nntpd in newslog . It should be located in $NEWSCTL. The entry will look like this:
news.debug -/var/lib/news/newslog |
Newsboot: Have this run (as news the news-database owner) when the system boots to clear out debris left around by crashes.
Add a Usenet mailer in sendmail: The mail2news program provided as part of the source code is a handy tool to send an e-mail to a newsgroup which gets digested as an article. You will have to add the following ruleset and mailer definition in your sendmail.cf file:
Under SParse1, add the following:
R$+ . USENET < @ $=w . > $#usenet $: $1 |
Under mailer definitions, define the mailer Usenet as:
MUsenet P=/usr/lib/newsbin/mail2news/m2nmailer, F=lsDFMmn, S=10, R=0, M=2000000, T=X-Usenet/X-Usenet/X-Unix, A=m2nmailer $u |
In order to send a mail to a newsgroup you will now have to suffix the newsgroup name with usenet i.e. your To: header will look like this:
To: misc.test.usenet@yourdomain. |
This, more or less, completes the configuration part.
To locally test the system, follow the steps given below:
post an article: Create a local newsgroup
cnewsdo addgroup mysite.test y |
As mentioned in "Section 2.4>", it becomes necessary to authenticate control messages to protect yourself from being attacked by pranksters. For this, you will have to configure the $NEWSCTL/controlperm file to declare whose control messages you are willing to honour and for what newsgroups alongwith their public key ID. The controlperm manpage shall give you details on the format.
This will work only in association with pgpverify which verifies the Usenet control messages that have been signed using the signcontrol process. The script can be found at ftp://ftp.isc.org/pub/pgpcontrol/pgpverify. pgpverify internally uses the PGP binary which will have to be made available in the default executables directory. If you wish to send control messages for your local news system, you will have to digitally sign them using the above mentioned signcontrol program which is available at ftp://ftp.isc.org/pub/pgpcontrol/signcontrol. You will also have to configure the signcontrol program accordingly.
If you are a leaf node, you will only have to send feeds back to your news provider for your postings in public newsgroups to propagate to the outside world. To enable this, you need one line in the sys and batchparms files and one directory in $NEWSARTS/out.going. If you are willing to transmit articles to your neighbouring sites, you will have to configure sys and batchparms with more entries. The number of directories in $NEWSARTS/out.going shall increase, too. Refer to first two sections of the chapter titled "Components of a running system>"for a better understanding of outgoing feeds. Again, you will have to determine how you wish to transmit the feed: UUCP or NNTP.
Control messages, discussed in detail earlier in Section 2.4>, instruct a Usenet server to take certain actions, like delete a message or create a newsgroup. If this facility is ``open to the public'', anyone with half a brain can forge control messages to create twenty new newsgroups, and then post thousands of articles into those groups. In the mid-nineties, we were hit by a storm of over 2,000 (two thousand) newgroup control messages, which rapidly taught us the danger of unprotected control messages and the protection against them.
The standard protection mechanism against this vulnerability is pgpverify, which can be downloaded from multiple Websites and FTP mirror sites by searching for pgpverify (the program) or pgpcontrol (the total software package). We have integrated this into our source distribution, so that our C-News works in a tightly coupled manner with pgpverify.
pgpverify works using public key cryptography, much like NoCeM, and all the official maintainers of respective Usenet group hierarchies sign control messages using their private keys. Your server will carry their public keys, and pgpverify will check the sign on each control message to ensure that it's from the official maintainer of the hierarchy. It will then act upon legit control messages and discard the spurious ones.
In today's nuisance-ridden Usenet environment, no sane Usenet server administrator receiving a feed of ``public'' hierarchies and control messages will even dream of running her server without pgpverify protection.
This directory is more popularly known as $NEWSCTL. It contains configuration, log and status files. There are no articles or binaries kept here. Let's see what some of the files are meant for. Control files are dealt in slightly greater detail in "Section 4.3>"
sys: One line per system/NDN listing all the newsgroup hierarchies each system subscribes to. Each line is prefixed with the system name and the one beginning with ME: indicates what we are going to receive. Look up manpage of newssys.
explist: This file has entries indicating articles of which newsgroup expire and when and if they have to be archived. The order in which the newsgroups are listed is important. See manpage of expire for file format.
batchparms: Details of how to feed other sites/NDN, like the size of batches, the mode of transmission (UUCP/NNTP) are specified here. manpage to refer: newsbatch.
controlperm: If you wish to authenticate a control message before any action is taken on it, you must enter authentication-related information here. The controlperm manpage will list all the fields in detail.
mailpaths: It features the e-mail address of the moderator for each newsgroup who is responsible for approving/disapproving articles posted to moderated newsgroups. The sample mailpaths file in the tar will give you an idea of how entries are made.
nntp_access/user_access: These files contain entries of servernames and usernames on whom restrictions will apply when accessing newsgroups. Again, the sample file in the tarball shall explain the format of the file.
log, errlog: These are log files that keep growing large with each batch that is received. The log file has one entry per article telling you if it has been accepted by your news server or rejected. To understand the format of this file, refer to Chapter 2.2 of the CNews guide. Errors, if any, while digesting the articles are logged in errlog. These log files have to be rolled as the files hog a lot of disk space.
nntplog: This file logs information of the nntpd giving details of when a connection was established/broken and what commands were issued. This file needs to be configured in syslog syslogd should be running.
active: This file has one line per newsgroup to be found in your news server. Besides other things, it tells you how many articles are currently present in each newsgroup. It is updated when each batch is digested or when articles are expired. The active manpage will furnish more details about other paramaters.
history: This file, again, contains one line per article, mapping message-id to newsgroup name and also giving its associated article number in that newsgroup. It is updated each time a feed is digested and when doexpire is run. Plays a key role in loop-detection and serves as an article database. Read manpage of newsdb, doexpire for the file format
newsgroups: It has a one-line description for each newsgroup explaining what kind of posts go into each of them. Ideally speaking, it should cover all the newsgroups found in the active file.
Miscellaneous files: Files like mailname, organisation, whoami contain information required for forming some of the headers of an article. The contents of mailname form the From: header and that of organisation form the Organisation: header. whoami contains the name of the news system. Refer to chapter 2.1 of guide.ps for a detailed list of files in the $NEWSCTL area. Read RFC 1036 for description of article headers .
>newsgroup directories: For each newsgroup hierarchy that the news server has subscribed to, a directory is created under $NEWSARTS. Further sub-directories are created under the parent to hold articles of specific newsgroups. For instance, for a newsgroup like comp.music.compose, the parent directory comp will appear in $NEWSARTS and a sub-directory called music will be created under comp. The music sub-directory shall contain a further sub-directory called compose and all articles of comp.music.compose shall reside here. In effect, article 242 of newsgroup comp.music.compose shall map to file $NEWSARTS/comp/music/compose/242.
control: The control directory houses only the control messages that have been received by this site. The control messages could be any of the following: newgroup, rmgroup, checkgroup and cancel appearing in the subject line of the article. More information to be found in "Section 2.4>"
junk: The junk directory contains all articles that the news server has received and has decided, after processing, that it does not belong to any of the hierarchies it has subscribed to. The news server transfers/passes all articles in this directory to NDNs that have subscribed to the junk hierarchy.
Over NNTP, since there is no batching, transfers happen one article at a time. Considering the (relatively) small size of an article compared to multi-megabyte UUCP batches, one would expect that an article would never pose a major problem while being transported; if it can't be pushed across in one attempt, it'll surely be copied the next time. However, we have experienced entire NNTP feeds getting stuck for days on end because of one article, with logs showing the same article breaking the connection over and over again while being transferred [1]. Some rare articles can be more than a megabyte in size, particularly in comp.binaries. In each such incident, we have had to manually edit the queue file on the transmitting server and remove the offending article from the head of the queue. Taylor UUCP, on the other hand, has never given us a single hiccup with blocked queues.
We feel that the overwhelming majority of servers offering the Usenet news service are at the leaf nodes of the Usenet news flow, not at the heart. These servers are usually connected in a tree, with each server having one upstream ``parent node'', and multiple downstream ``child nodes.'' These servers receive their bulk incoming feed from their upstream server, and their users can tolerate a delay of a few hours for articles to move in and out. If your server is in this class, we feel you should consider using UUCP over TCP and transfer compressed batches. This will minimise bandwidth usage, and if you operate using dialup Internet connections, it will directly reduce your expenses.
A word about the link between mesh-patterned newsfeed flow and the need to use NNTP. If your server is receiving primary --- as against trickle --- feeds from multiple next-door neighbours, then you have to use NNTP to receive these feeds. The reason lies in the way UUCP batches are accepted. UUCP batches are received in their entirety into your server, and then they are uncompressed and processed. When the sending server is giving you the batch, it is not getting a chance to go through the batch article by article and ask your server whether you have or don't have each article. This way, if multiple servers give you large feeds for the same hierarchies, then you will be bound to receive multiple copies of each article if you go the UUCP way. All the gains of compressed batches will then be neutralised. NNTP's IHAVE and SENDME dialogue in effect permits precisely this double-check for each article, and thus you don't receive even a single article twice.
For Usenet servers which connect to the Internet periodically using dialup connections to fetch news, the UUCP option is especially important. Their primary incoming newsfeed cannot be pushed into them using queued NNTP feeds for reasons described in the above paragraph These hapless servers are usually forced to pull out their articles using a pull NNTP feed, which is often very slow. This may lead to long connect times, repeat attempts after every line break, and high Internet connection charges.
On the other hand, we have been using UUCP over TCP and gzip'd batches for more than five years now in a variety of sites. Even today, a full feed of all eight standard hierarchies, plus the full microsoft, gnu and netscape hierarchies, minus alt and comp.binaries, can comfortably be handled in just a few hours of connect time every night, dialing up to the Internet at 33.6 or 56 Kbits/sec. We believe that the proverbial `full feed' with all hierarchies including alt can be handled comfortably with a 24-hour link at 56 Kbits/sec, provided you forget about NNTP feeds. We usually get compression ratios of 4:1 using gzip -9 on our news batches, incidentally.
INN and CNews are the two most popular free software implementations of Usenet news. Of these two, we prefer CNews, primarily because we have been using it across a very large range of Unixen for more than one decade, starting from its earliest release --- the so-called ``Shellscript release'' --- and we have yet to see a need to change.[2]
We have seen INN, and we are not comfortable with a software implementation which puts in so much of functionality inside one executable. This reminds us of Windows NT, Netscape Communicator, and other complex and monolithic systems, which make us uncomfortable with their opaqueness. We feel that CNews' architecture, which comprises many small programs, intuitively fits into the Unix approach of building large and complex systems, where each piece can be understood, debugged, and if needed, replaced, individually.
Secondly, we seem to see the move towards INN accompanied by a move towards NNTP as a primary newsfeed mechanism. This is no fault of INN; we suspect it is a sort of cultural difference between INN users and CNews users. We find the issue of UUCP versus NNTP for batched newsfeeds a far more serious issue than the choice of CNews versus INN. We simply cannot agree with the idea that NNTP is an appropriate protocol for bulk Usenet feeds for most sites. Unfortunately, we seem to find that most sites which are more comfortable using INN seem to also prefer NNTP over UUCP, for reasons not clear to us.
Our comments should not be taken as expressing any reservation about INN's quality or robustness. Its popularity is testimony to its quality; it most certainly ``gets the job done'' as well as anything else. In addition, there are a large number of commercial Usenet news server implementations which have started with the INN code; we do not know of any which have started with the CNews code. The Netwinsite DNews system and the Cyclone Typhoon, we suspect, both are INN-spired.
We will recommend CNews and NNTPd over INN, because we are more comfortable with the CNews architecture for reasons given above, and we do not run carrier-class sites. We will continue to support, maintain and extend this software base, at least for Linux. And we see no reason for the overwhelming majority of Usenet sites to be forced to use anything else. Your viewpoints welcome.
Had we been setting up and managing carrier-class sites with their near-real-time throughput requirements, we would probably not have chosen CNews. And for those situations, our opinion of NNTP versus compressed UUCP has been discussed in Section 12.1>
Suck and Leafnode have their place in the range of options, where they appear to be attractive for novices who are intimidated by the ``full blown'' appearance of CNews+NNTPd or INN. However, we run CNews + NNTPd even on Linux laptops. We suspect INN can be used this way too. We do not find these ``full blown'' implementations any more resource hungry than their simpler cousins. Therefore, other than administration and configuration familiarity, we don't see any other reason why even a solitary end-user will choose Leafnode or Suck over CNews+NNTPd. As always, contrary opinions invited.
ftp://rtfm.mit.edu/pub/usenet-by-hierarchy/news/software/b/Usenet_Software:_History_and_Sources
Date: Tue, 28 Dec 1999 09:00:19 GMT Supersedes: <FMMECL.58s@tac.nyc.ny.us> Expires: Fri, 28 Jan 2000 09:00:19 GMT Message-ID: <FnG10J.HAo@tac.nyc.ny.us> From: netannounce@deshaw.com (Mark Moraes) Subject: Usenet Software: History and Sources Newsgroups: news.admin.misc,news.announce.newusers,news.software.readers,news.software.b,news.answers Followup-To: news.newusers.questions Approved: netannounce@deshaw.com (Mark Moraes) Archive-name: usenet/software/part1 Original-from: spaf@cs.purdue.edu (Gene Spafford) Comment: edited until 5/93 by spaf@cs.purdue.edu (Gene Spafford) Last-change: 9 Feb 1998 by netannounce@deshaw.com (Mark Moraes) Changes-posted-to: news.admin.misc,news.misc,news.software.readers,news.software.b,news.answers |
<ftp://rtfm.mit.edu/pub/usenet/comp.os.msdos.mail-news/intro> <ftp://rtfm.mit.edu/pub/usenet/comp.os.msdos.mail-news/software> |
This section fills in gaps which were hard to classify under any of the previous chapters.
badexpiry: utility to look for articles with bad explicit Expiry headers
checkactive: utility to perform some sanity checks on the active file
cnewsdo: utility to perform some checks and then run C-News maintenance commands
controlperm: configuration file for controlling responses to Usenet control messages
explode: internal utility to convert a master batch file to ordinary batch files
mergeactive: utility to merge one site's newsgroups to another site's active file
news(5): description of Usenet news article file and batch file formats
newsflag: utility to change the flag or type column of a newsgroup in the active file
newsmaint: utility scripts used by Usenet administrator to manage and maintain C-News system
newsoverview(8): library functions of the NOV library and the utilities which use them
report: utility to generate and send email reports of errors and events from C-News scripts
nntpxmit: The NNTP batch transmit program for outgoing push feeds
This very interesting paper has been mentioned in the section titled "Usenet software: a historical perspective>". It is titled ``News Need Not Be Slow'', and is available from ftp://ftp.cs.toronto.edu/doc/programming/c-news.* or from our Website (http://www.starcomsoftware.com/proj/usenet/doc/c-news.{ps,pdf}).
It focuses on B News, analyses it for performance, and demonstrates how specific changes in design and implementation can speed things up. It is well-written, and is educative in many areas independent of Usenet news.
We also offer you an integrated source distribution of C News, NNTPd, as discussed earlier in the section titled "Setting up C News + NNTPd>". This integrated source distribution fixes some bugs in the component packages it includes, and it comes pre-configured with ready made configuration files which allow all components to be compiled and installed on a Linux server in a manner by which they can work together (e.g. key directory paths are specified consistently across all components, etc.) This is available at http://www.starcomsoftware.com/proj/usenet/src/
The URL http://www.starcomsoftware.com/proj/usenet/src/archives/ holds the original sources of some of the software components we base our distribution on. These include C News (c-news.tar.Z), NNTPd (nntp.1.5.12.1.tar.Z), and Nestor (nestor.tar.Z). Other components, like pgpverify are maintained by their current maintainers and can be obtained from their respective sites. Therefore, they are not included in our archives.
The URL http://www.starcomsoftware.com/proj/usenet/doc/ carries copies of some of the important technical articles and Usenix papers on the subject of the Usenet.
We will endeavour to answer all queries sent to usenet@starcomsoftware.com, pertaining to the source distribution we have put together and its configuration and maintenance, and also pertaining to general technical issues related to running a Usenet news service off a Unix or Linux server.
We may not be in a position to assist with software components we are not familiar with, e.g. Leafnode, or platforms we do not have access to, e.g. SGI IRIX. Intel Linux will be supported as long as our group is alive; our entire office runs on Linux servers and diskless Linux desktops.
You are not forced to be dependent on us, because neither do we have proprietary knowledge nor proprietary closed-source software. All the extensions we are currently involved in with C-News and NNTPd will immediately be made available to the Internet in freely redistributable source form.
Hema Kariyappa co-authored the first couple of versions of this HOWTO, starting with v2.0.
Copyright (c) 2002 Starcom Software Private Limited, India
You may create a derivative work and distribute it provided that you:
[1] | This lack of a restart facility is something NNTP shares with its older cousin, SMTP, and we have often seen email messages getting stuck in a similar fashion over flaky data links. In many such networks which we manage for our clients, we have moved the inter-server mail transfer to Taylor UUCP, using UUCP over TCP. |
[2] | One of us did his first installation with with BNews, actually, at the IIT Mumbai. Then we rapidly moved from there to CNews Shellscript Release, then CNews Performance Release, CNews Cleanup Release, and our current release has fixed some bugs in the latest Cleanup Release. |