(From: Erland Sommarskog <sommar@sophocles.algonet.se>)
Q: We started a new news server just a month ago. SS-20, lots of memory,
   23 GB of news spool. The first week it ran fast as a jaguar, but now
   it crawls like a snail. It takes forever to connect, we can't keep
   up with our feeds which have to flush our queues, it takes to forever
   to connect. When we monitor the box it spends an awful lot of time
   in kernel mode, and seems to be doing a tremendous amount of disk
   access. Where did we go wrong? If it matters, we're not running
   expire, but dexpire and histtrim.

A: The problem is with the history database. This database consists of
   three files: history, history.pag and history.dir. The dir files
   contains a hash table. For optimal performance, the hash file must
   not be too small, or you will get many collisions. The initial size
   of the history database is based on Usenet traffic as it was a
   couple of years ago, and no one was considering a 23 GB spool. Now,
   for people who is running expire this is not a problem, because when
   you run expire, the history file is rebuilt, and the size for the
   new database is taken from the old one. But if you are running
   dexpire and histtrim, this never happens, and innd will spend most
   of its time reading overflow buckets.
     What to do then? Rebuild the history database. And the simlpest
   way to do it, is to run expire. A simple approach is to run expire 
   every now and then, with an expire.ctl that safe won't expire very 
   many articles. (But add a short expiration time on some junk group,
   so that expire get something to work with.) 
   A better approach is to discontinue use of histtrim, and run expire 
   daily basis. Dexpires produces an output which you easily can trans-
   form into an expire.ctl. I discourage use of histtrim, since it
   may delete articles from history and yet are on the system.

(From: James Brister <brister@vix.com>)
	A good indicator of your performance characteristics would be how much
smaller is the number generated by this,

   head -1 /var/news/etc/history.dir | perl -ane 'print 2 ** $F[7], "\n";'

than the size of your history text file. If it's bigger you're OK. if it's
smaller, then lookups for the message ids at the tail of the history text
file (past the byte indexed by the number just generated) will be much
slower than for those at the front.

See also: #7.6, #7.7 and #7.41

© 2008 FAQS.ORG. All rights reserved.