|Performance Tuning Tips for the InterNetNews Server on Compaq Tru64 UNIX
This document provides tips for tuning the InterNetNews server (INN) packaged with Internet Express (formerly known as Internet AlphaServer System Software (IASS)). If you are not using the Internet Express version of INN, visit the link given in Where to Find More Information to obtain information on tuning the compilation process.
A news server needs sufficient memory to operate efficiently. Memory is consumed by each news reader that is connected to the server. Large amounts of memory are needed to support housekeeping operations, such as the expire(8) operation.
The recommended minimum amount of memory for a news server is 512MB. You can use the vmstat(1) command to determine the amount of free memory and paging activity on your server.
It is also important to allocate sufficient swap space. A common rule of thumb is to have 1.5 times the resident memory size allocated for swap space. Insufficient swap space can slow down a system and cause failures in the expire(8) process.
The most critical task when establishing a news server is to properly set up the file systems that the news server will use. Unless these file systems are properly configured and tuned, other tuning activities will have little impact on the performance of the news server.
Consider the following when tuning file systems for the news server:
Type of File System
Compaq recommends the Advanced File System (AdvFS) for use with INN.
The recommended minimum disk space allocation for the news spool area is 18GB; allocate 32GB of disk space to maintain articles for 7-14 days. Allocate sufficient space to allow the spool area to grow; the volume of news articles has steadily increased over time.
The following features of AdvFS facilitate management of the spool area:
- Ability to add or remove disk devices while the file system is active
- Transparently spans multiple devices
- Automatically balances the load across available devices
With AdvFS, you can contain the news spool area in a single mounted file system. This allows you to manage the news space more easily; with multiple file systems, you must manually balance the load on each file system.
Location of Configuration and Spool Areas
Compaq recommends that the news configuration area (/var/news/etc) reside on a disk that does not contain any of the news spool area (/var/spool/news).
INN uses a number of configuration files located under the /var/news/etc directory (which when installed with Internet Express may be a symbolic link to /data/news/etc). Two of these configuration files, active and history, are frequently used and are very large. If these configuration files are located in the same area as the news spool, access times to the disk drive will suffer delays.
Distribution of Configuration and Spool Areas
Compaq recommends that you spread the news spool area over as many disk devices and SCSI buses as possible.
Each time a disk device handles a read request, there is a time lag. If you spread I/O operations across multiple devices, overall throughput will be improved. The UNIX kernel allows multiple pending I/O requests.
Adding SCSI buses has the effect of increasing the maximum I/O bandwidth between the disks and the processor.
Provisions for Disaster Recovery
Compaq recommends that you use Redundant Array of Inexpensive Devices (RAID) technology to provide for failure of a disk drive.
Because a news spool area is normally a large area that is undergoing constant change, it is very difficult to use a typical disaster recovery mechanism (such as a tape backup) to recover a file system damaged by the loss of a disk drive.
If restarting the news spool area from scratch is not an option, RAID technology is the optimal solution for providing disaster recovery. RAID technology tolerates the failure of one disk drive (sometimes more) without compromising the file system. RAID level 0¾ which performs mirroring (that is, keeping two identical copies of the data) ¾ has additional performance benefits that improve news serving. Writing to the news area is slightly slower with mirroring (because the data needs to be written twice), but read performance on a UNIX system can be improved by alternating read requests between the mirrored devices.
Compaq recommends Logical Storage Manager (LSM), which is a software product (see http://www.unix.digital.com/storage/ ), and hardware solutions (see http://www.compaq.com/products/storageworks/) to implement RAID technology on your system. Both of these solutions present to the operating system a virtual disk device, hiding the detail of physical devices and mirroring operations from the file system.
The AdvFS file system provides for spanning of the file system across multiple devices, and is recommended for use in conjunction with these RAID solutions.
Although it is possible to accumulate many devices into one very large virtual device, this is not recommended. Instead, create many mirrored pairs of disks, with each pair of disks presented to the file system as one virtual disk. This configuration offers the following advantages:
- AdvFS performs better with the larger number of virtual disks since it can schedule multiple I/O operations simultaneously.
- Smaller virtual disks allow for easier maintenance of the file system. Larger virtual devices complicate the AdvFS add and remove operations.
Restricting Access to the News Server
By restricting access to the news server to members of your user community, you can protect your system resources from unauthorized use. You can use the following methods to restrict access to the news server:
INN supports restricting user access to the news server by the host and/or domain of the client machine. By default, Internet Express installs INN so that access is restricted to the domain of the news server.
You can use the Internet Express Administration utility to change these access controls (or you can edit the /var/news/etc/nnrp.access file), and then restart the news server.
Password Controlled Access
INN supports restricting user access to the news server by a password. By default, Internet Express does not enable this feature.
You can use the Internet Express Administration utility to enable password access (or you can edit the /var/news/etc/nnrp.access file), and then restart the news server.
Tuning Newsgroup Expiration
You can significantly impact the performance of your news server by tuning the following:
- Article expiration
- Article history length
You can use the Internet Express Administration utility to tune article expiration and history length (or you can edit the /var/news/expire.ctl file), and then restart the news server.
Article expiration defines the length of time an article will be retained in the news spool space. With Internet Express, article expiration defaults to four days, that is, an article will be removed from the spool area after four days, unless the article contains an internal expiration date that is sooner than that. A four-day retention period is fairly short, and it is likely you will want to increase the retention period. (Note: A 7-day article expiration requires approximately 40GB of spool space).
Since the increase in disk space that corresponds to a longer retention period can be considerable, it makes sense to take a close look at the newsgroups you are carrying. The articles in some newsgroups (for example, test or weather) should be allowed to expire before the end of the general retention period. By individually tuning the article expiration for these groups, you can free up valuable space.
If your news server is not feeding other news sites, you can save a considerable amount of disk space by specifying a short article expiration period for both the junk and control newsgroups. The junk newsgroup contains all the articles for newsgroups that do not have entries in the active(5) file. An expiration time of one day is adequate. The control newsgroup contains special news articles of interest to news administrators, such as announcements for newly created newsgroups, cancellation notices, and so on. The following example shows how to specify two- or one-day article expiration periods for the weather, test, control, and junk newsgroups in the expire(5) file:
When an article arrives at a news server, its message ID is stored in the history(8) database. The news daemon (innd) checks this database to identify repeated articles and remove them. The length of time that a message ID is stored in the history(8) database is specified in the expire.ctl(8) file. By default, INN as configured with Internet Express uses a 14-day history value. Increasing this value increases the likelihood that a repeated article will be properly discarded. The drawback to increasing the history value is that the size of the history database will also increase¾ by about 300,000 message IDs per day. This increase in database size increases the processing time for incoming articles and article expiration.
You can modify the FLAGS= line in the /usr/news/bin/rc.news file to specify the innd daemon's -c flag. This flag, when used in conjunction with the history value, helps to ensure that older articles are properly discarded. The number of days you specify as the argument for the -c flag should be the same as the history value. For example, assuming the history value is 7, add the following line to the rc.news file:
After editing the rc.news file, you must restart the news server before the changes will take effect. You can use the Internet Express Administration utility to restart the news server, or use /sbin/rc3.d/S85innd. A history value of seven days should speed processing time considerably.
Where to Find More Information
INN Home page
PGP verification of control messages
Dave Barrs INN page