4. Setting up CNews + NNTPd

4.1. Getting the sources and stuff

4.1.1. The sources

C-News software can be obtained from ftp://ftp.uu.net/networking/news/transport/cnews/cnews.tar.Z and will need to be uncompressed using the BSD uncompress utility or a compatible program. The tarball is about 650 KBytes in size. It has its own highly intelligent configuration and installation processes, which are very well documented. The version that is available is Cleanup Release revision G, on which our own version is based.

NNTPd (the NNTP Reference Implementation) is available from ftp://ftp.uu.net/networking/news/nntp/nntp.1.5.12.1.tar.Z. It has no automatic scripts and processes to configure itself. After fetching the sources, you will have to follow a set of directions given in the documentation and configure some C header files. These configuration settings must be done keeping in mind what you have specified when you build the C-News sources, because NNTPd and C-News must work together. Therefore, some key file formats, directory paths, etc., will have to be specified identically in both software systems.

The third software system we use is Nestor. This too is to be found in the same place where the NNTPd software is kept, at ftp://ftp.uu.net/networking/news/nntp/nestor.tar.Z. This software compiles to one binary program, which must be run periodically to process the logs of nntpd, the NNTP server which is part of NNTPd, and report usage statistics to the administrator. We have integrated Nestor into our source base.

The fourth piece of the system, without which no Usenet server administrator dares venture out into the wild world of public Internet newsfeeds, is pgpverify.

We have been working with C-News and NNTPd for many years now, and have fixed a few bugs in both packages. We have also integrated the four software systems listed above, and added a few features here and there to make things work more smoothly. We offer our entire source base to anyone for free download from http://www.starcomsoftware.com/proj/usenet/src/news.tar.gz. There are no licensing restrictions on our sources; they are as freely redistributable as the original components we started with.

When you download our software distribution, you will extract it to find a directory tree with the following subdirectories and files:

  • c-news: the source tree of the CR.G software release, with our additions like pgpverify integration, our scripts like mail2news, and pre-created configuration files.

  • nntp-1.5.12.1: the source tree of the original NNTPd release, with header files pre-configured to fit in with our configuration of C-News, and our addition of bits and pieces like Nestor, the log analysis program.

  • howto: this document, and its SGML sources and Makefile.

  • build.sh: a shellscript you can run to compile the entire combined source tree and install binaries in the right places, if you are lucky and all goes well.

Needless to say, we believe that our source tree is a better place to start with than the original components, specially if you are installing a Usenet server on a Linux box and for the first time. We will be available on email to provide technical assistance should you run into trouble.

4.1.2. The key configuration files

Once you get the sources, you will need some key configuration files to seed your C-News system. These configuration files are actually database tables, and are changing frequently, whenever newsgroups are created, modified or deleted. These files specify the list of active newsgroups in the ``public'' Usenet. You can, and should, add your organisation's internal newsgroups to this list when you set up your own server, but you will need to know the list of public standard newsgroups to begin with. This list can be obtained from the same FTP server by downloading the files active.gz and newsgroups.gz from ftp://ftp.uu.net/networking/news/config/. You can create your own active and newsgroups files by retaining a subset of the entries in these two files. Both these are ASCII text files.

Getting the sources from our server will not obviate the need to get the latest versions of these files from ftp.uu.net. We do not (yet) maintain an up-to-date copy of these files on our server, and we will add no value to the original by just mirroring them.

4.2. Compiling and installing

For installing, first make sure you have an entry for a user called news in your /etc/password file. This is setting the news-database owner to news. Now download the source from us and untar it in the home directory of news. This creates two main directories viz. c-news and nntp. To install and compile, run the script build.sh as root in the directory that contains the script. It is important that the script run as root as it sets ownerships, installs and compiles the source as user news. This is a one-step process that puts in place both the C-News and the NNTP software, setting correct permissions and paths. Following is a brief description of what build.sh does:

4.3. Configuring the system: What and how to configure files?

Once installed, you have to now configure the system to accept feeds and batch them for your neighbours. You will have to do the following:

4.4. Testing the system

To locally test the system, follow the steps given below:

4.5. pgpverify and controlperms

As mentioned in "Section 2.4>", it becomes necessary to authenticate control messages to protect yourself from being attacked by pranksters. For this, you will have to configure the $NEWSCTL/controlperm file to declare whose control messages you are willing to honour and for what newsgroups alongwith their public key ID. The controlperm manpage shall give you details on the format.

This will work only in association with pgpverify which verifies the Usenet control messages that have been signed using the signcontrol process. The script can be found at ftp://ftp.isc.org/pub/pgpcontrol/pgpverify. pgpverify internally uses the PGP binary which will have to be made available in the default executables directory. If you wish to send control messages for your local news system, you will have to digitally sign them using the above mentioned signcontrol program which is available at ftp://ftp.isc.org/pub/pgpcontrol/signcontrol. You will also have to configure the signcontrol program accordingly.

4.6. Feeding off an upstream neighbour

For external feeds, commercial customers will have to buy them from a regular News Provider like dejanews.com or newsfeeds.com. You will have to specify to them what hierarchies you want and decide on the mode of transmission, i.e. UUCP or NNTP, based on your requirements. Once that is done, you will have to ask them to initiate feeds, and check $NEWSARTS/in.coming directory to see if feeds are coming in.

If your organisation belongs to the academic community or is otherwise lucky enough to have an NDN server somewhere which is willing to provide you a free newsfeed, then the payment issue goes out of the picture, but the rest of the technical requirements remain the same.

One problem with incoming NNTP feeds is that it is far easier to use (relatively) efficient NNTP inflows if you have a server with a permanent Internet connection and a fixed IP address. If you are a small office with a dialup Internet connection, this may not be possible. In that case, the only way to get incoming newsfeeds by NNTP may be by using a highly inefficient pull feed.

4.7. Configuring outgoing feeds

If you are a leaf node, you will only have to send feeds back to your news provider for your postings in public newsgroups to propagate to the outside world. To enable this, you need one line in the sys and batchparms files and one directory in $NEWSARTS/out.going. If you are willing to transmit articles to your neighbouring sites, you will have to configure sys and batchparms with more entries. The number of directories in $NEWSARTS/out.going shall increase, too. Refer to first two sections of the chapter titled "Components of a running system>"for a better understanding of outgoing feeds. Again, you will have to determine how you wish to transmit the feed: UUCP or NNTP.

4.7.1. By UUCP

For outgoing feeds by UUCP, we recommend that you start with Taylor UUCP. In fact, this is the UUCP version which forms part of the GNU Project and is the default UUCP on Linux systems.

A full treatment of UUCP configuration is beyond the scope of this document. However, the basic steps will be as follows. First, you will have to define a "system" in your Usenet server for the NDN (next door neighbour) host. This definition will include various parameters, including the manner in which your server will call the remote server, the protocol it will use, etc. Then an identical process will have to be followed on the NDN server's UUCP configuration, for your server, so that that server can recognize your Usenet server.

Finally, you will need to set up appropriate cron jobs for the user uucp to run uucico periodically. Taylor UUCP comes with a script called uusched which may be modified to your requirements; this script calls uucico. One uucico connection will both upload and download news batches. Smaller sites can run uusched even once or twice a day.

Later versions of this document will include the uusched scripts that we use in Starcom. We use UUCP over TCP/IP, and we run the uucico connection through an SSH tunnel, to prevent transmission of UUCP passwords in plain text over the Internet, and our SSH tunnel is established using public-key cryptography, without passwords being used anywhere.

4.7.2. By NNTP

For NNTP feeds, you will have to decide whether your server will be the connection initiator or connection recipient. If you are the connection initiator, you can send outgoing NNTP feeds more easily. If you are the connection recipient, then outgoing feeds will have to be pulled out of your server using the NNTP NEWNEWS command, which will place heavy loads on your server. This is not recommended.

Connecting to your NDN server for pushing out outgoing feeds will require the use of the nntpsend.sh script, which is part of the NNTPd source tree. This script will perform some housekeeping, and internally call the nntpxmit binary to actually send the queued set of articles out. You may have to provide authentication information like usernames and passwords to nntpxmit to allow it to connect to your NDN server, in case that server insists on checking the identity of incoming connections. (You can't be too careful in today's world.) nntpsend.sh will clean up after an nntpxmit connection finishes, and will requeue any unsent articles for the next session. Thus, even if there is a network problem, typically nothing is lost and all pending articles are transmitted next time.

Thus, pushing feeds out via may mean setting up nntpsend.sh properly, and then invoking it periodically from cron. If your Usenet server connects to the Internet only intermittently, then the process which sets up the Internet connection should be extended or modified to fire nntpsend.sh whenever the Internet link is established. For instance, if you are using the Linux pppd, you can add statements to the /etc/ppp/ip-up script to change user to news and run nntpsend.sh