DNEWSLINK(8)

NAME
dnewslink - Diablo program to feed news to remote systems

SYNOPSIS
dnewslink -s xxx...xxxx -n -i -D -f -p -P port -d[#] -b batchfile/template -r -S beginSeq -N numSeq -H -L -A[classes] -h remote-host -w delay -t timeout -l logFrequency -c reconnectFrequency -T txbufsize -R rxbufsize -I rxbufsize -B ip

DESCRIPTION
DNewslink generally takes queue files from the dqueue directory that were generated by the diablo server and uses them to transfer articles from the news spool to a remote host. There are a horrendous number of options covering four modes of operation. DNewslink can transmit articles based on a discrete queue file, can transmit using multiple queue files based on a template, can transmit based on input from a pipe, or can sit on a realtime queue file to transmit a feed in near real time. DNewslink will usually rewrite the queue file with any entries that had to be defered. When DNewslink connects to a remote host, it will attempt to negotiate a check/takethis streaming connection and, if that fails, will fallback to the non-streaming ihave protocol.

DNewslink locks queue files as it processes them, allowing multiple dnewslinks to operate in parallel on different queue files without the possibility of creating a conflict. DNewslink uses a sophisticated non-blocking I/O model to keep the TCP pipeline full for streaming feeds and is capable of generating serious bandwidth. The options are described below.

-s xxx...xxxx

This option reserves space in the argv array, allowing dnewslink to store realtime status in it such that it shows up in a ps. At least 23 characters must be given to reserve sufficient space.

-n

This option is a NOP. It does nothing, but is often used by other programs (aka dspoolout) to placehold optional arguments.

-i

This option disables the streaming negotiation, forcing the non-streaming ihave protocol to be used. If not given, DNewslink will attempt to negotiate the streaming protocol and fallback to the non-streaming protocol if the negotiation fails.

-D

This option causes dnewslink to detect when the batch file it is operating on is deleted out from under it. DNewslink will not attempt to rewrite the batchfile, or will rewrite and remove the batchfile if it detects this case and the -D option is given.

-f

This option has been depreciated.

-p

This option specifies that the input is coming from a pipe rather then the batchfile (see -b option). A batchfile must still be specified, but it will be used solely for the rewriting of defered lines.

-P port

This option specifies the port on the destination host dnewslink should connect to. The default is port 119 (nntp).

-d [level]

This option turns on debugging. Normally something like -d2 is specified when dnewslink is run manually in order to trace the protocol interaction.

-b batchfile/template

This option specififes the batchfile. The batchfile specification depends on the mode of operation. For a pipe input, this is the spill file. With a normal batch, this is simply a batch file such as 'label.S04522'. When combined with -S, this is a template (usually 'label.S%05d'). For a realtime feed, this is the name of the realtime queue file... usually just 'label'.

-r

This option turns on realtime operation, indicating that the batchfile is being actively appended to by another program (aka the diablo server). This option enables a number of optimizations including detecting when dspoolout flushes/renames the active file. When this occurs, DNewslink will attempt to reopen/lock the new realtime queue file and, when this succeeds, will delete the renamed file. However, if deferals had to be made, DNewslink will not remove the renamed file which usually results in dspoolout rerunning the file as a normal spool file.

-S beginSeq

This option turns on multi-batch sequencing, causing DNewslink to attempt to feed several batch files in sequence. If given, you must specify the beginning sequence number in this option and the batchfile name (-b option) must contain a template rather then a filename.

-N numSeq

This option specifies the number of sequential sequence numbers to attempt to feed out. DSpoolout usually specifies 32 here. If there are more then 32 batch files or if dnewslink gets interrupted before completion, dspoolout will simply start another one. For realtime batches dspoolout specifies a higher number which reflects the number of new queue files it is allowed to rotate through before exiting. For queue-delayed batches (qNNNN option in dspoolout.ctl), dspoolout will specify an appropriate number based on the number of queue files.

-H

This option tells dnewslink to generate a header-only feed, used when feeding the diablo reader system and often coupled with diablo support for Xref: headers to assign article numbers. The remote end must support sent to a normal feed and propogating outward. Beyond that, the header-only feed uses the same ihave/check/takethis protocols that a normal feed uses.

-L

This option will enable generation of Lines: headers in headfeed mode (see above); the reason for this option is to always have a meaningful lines count in overview. Specifying the 'genlines' option in dnntpspool.ctl will make dspoolout pass -L to dnewslink.

-h remote-host

This option specifies the remote system to connect to. See also: -P option.

-w delay

This option specifies the connection fail / reconnection delay in seconds. The default is 10 seconds.

-t timeout

This option specifies the inactivity timeout. The default is 10 minutes.

-l logFrequency

This option specifies the log frequency. The default is to log every 1000 transactions.

-c reconnectFrequency

This option specifies the forced disconnect/reconnect frequency to cause the remote end to generate a log line. The default is to close and reopen the connection every 1000 transactions.

-T txbufsize

-R rxbufsize

These options set the TCP receive and transmit buffer sizes. DNewslink will impose a minimum receive buffer size to prevent pipeline lockups. This option is normally used to tune the system. You can usually reduce the receive buffer size to 4096 bytes without effecting efficiency. Some people may want to increase the transmit buffer size from the system defaults. These same options can be given to dspoolout and will be passed to dnewslink.

-I

This option disables the "check" NNTP command. Rather then check each message id, dnewslink will simply push the article out with "takethis", assuming a streaming feed was negotiated. Useful for header-only feeds.

-A [classes]

This option turns on article logging to <logpath>/feedlog.<hostname>. Classes is is either comma-separated list of any combination of "accept", "reject", "defer", "refuse" and "error", or "all". Not that specifying "refuse" can generate huge logfiles. Default is "all" if classes is not specified, and no logging if this option is not given.

-B

This option sets the source ip address for outbound connections. It can be either an ip address or a hostname which will be resolved by DNS.

TYPICAL USE

DNewslink is run by dspoolout and usually operated manually only for debugging purposes. Typical usage is something like: dnewslink -b fubar.S23343 -h news.fubar.com

SEE ALSO
diablo(8), dicmd(8), didump(8), diload(8), dnewslink(8), doutq(8), dexpire(8), dexpireover(8), diconvhist(8), dilookup(8), dspoolout(8), dkp(8), dpath(8), diablo-kp(5), diablo-files(5)