RED - Listserv Redirector (C)1988 Turgut Kalfaoglu What is RED? RED provides the SIMTEL-20 files, and directory listings, with its own cache, where it keeps its most recently requested files. It reduces the network load by providing the cache, and by providing directory listings locally, instead of through a distant list server.. It is a machine (process) that runs disconnected from a terminal. Currently, the six sites that run this software are called: In Denmark: TRICKLE@DKTC11 In Turkey: TRICKLE@TREARN In Italy: TRICKLE@IMIPOLI In Belgium: TRICKLE@BANUFS11 In Austria: TRICKLE@AWIWUW11 In Germany: TRICKLE@DB0FUB11 You are urged to use the one that is closer to your location. In this tutorial, we will be using 'TELL TRICKLE AT TREARN', but this can be replaced with 'TELL TRICKLE AT ' We also will use the 'TELL' command to send a single line message. It should be replaced with whatever is appropriate for your system. (Like XMIT , SEND, etc.) If you are on a node that cannot reach a TRICKLE directly, for example, a JANET node, then you must send MAIL files to the server. Simply put the commands, one per line, into the text portion of your mail. What Does it Provide? A Milnet node, SIMTEL20.ARMY.MIL at White Sands Missile Range, New Mexico contains a large selection of public domain and 'shareware' software. This DECsystem-20 machine, running the Tops-20 operating system provides many files of interest, especially to CP/M and MSDOS users. The collection is open to public, anyone may obtain copies of this of the files using the Internet file transfer protocol, FTP. However, this protocol is not available to Bitnet, or EARN sites. For this reason, two servers in The United States, who have a connection both to Milnet and to Bitnet, provide us with these files. However, since both of these servers are in The US, the requests of these files puts a burdon on these two servers. The solution was to create a server here in Europe that could provide the files requested, send the directory listings, and also keep the recently requested files, in case someone else wishes to have the same file. We, the server operators, would like to stress that we have no affiliation with the US Army, nor with White Sands Missile Range. These servers are made available in the true spirit of volunteerism, without any outside sponsorship for the service. The Trickle, and The US servers support the following directories: CPM Software and information for CP/M users. Contributions are gathered from a variety of sources, including the members of the Info-CPM electronic mail discussion group. This archive is updated very frequently. MSDOS Software and information for PC-DOS and MS-DOS system users. This archive is updated very frequently. PC-BLUE Software and information for PC-DOS and MS-DOS system users. The archive contains files distributed by the PC-Blue Users Group. New files are added as they become available. SIGM Software and information for CP/M users. The archive contains the files distributed by the SIG/M Users Group. New files are added as they become available. MISC Software and information for miscellaneous systems (mostly large systems like IBM/370 and DEC VAX). Contributions are gathered from a variety of sources. How does it work? It provides faster file delivery than LISTSERV@RPICICGE because it holds the most recently requested files, and it also asks its peers, to find out if any of them has the file. It has two major commands. /PDDIR and /PDGET.. As the names indicate, /PDDIR provides the names of the files, and /PDGET delivers files. How to use /PDDIR: On IBM/VM systems, you can get a list of the 'major directory' names by simply typing this command: TELL TRICKLE AT TREARN /PDDIR RED should now send you a list of the major directory names. Now, you can obtain a list of the sub-directories of any of the displayed names by putting the name between less-than and greater-than symbols.. For example, TELL TRICKLE AT TREARN /PDDIR RED will mail you a file containing the names of the sub-directories. Once you choose a subdirectory to examine, type in the main directory name, a period, then the name of the subdirectory name. For example, if you chose SYSUTL sub-directory of MSDOS, then you may type this: TELL TRICKLE AT TREARN /PDDIR RED will first notify you of the number of files found, then will go ahead and mail you this list.. Once you choose your file(s) to order, then read on.. How to use the /PDGET command: Once you have a filename in hand, then tell RED to send you this file by providing it with the full directory name, and the filename.. If you wish to order a game called MADMAX.ARC that resides in the directory, then you may type in this command: TELL TRICKLE AT TREARN /PDGET MADMAX.ARC naturally, the above is only an example, and the file may or may not be present if you send the above command. Receiving your file in a different format: Normally, SIMTEL files are sent AS-IS, meaning, ASCII. If you would like to receive your file in a different format, you may want to append any of the below to the end of any of your your commands: (EBC80 Converts the file to 80-Column EBCDIC format (EBC32 Converts the file to 132-Column EBCDIC format (UUE uuencodes the file (OLD Sends the file using DISK DUMP or PUNCH format (SF Supresses BITSEND, and forces SENDFILE to be used for the transfer. (MAIL Forces results to be sent via MAILER. (This option is automatically added for MAIL command files) (XXE XXDECODES the file that is to be sent to you. (HEX Turns the file to HEX format - use it if even (XXE doesn't work for you. (BTOA BTOA-Encrypts the file. Useful for Unix systems. You may also wish to combine several options together. For example, to receive a directory listing in PUNCH format, and UUEncoded, TELL TRICKLE AT TREARN /PDDIR (OLD UUE To receive a file in in EBCDIC format, you may enter a command that looks like this: TELL TRICKLE AT TREARN /PDGET MADMAX.ARC (EBC80 However, it is not useful at all to receive an .ARC file in EBCDIC format. The above is not a terribly good example. Other commands: /NEWS sends you our 2-page newsletter. /STAT provides you the statistics of usage. /HELP sends you this file. /IMDAT sends the Turkish version of this help file. Note that this command is only valid for the TREARN server. /CAC Sends you a list of the files that are stored on its disk right now. These files can be sent faster than the other files. /OPS displays the RED operators /QUO Shows you the RED's quota, and how much of that quotait has used. Once RED exceeds its quota, it cannot order files, until it receives some of the requested files. /SUB Allows you to subscribe to a directory. Whenever a new listing comes in, RED will send you a file containing the names of the new files. /UNSUB is to stop RED from sending you the new files listings. Please issue this command if you will not use the server anymore. /NEW nnn This command, displays the files that have arrived within 'nnn' days, in the 'dirnam' directory. If 'nnn' is omitted, it defaults to the last time you issued this command, for that directory. If you are issuing this command for the first time, then it simply looks for files that are at most a month old. /POLL forces RED to check its peer servers Delay Periods: If the file that you requested already exists in the cache directory, then you may expect to receive your file within a few hours. However, the system that RED is running is often slowed down by the other processes that are running. This negatively affects the response time of RED. If the file requested does not existin the cache directory, then RED will have to order this file from its list server.. If this is the case, the response time of RED is dependent upon the list server. RED will give up waiting for a file after five to twelve days after its request. Sending files to RED: RED now accepts command files in either MAIL or NOTE format. Use NOTE or MAIL utilities to prepare your command file. The files may contain any number of instructions, one per line. Note that RED will abort processing a file if it detects an invalid command within the file. The server has a 'likewise' habit, and will MAIL back your files, using the default UUENCODING, unless you tell it otherwise, if you request your file via mail. This is done for those of us, who are not on Bitnet, and keep forgetting to put the (MAIL at the end of the command. How to DONATE files to Simtel Archives: Files that you receive from here are sent from another network called ArpaNet. The person-in-charge for the programs is: Keith Petersen . Since it's another network, you will need to use MAIL to send the message. He urges that you talk to him before you send in the file, so that he can check where it should be put, if it already exists, etc. After getting his approval, you need to UUENCODE your file (perhaps using PDUTIL), then MAIL it to him. Format of the files that comes with /PDGET command: For the below chart, we shall assume that you have not placed any conversion options at the end of your command. If you have used: You can expect the file to arrive: ----------------------------------------------------------- 'tell' style message: BITSEND, NETDATA format. ----------------------------------------------------------- MAIL command file: UUENCODED, in numbered pieces. ----------------------------------------------------------- A regular file, or Just like 'tell' messages, IBM's NOTE command: replies in BITSEND, NETDATA. ----------------------------------------------------------- RED will send the files in a NETDATA format, -unless you use the (OLD option-. On IBM systems, these files can be LOOKed and RECEIVEd, but the PEEK command cannot handle NETDATA format properly. However, since most files are ASCII, it is of little use to LOOK at them. The files will mostly be either in Binary format, in ASCII format, or in EBCDIC format. The binary files are recognized by the '.BIN','.EXE', '.COM ', '.ARC', '.LBR' suffix in their names. These files are machine- specific.The files that are in ASCII format can be converted to EBCDIC (so that they can be used on an IBM system) by running a conversion program called PDUTIL. If your installation does not have either of these files, Either I, or the LISTSERV@RPICICGE can provide it. Simply say: TELL LISTSERV AT RPICICGE GET PDUTIL PACKAGE The .ARC format: ARC is a special compression method that provides substential reduction on file size. There are one or more files contained within an .ARC archive. In order to extract the files from an archive, you will need a utility called ARC or PKXARC.These are available from directory. The actual file names of these files vary, but you may try: TELL TRICKLE AT TREARN /PDGET PK361.EXE or: TELL TRICKLE AT TREARN /PDGET ARC512.EXE If these attempts fail, it will probably mean that the file version has changed, and so has the file name. You may wish to try TELL TRICKLE AT TREARN /PDDIR and guess the new name of these files. Once you receive either ARC or PKXARC and an .ARC file, transmit them to your personal computer and issue the following command: ARC X or PKUNPAK There is also a second utility called ARCUTIL, which runs on the VM systems, and extracts files. It also provides ASCII to EBCDIC conversion of the extracted files. To request ARCUTIL, enter: TELL TRICKLE AT TREARN /PDGET ARCUTIL.LBR The directories of SIMTEL change often. So, the above files may or may not be in the same directories as I have indicated. So you may have to do some searching to find them.. A good place to check would be the directory, where additional help, and the mentioned archive managers can be found. How to use the BITSEND/BITRCV: >From now on, RED is sending its files in a special format called BITSEND - Unless you include the (SF option while issuing your command. In this format, the files that are sent are broken into smaller pieces, if the entire file is too big to be sent. If the file you request is over the size limit, then the server will first send you a file called BITCTRL - this is the control file, where BITSEND has written the protocol used, the number of pieces that make up that file, etc. ------------------------------------------------------- Important: You should NOT 'RECEIVE' any files that have BITCTRL or just numbers as filetype! ------------------------------------------------------- If you wait a little longer, the rest of the file will arrive, the filename will be the same as the BITCTRL file, but the filetype will consist of just numbers. Once you have all the pieces that make up that file, you can then issue the BITRCV command. You must also specify the 'spool ID' of the file that has BITCTRL as filetype. From RDRList, you can simply type BITRCV in front of the file that has BITCTRL as filetype. If you get back an error message, saying "UNKNOWN CP/CMS COMMAND" it simply states that your installation does not have the BITRCV program. Contact your network manager/system operator. Note: BITRCV EXEC can be obtained from your country NETSERV as well. Simply send "GET BITRCV EXEC" to your NETSERV. ----------------------------------------------------------- VAX users: There is also an identical file for your installation, however, it is called 'BITRCV COM', and can be obtained from a NETSERV by issuing GET BITRCV COM to your country NETSERV. ----------------------------------------------------------- If you get back an error saying that not all of the file is in the reader, it simply means that you have to wait a little longer for the rest of the file to arrive. How to receive the file that arrives: Once the file you requested arrives, and is stored on your disk, you will most likely wish to 'download' this file to your personal computer. There are many types of mainframe computers, many kinds of personal computers, so it is impossible for me to give you direct, and precise directions. However, here are some clues: * If you have a PC with a 3270 Emulation program, and an IBM mainframe, you should request your files from the server without options, or with (SF option, and use the built-in transfer protocol of the emulation program, without any options again. * If you have KERMIT at your installation, request your file without options, or with (SF option, then set the KERMIT's FILE-TYPE to BINARY before transferring your file. * Remember that if you send a MAIL command file to the server to request your file, the file will arrive in UUENCODE format, since the mailer cannot process binary files - unless you specify (XXE or (HEX in the command line. How Does the Cache Work? Imagine that you ask for a file, and the server brings this file from United States for you. Thinking that others may wish to have this file as well, the server keeps this file in an area called 'cache.' When someone else requests this file, the server simply uses the stored copy of the file, instead of asking for the file again from overseas. All the servers that you see on top of this document have different files in their caches. So, if you wish to see the files they are holding right now, you will have to issue '/CAC' to each one of them. Note that a file does not stay in cache forever. As new files arrive, the older ones are deleted to make room. The Amazing Life of a /PDGET request: Once you order your file via /PDGET, the server will first check its local cache listings. If the file is not there, then it will check the SIMTEL20 listings to ensure that a such file indeed exists. After this, the server sends the request to all other servers, asking them if they have your file in cache. If a server replies 'YES!', then that server has to send you the file. Everything fails: none of the servers have your file, or even some servers don't respond. Your server will give up waiting for a reply in a day, and order your file from the United States. Once the file requested arrives from there, it will be sent to you, and put into the cache directory. Quite a trip for one /PDGET command. Quotas, and Other Ugly Limitations You may be surprised that even though most TRICKLE servers have some kind of quota, we still get several hundred requests daily. Without them, this number may easily rise to thousands. The impact of a such usage rate on the local computer can be very 'tiring.' So, the following quota schemes have been implemented: 1) Total outstanding bytes quota: This quota is not really put by the server's operators. It is the amount that a TRICKLE server can order from The United States. This is currently set at 10 megabytes for most servers. 2) Prime times: Some of the servers, do not function during the day, they record the commands received, and process these commands later, when the load on the computer is low. 3) User request limitations: Most TRICKLE servers have a limit on how many requests a user can make on the server per day. The request can be a simple /OPS command, or a file order via /PDGET. It still counts as one. The server will warn you that you are approaching the limit, once you have 3 more commands left. 4) Outstanding files per user: This scheme is also employed by some servers, and it limits the number of files a user can order from The United States. 5) Delayed Sendfile: This last scheme is simple: it delays sending your file until a specified time comes. Usually at night, when the network load is low. If a site uses delayed sendfile, you will see a '* Your file will be mailed' notice, instead of '* Your file is being mailed.' A Last Word on Options: Some of the options are not compatible, such as (MAIL SF, and should not be used together - the behavior of the server may be unpredictable. Also, the (SF option may result in a file that is too large for shipment. If that is the case, a network control program may detect it, and delete it before it reaches you. Use (SF carefully. Also, (SF and (OLD options would be ignored if you send in your commands in a MAIL file. If you wish to receive your files AS-IS, and still be able to put your commands in a file, you can either send a NOTE to the server, or simply create a file using your editor, then send this file directly to the server, without first going through the mailer. VAX/VMS Users: If your host is a DEC VAX system running VMS with Jnet networking software, you can avoid the need for uuencoding. You can tell the Jnet software to bypass the usual EBCDIC/ASCII conversion, but there are a few additional steps needed before downloading a file. * Receive the file with the Jnet command RECEIVE/BINARY. The BINARY modifier suppresses the normal EBCDIC/ASCII conversion. Let's assume that the file is called MYFILE.ARC. This file, as received, is almost correct; there may be an error is how VMS interprets the records. * Generate an FDL file for MYFILE.ARC using: ANALYZE/RMS/FDL MYFILE.ARC * Edit the FDL file with the command EDIT/FDL MYFILE Examine the CARIIAGE_CONTROL setting. Change it to NONE. Exit the editor. * Use the edited FDL to correct carriage control interpretation errors in the original MYFILE.ARC: CONVERT/FDL=MYFILE.FDL MYFILE.ARC FIXED_MYFILE.ARC * Download the FIXED_MYFILE.ARC as a binary file using any reliable means. (For VAX Kermit, use SET FILE TYPE BINARY command before starting the download.) Puzzled? We now have an online discussion list that gives assistance on the server. To join this list, simply send the following command to either LISTSERV AT TREARN, or LISTSERV AT DB0FUB11: SUB RED-UG Myfirstname Mylastname Also, you may get in touch with your local TRICKLE operator. You can get his network address by using the /OPS command. We wish you great benefits from using TRICKLE - we know that the software it provides can accomplish that.