Conference 7.286::fddi

1284.0. ""Striping" drivers for faster FDDI" by MUDDY::WATERS () Tue Mar 22 1994 07:56

Article 2466 of comp.dcom.lans.fddi:
Path: nntpd.lkg.dec.com!crl.dec.com!crl.dec.com!decwrl!hookup!swrinde!
	ihnp4.ucsd.edu!sdcc12!cs!jkay
From: [email protected] (Jon Kay)
Newsgroups: comp.protocols.tcp-ip,comp.unix.ultrix,comp.sys.dec,
	comp.org.usenix,comp.dcom.lans.fddi,comp.unix.osf.osf1
Subject: Port of Checksum Redundancy Avoidance to Alpha OSF/1
Message-ID: <[email protected]>
Date: 19 Mar 94 01:38:40 GMT
Sender: [email protected]
Followup-To: comp.protocols.tcp-ip
Organization: UCSD Computer Systems Lab
Lines: 190
Nntp-Posting-Host: cosenza.ucsd.edu
Xref: nntpd.lkg.dec.com comp.protocols.tcp-ip:31548 comp.unix.ultrix:25276 
	comp.sys.dec:23324 comp.org.usenix:5149 comp.dcom.lans.fddi:2466 
	comp.unix.osf.osf1:2645


	This note is to announce availability of implementations of
checksum redundancy avoidance and a striping network interface driver
for Alpha OSF/1.  It is available for anonymous FTP from

ucsd.edu:/pub/csl/fastnet

	A while back Joe Pasquale and I published a paper on checksum
redundancy avoidance in the Winter '93 Usenix and released an
implementation of it for Ultrix.  This is a port of that software to
OSF/1.  Checksum redundancy avoidance is simply the observation that
for  TCP/IP data transmission over LANs, the software data checksums
are redundant with the LAN CRC, and it is worth avoiding doing the
software checksum in that case.  Checksumming is still done on routed
packets because then the software checksum is no longer redundant with
any CRC. 
	One problem we have with network research on Alphas is that
stock OSF/1 can already saturate an FDDI network, so we have to use a
striping driver to get higher bandwidths to see the effects of our
modifications.  Our solution is the striping, or concatenation driver.
This makes multiple FDDI drivers look like a single, much faster
driver.  We call this driver cat0 (for concatenation driver), and the
source code resides in io/dec/netif/if_cat.c.  It is included in the
distribution.
	Between the concatenation driver and checksum redundancy
avoidance, we are able to achieve 160 Mbps of throughput between two
Alpha 3000/500s, each equipped with two DEFTA FDDI interfaces striped
with cat0.  We are able to achieve 210 Mbps of throughput on the
loopback interface of a 3000/500.
	Note: the checksum elimination algorithm is experimental.  
	If you do anything with the software, or have comments on the
checksum elimination algorithm, I would be interested in hearing about it
(our addresses may be found at the bottom).
	There are miscellaneous other goodies likely to be of interest
to those interested in high performance networking: a SIGCOMM '93
paper on why realistic packet processing is not dominated by data
touching overheads (sigcomm93.ps), the old Ultrix 4.2a checksum
redundancy implementation in both patches to source and object files
(ultrix-4.2a-patches and ultrix-4.2a-BINARY, and a short monograph on
choosing packet size for best efficiency (BUFFERS).  Also, various
other software and papers written in the UCSD Computer Systems Lab can
be found in adjacent directories on the FTP server.  Still other
papers can be found on cs.ucsd.edu:/pub/csl.

There are several files in the distribution:

README - this note
DOC - documentation
BUFFERS - optimal I/O buffer size considerations
usenixw93.ps - the Usenix paper
sigcomm93.ps - a SIGCOMM paper on why data touching overheads don't
		dominate processing of realistic network workloads.
osf-1.3-patches - patches to Alpha OSF/1 1.3 source code implementing
		the improvements. 
osf-1.3-OBJS.tar.Z - object files for Alpha OSF/1 1.3 implementing the
		improvements. 
ultrix-4.2a-patches - patches to Ultrix 4.2a source code implementing
		the improvements. 
locore_patch - patch to Ultrix 4.2a MIPS /sys/machine/mips/locore.o
		(in BINARY distribution!) implementing RISC checksum.
ultrix-4.2a-OBJS.tar.Z - kit of object files to implement improvements
		on binary kernel.
ccsum.c	-	Extremely fast checksum routine for MIPS processors.


UNSUPPORTED OS's
	Oh, yeah: what happens if you aren't running exactly the same
version of OSF/1 or Ultrix than is supported here?  Good question.
This stuff will PROBABLY work with other releases since the TCP/IP
code doesn't seem to change much from release to release, but then
again it might not work.  I would be interested in hearing if you try
it out and it doesn't work - maybe some arrangements can be made.
	For that matter, it ought to be pretty easy to port the Ultrix
patches to any Berkeley Tahoe based OS, and it ought to be pretty easy
to port the OSF/1 patches to any Berkeley Reno or 4.4 based OS.


INSTALLATION

OSF/1 BINARY DISTRIBUTION:

	osf-1.3-OBJS has a BINARY subdirectory.  Copy the object files
in BINARY into the BINARY build directory of your kernel object tree,
being sure to first move old versions aside.   
	Next, some patches must be made.  Change directory to the top
directory of your kernel tree (normally /sys), and run

patch -p0 < osf-1.3-OBJS/patches

	The next step is to reconfigure your kernel.  If you are
running the GENERIC or GENERIC.rt kernel, this has been done for you.
Otherwise, use the same kernel configuration file (in /sys/conf on
Alphas and/sys/conf/mips on DECstations) as you already use EXCEPT you
need to specify that you want to use the checksum redundancy avoidance
and other performance-enhancing patches.  This is done by adding two
lines to the kernel configuration file:

options		SEQUOIA_FASTNET     	# faster networking
pseudo-device cat 1			# concatenation driver

Then follow normal procedures for building a kernel.


ULTRIX BINARY DISTRIBUTION:

	Copy in the object files in BINARY into /sys/MIPS/BINARY,
being sure to first move old versions aside.  Now, since locore.o is
typically recompiled whenever you reconfigure your kernel (even from
binary distribution!), you have to patch your locore.s using the patch
in locore_patch.
	The next step is to reconfigure your kernel.  Use the same kernel
configuration file (in /sys/conf/mips) as you already use EXCEPT you
need to specify which checksum improvement algorithm, if any, you wish
to use.  This is done by adding an option to the kernel configuration file:

options		SEQUOIA_FASTNET      # faster networking

Then follow normal procedures for building a kernel.
	Note: the object files are compiled in such a way that all the
improvements, even the experimental checksum elimination algorithm, are
enabled.  It is, however, possible to set things up so that you only use the
RISC checksum (more than halving the benefit of this package).  Since the
RISC checksum is completely contained in locore.s, and no other enhancements
are in there, one could only install that patch and avoid installing any of
the object files.

	Oh, yeah: what happens if you aren't running Ultrix 4.2a?  Good
question.  This stuff will PROBABLY work with other Ultrix releases
since the TCP/IP code doesn't seem to change much from release to
release, but then again it might not work.  I would be interested in
hearing if you try it out and it doesn't work - maybe some arrangements 
can be made.


OSF/1 SOURCE DISTRIBUTION:

	Change directory to the top directory of the kernel source
code to be patched.
	Make the patches to your source code, using 'patch -p0' and
the appropriate patches file.  If you are using source code of a
variant OS version, you may want to put in the patches by hand.  
	Next step is to reconfigure your kernel.  The patches
automatically configure the BINARY, BINARY.rt, GENERIC, and GENERIC.rt
kernel configuration files to add support for the SEQUOIA_FASTNET
option and cat0 concatenation interface.  If you are using a different
configuration file, you need to add the following two lines to it:

options		SEQUOIA_FASTNET     	# faster networking
pseudo-device cat 1			# concatenation driver

(the SEQUOIA_ prefix is to ensure that names of kernel enhancements from the
Sequioa 2000 project don't run afoul of other options invented
elsewhere).  
	Then build your kernel normally, starting with setup in the
kernel.sh script, because the patches don't modify the shadow in the
../obj directory.


ULTRIX SOURCE DISTRIBUTION:

	Make the patches to your source code, using the
ultrix-4.2a-patches file.  If you aren't using Ultrix 4.2a source
code, you may want to put in the patches by hand.  Then build your
kernel normally.

	Next step is to reconfigure your kernel.  Use the same kernel
configuration file (in /sys/conf/mips) as you already use EXCEPT you
need to specify which checksum improvement algorithm, if any, you wish
to use.  You do this by adding an option to your kernel.  There are
four options (if you specify none, you get a "normal" kernel): 

options		SEQUIOA_FASTNET      # all nifty new algorithms in one option
options		SEQUIOA_FASTSUM      # just RISC checksum
options		SEQUIOA_NOSUM        # just checksum elimination
options		SEQUOIA_NOSMALLBUFS  # just if_mtu reduction to 4k

(the SEQUOIA_ prefix is to ensure that names of kernel enhancements from the
Sequioa 2000 project don't run afoul of other options invented
elsewhere).  You can mix and match those options, but SEQUOIA_FASTNET
will save you time.


Addresses:

	Jonathan Kay			Professor Joseph Pasquale
	[email protected]			[email protected]

	Department of Computer Science and Engineering
	University of California, San Diego
	La Jolla, CA 92093-0114

Ya sure.

udp:
        46208824 total udp requests
        0 incomplete headers
        0 bad data length fields
        13 bad checksums

Of course, some of those may have been caused by people implementing the
bug in RFC 1141... :-)

On the other hand, the statement about checksum being redundant is false.
More precisely, it is false in the common case where the checksum is implemented
in the host.  If the checksum is done as a "hardware assist" on the adapter,
then indeed it is redundant, but that is also the case where you aren't going
to get any performance gain from eliminating it!

As for the "striping" notion, that's interesting.  Of course, you don't need
it if you use DECnet/OSI, since there it is a standard feature (called
"equal cost path splitting").

	paul