Conference netcad::hub_mgnt

2762.0. "DS900EF Stop Forwarding Ethernet Broadcast!" by HGRNCC::FARADAYCHONG () Fri Sep 15 1995 05:41

    The following incidences occurred in one of our most prestigious
    customer.
    
    Is this known problem?
    Any resolution or suggestion?
    
    Faraday
    
Problem: DECswitch 900EF failed to forward Ethernet broadcast frames

Due to unknown reason, some of our DECswitch 900EFs failed to forward
Ethernet broadcast frames, and therefore ARP, Bootp and other protocols
rely on broadcast were failed across the switches.  The only solution is to
reboot them.

We have come accross this problem 3 times during the last week.
With the help of Sniffer LAN analyser, we observed that only Ethernet
broadcast frames (i.e. 0xffffff-ffffff) were blocked, other network traffic
accross the switches, including uni-cast and multi-cast, were normal during
the presence of the problem.

We have a total of 9 DECswitch 900EFs installed, 7 of them are connected
to the backbone network (4 via FDDI and 3 via Ethernet), the other 2 are
used inside subnets.  The following diagram illustrates the switches
connection on the backbone:

               FDDI backbone
        +=======+=======+=======+
        |       |       |       |
        |       |       |       |   Ethernet      FDDI
	S1	S2	S3	S4------+------S5====== NT server
	|	|	|	        +------S6====== NT server
	|	|	| Ethernet      +------S7====== NT server
        |       |       |
      NTs &    NTs   subnets
     subnets

Switches configurations:

		S1	S2	S3	S4	S5	S6	S7

up time	      >7 mth  > 6mth  > 3mth   2 week  2 week  2 week  2 week

up link	       FDDI    FDDI    FDDI    FDDI    Ether   Ether   Ether

Firmware       1.2.3   1.4.0   1.4.0   1.5.0   1.5.0   1.5.0   1.5.0

rate limiting
for broadcast  enable  enable  enable  disable disable disable disable

rate limit     200pps  200pps  200pps   -	-	-	-

Other filter   DECnet  DECnet  DECnet  none    none    none    none
for rate       endnode endnode endnode
limiting       hello   hello   hello

The 3 events are being analysed and summarized as follows:

1.	02:00 Sep 08

	S2 and S4 hit the problem.  Both of them resumed normal after
	reboot.  S4 is replaced by another new DECswitch 900EF, namely
	S4(R) (firmware 1.5.0 with no rate limiting enable), Ethernet
	segments connected by S2 (mainly connected NT servers using
	DEC Alpha) were moved from S2 to S4(R) also.  After that, S2
	was idle and nothing was connected other than the FDDI uplink,
	while S4(R) connected 3 DECswitch 900EFs and some NT servers.

2.	03:30 Sep 11

	S2 and S4(R) hitted the same problem again.  Reboot S2 and S4(R).

3.	Before 08:00 Sep 13

	S2 and S4(R) hitted the same problem again.  Reboot S2 and S4(R).

We have no more spare DECswitch 900EF on hand, no replacement was done 
after the first event.  However, the fault is hard to spot, we have
logged a call to the Digital fault report centre (Ref. is B90293), what
the engineer suggested was to swap equipment, but we believed that the problem
should not be individual (We have a total of 3 DECswitches hit the same
problem), it seems to be a firmware bug, is there any update patch for
DECswitch 900EF other than version 1.5.0?

Since DECswitch 900EFs are the core of our backbone network, its failure
cause a lot trouble to hundreds of users (can't boot, print, ARP lookup
etc.), quick response to this problem is appreciated.

    Other sites have had similar problems with the DS900EF(DB900MX)
    modules.  Engineering has said that the switch gets fried due to bad
    packet on the network (suposedly the packet has an invalid source
    address).  
    	Engineering says to find the node(s) sending the bad packets and
    fix it.
    
    	My question.....is why does the bridge get fried just because a bad
    packet shows up on the net???

    I would like to any known software/hardware "fried" DS900EF/DB900MX?
    
    At least, this help us start looking for potential sources of problem.
    
    Faraday

    That path (dechub-boot) does not currently exist on cigam1.  Can you
    fix that please or provide a pointer to another location?
    
    Thanks!
    
    Frank

    re .4
    
    I couldn't access the file too from IP, but it here via DECnet with
    path cigam1::"/dechub-boot/defba_152.bin".
    
    Faraday

    
    We have tried to copy the defba_152.bin from cigam1.dechub.lkg.dec.com
    and uploaded to a switch 900EF successfully.  However, it is found that
    image verison is still v1.50 and this file size is also equal to v1.50.
    
    Both files have been compared by DIFF command of VMS and result are as
    follows :-
    
    $ diff DEFBA_150.BIN DEFBA_152.BIN
    
    Number of difference sections found: 0
    Number of difference records found: 0
    
    DIFFERENCES /IGNORE=()/MERGED=1-
        DSA4:[FAL$SERVER]DEFBA_150.BIN;1-
        DSA4:[FAL$SERVER]DEFBA_152.BIN;1
    
    Is the defba_152.bin a patch file to fix the customer's problem ? 
    Please help to verify this problem sooner.
    
    Thanks a lot !
    
    Regards,
    Ivan Cheng
    

re. .3, Bill,

can you please explain what problem was fixed? And did these symptoms get
triggered by an invalid source address (multicast bit set is the only I can
think of)? Is this ratelimiting dependent?

thanx, greg.

we too have customers with 900EFs in very critical lans.

    re .6                                                        
    As a sanity check I just copied the v1.5.0 and v1.5.2 images and they ARE
    different (you had me scrambling there for a minute).  
    
    And the version identifier was updated so the "Show Current Settings" option 
    from the console will display the version as "1.5.2".
                                                                 
    re .7                                                        
    V1.5.0 had a "bug" in the FDDI adapter initialization code that caused it to 
    NOT drop packets containing a multicast SA.  v1.5.1 contained a fix that 
    initialized the FDDI MAC control register to the correct value.

    Testing at two customer sites determined that 1.5.1 did not solve the 
    problem.  Even by turning on the hardware filter for multicast SAs on the 
    FDDI port in v1.5.1, some packets with multicast SAs were leaking
    through.  In v1.5.2, an additional check is made for packets received
    with  multicast SAs on the FDDI port and they are discarded and
    counted.   Any packets with multicast SAs received on the ethernet ports 
    have always been discarded from the earliest version of the firmware. 
                   
    Bill