Conference spezko::cluster

   What are the allocation classes of the AlphaServer 2100, the VAX 6000-510,
   and the DSSI disk ISE?  (This value should be non-zero, and should match
   across all three.)

   Be aware that since the AlphaServer 2100 must be in a non-zero disk
   allocation class to serve the DSSI disk, you will need to make sure that
   the disk unit numbers are unique across all disks in that allocation
   class across all nodes and all HSx controllers and all DSSI ISEs in that
   allocation class -- this includes both the DSSI disks *and* the local
   SCSI disks.  The unit numbers must be unique regardless of the device
   driver prefixes used: DU, DK, etc.

   To alter the disk unit numbers, it's likely easiest to use the DSSI ISE
   parameters to alter the disk unit number on the DSSI spindle, but some
   Alpha systems do include a mechanism to alter SCSI unit numbers via
   SYSGEN parameter.

   If you alter an allocation class, you will need/want to reboot all nodes
   in order to pick up the change consistently...

	Thanks for your answer.

Customer will change these unit numbers.

> The unit numbers must be unique regardless of the device
>   driver prefixes used: DU, DK, etc.

Could you give some more details about the reason why ?

regards,

Maria.

:> The unit numbers must be unique regardless of the device
:>   driver prefixes used: DU, DK, etc.
:
:Could you give some more details about the reason why ?

   Because it is a requirement -- the lower levels of the VMScluster
   protocols look only at the allocation class and the unit number
   when identifying devices.  If there are devices with unit-number
   overlaps within a particular allocation class, some of the low-level
   VMScluster code can potentially become "confused" as to the uniqueness
   of these devices, with, uh, "interesting" results.

>>> The unit numbers must be unique regardless of the device
>>>   driver prefixes used: DU, DK, etc.
>>
>>Could you give some more details about the reason why ?

Steve's answer in .3 is mostly correct:  MSCP creates a unit identifier out of
the allocation class, unit number, and controller letter.  This is how the MSCP
server identifies a unit, and how DUDRIVER specifies a unit as a target for an
I/O operation.

The other reason this is so is that on the client system, DUDRIVER is the class
driver for the device, regardless of whether it's a DU unit or a DK unit. 
DUDRIVER is the only disk class driver that knows how to speak the MSCP
protocol, so it's the only class driver that can be used on a client system, and
so all disks that want to be distributed have to conform to DUDRIVER's rules.
And one of these rules is there must be a unique allocation class/unit number
for every device in the cluster.

If there are two devices that have the same allocation class/unit number
combination, DUDRIVER on the various client nodes will pick one, and all I/O
will go to that volume.  Which one is picked cannot be reliably predicted, and
will likely change as the cluster configuration changes due to state
transitions, nodes coming and going, and even transient events on the ethernet
or FDDI.  DUDRIVER on different clients may (probably will) make a different
choice -- client A will access one volume and client B will access the other,
even though they both may believe they're accessing the same one.  This can lead
to serious data corruption; one of the "interesting" results Steve mentioned in
.3.

John

  .4> If there are two devices that have the same allocation class/unit number
  .4> combination, DUDRIVER on the various client nodes will pick one, and all   
  .4> I/O will go to that volume. 

I once came across an interesting variant of this situation.

There were two disks in the VMSCluster which had the same Unit number and
allocation class.  Neither of these disks was served and the customer believed
that this meant they were not at risk.   System A could only access its local
$2$DUA24 and system B could only access a different local $2$DUA24.  There was
"no possibility" of data corruption because each drive was only accessed by one
system which knew what it was doing.

The fascinating XQP bugecheck was caused by the fact that the two systems were
both working on files with the same FID at the same time.  One system was in the
process of creating a new file using the unused FID, the other system was in the
process of opening an already existing file with the same FID on its disk.  The
system creating the file attempted to gain the appropriate lock for its newly
created file.  Discovered that another system had a lock on this "not-yet
existing" file and promptly bug-checked!

StuartR

This is my day for editorial corrections, I guess....

My reply in .4 is incorrect. There is no requirement that device unit numbers be
unique in a cluster as long as the allocation class rules are followed.  In
fact, on the STAR cluster, one of the VMS development clusters, there are two
devices named DKA200 as I write this.  Different allocation classes, of course.

The MSCP unit identifier contains, in addition to the allocation class, unit
number, and controller letter, the device type letters, the "dd" part of the
ddcu device name.  These are encoded in the 64-bit MSCP unit ID.  (You will
notice this if you look in the right places in the code, as I did this morning!)
This shows that you can have devices named DUA100 and DKA100 in the same cluster
at the same time, even within the same allocation class.

I also looked at the manual (What?  Not the manual!), and there's no requirement
there for unique unit numbers across the entire cluster.  Just unique names.
Unique SCSI unit numbers are (today, anyway) well-nigh impossible in any largish
cluster, and anyway, if there were unique unit numbers, there'd be no need for
allocation classes.

Sorry for any confusion my earlier reply caused.

John

    
    	John,
    
    	Isn't the problem with unique allocation/mscp unit numbers just
    in the mount verification routines? Did you look there?
    
               					Bill

    While this has no bearing on the problem in .0, I should mention
    that it appears that V7.1 introduced a "feature"...  It appears that
    there is an error in DUDRIVER that can lead to corruption of the 
    IO database if there are duplicate device/unit numbers (but different
    allocation classes).  As a result, device will not mount verify 
    correctly nor will they MSCP fail over correctly.  There were also
    problems reported with MOUNTing a device in this state (not
    surprising).
    
    The customer that IPMTed this problem has renumbered all of the disks
    so that they have unique unit numbers, and has not seen the problem
    since.  (CFS.49819)
    
    This problem is in V7.1 and V6.2 COMPAT and CLUSIO kits.  It will not
    be addressed in the upcoming DRIV01_071 kit, which will have other
    fixes for DUdriver and TUdriver.
    
    Cheers,
    jeff
    

    you mentioned DUDRIVER. Can I assume that it does not affect then
    SCSI devices, as this seems to be the more general cases with over-
    lapping unit numbers?
    
    /cmos

DUDRIVER is involved when talking to MSCP served SCSI devices.

    
:    	Isn't the problem with unique allocation/mscp unit numbers just
:    in the mount verification routines? Did you look there?

   There was a long discussion on this a while back somewhere in this
   conference, and the limitation (from memory) was in one of the MSCP
   failure paths.  It may well be mount verification...  It was rather
   obscure, but it had been occasionally seen...