Conference noted::hackers_v1

505.0. "How to juggle queued EX-mode locks around...???" by FALEK::FALEK (ex-TU58 King) Mon Jun 22 1987 19:45

    
    I have an application that consists multiple servers, one per
    VAXcluster node. At startup time, each server $ENQ's a request for
    an EX-mode lock on resource "MASTER" , specifying "MASTER_AST" as
    the completion AST. Only one server can get the lock granted in
    EX-mode, so the other requests wait in queue.
    
    I want to be able to juggle the locks so that I can control which
    server in the cluster holds "MASTER" in EX-mode. (the other servers
    should be in queue for it)
    
    For communication between the user-interface and the servers I added
    another AST (The "UI_AST") to the servers and have a mechanism whereby
    the user-interface can cause all servers to get the "UI_AST" when
    it sends them a message via the "UI" lock-value block; The message
    consists of the nodename of the particular server that should acquire
    the "MASTER" lock in EX-mode (this part works).
    
    Now for the hard part: How to juggle the EX-mode locks...
    The UI_AST compares the server's own nodename with the desired
    nodename. If they match, it simply exits the AST. If they
    don't match, it $DEQ's  the queued request for exclusive ownership
    of "MASTER" resource and then queues a new request. I reasoned that
    since the documentation says lock conversions go ahead of ungranted
    locks, the designated server's already queued EX-mode request should
    always complete before any new request for that resource.
    
    Well, I'm having a problem getting this mechanism to work, and I
    think it is because I'm sharing the lock status blocks in COMMON
    between the server and its two AST routines - but I'm not sure.
    
    Here's what I'm seeing (in the case of a server that SHOULDN'T get
    the lock):

    The UI_AST $DEQ's the old EX-mode lock request (call it lockid #1)
    Then it $ENQ's a new request (call it lockid #2)
    Then it exits the UI_AST.
    .
    Then the "MASTER" AST fires.
    I check the status in the LSB.
    The lock-id I see there is #2 (should it be #1?)
    The status is usually SS$_ABORT, (correct for lockid #1 ?)
    	but occasionally it is SS$_NORMAL EVEN THOUGH THE SERVER DOESN'T
           REALLY HAVE THE LOCK!

    Anyone have any suggestions on a good way to do this? I'm probably
    overwriting my LSB for lock request #1 by using it again for lock
    request #2 - and the MASTER_AST has no way of knowing...
    
    Is there a way to do it without actually $DEQing the lock so I can
    always keep just a single LSB for the lock throughout?
    
    What IS the best way to shift ownership of the EX-mode resource at
    will?

    	any help would be much appreciated...   Lou Falek
    
	[ Also posted to : VMSnotes 862.0 ]		

    I figured out how and have got it working, so I'll post the answer
    here. It works like this:
    
    User-interface program sends a message to all servers via a lock value
    block when it desires to cause a different server to have the MASTER
    lock granted in EX-mode. (all servers hold the user-interface resource
    in CR-mode specifying blocking AST UI_AST. UI_AST downgrades UI-lock
    to NL-mode, waits a moment, then waits for UI-lock to be granted
    in CR-mode again and reads value block to get the node where the
    server should get the MASTER-lock in EX)
    
    UI_AST on each server compares server's node with desired node...
    
    If they match, merely exit AST, we are already queued for MASTER
    resource in EX-mode, and will get it soon.

    If we already hold the MASTER EX-lock, $DEQ the EX-mode lock and $ENQ a
    new request for it.    

    If we are waiting for the EX-lock, $DEQ the lock request (LCK$M_CANCEL)
    and exit AST.  Our completion AST for the EX-request will fire but
    the status in the LSB will be SS$_ABORT. In that case, queue a new
    request for the MASTER resource in EX- from within the MASTER
    completion AST, and exit the AST.  (see... only one pending operation
    at a time, so the LSB doesn't get over-written in transit.)

    Best of all, it works.
    
    By the way, I briefly tried to do this with lock upgrades from NL
    and EX-NL downgrades but got bitten by the fact that NEW $ENQW's
    for a lock on the MASTER resource in NL-mode are blocked as soon
    as there is a request for an upgrade from NL-EX in the conversion
    queue. (So I couldn't start a third server). This behavior has always
    seemed a bit bizarre to me, but I suppose there are reasons...