Conference vaxaxp::vmsnotes

221.0. "dirct I/O, RWAST and global section" by UTOPIE::BRAUN_R () Wed Feb 19 1997 10:56

    Hi all,
    
    posted the note below in other confs, if anybody could help ?
    
    Any help will be highly appreciated.
    
    Thanks,
    Ralph
    
    
    
    
    
    
    
          <<< HELP::USER0:[NOTES$LIBRARY]DECNET-OSI_FOR_VMS.NOTE;1 >>>
                          -< DECnet/OSI for OpenVMS >-
================================================================================
Note 3875.0                X.25 direct I/O and RWAST ?                 4 replies
UTOPIE::BRAUN_R                                      61 lines  18-FEB-1997 13:37
--------------------------------------------------------------------------------
(cross-posted in DECNET-OSI_FOR_VMS, X25PSI and X25_AVMS)


Hi all,

sorry, if some of the following information is inaccurate, also showing
my shameful poor knowledge of VMS programming:


Our customer, a software partner company has the following problem with
X.25, buffered I/O and direct I/O:

VAX4000-60 + DECstation 2000-300, OpenVMS 6.2, DNOSI 6.3, ECO3    X.25 V1.0-E

(but the problem seems to be same on AXP)


application declares as X25 Destination
application accepts a connection (X25 Confirm)
application sets an asynch. Read-QIO to receive a new 
  packet within as AST-routine. 

application maps a global section 
application copies data to this section.
application tries to remove this section via $DELTVA from its address space.


--> $DELTVA-service is unable to finish, hangs forever: only 
    an incoming X.25-packet (thereby finishing the direct I/O)
    is interupting (and finishing) this hangup.


But this hangup only occurs, if the global section is shared 
by "many" processes at the same time.
If the global section is initialized and used by this test program only,
the problem will not occur.
If there is only one other test program initializing the section and the
application above is only accessing the section, the problem will not
occur, too. 


 - process is hanging in RWAST-state, how to find the reason via
   SDA (VMS-guy told me, this is very specific depending on the
   situation)
      (some pointer to SDA-sniffing required)


 - customer wants to know about the potential problem of direct
   I/O (X.25):  with DECnet (buffered I/O) instead of X.25
   the same application (X.25-calls changed to DECnet-calls)
   does not hang.

   According to the customer, the problem could be due to 
   direct I/Os  (??????) 


Any help will be highly appreciated.

Thanks,
Ralph

================================================================================
Note 3875.1                X.25 direct I/O and RWAST ?                    1 of 4
RMULAC.DVO.DEC.COM::S_WATTUM "Scott Wattum - FTAM/V" 18 lines  18-FEB-1997 14:30
--------------------------------------------------------------------------------
This would appear to be more of an OpenVMS question, wouldn't it?

In any event, my Internals and Data Structures manual seems to be pretty clear
on one point:  "If the process has direct I/O in progress, its I/O must complete
before this page can be deleted."

The text goes on to describe how the process will be put into RWAST and under
what circumstances (which appear to include the times when the page is mapped to
more than one process - which would plssibly explain what your customer is
seeing).

So, at an admittedly quick glance (and a not very good understanding of this
aspect of OpenVMS memory management), this looks like expected behavior that's
independant of X.25, but directly related to direct I/O.  I think if you need a
better answer, you should check an OpenVMS conference.

fwiw,
--Scott
================================================================================
Note 3875.2                X.25 direct I/O and RWAST ?                    2 of 4
UTOPIE::BRAUN_R                                       8 lines  19-FEB-1997 04:26
                             -< no common memory >-
--------------------------------------------------------------------------------
    Scott,
    
    thanks a lot for your answer, but according to the customer, the global 
    section has nothing in common with the memory buffer being used for the 
    direct I/O.
    
    Thanks,
    Ralph
================================================================================
Note 3875.3                X.25 direct I/O and RWAST ?                    3 of 4
RMULAC.DVO.DEC.COM::S_WATTUM "Scott Wattum - FTAM/V" 12 lines  19-FEB-1997 07:52
--------------------------------------------------------------------------------
I don't think it matters.  The text in the VMS internals manual did not seem to
care whether the memory was directly related to an I/O transfer or not - I
suspect this may be because VMS cannot tell during the page deletion process
whether the page could potentially be involved in a direct I/O operation (as a
buffer, or what have you).

This is why I suggest an OpenVMS conference - you're going to find people that
really *know* memory management and can confirm for you whether this is expected
- my quick look at the internals manual suggests that this is normal.

fwiw,
--Scott
================================================================================
Note 3875.4                X.25 direct I/O and RWAST ?                    4 of 4
UTOPIE::BRAUN_R                                       3 lines  19-FEB-1997 10:23
--------------------------------------------------------------------------------
    Thanks a lot, will squeeze the VMS guys.
    
    Ralph

    This is what we do from a memory management point of view if you try to
    delete a global page:
    
     - check if the page currently has an elevated reference count
       indicating some direct I/O in progress
    
        - if not: go ahead, delete the page from this process's virtual
    	  	  address space
    
        - else:   check if the process has any direct I/O outstanding
    
    		  - if not: the current process cannot need this page for
    			    I/O (some other process does); go ahead...
    
    		  - if yes: place the process in RWAST and wait for an I/O
    			    to complete; then try again to delete this page
    			    restarting from the beginning
    
    There is just no reasonable way to figure out if any of our outstanding
    direct I/Os is really going to this page.  We'd have to scan all
    outstanding IRPs for all channels.  While this may be possible in a
    dump, synchronizing this on a running system would be an "interesting"
    job.
    
    The whole idea is that direct I/O doesn't take "forever".  Any I/O that
    can possibly take a very long time should be buffered.

    re .0
    
    You and .1 are almost certainly correct that the wait state is caused
    by the pending direct I/O. It seems highly desirable for all I/Os to be
    to system address space if there is an unlimited (e.g. pending read) or
    large delay in completion of the I/O. (Bufferred I/O gives you this
    implicitly.) I am surprised that X.25 is using direct I/O. Any chance
    that the X.25 software gives you the choice of direct v. bufferred? Try
    contacting the people who support the X.25 software.
    
> - process is hanging in RWAST-state, how to find the reason via
>   SDA (VMS-guy told me, this is very specific depending on the
>   situation)
>      (some pointer to SDA-sniffing required)
                                                        
    Try STARS for a pointer to analysing RWAST.

    Thanks!
    
    Just a short question "to VMS" (before I'm going to squeeze the 
    X.25 guys):
    
    Provided, that use of direct I/O can not be changed in the X.25 product
    (which is very probable):
    
    Is there any way to prevent from via a bypass:
    
    Customer has to use this X.25-Call and the global section due to
    program design, would it help to e.g. lock the section or something
    like this ?
    
    
    Thanks,
    Ralph
    
    
    
     

    re .3
    
    I don't understand your question.  In .0 you said that the application
    hangs when they try to delete the global section ($deltva).  As long as
    they don't do this prior to shutting down X.25, they should be all set.
    
    Some of the section pages are locked for I/O, that appears to be the
    problem.  I don't see how additional locking could help here.
    
    
    -andy

re .3
    
>    Is there any way to prevent from via a bypass:
    
    Not within VMS that I know of.
    
    If the feature exists it would have to be provided by the X.25 device
    driver.

    re 482.0
    
    You just can't do it. Page deletion _cannot_ occur while direct I/O is in
    progress. Direct I/O by name and nature is direct from the I/O device
    to the page in question. Since VMS does not deem it appropriate to scan
    all pending direct I/Os to work out whether any one particular page is
    the target of a direct I/O (and even if it did, the synchronisation to
    avoid race conditions and the like would be hairy), VMS does not allow
    a page to disappear out from underneath an I/O. To allow this would be
    to invite horrible corruption and loss of integrity.
    
    I personally consider it to be a bug in the X.25 product if it uses
    direct I/O that can stall indefinitely. If you can't get joy from the
    X.25 group about changing this and there are no options already within
    the product then the application design MUST change. No Ifs. No Buts.
    If the customer wants Digital to help them with that design change, I
    hope you can extract some money from them.

    If the customer were to use some other intermediary to copy the data
    to the global section, for example some other process that gets the
    data from a mailbox, the I/O would be in a different process from
    that which was doing the mapping and buffering would be achieved.
    
    Were the communication done by a global area that was common to both
    processes and mapped all the time by both, it might even be faster,
    but I'm not going to try (with no info) to design the thing on the
    fly. Point is, the application can change its "copy to global section"
    logic and avoid the issue. The customer does need to understand the
    ground rules of direct I/O, but that should not prevent him solving
    his problem.
    

    Thanks!
    
    I'll forward this to customer (and also try to get something from the
    X.25 guys).
    
    Regards,
    Ralph