Conference bulova::decw_jan-89_to_nov-90

1939.0. "SESMGR dies when DECterm contr. can't open disp." by CSC32::C_ROSE () Mon Dec 18 1989 17:27

Op-sys Version: V5.1
Component     : VMS-DECWINDOWS
Component Vers: V1.0

TITLE: SESMGR Dies When DECwindows DECterm Controller Can't Open a Display

If the DECterm controller is not able to successfully open a display
for a new DECterm, the Session Manager will crash. The DECterm
controller and DwtDECTermPort do not handle the failure correctly.
Commands to reproduce and error messages:

     1.  Log in, get Session Manager, and FileView.
     2.  Select a file with FileView - Edit this file using EVE.
     3.  After EVE is running, go to the Session Manager control panel
         and select the CREATE TERMINAL WINDOW.
     4.  Do this until the Session Manager vanishes.

         ACCVIO, PC=000CBDC1, V/A=xxxxx

WORKAROUND:
Keep GBLPAGES with plenty of free pages for DECwindows.




Here is some extra info which may or may not be of help:

In PTYSUB.C of the DECterm controller, any failure to open the display
will result in FALSE being returned from the create() routine (line
9893 in PTYSUB).  FALSE is equal to 0.  This value is returned to
respond_to_message in TEA.C, which checks the status return against
TRUE (1) (line 9751 in TEA.C).  When this code sees FALSE, it calls
send_error, which sets up a return mailbox message to DECWTermPort,
indicating a message type of DWT$K_ERROR, and the status code of FALSE
(0).

The problem occurs when we get back into DECWTermPort.  This code
checks the answer_type, and if it is DWT$K_ERROR, then the
answer_status from the return mailbox message is returned to the
caller WITHOUT updating the return_length parameter passed in (line
2249 in DECWTermPort).  However, 0 on this side of the mailbox is
equivalent to DWT$K_SUCCESS.  The switch statement at line 2261
replaces the DWT$K_SUCCESS with SS$_NORMAL, which is passed back to
the caller.

When the Session Manager gets the successful status back, it attempts
to move the returned device name (which was never set up correctly)
using the return_length parameter, which was never updated.  The
return_length parameter contains garbage, which causes the MOVC5 at
0CBDC1 to ACCVIO, and the Session Manager vanishes.

A solution would be to return some value other than FALSE from create
when the open_display fails.

    Please submit an SPR (this is released software).
    
    - Andy ��� Leslie
      VMS CSSE Newbury

This (.0) is a copy of most of the important information from a recently
submited and verifyed SPR. I thought I would add it to this conference, so
that DECwindows Version 1.0 users could recognize this as a potential problem.

Any comments about when this might be fixed?