Conference annecy::ftso

102.0. "Copy in FTPMODE - LRL not set with variable record length" by SIOG::M_CRONIN () Fri Feb 07 1997 19:17

Hi,
    
    
    I have copied this from the UCX notes conference.
    
    Is there a way to get FTOS to copy variable length record files in
    FTPMODE so that FTSO does not enable VMS_PLUS mode .
    
    Alos, if so, will it be possible to copy binary files using FTSO in
    FTPMODE ?
    
    Emmanuel - any suggestions ?
    
    Thanks in advance,
    
    Mary.
    
    
    
              <<< LASSIE::UCXAXP$DKA0:[NOTES$LIBRARY]UCX.NOTE;1 >>>
                      -< DEC TCP/IP Services for OpenVMS >-
================================================================================
Note 5192.0  Variable length record attributes converted after ftp/pu  9 replies
SIOG::M_CRONIN                                       48 lines   4-FEB-1997 07:49
--------------------------------------------------------------------------------
    Hi,
    
    I have a customer running UCX V4.0 on OpenVMS AXP V6.2. When they use
    ftp to copy files with variable length records to another OpenVMS V6.2
    system running UCX V4.0 the record format in the file header changes.
    
                    
    I have tried this with two different file types with the following
    results:
    
    
    File A:
    
    Record Format: VFC, 2 byte header, maximum 0 bytes, longest 80 bytes
    On remote side changes to Record Format: VFC, 2 byte header, maximum 0 bytes, longest 0 bytes
    
    
    File B:
    
    Record Format: Variable length, maximum 0 bytes, longest 722 bytes
    On remote side changes to 
    Record Format: Variable length, maximum 0 bytes, longest 0 bytes
    
    If I convert the file manually on the remote side it corrects the
    value of longest bytes. It looks like the file is not being written
    correctly.
    
    I have tried this using
    
    ftp> put/convert
    ftp> put/fdl
    
    and also with the logical UCX$FTP_CONVERT_FILE defined as TRUE.
    
    My understanding of this logical is that 'users retain the
    RMS-formatting information or their files'. Shouldn't this mean the
    maximum bytes value remains the same ?
                                    
    Thanks in advance,
    
    Mary.
    
    
    
    
    
    
    
================================================================================
Note 5192.1  Variable length record attributes converted after ftp/put    1 of 9
UCXAXP::GEMIGNANI                                    19 lines   4-FEB-1997 22:13
--------------------------------------------------------------------------------
    The problem is that when copying files from UCX to UCX, a special mode
    is used where the file's FDL is sent before the file's binary contents.
    
    The LRL (longest record length) field is maintained by RMS as records
    are inserted into the file.  It is a value that is generated from its
    experience in loading the file.
    
    When the file is written into as binary blocks (such is the case here),
    the LRL attribute is not determinable.  Note that this is an ATTRIBUTE
    on the file; it does not occupy space within the file's body.
    
    As a result, RMS writes it as zero (as it would for any block I/O
    written file).  While this causes file attribute mismatches when files
    are copied, there should be no problem accessing the file.  In
    addition, any appends which occur on the file will cause LRL to be
    updated.  In normal programming, I make accomodations for LRL, LRL=0,
    and possibly larger-than-LRL records in the file.
    
    Have you experienced any problems reading data from these files?
================================================================================
Note 5192.2  Variable length record attributes converted after ftp/put    2 of 9
ZUR01::DEMETRIOU                                     17 lines   5-FEB-1997 02:49
                        -< FTP&&CONVERT != FAB+(n*RAB) >-
--------------------------------------------------------------------------------
    hello,
    
    
    you mention VFC right?
    I do not think that FTP can handle such files!
    A file with this format (VFC) has two distinct records:
    The first one describes the print format of the file which
    is handle by VMS utilities such as print and type but
    not CONVERT or editors! and the second one is the record 
    format of the plain data. 
    So , if you use convert on this file or edit it , the first
    record is gone forever!
    To handle correctly this file format you need a program 
    doing multistreamming (one FAB and two or more RABs).
    
    regards
    Phyl(actis)
================================================================================
Note 5192.3  Variable length record attributes converted after ftp/put    3 of 9
UCXAXP::GEMIGNANI                                    34 lines   5-FEB-1997 03:29
--------------------------------------------------------------------------------
    
    First of all, when copying from UCX-to-UCX in VMS-Plus mode, the entire
    body of the file is copied.  The problem here is that the LRL attribute
    is stored in the file header, not in the body of the file.  Also, RMS
    is the one responsible for maintaining it (or anyone else who pretends
    to be RMS) and, when performing block I/O (which is what we are doing),
    RMS doesn't apply the notion of a record to the data at all.  Hence,
    LRL is not maintained (only in record mode).
    
    Secondly, as far as the VFC formatting goes, the VFC header is part of the
    record; it does not exist in a `different' record at all.  When RMS
    reads the record, it places the variable data into the buffer which you
    describe in rab$l_ubf and rab$w_usz, and the header (which is always
    fab$w_fsz bytes in size) into the buffer pointed to by rab$l_rhb.
    If fab$l_rhb is not filled in, then RMS won't return the header. That's
    why most people don't "see it"; they don't recognize the header and
    account for it.
    
    When a file is `print file format', then rab$b_rat has FAB$M_PRN set in
    it, fab$b_rsz is two, and each of the bytes in the fixed header
    describes what to place before and after each record.  Usually, you see
    a 0x01 0x8d in there, which is "insert one line feed before the record"
    (0x01), and a "carriage-return afterwards" (0x80 | 0x0d=cr).  It looks
    like the latin form of a fortran record on VMS.
    
    By the way, I don't like print file format at all.  I've had to make it
    work properly for several of the UCX components and it doesn't fit
    perfectly into the Unix concept of a stream[_LF] file.  Why this
    became the standard format of log files is beyond me.
    
    No, a file of this type has one distinct record.
    
    The FHZ (fixed header size) is an area which actually exists in the
    beginning of the record and, to read it, you must point 
================================================================================
Note 5192.4  Variable length record attributes converted after ftp/put    4 of 9
UCXAXP::GEMIGNANI                                     7 lines   5-FEB-1997 03:34
--------------------------------------------------------------------------------
    By the way, CONVERT does handle VFC files.
    
    See the /FIXED_CONTROL qualifier.
    
    (I know this because I wrote RMSCNV, the PDP-11 version of the
    utility.  Gosh, I was in my early 20s without even a hint of
    a bald spot!)
================================================================================
Note 5192.5  Variable length record attributes converted after ftp/put    5 of 9
SIOG::M_CRONIN                                       25 lines   5-FEB-1997 05:54
                              -< Sort-e-bad_lrl >-
--------------------------------------------------------------------------------
    Hi,
    
    Thankyou for your replies.
    
    Re .1 Yes, we are having problems accessing these files. A command
    procedure tries to sort the copied file(s) and gets the following
    errors:
    
    %SORT-E-BAD_LRL, record length (722) greater that specified longest
    length record
    -SORT-E-READERR, error reading 'filename'
    
    The SORT command is:
    
    $ SORT/KEY=(POSITION:1,SIZE:2,DESCENDING) 'inputfile 'outputfile
    
    I do not know if they can open the files from their COBOL programs
    since the sort fails.
    
    Is there any way around this problem without having to modify the
    customer's code ?
                     
    Thanks in advance,
    
    Mary.
================================================================================
Note 5192.6  Variable length record attributes converted after ftp/put    6 of 9
SIOG::M_CRONIN                                       31 lines   5-FEB-1997 07:25
                         -< Turn off VMS-Plus mode ? >-
--------------------------------------------------------------------------------
    Hi,
    
    I have managed to copy the file and have the LRL value set by turning
    off VMS-Plus mode in ftp.
    
    FTP> OPEN remote_host
    FTP> DISABLE vms_plus
    FTP> PUT filename
                                   
    The remote file had LRL correctly set and I could sort it.
                        
    It will not be possible to interactively disable VMS-plus mode as I did
    above.  The customer does not use ftp directly. They use FTSO on the
    local side in 'ftpmode' which causes it to use UCX's ftp on the remote
    side.
    
    Is there a way to permanently disable VMS-Plus mode on a process-wide
    basis  ?
    
    We will still need to be able to copy binary files therefore we will
    need to be able to use VMS-plus mode for some copies.
    
    What is the effect of disabling VMS-plus mode ?
    	i.e. Will it affect performance ?
    	     How will it prevent copying of binary files ?
    
    
    Thanks in advance,
    
    Mary.
    
================================================================================
Note 5192.7  Variable length record attributes converted after ftp/put    7 of 9
SPECXN::DERAMO "Dan D'Eramo"                         20 lines   5-FEB-1997 11:53
--------------------------------------------------------------------------------
	re .5
        
>    %SORT-E-BAD_LRL, record length (722) greater that specified longest
>    length record
>    -SORT-E-READERR, error reading 'filename'
        
        I think that if the LRL is stored as 0 in the file header then
        SORT has to guess, and will complain if a record greater than
        the guess is discovered.
        
        You should be able to specify the input file qualifier as
        something like /FORMAT=(RECORD_SIZE:nnn) once you know an
        upper bound nnnn (here, at least 722) on the true record
        length.
        
        Likewise you can set the file header with
        
        	$ SET FILE/ATTRIBUTE=(LRL:nnnn) filename
        
        Dan
================================================================================
Note 5192.8  Variable length record attributes converted after ftp/put    8 of 9
UCXAXP::GEMIGNANI                                    10 lines   5-FEB-1997 21:08
--------------------------------------------------------------------------------
    Two possibilities:
    
    	(1) Convert the file before running your sort
    
    	(2) Disable VMS+ mode by placing the command:
    
    		DISABLE VMS_PLUS
    
    	    in SYS$LOGIN:FTPINIT.INI
    
================================================================================
Note 5192.9  Variable length record attributes converted after ftp/put    9 of 9
SIOG::M_CRONIN                                       30 lines   7-FEB-1997 13:05
                        -< Anything other suggestions >-
--------------------------------------------------------------------------------
    Hi,
    
    Disabling VMS_PLUS in FTP$INIT.INI works for copying files using just
    ftp (i.e. the LRL is set). However when I copy the file using FTSO the LRL
    is not set. I think this is because FTSO enables VMS_PLUS mode itself.
    
    $ SORT input/FORMAT=RECORD:nn output
    Works - ie it sorts it and set the LRL.
    The problem with this is that the customer will not always know what
    the LRL should be and there are several procedures within a large
    application doing various sorts.
    
    $ CONVERT before sorting the file also works, but again this means
    changing several command procedures
    
    Will ftp be able to copy binary files correctly if VMS_PLUS mode is
    disabled ?
    
    Is there any way to get RMS to write the file on the remote side so
    that LRL is set (eg setting something on the remote system).
    
    I'll cross post this in the FTSO conference.
    
    Any other suggestions VERY welcome.
    
    Thanks in advance,
    
    Mary.
    
      
    

	Mary,

	You need to use FTSO in Normal mode (FTSO on both side) in you want
	to deal with LRL.

	Rgds,

	Didier