Conference noted::hackers_v1

Run the program in BATCH, with /KEEP.  Specify the output file as SYS$OUTPUT.

As the program runs, the log file fills with the output from the program,
which VMS flushes from time-to-time.  (There's a command which sets this,
but I don't know what it is -- default is every minute or so).

/KEEP means you won't lose the output after the job completes; you may also
want to specify /NOPRINT.

				JENelson

re: .1

Nice, but the program has 2 output streams, one giving status info. goes
to SYS$OUTPUT, the other giving the "results" goes to a predefined file.
This is the one which gets write-locked.

Diarmuid.

Normally, when a file is open/created for PUT access, it has SHR=NIL in the
$FAB (this is the default if your ask for WRT access to the file, if you
only ask for RD then the default is SHR=RD).
 
Instead, explicitly specify SHR=RD.

I believe this is how the .LOG files are created, so that normal toys like
TYPE etc. can still read them (although perhaps they specify SHR=WRT as well).
                                                                            
/Bevin

re: .3 

Unfortunately we only have the .exe of the program (its an outside vendor's
software) which means we can't change the code itself.

Other applications for this kind of thing would be to check how much of that
large file that you've being copying accross the net for the last n hours
has actually arrived. Also things like reading the OPERATOR.LOG file etc.

Diarmuid.
                                              

Make a temporary copy of the file with

	$ BACKUP/IGNORE=INTERLOCK filename.typ;vers tempfilename.typ

You cannot guarantee that the copy will be internally consistent, though.

Dave C.

VMS V4.4 is supposed to bring some releive in this area.
I still have to study exactly how.

If you could get hold of the sources, the SHR=UPI option
could do the job.

The network copy:

	I suggest you have a look at the tool FTSV it is
	available throigh geneva and through the toolshed.
	It has build in functions to show how for it got,
	within how much time/retries. You'll like it.

Hein.

	> Other applications . . . would be to check how much of that
	> large file that you've [been] copying [across] the net for the
	> last n hours has actually arrived.

	    I've never had any problem doing that, I just use DIRECTORY
	with the /SIZE=ALL qualifier.
			<_Jym_>

re .-1
	$dir/siz=all won't work (at least on a VMS-to-VMS copy), because
the receiving side will preallocate the disk space giving you a
directory listing like 0/1234 even from the beginning.

The BACKUP suggestion in .5 gives the following results...

$ BACKUP /IGNORE=INTERLOCK OUTFILE.DAT;1 OUTFILE.TMP
%BACKUP-E-OPENIN, error opening DUA1:[DONNELLY]OUTFILE.DAT;1 as input
-SYSTEM-W-ACCONFLICT, file access conflict
%BACKUP-W-NOFILES, no files selected from DUA1:[DONNELLY]OUTFILE.DAT;1

Thanks for your suggestions; the quest goes on...

Diarmuid.
                           

If the name of the "predefined file" could be specified as a logical name
and if the file format is sequential, you could have your .EXE write its
output to a mailbox and then set up another program to copy from the mailbox
to a sharable file.

Another idea:  Even without the sources, it should be not difficult to
locate the code that opens the FAB to the predefined file from the debugger.
(It will be in one of the stack frames above a call to SYS$OPEN).  You
could then PATCH the .EXE to modify the SHR info just before the call.

/Bill Fisher  
                                           

re: .9
You need either SYSPRV or ownership of the volume for /IGNORE=INTERLOCK to
be effective.  (One of the worst abuses of the VMS security system, that one -
if I want to see my inconsistent file, why shouldn't I be able to?)

							-- Jerry

	Re: .11, I suspect it may be because they would not be able
to make high water marking work without that restriction. Reading
an inconsistent file with /IGNORE=INTERLOCK means ignoring the
current high water mark (since that is probably at block 1 anyway).
Admittedly the current high water marking is not "real" high
water marking, but maybe they are just getting you used to the
restriction before they have to apply it for real.

	Of course for a disk which does not have high water marking
you are probably right that there is no good reason for the privilege
restriction.

	Dave

On our V3.7 VMS system MMS does keep its log file locked during the execution.  
How I wish to look at the log file and find out what it was doing.  I finally 
gave up and defined a command e.g. 
   BRANDNEW :== DIRECTORY/CREATED/SINCE=-00:05
and use it on MMS's working directory and try to make some sense out of it.

Next time I will try BACKUP/IGNORE with SYSPRV set and see what happens.

- Vikas

re .12

	Not necesarrely (sp) Dave! I've been wanting this feature most
	often to read the start of the file where something else is
	activly appending it. If would also be relativly easy for an
	application to update the EOF pointer through RMS to indicate
	how far others can safely read.

Re .original

	Could some of the writing processes be fooled into writting to
	an indexed file? Anyway, 4.4 is supposed to be of big help.

Hein.

re: .13
If the MMS log file is a batch log, use SOS to read it:
$ EDIT/SOS/READ FOOBAR.LOG
then use "P" commands to read it.
[This works since SOS will ignore the EOF marker. You will see crud after
the real log contents. Good demo of what highwater marking is all about]
	-Rick

re .13, .15
	The log file produced by a batch job *is* readable, as of V4.0.
	The TYPE command will read it, as will TPU or MERGE/NOCHECK
	(note that TYPE gives no warning if the file is locked, while
	the others do).  Note that DCL flushes the output file once a
	minute (by default -- see HELP SET OUTPUT_RATE), so you won't
	see any output for the first minute or so.

The compatability mode DMP used to be good for this type of thing but I don't
think it works on V4+.

tom_p

re:  High-water marking

Not relevant; the privilege restriction has been there since at least V2,
probably since early base levels.  (Curiously, it may be the only thing VMS
picked up from RSTS.  RSTS has a "read-regardless" mode that also requires
privilege, even for files you actually have read access to.)  High water
marking is much more recent (V4), and making it work correctly with 
IGNORE=INTERLOCK would be trivial (the reader is given all zeros when
he accesses anything beyond the high water mark.)
							-- Jerry

re: .18

	Giving the reader all zeros when he reads beyond the high
water mark seems not to be quite as simple as you think, since
it is not implemented that way even for files for which you have
full access (that is, no-one else sharing it). It is instead
currently implemented by writing zeros to the disk blocks when
a file is extended, which is why it is currently a significant
performance hit.

	The high water mark would exist on disk in the file header,
and for the time a file was open for write, in the context of
the writing process. To update the disk copy continuously would
give a performance degradation even worse than the current
implementation.

	The only reasonable way of passing the high water mark
between two processes then would be via a lock value block. As the
reading and writing processes would be operating asynchonously
to implement this correctly would mean an exchange of locks
between the two processes for every read access beyond the last
seen high water mark, since the read operations are not necessarily
sequential.

	Given a choice between versions of XQP of
1) Current "high water marking" with its performance hit.
2) Real high water marking and the privilege restriction.
3) Real high water marking and lots of lock exchanges between XQPs
I would choose 2.

------------------------------------------------------------------

	The above was just an attempt to justify a rather off-the-cuff
guess as to the reason. I suspect the real reason may be more like
the following.

	I can have a programme installed with privileges that
processes data and then allows me to see a subset. It may use
work files to process that data. If I could just go to another
process and dump out the work files then the programme could
not effectively enforce that I should only see the subset.
I think it may be just an ambiguous use of the FIB$M_NOLOCK bit
(which is where the privilege restriction comes in).

	Dave.

Too bad no one bothered to find out how other people have solved this problem.
Neither Tops-10 nor Tops-20 ever had this problem in the first place.
				/AHM

I'm not sure, but I think this has gotten slightly off-track.  I'm still
curious:  is there some way to have the program at least update a file it's
writing periodically to show where it's at?   (I don't have any of the
documentation around)  Will the RMS $Flush routine do this?  If so, does
anybody have any example of using $Flush (or any other of the related RMS
routines) from a high-level language, say FORTRAN?

SYS$FLUSH will update the file header with the current size of the file.
It will also write any modified buffers to the file.  In order to use this
from a HLL, you must use a USEROPEN and save the RAB address.

tom_p

From FORTRAN you can use the FOR$RAB function to return the RAB address.
	-Rick

        Pascal has a similar function (Rab_Ptr := Pas$Rab(File_Variable);)
                                                  
        	/dave

    
    How do you create a file that can be read by other users?
    
    RMS says you can only share a sequential file if it has fixed-length
    512 byte records - but log files are not bound by such strict
    rules.
    
    Mark

    "Sharing" in the sense of multiple WRITERS.  For one writer & many
    readers, you just have to open the file according to convention.
    Refer to 4.0 RMS reference manual, section 5.13, p5-9.

    I still have the same problem with reading the "locked" files. 
    I tried BACKUP/INTERLOCK but got very interesting results.  I have
    tried numerous other variation but have still no luck at all. Probably
    I could modify my command file so that it will not use /OUTPUT qualifier
    on MMS but let the batch processor handle the output to the batch
    log file.

    There *has* to be a way to read that file at my own risk.
    
- Vikas

P.S  The output from BACKUP is listed    
    
$ BACKUP/IGNORE=INTER/LOG NG:CTBUILD.LOG HM:FOO.BAR;1

%BACKUP-W-ACCONFLICT, SERPNT_SRC1:[CTAB.V31.FT1]CTBUILD.LOG;37 is open for write by another user
%BACKUP-S-CREATED, created DUA1:[SONTAKKE]FOO.BAR;1
%BACKUP-E-OPENOUT, error opening DUA1:[SONTAKKE]FOO.BAR;1 as output
-RMS-E-FEX, file already exists, not superseded
%BACKUP-E-OPENOUT, error opening DUA1:[SONTAKKE]FOO.BAR;1 as output
-RMS-E-FEX, file already exists, not superseded
%BACKUP-E-OPENOUT, error opening DUA1:[SONTAKKE]FOO.BAR;1 as output
-RMS-E-FEX, file already exists, not superseded
%BACKUP-E-OPENOUT, error opening DUA1:[SONTAKKE]FOO.BAR;1 as output
-RMS-E-FEX, file already exists, not superseded
%BACKUP-E-OPENOUT, error opening DUA1:[SONTAKKE]FOO.BAR;1 as output
-RMS-E-FEX, file already exists, not superseded
%BACKUP-E-OPENOUT, error opening DUA1:[SONTAKKE]FOO.BAR;1 as output
-RMS-E-FEX, file already exists, not superseded
%BACKUP-E-OPENOUT, error opening DUA1:[SONTAKKE]FOO.BAR;1 as output
-RMS-E-FEX, file already exists, not superseded
%BACKUP-E-OPENOUT, error opening DUA1:[SONTAKKE]FOO.BAR;1 as output
-RMS-E-FEX, file already exists, not superseded
%BACKUP-E-OPENOUT, error opening DUA1:[SONTAKKE]FOO.BAR;1 as output
-RMS-E-FEX, file already exists, not superseded
%BACKUP-E-OPENOUT, error opening DUA1:[SONTAKKE]FOO.BAR;1 as output
-RMS-E-FEX, file already exists, not superseded
%
*Interrupt*

    what happens if ou specify a version number on the input file in
    the BACKUP command?
    
    Danny

    RE: .28
    
    Specifying the version number on the input file allowed me to copy the
    file using BACKUP.  Unfortunately since the original file is showing up
    as 0/30, the new one also is 0/30.  I plugged the eof pointer using PIP
    file.tmp/EOF.  Now if I try to read the file, I get all weird kind of
    errors. I suspect that the data was not yet written to the disk.  I
    dumped first few blocks but all it contained were nulls. 
    
$ typ hm:foo.bar
%TYPE-F-WRITEERR, error writing SYS$OUTPUT:.;
-RMS-F-SYS, QIO system service request failed
-SYSTEM-F-MBTOOSML, mailbox is too small for request

$ mcr dmp ti:=hm:foo.bar/as
DMP -- TI:=HM:FOO.BAR/AS
DMP -- FATAL - Page header line too long

    Re .29:
    
    	Vikas, Your background is showing...
    
    		>PIP file.tmp/EOF	--> $SET FILE/END file.tmp
    
    		>DMP ti:=hm:foo.bar/as	--> $DUMP hm:foo.bar
    
    
    	Admittedly, I use PIP /EOF every so often because it, contrary
    	to the VMS implementation allows a block&byte parameter. And
    	then there is /FI which accepts a file-id and also does not
    	seem to have a VMS counterpart.
    
    Hein.

    I ran a little test against my OPERATOR.LOG, and it worked like
    a charm...TYPE and all...
    
    Could the writing application have a problem?
    
    Danny

    It must indeed be the feature ( I never call anything a bug :-) of the
    writing application.  Unless specifically told to, I doubt if RMS will
    automatically dump its buffer to the file.  So if the application
    program has large RMS buffers, it could be hours before any data would
    be actually written to the disk.  Admittedly I do no see any reason for
    the application program to go out of its way to write the buffers
    periodically.  Anyway, I am going to rewrite that command procedure and
    let batch processor write the log file. I belive VMS changed the batch
    processor to periodically flush the log file since V4.0. 
    
- Vikas

    RMS will write the buffers (when they fill) to the file, but usually
    never has any reason to update (sys$flush) the file header.

    Re: Flushing a log file; see HELP SET OUTPUT_RATE.