| Hi Derek,
To follow on to what Mark mentioned in .1, our system discovered this
same feature during DIAMOND Field Test activity last November. We use
the split Sender/Fetcher process due to the volume of correspondence
passing through our system and noticed that periodically the Fetcher
would take an excessive amount of time to process one message (and
back up incoming traffic (600-800 deep!) accordingly!).
Investigation determined that the behaviour was due to the addressee
list expansion Mark described in .1, exacerbated by some of the mass
mailings addressed to four hundred (400) or so Digits at a time.
Exhaustive testing of the Fetcher DAF and DOCDB initial sizings finally
yielded the values we have used since, on both Version 2.4 and Version
3.0 systems. We see no more expansion of the DAF beyond the initial
value. We also use a modified (actually for V2.3) OAMTIMAIL.COM to
place new, empty DAF and DOCDB files after _each_ Fetcher run. After
the <FF>'s are a copy of the three files:
OA$LIB:OAMTIMAIL.COM, OA$LIB:FETCHER.FDL, OA$LIB:DOCDB.FDL
Hth,
Ron
$ !
$ ! ALL-IN-1 Mail fetcher command procedure
$ !
$ set verify
$ set noon
$ show daytime ! Write the time
$ !
$ ! If OA$MTI_SFLOG is defined we can log run information such as node, time,
$ ! Sender/Fetcher, and what we're up to.
$ !
$ start_time = f$time()
$ Node := "''F$GETSYI("NODENAME")'"
$ DEB$LOG := F
$ IF "''F$LOGICAL("OA$MTI_SFLOG")'" .EQS. "" THEN GOTO PNAME
$ IF F$SEARCH("OA$MTI_DATA:MTI$DEBUG.LOG;1") .EQS. "" THEN -
$ CREATE OA$MTI_DATA:MTI$DEBUG.LOG;1
$ OPEN/APPEND/SHARE/ERROR=PNAME LOGFILE OA$MTI_DATA:MTI$DEBUG.LOG;1
$ DEB$LOG := T
$ WRITE LOGFILE -
"''start_time' %FETCHER-I-START, Fetcher started run on ''node'"
$ start_time = f$extract(12,5,start_time)
$ !
$ ! Set our process name up
$ !
$PNAME:
$ SET PROCESS/NAME="A1 Fetcher"
$ !
$ ! Rename all our .LOG files if version > 32700. (Job would fail if highest
$ ! version = 32767)
$ !
$ IF F$PARSE (F$SEARCH ("SYS$LOGIN:OAMTIMAIL.LOG"),,,"VERSION")-";" .LT. 32700 -
THEN GOTO Wait_loop
$ !
$ ! Rename all log files to low version nunbers.
$ !
$Search:
$ !
$ FILE = F$SEARCH ("SYS$LOGIN:OAMTIMAIL.LOG")
$ IF F$LENGTH (FILE) .EQ. 0 THEN GOTO Rename
$ RENAME/NEW_VERSION 'FILE' SYS$LOGIN:OAMTIMAIL_RENAME.LOG
$ GOTO Search
$ !
$Rename:
$ !
$ FILE = F$SEARCH ("SYS$LOGIN:OAMTIMAIL_RENAME.LOG")
$ IF F$LENGTH (FILE) .EQ. 0 THEN GOTO Wait_loop
$ RENAME/NEW_VERSION 'FILE' SYS$LOGIN:OAMTIMAIL.LOG
$ GOTO Rename
$ !
$Wait_loop:
$ !
$ ShutDown = "''F$LOGICAL("OA$IMAGE")'"
$ !
$ ! Wait if system not started yet
$ !
$ IF ShutDown .NES. "" THEN GOTO System_up
$ !
$ Wait 0:5:0 ! Wait for 5 minutes
$ !
$ GOTO Wait_loop ! and try again
$ !
$System_up:
$ !
$ ! Set delay time for rescheduling using a local Message Router
$ !
$ wait_time := "+0:5"
$ queue:='f$logical("OA$MTI_QUEUE")
$ !
$ ! Look in the system permanent symbol table to find the fetcher control
$ ! record. If the record is not defined, or is set to something illegal,
$ ! we'll notify the operator, and drop off the queue.
$ !
$ @OA$LIB:SYSTEM_PST R OA$MTI_FETCHER_CONTROL
$ ShutDown = pst_value
$ !
$ ! Check for SHUTDOWN request
$ !
$ IF ShutDown .EQS. "STOPPED" THEN GOTO FIN
$ !
$ ! Check for HOLD request
$ !
$ IF ShutDown .EQS. "ON HOLD" THEN GOTO RESCHED
$ !
$ ! Check we're still running
$ !
$ IF ShutDown .NES. "RUNNING" THEN GOTO ABORT_FETCHER
$ !
$ ! Ensure that Fetcher and Sender are not running elsewhere. If we can't open
$ ! the locked files we resubmit and exit.
$ !
$!~~ IF F$SEARCH("OA$MTI_DATA:SENDLOCK.DAT;1") .EQS. "" THEN -
$!~~ CREATE OA$MTI_DATA:SENDLOCK.DAT;1
$!~~ OPEN/READ/WRITE/ERROR=RESCHED1 LOCKFILE1 OA$MTI_DATA:SENDLOCK.DAT;1
$
$ IF F$SEARCH("OA$MTI_DATA:FETCHLOCK.DAT;1") .EQS. "" THEN -
$ CREATE OA$MTI_DATA:FETCHLOCK.DAT;1
$ OPEN/READ/WRITE/ERROR=RESCHED1 LOCKFILE2 OA$MTI_DATA:FETCHLOCK.DAT;1
$ !
$ ! We're OK - reschedule the job first and then run this one
$ !
$ GOTO RESCHED
$ !
$ ! Stop A1 being run if sender or fetcher is already running
$ !
$RESCHED1:
$ ShutDown := "ON HOLD"
$ !
$ ! Take a look in the queue to find our process name
$ !
$RESCHED:
$ !
$ ! Get the current username
$ uname = f$getjpi("","username")
$ uname = f$edit(uname,"COLLAPSE")
$!
$! Now set up the strings we look for appropriately . . .
$!
$ holding = "Holding"
$ pending = "Pending"
$ !
$ ON ERROR THEN CONTINUE
$ ASSIGN/USER QUEUE.TMP SYS$OUTPUT
$ SHOW QUEUE/ALL/BATCH 'queue
$ OPEN/READ/ERROR=30 QUEUE QUEUE.TMP
$10:
$ READ/END=20 QUEUE LINE
$ IF 'F$LOCATE("OAMTIMAIL",LINE) .EQ. 'F$LENGTH(LINE) THEN GOTO 10
$ IF 'F$LOCATE(uname,LINE) .EQ. 'F$LENGTH(LINE) THEN GOTO 10
$ IF 'F$LOCATE(holding,LINE) .NE. 'F$LENGTH(LINE) THEN GOTO 15
$ IF 'F$LOCATE(pending,LINE) .EQ. 'F$LENGTH(LINE) THEN GOTO 10
$15: ! Job is already holding so we won't resubmit
$ CLOSE QUEUE
$ DELETE QUEUE.TMP;*
$ GOTO RUN_FC
$20:
$ CLOSE QUEUE
$ DELETE QUEUE.TMP;*
$30:
$ !
$ ! Reshedule the batch job
$ !
$ ON ERROR THEN GOTO SubmitFail
$ ON WARNING THEN GOTO SubmitFail
$ !
$ SUBMIT OA$LIB:OAMTIMAIL/RESTART/NOPRINT/NONOTIFY/AFTER="''wait_time'" -
/QUEUE='queue'
$ !
$ ! Now check that we're not just ON HOLD
$ !
$RUN_FC:
$ !
$ ON ERROR THEN CONTINUE
$ ON WARNING THEN CONTINUE
$ !
$ IF ShutDown .EQS. "ON HOLD" THEN GOTO FIN
$ !
$ ! See if there's anything in the FETCHER QUEUE record in the pending file.
$ ! If there is, we've got to run ALL-IN-1 anyway.
$ !
$ IF DEB$LOG THEN WRITE LOGFILE -
"''f$time()' %FETCHER-I-CHECK, Checking for messages"
$ RUN OA$LIB:FETCHCHECK
$ IF $STATUS .EQ. 3 THEN GOTO RUN_A1
$ !
$ ! Ask Message Router if there is anything to fetch, to save firing up
$ ! ALL-IN-1, if MR is on this node. If it is on another node, we've got
$ ! to run A1 anyway, because we need the mailbox password. If it's on another
$ ! node, OA$MTI_MR_NODE will be the nodename.
$ !
$ IF "''F$LOGICAL("OA$MTI_MR_NODE")'" .NES. "" THEN GOTO RUN_A1
$ MBX = F$LOGICAL("OA$MTI_MAILBX")
$ !
$ ! Work out the correct DECNET object to link to
$ !
$ ON WARNING THEN GOTO OPNLNKERR
$ MRN = F$LOGICAL ("MR$NODE")
$ IF ''F$EXTRACT(0,5,MRN)' .eqs. """""::""" THEN GOTO NONODE
$ OPEN/READ/WRITE/ERROR=OPNLNKERR MRLINK 'MRN'
$ GOTO LINKOPEN
$NONODE:
$ OPEN/READ/WRITE/ERROR=OPNLNKERR MRLINK 0'MRN'
$LINKOPEN:
$ ON ERROR THEN GOTO LNKERR
$ !
$ ! We don't use the password, because this job must run under the same VMS
$ ! account as Manager and Postmaster.
$ !
$ WRITE MRLINK "I''MBX'"
$ READ MRLINK ANSWER
$ IF "''F$EXTRACT(0,1,ANSWER)'" .EQS. "I" THEN GOTO LNKIDENT
$ CLOSE MRLINK
$ GOTO LNKERR
$ !
$LNKIDENT:
$ !
$ WRITE MRLINK "Q"
$ READ MRLINK ANSWER
$ CLOSE MRLINK
$ IF ANSWER .NES. "C0" THEN GOTO MESSAGE
$ WRITE SYS$OUTPUT "No messages to fetch"
$ IF DEB$LOG THEN WRITE LOGFILE -
"''f$time()' %FETCHER-I-NOMSGS, No messages to fetch from MR"
$ GOTO FIN
$ !
$MESSAGE:
$ !
$ IF "''F$EXTRACT(0,1,ANSWER)'" .NES. "C" THEN GOTO LNKERR
$ IF DEB$LOG THEN WRITE LOGFILE -
"'f$time()' %FETCHER-I-FETCH, Fetching messages"
$ WRITE SYS$OUTPUT "There are messages to be fetched"
$ GOTO RUN_A1
$ !
$LNKERR:
$ !
$ WRITE SYS$OUTPUT -
"Error in connection to Message Router - returned message was ''answer'"
$ IF DEB$LOG THEN WRITE LOGFILE -
"''F$TIME() %FETCHER-F-MRERR, Message Router returned ''answer'"
$ GOTO FIN
$ !
$OPNLNKERR:
$ !
$ MESSAGE = F$MESSAGE($STATUS)
$ WRITE SYS$OUTPUT -
"Error occurred while creating link to message router - message was ''MESSAGE'"
$ IF .NOT. DEB$LOG THEN GOTO FIN
$ TIME = F$TIME()
$ WRITE LOGFILE -
"''TIME' %FETCHER-F-MROPN, Failed to open link"
$ WRITE LOGFILE -
"''TIME' ''MESSAGE'"
$ GOTO FIN
$!
$RUN_A1:
$ !
$ ! Run ALL-IN-1 and call the fetch routine.
$ !
$!~~ ALLIN1/NOINIT/USER=POSTMASTER/PASSWORD=DECMAIL
$ ALLIN1/NOINIT/USER=A1FETCHER
OA$INI_INITIALIZE
GET CLI$WHERE = OA$PROFIL_DIRECT
DO OA$LIB:FETCHER_REFILE
MAIL MTI_FETCH_REMOTE
$ !
$!------------------------------------------------------------------------
$! ALWAYS replace the Fetcher's DAF.DAT and DOCDB.DAT with empty files
$! as there should be NO documents in the Fetcher's file cabinet!
$!
$! Use CREATE/FDL to set up new, empty .DAT files - 12/04-91 - Ron/John
$!
$ CREATE/FDL=OA$LIB:FETCHER 'where'DAF.DAT
$!
$ CREATE/FDL=OA$LIB:DOCDB 'where'DOCDB.DAT
$!
$ PURGE/LOG 'where'*.DAT ! Keep 1 version
$!
$ RENAME/LOG 'where'DAF.DAT; 'where'DAF.DAT;1
$!
$ RENAME/LOG 'where'DOCDB.DAT; 'where'DOCDB.DAT;1
$!
$!------------------------------------------------------------------------
$ GOTO FIN
$ !
$ SubmitFail:
$ ABORT_FETCHER:
$ !
$ ON ERROR THEN CONTINUE
$ ON WARNING THEN CONTINUE
$ REPLY/BELL/TERMINAL='f$logical("OA$MTI_OPER") "Please re-submit ALL-IN-1 Mail Fetcher"
$ CREATE MAIL.TMP
Please re-submit the ALL-IN-1 Mail Fetcher. Examine the log file OAMTIMAIL.LOG
in the login directory of your ALL-IN-1 system for more detail.
$ MAIL/Subject="ALL-IN-1 Mail Fetcher Failure" MAIL.TMP System
$ Delete MAIL.TMP;
$ !
$FIN:
$ IF DEB$LOG THEN WRITE LOGFILE -
"''f$time()' %FETCHER-I-END, End of ''start_time' fetcher run on ''node'"
$ IF DEB$LOG THEN WRITE LOGFILE ""
$ ON ERROR THEN CONTINUE
$ ON WARNING THEN CONTINUE
$ PURGE SYS$LOGIN:OAMTIMAIL.LOG/KEEP=5
$ IF "''F$LOGICAL("LOCKFILE1")'" .NES. "" THEN CLOSE LOCKFILE1
$ IF "''F$LOGICAL("LOCKFILE2")'" .NES. "" THEN CLOSE LOCKFILE2
$ IF "''F$LOGICAL("LOGFILE")'" .NES. "" THEN CLOSE LOGFILE
IDENT " 4-DEC-1991 09:50:32 VAX/VMS ANALYZE/RMS_FILE Utility"
SYSTEM
SOURCE VAX/VMS
FILE
ALLOCATION 774
BEST_TRY_CONTIGUOUS yes
BUCKET_SIZE 63
CLUSTER_SIZE 2
CONTIGUOUS no
EXTENSION 63
FILE_MONITORING no
GLOBAL_BUFFER_COUNT 0
ORGANIZATION indexed
PROTECTION (system:RWED, owner:RWED, group:RE, world:)
RECORD
BLOCK_SPAN yes
CARRIAGE_CONTROL carriage_return
FORMAT variable
SIZE 2000
AREA 0
ALLOCATION 702
BEST_TRY_CONTIGUOUS yes
BUCKET_SIZE 63
EXTENSION 63
AREA 1
ALLOCATION 72
BEST_TRY_CONTIGUOUS yes
BUCKET_SIZE 12
EXTENSION 0
KEY 0
CHANGES no
DATA_KEY_COMPRESSION no
DATA_RECORD_COMPRESSION no
DATA_AREA 0
DATA_FILL 100
DUPLICATES no
INDEX_AREA 1
INDEX_COMPRESSION no
INDEX_FILL 100
LEVEL1_INDEX_AREA 1
NAME "DAF_KEY"
NULL_KEY no
PROLOG 3
SEG0_LENGTH 65
SEG0_POSITION 0
TYPE string
IDENT " 5-MAR-1990 09:08:21 VAX/VMS ANALYZE/RMS_FILE Utility"
SYSTEM
SOURCE VAX/VMS
FILE
CONTIGUOUS no
GLOBAL_BUFFER_COUNT 0
ORGANIZATION indexed
PROTECTION (system:RWED, owner:RWED, group:RE, world:)
RECORD
BLOCK_SPAN yes
CARRIAGE_CONTROL carriage_return
FORMAT fixed
SIZE 500
AREA 0
ALLOCATION 48
BEST_TRY_CONTIGUOUS yes
BUCKET_SIZE 3
EXTENSION 12
AREA 1
ALLOCATION 12
BEST_TRY_CONTIGUOUS yes
BUCKET_SIZE 3
EXTENSION 6
AREA 2
ALLOCATION 24
BEST_TRY_CONTIGUOUS yes
BUCKET_SIZE 3
EXTENSION 12
KEY 0
CHANGES no
DATA_KEY_COMPRESSION yes
DATA_RECORD_COMPRESSION yes
DATA_AREA 0
DATA_FILL 75
DUPLICATES no
INDEX_AREA 1
INDEX_COMPRESSION yes
INDEX_FILL 75
LEVEL1_INDEX_AREA 1
NAME "DOCUMENT"
NULL_KEY no
PROLOG 3
SEG0_LENGTH 36
SEG0_POSITION 0
TYPE string
KEY 1
CHANGES no
DATA_KEY_COMPRESSION yes
DATA_AREA 2
DATA_FILL 75
DUPLICATES yes
INDEX_AREA 2
INDEX_COMPRESSION yes
INDEX_FILL 75
LEVEL1_INDEX_AREA 2
NAME "DOCNUM"
NULL_KEY no
SEG0_LENGTH 6
SEG0_POSITION 36
TYPE string
KEY 2
CHANGES yes
DATA_KEY_COMPRESSION yes
DATA_AREA 2
DATA_FILL 75
DUPLICATES yes
INDEX_AREA 2
INDEX_COMPRESSION yes
INDEX_FILL 75
LEVEL1_INDEX_AREA 2
NAME "TITLE_HASH"
NULL_KEY no
SEG0_LENGTH 6
SEG0_POSITION 42
TYPE string
...end...
|