[Search for users] [Overall Top Noters] [List of all Conferences] [Download this site]

Conference iosg::all-in-1_v30

Title:*OLD* ALL-IN-1 (tm) Support Conference
Notice:Closed - See Note 4331.l to move to IOSG::ALL-IN-1
Moderator:IOSG::PYE
Created:Thu Jan 30 1992
Last Modified:Tue Jan 23 1996
Last Successful Update:Fri Jun 06 1997
Number of topics:4343
Total number of notes:18308

2756.0. "Oversized Shared Directory Problems" by FLEX7::ALLINGHAM_PD (It's not easy having a good time!) Wed May 26 1993 09:33

    I am at a customer site at present who has allowed his shared
    directories to get horribly over sized.
    
    This means that his dir files are not being read in XQPCACHE and
    consequently there are more I/O's around than there should be.  This is
    particularly true of OA$MTI_DATA (DATA_SHARE.DIR) which is currently
    186 blocks in size - presumably there are excessive I/O's when
    accessing files like pending and profile.
    
    My question is...
    
    Years ago there used to be a cuddly little executable called COMDIR.EXE
    kicking around that compressed directories without the need for lengthy
    BACKUP operations to rebuild the oversized directory structures.
    
    Is there anything about these days, preferably compiled under and
    operating system later than 4.0, that can do the same job.  Does anyone
    have any cute ways of dealing with this problem?
    
    Regards,
    
    Peter.
    
    P.S. Is there a SYSGEN parm that can up the size of a directory that
    can be read in CACHE?  
T.RTitleUserPersonal
Name
DateLines
2756.1A few renames should do the trick!BRUMMY::MARTIN::BELLMartin Bell, NTCC, Birmingham UKWed May 26 1993 09:5126
Peter,

you don't actually need COMDIR to compress a directory. Close ALL-IN-1
down, rename the directory file to a temporary name, eg

RENAME SHARE123.DIR SHARETEMP.DIR

create a new, empty, directory

CREATE/DIR [.SHARE123]

(checking the ownerships, ACLs etc)

then rename the files from the old to the new, eg

RENAME [.SHARETEMP]*.*.* [.SHARE123]*.*.*

Which should improve the internal structure, and most probably reduce
the size.

Best to write a command file to do the work if you have many directories
to compress,

cheers,

mb
2756.2It's so obvious it *has* to be a trap!FLEX7::ALLINGHAM_PDIt's not easy having a good time!Wed May 26 1993 10:098
    Dear Martin,
    
    I grovel in mortification!  I really ought to have thought of that
    myself - sometimes the simple solution eludes you just for fun!
    
    Thanks for that.
    
    Cheers Peter.
2756.3See VMSTUNINGIOSG::DAVISMark DavisWed May 26 1993 10:1015
    
    You can look at VMSDEV::VMSTUNING note 1101 (KP7 to add) for a
    discussion on the effect of directory size on performance. The general
    conclusion is that directory size will normally only affect performance
    when you are doing wild card operations. Moreover it only when a file
    is opened that the directory is accessed and not for subsequent reads.
    So that PENDING.DAT will only be opened once during a session. Other
    data files such as PROFILE.DAT may be opened a few times depending on
    whether they fall out of cache but not frequently.
    
    				Mark
    
    				
    
    
2756.4Give me back COMDIR....NEWVAX::MZARUDZKII AXPed it, and it is thinking...Wed May 26 1993 14:557
    re -.few
    
    
    COMDIR was fast, RENAME or BACKUP on fragmented directorys takes
    forever. Plus, when you start seeing fraged dirs I'm sure the disk
    indexf itself ain't to far behind.
    -Mike Z.
2756.5BRUMMY::MARTIN::BELLMartin Bell, NTCC, Birmingham UKWed May 26 1993 16:1813
Peter,

	as .4 says, i suppose i should have added the fact that if you have
large directory files, then there is a strong possibility that the disk is
fragmented anyway, so you would have to backup and restore to sort things
out properly.

If you do get a copy of COMDIR, make sure that it is one of the later
versions that does the re-create in one pass, as earlier versions _could_
corrupt the directory file if it took more than one I/O (or something like
that).

mb
2756.6COMDIR was cool...FLEX7::ALLINGHAM_PDIt's not easy having a good time!Wed May 26 1993 16:599
    If anyone's got a copy hanging around please let me know and I'll give
    you a destination to copy it to.
    
    I looked in the VMSTUNING conf and got a lot of folks back in 1991
    having an argument!  Not very helpful...
    
    Thanks for the replies so far...
    
    Peter.
2756.7Well it is your disc after all :-)IOSG::PYEGraham - ALL-IN-1 Sorcerer's ApprenticeThu May 27 1993 09:5020
    We had a similar piece of code to the revised COMDIR in the original
    version of the ALL-IN-1 directory compression program, but since we got
    very bad vibes from VMS engineering about it, we removed it.
    
    FWIW, COMDIR is in the Toolshed (METOO::SW_TOOLS_CATALOG) note 1263,
    although it's a V4.5 tested kit as a previous respondent said. Perhaps
    the MAYDAY::COMDIR (Press KP7 etc.) conference has a later version.
    
    RE .6
    
    <<< I looked in the VMSTUNING conf and got a lot of folks back in 1991
    <<< having an argument!  Not very helpful...
    
    If 1991 was when the directory caching algorithm was last updated, then
    perhaps it is helpful....
    
    Maybe the online disc defragmentors make this sort of activity
    unnecessary?
    
    Graham
2756.8Further COMDIR info....IOSG::PYEGraham - ALL-IN-1 Sorcerer&#039;s ApprenticeThu May 27 1993 09:547
    Especially see note 27 of the COMDIR conference if you are on a version
    of VMS earlier than V5.5.
    
    Since COMDIR is written/supported by someone who is on the node used by
    RMS engineering, I presume it must be OK to use.
    
    Graham
2756.9Miscellaneous CommentsSCOTTC::MARSHALLSpitfire Drivers Do It ToplessFri May 28 1993 17:2320
A few comments:

Whatver method you use to compress the directory, make sure ALL-IN-1 is shut
down while you do it.  Otherwise you could get directory entries being deleted
when they shouldn't be, directory entries still there when they should have
been deleted, etc.

From my experience of talking to VMS engineering about COMDIR-type programs,
they do not support that method of directory compression for use at customer
sites.  Maybe the rules have changed, but make sure you get official approval
before using internal tools on a customer's disk!  Basically (from what little I
remember), the risk is that the code compresses the directory in situ; if
anything goes wrong (eg the system crashes) part way through, the directory is
corrupted.  It could well be that things have changed now, you'd have to check.

If $RENAME is too slow, you may be able to get better performance using
$SET FILE/ENTER (You'd have to experiment, I can't remembe the relative
performance of the two methods)

Scott