Conference vaxaxp::vmsnotes

631.0. "INDEXF.SYS size again!!" by CHOWDA::GLICKMAN (writing from Newport,RI) Thu May 22 1997 16:23

    One of my customers read this article, wanting to reduce the size of
    INDEXF.SYS on some disks where massive clean up was done.  She is
    running OpenVMS 6.1 VAX. She wasn't sure that performing the
    BACKUP/IMAGE, INITing the disk and BACKUP saveset disk:[*...]
    would work with this version of OpenVMS. Could someone let me know
    if this will work? And if so, are there performance gains to be 
    achieved?
    
    Thanks.
    
[OpenVMS] BACKUP Restore of Disk Does Not Shrink Size of INDEXF.SYS
Last Technical Review: 31-JUL-1995                            Size: 119 lines
    

     Any party granted access to the following copyrighted information
     (protected under Federal Copyright Laws), pursuant to a duly executed
     Digital Service Agreement may, under the terms of such agreement copy
     all or selected portions of this information for internal use and
     distribution only. No other copying or distribution for any other
     purpose is authorized.

Copyright (c) Digital Equipment Corporation 1995.  All rights reserved

RPODUCT:    OpenVMS VAX, Version 5.5-2 and above
            OpenVMS Alpha, Version 6.1 and above

COMPONENT:  Disk Management

SOURCE:     Digital Equipment Corporation


PROBLEM:

When the number of files and/or directories on a disk is smaller than
the number of headers in INDEXF.SYS, a significant amount of disk
space may be wasted.  The space is wasted on file headers that will
never be used (1 header = 1 block).

INDEXF.SYS can grow but never shrink.  The recommended solution for
reducing the size of this file is to make an image BACKUP of the disk,
re-initialize it, and then execute a non-image restore.

A non-image restore allocates headers from INDEXF.SYS only as needed.
Thus, it should result in the number of file headers in INDEXF more
closely matching the actual number of files and/or directories on the
disk.

However, as of OpenVMS VAX V5.5-2 and OpenVMS Alpha V6.1, this is no
longer the case.  For example, a non-image restore of a disk with
only a 1000 files may result in an INDEXF.SYS of 50,000+ blocks.  In
this case, INDEXF.SYS would contain approximately 49,000 unused
headers.

An image restore can NEVER be used to reduce the size of INDEXF.SYS.


DIGITAL RESPONSE:

This is expected behavior as of OpenVMS VAX V5.5-2 and OpenVMS Alpha
V6.1.


WORKAROUND:

Since the behavior is due to the normal expansion of INDEXF.SYS by the
file system, do not allow the file system to expand INDEXF.

Estimate the approximate number of files to be restored to the disk.
Use this number in the INITIALIZE command to pre-extend INDEXF.SYS and
execute the non-image restore of the disk.

For example, assume a disk currently has an INDEXF.SYS of 50,000
blocks (even though the disk will never have more than 2000 files on
it).  In this case, 48,000 file headers (i.e. blocks) are wasted.

To create a more realistic INDEXF.SYS:

   1.  Make an image BACKUP of the disk.

            $ BACKUP/IMAGE disk: device:saveset/SAVE_SET

   2.  Re-initialize the disk:

            $ INITIALIZE/HEADERS=2000/MAXIMUM_FILES=4096 disk: label

   3.  Execute a non-image restore of the disk:

            $ BACKUP/VERIFY device:saveset/SAVE_SET disk:[*...] -
               /BY_OWNER=ORIGINAL

Step 2 is critical in this procedure.  It pre-allocates 2000 file
headers in INDEXF.SYS and sets the maximum number of files allowed to
4096.  The number 4096 was chosen because any number up to 4096 uses 1
block of space in INDEXF.SYS.  This minimizes wasted space.

When the non-image restore is performed, all necessary file headers
are already pre-allocated so the file system has no reason to extend
INDXF.SYS.


ANALYSIS:

Previous to OpenVMS VAX V5.5-2 and OpenVMS Alpha V6.1, an extension of
INDEXF.SYS was hard-coded to be a 1000 blocks (rounded up to the
nearest cluster size).  As of OpenVMS VAX V5.5-2 and OpenVMS Alpha
V6.1, this is no longer true.  When INDEXF.SYS is extended, the size of
the extension is variable.

There are several factors involved in the extension.  A major
consideration is the amount of free space on the disk.  The reasoning
is that when there is a lot of free space, there is a potential to
create a lot of new files.  Since each of these potential new files
requires a file header, the file system allocates a large extension to
get a large number of file headers.

Conversely, if the disk is almost full, the file system allocates a
small number of headers since the disk will probably run out of space
before many new files are created.

There is a problem with the non-image restore.  The default number of
file headers on a newly initialized disk is so small (16), that all
available file headers are used up after restoring only a few files.

This forces an almost immediate extension of INDEXF.SYS.  Since the
file system has no way of knowing that only a limited number of
additional files are going to be restored, it extends INDEXF.SYS based
on an algorithm that is seeing an essentially empty disk.  As a
result, an extension of 60,000 blocks is not unusual.

   There are no performance gains here, beyond those one would expect
   to obtain through the disk defragmentation operation.

   And unless the index file is both huge and very sparse, this project
   isn't likely worth the effort -- this may save some disk space, for
   that interval while the extra file headers are not needed.

   As for how to do this, you cannot use BACKUP/IMAGE, as it preserves
   FIDs, which is at the root of the restriction here.  See 616.* for a
   previous discussion on this particular topic, and some of the steps
   to transfering files over to a disk with a smaller index file.

   (In some ways, 616.* is the opposite approach to what is proposed
   here; the larger cluster factor proposed in 616.* reduces the size
   of the storage bitmap and would tend to reduce the number of file
   headers required.  I'd *not* recommend the 256 block cluster factor,
   however.)