[Search for users] [Overall Top Noters] [List of all Conferences] [Download this site]

Conference azur::mcc

Title:DECmcc user notes file. Does not replace IPMT.
Notice:Use IPMT for problems. Newsletter location in note 6187
Moderator:TAEC::BEROUD
Created:Mon Aug 21 1989
Last Modified:Wed Jun 04 1997
Last Successful Update:Fri Jun 06 1997
Number of topics:6497
Total number of notes:27359

3037.0. "Disk space usage of an mcc Rdb database" by TAV02::HAGAR (Hagar Ronen, Israel VIA Support) Wed May 20 1992 12:05

Hello,

I am an Rdb/VMS specialist who has been asked to help size a DECmcc database
for our comm experts.

I didn't get very far in my calculations before I ran into things that deeply
puzzle me (to say the least), and I'm wondering if someone here might be
able to explain them to me.

The database in question is going to have about 121000 node records and
about 303000 line and circuit records. From a sample of the data it seems
that the average record size of node entries will be about 500 bytes and
the average record size of line and circuit entries will be between 300 and
400 bytes.

The maximum record size defined for the tables is some 5000 for the nodes,
and over 4000 for the line and circuit. The page size of the database is
2 blocks, which means that there will never be more than one record per
page. For the line and circuit tables this means that only about 35% of the
space can ever be used. For the node table it goes up to 50%. For the
customer question this means that at least 1,500,000 disk blocks are needed
to store this database. This is absurd.

On top of this there are no indices defined in the mcc database, which
means that any access to the tables has to do a full sequential scan.

Being, as I said, an Rdb expert, this amazes me. I can suggest ways of 
redesigning the database in a way that will improve the situation - we could
store the data in mixed areas or up the page size to 20 or 30 blocks, and of
course we could add indices to speed up queries.

What I'm trying to find out from you people out there is this:

1. How do other customers manage to live with large DECmcc databases? Has
   anyone dealt with these problems before? What were the possible solutions?

2. If I were to change the database attributes (make it multi-file, change 
   the page size etc.), what implication would this have on maintenance
   activities that concern this database? Under which circumstances is the
   database re-created (thus making it necessary to modify it again)? 

3. Would it be a good idea to define indices for the tables? If it isn't
   a bad idea, where can I get information about the queries made against 
   the database so I'll know which indices to define?

Thanks for any help you can give,

Hagar (Israel IM Support).

p.s. A similar note is being entered in the Rdb_41 notes file.
T.RTitleUserPersonal
Name
DateLines
3037.1TOOK::SHMUYLOVICHWed May 20 1992 18:1929
 Some details how Exporter FM creates an RDB:

  1. Exporter tries to create an RDB every time when exporter background
     process restarts ( note that the first time it's waiting for the
     first Export directive). 
     An RDB file name is an argument to the Export command and to the
     background process. It gives an ability to export data to the
     different RDB files.

  2. Exporter tries to create a table in every Export directive.
     On VMS the table name is built from an entity class, for example,
     MCC_NODE4 or MCC_NODE4_LINE.

     The columns names and datatypes are obtained from the MCC Data Dictionary.
     Same of the attributes are not returned by the corresponding AMs or their
     data types are not supported yet by the Exporter so these columns are
     empty.

  3. When Exporter collects required data it INSERTs data into the 
     corresponding table.

     
I'm not an RDB expert so my question could be naive. Is it possible instead 
of customizing Exporter's RDB file just to copy information you need to
another RDB file?

	Sam
    
3037.2Might be a good ideaTAV02::HAGARHagar Ronen, Israel VIA SupportThu May 21 1992 07:1228
re .1

Your naive idea might be a great one. I need to understand more to know.

>>  1. Exporter tries to create an RDB every time when exporter background
>>     process restarts 

I'm sorry this is chinese to me. How often does this happen? Does it happen
at fixed intervals or with any predictable schedule? What happens to the old
database when a new one is created?

>>     ( note that the first time it's waiting for the
>>     first Export directive). 

??

>>     The columns names and datatypes are obtained from the MCC Data Dictionary.
>>     Same of the attributes are not returned by the corresponding AMs or their
>>     data types are not supported yet by the Exporter so these columns are
>>     empty.

How do I know which columns will always be empty? If I do create my own database
and move the data into it, all problems would be over (maybe) if I could omit 
these fields from the tables.

Thanks very much,

Hagar.
3037.3TOOK::SHMUYLOVICHThu May 21 1992 18:5462

>>>  1. Exporter tries to create an RDB every time when exporter background
>>>     process restarts 
>
>I'm sorry this is chinese to me. How often does this happen? Does it happen
>at fixed intervals or with any predictable schedule? What happens to the old
>database when a new one is created?
>
>>>     ( note that the first time it's waiting for the
>>>     first Export directive). 
>
>??

    Sorry, I thought that you are familiar with MCC at least in general.

 The user can ask MCC to export data to the RDB file any time he wants. 
 He is able to specify an RDB file name in every export request. 
 So it's up to the user what RDB file to use, how many entities to export 
 in the same RDB file.
 For example, if you want to export the information from 100 nodes it's
 possible to use one RDB file for all of the nodes or to use a hundred 
 different files  (the user has to define his strategy). 

 I'll not go into the details of the Exporter FM (an MCC functional module
 responsible for exporting data out of MCC into the RDB) but the main point
 is that an attempt to create an RDB file may occur several times and is not
 predictable. If a given RDB file already exists (has been created earlier)
 the Exporter uses this existing file.

 Again, I'm not an RDB expert but it seems to me that it's possible to create 
 an RDB file prior to specifying export directive in MCC and let Exporter
 to use this file. If this is correct you can create an RDB file in any way 
 you want (making it multi-file, choose the page size etc).


>>>     The columns names and datatypes are obtained from the MCC Data Dictionary.
>>>     Same of the attributes are not returned by the corresponding AMs or their
>>>     data types are not supported yet by the Exporter so these columns are
>>>     empty.
>
>How do I know which columns will always be empty? If I do create my own database
>and move the data into it, all problems would be over (maybe) if I could omit 
>these fields from the tables.


 On VMS every RDB table (table not a file!) corresponds to an object class,
 for example, node4, snmp, or node4 line etc.
 Every column in the RDB table represents a particular attribute for a given
 object class.
 The MCC Data Dictionary defines attributes, their names and datatypes for each
 object class.

 Not each instance of a given class returns ALL set of defined attributes.
 It's impossible to predict which columns will be empty. It seems to me easier
 to move only columns you need.

 Hope this helps, if you need more information you can contact me directly at
 TOOK::SHMUYLOVICH

	Sam
    
3037.4Data is for reportingWELCLU::MCCALLUMFri May 22 1992 12:3611
    
    One has to remember that the exported data from DECmcc is primarily
    used for long term reporting and thus querying the database is not 
    something that was expected to be done often. This is not an
    interactive query type application - more a way of getting data out
    for reporting purposes and reports are probably monthly events after
    which the data is deleted.
    
    I doubt if much time was spent 'designing' the database.
    
    DMcC.