[Search for users] [Overall Top Noters] [List of all Conferences] [Download this site]

Conference azur::mcc

Title:DECmcc user notes file. Does not replace IPMT.
Notice:Use IPMT for problems. Newsletter location in note 6187
Moderator:TAEC::BEROUD
Created:Mon Aug 21 1989
Last Modified:Wed Jun 04 1997
Last Successful Update:Fri Jun 06 1997
Number of topics:6497
Total number of notes:27359

5618.0. "Historian/exported rdb file size" by TROOA::BALDOCK (Chris Baldock) Fri Sep 24 1993 17:42

    
    A simple question.  Why is the data file collected by the historian
    larger than the exported rdb file by a factor of 10.  For example,
    the customer is collecting bridge statistics.  The historian file is
    200,000 blocks while the exported rdb file is only 20,000 blocks.
    Why?
    
    Thanks.
    
    Chris
    
T.RTitleUserPersonal
Name
DateLines
5618.1TROOA::BALDOCKChris BaldockThu Sep 30 1993 10:3920
    
    I'd like to add a little more information to the previous note in
    the hope of getting some comments or opinions.
    
    The customer collected bridge statistics and then wanted to export
    them to an rdb database.  It took seven (7) days and nights for
    the operation to complete on a VAXserver 3100/10e.  There were at
    most a dozen (12) bridges and the system wasn't performing any
    other operation.  Data was collected at 15 minute intervals.
    Now for the questions:
    
    1.  Why did it take to long for such a simple task?
    
    2.  Why was the historian file 200,000 blocks while the rdb
    	database was only 40,000?  What is taking up the "extra"
    	space in the history files?
    
    Thanks.
    
    Chris
5618.2TROOA::BALDOCKChris BaldockFri Oct 01 1993 10:085
    
    Is there anybody out there or did we all jump onto the Netview
    bandwagon? ;-)
    
    Chris_who_still_needs_to_support_mcc
5618.3TOOK::MINTZErik MintzFri Oct 01 1993 14:227
There are still people here, but support requests are swamping our
remaining staff.  We are relying on the formal support channels to
prioritize requests to engineering.  See note 7 (and the conference banner)
for details.

-- Erik

5618.4SISE::SAMMon Oct 04 1993 11:1572

	First of all your customer is not able to record statistics. As it's
 described in the Historical Data Use manual the Historian does not allow
 to record this attribute partition. Statistics can be calculated on previously
 recorded data. So you have to record all data needed for this calculation
 ( as far as I remember the Performance Analyzer manual describes what 
 partitions per entity class have to be recorded). Also I would recommend
 to look into description of a past time data flow in SRM.  

 When the Exporter FM receives a past time export request it issues a past
 time Show commands for every attribute partition for a given target entity. 
 The Information Manager (IM) tries to get requested data (except statistics)
 from the historical MIR. If this data are not available in the MIR the IM 
 sends this request over the network. The past time Show statistics is handled
 by the Performance Analyzer FM (PA). It issues all necessary Show requests
 which are processed by the IM in the described above form. 

  Now about your questions:  
	

>    1.  Why did it take to long for such a simple task?
	
	 I don't know what export request has been used by your customer: 
	 begin time, end time, export period define number of samples
	 exported to RDB. Also time needed to process each request depends
	 on the data availability in the historical MIR. Anyway 7 day does 
	 not look right to me unless your customer tried to export a huge 
	 amount of data.

>    2.  Why was the historian file 200,000 blocks while the RDB
>    	database was only 40,000?  What is taking up the "extra"
>    	space in the history files?

	 Are you sure that Historical MIR contains only data later exported
	to RDB?

	The structure of a record is different in RDB and in the historical 
	MIR but I do not think that historical record can be 5 times bigger
	that RDB record.

	Here are differences:

	 a. Disk space allocation schema is different:
	    - RDB allocates only what is necessary to hold data;
	    - MIR allocates by constant number of blocks.

         b. the historical record contains extra information (ILV coded data);

	 c. exporter does not write into RDB data with constructed data types.
	


	Unfortunately we are not able to give you more precise answer but here
	is a description of every simple  experiment which can give these data:

	 1. Create a new domain;
	 2. Set up a recordings for 1 entity for all partitions. Please,
	    look at Performance Analyzer manual to learn what additional
	    entities must be recorded for statistical calculation.
	 3. After a period of time stop  these recordings;
	 4. Using Show Record command you can look how many samples are 
	    recorded;
         5. Set up a past time export command into a  new RDB data base. 
	    Do not specify an export period less that polling period in the 
	    record command.


		Sam


    
5618.5TROOA::BALDOCKChris BaldockSat Oct 09 1993 14:2910
    
    How do you keep the recording of partions synchronized?
    
    If I want to calculate statistics for bridges I have to record
    two partitions.  How to I keep the two recordings synchronized
    so that the statistics are meaningful?
    
    Thanks.
    
    Chris
5618.6why syncronize??CTHQ::WOODCOCKSun Oct 10 1993 21:0534
>    How do you keep the recording of partions synchronized?
>    
>    If I want to calculate statistics for bridges I have to record
>    two partitions.  How to I keep the two recordings synchronized
>    so that the statistics are meaningful?
>    
>    Thanks.
>    
>    Chris

Hi Chris,

So far I've never seen a need to 'syncronize' partition polls. The different
partitions being polled are used differently in the calculations. Example:

To get NODE4 CIRCUIT stats you need to record

	NODE4 LINE Char (poll once/day)
	NODE4 CIRC Char (poll once/day)
	NODE4 CIRC Counters (poll once/hour)

The performance analyzer only requires a single attribute from LINE Char which
is LINE SPEED. It only requires a single attribute from CIRC Char which is
the type of circuit. Both of these attributes are assumed to not change over
time and therefore even a single poll per day is enough for calculations. As
for the counters, the tool requires these as frequently as the user would like
outputs (specified as DURATION in commands). We have the above described
polling set up so we can produce metrics to the granularity of HOURLY. Counters
are *usually* the only partition which requires frequent polling for any of the
entities I've tried. (Stats for NODE4 circuits, IP interfaces, NODE HDLC 
Stations).

cheers,
brad...
5618.7performanceTROOA::BALDOCKChris BaldockTue Oct 26 1993 18:1113
    
    Jumping back to note .0, the customer reran their timing test but
    the performance was no better.  This time they polled the counters
    partition for a single bridge at 15 minute intervals for 13 hours.
    The export consumed 5 minutes of CPU time on a VAXserver 3100/10e,
    12 minutes of elapsed time.  That is for 52 records (4*13).
    
    Is this standard performance or have I missed something?
    
    Thanks.
    
    Chris