[Search for users] [Overall Top Noters] [List of all Conferences] [Download this site]

Conference azur::mcc

Title:DECmcc user notes file. Does not replace IPMT.
Notice:Use IPMT for problems. Newsletter location in note 6187
Moderator:TAEC::BEROUD
Created:Mon Aug 21 1989
Last Modified:Wed Jun 04 1997
Last Successful Update:Fri Jun 06 1997
Number of topics:6497
Total number of notes:27359

6437.0. "DATA HOLES IN RDB DATABASE" by GIDDAY::HONSCH (If in doubt blame Murphy) Wed Apr 03 1996 04:30

    DECmcc Version 1.3.
    My knowledge of RDB/MCC is very limited!. 
    Situation customer requires reports on various HUBS and ROUTERS for which a
    Microsoft Access (V1.1) application was written. DECmcc exports to the
    RDB database then Access imports from RDB to create the report.
    
    Problem: There are data holes in the RDB database. The customer can
    produce his reports for say 6am to 6pm and finds that the only data is
    for the period 9am to 12am. I have called in Access/RDB/VMS experts
    and the answer to date is that DECmcc seems to be the culprit. 
    
    I am aware that at this time my request is very simplistic but if
    anyone has any ideas or suggestions please ask questions that I can ask
    of the above experts.
    
    Fred Honsch (Adelaide S.A.) 
T.RTitleUserPersonal
Name
DateLines
6437.1been there, done that!SCASS1::GALVINThe Energizer Bunny's Trainer...Tue Apr 09 1996 08:3271
    Hi there,
    
    Well, I'll start with a caveat... I used the EXPORTER/REPORTER 3 years
    ago... At the time, we had limited system resources and saw all kinds
    of problems...  ok, that being said.
    
    I assume, *correctly*? that you're using an OpenVMS platform and this
    is not an Ultrix implementation, yes?  I don't think Rdb ran on
    Ultrix...
    
    That being said, you need to check things like the
    MCC_EXPORTER-whatever.LOG file(s).  You should check to ensure that
    what you think you're exporting is actually being exported... It's been
    some time, but I believe you can check the status of your exporting?
    
    You might want to check the obvious stuff... are the rdb database files
    being grown to the full extent you think they should be, or are you
    running out of diskspace, etc.  Do a SHOW ERR, to ensure you're not
    seeing hard disk errors...
    
    Check your batch queues, you need to setup a batch queue that has the
    Rdb database filename in it, don't you?  Make sure you have the proper
    quotas, ie:  WSQUOTA or whatever for your queues...
    
    I seem to remember that if you try to export too much stuff, you start
    losing things... for example, you might want to export line counters
    for a routing circuit, as opposed to just blankedly exporting all
    attributes of a router...
    
    Sounds like you're sure that the REPORTER is in fact setup properly? 
    You have your CDD$TOP or whatever pointers setup properly in
    SYLOGIN.COM?  I've made mistakes like this in the past...
    
    You didn't mention bridges, but you know you can't do much with
    bridges?  Never implemented...
    
    You state that you're seeing a smaller period of time than you are
    requesting, which leads me to believe it's either a typo... I make 'em
    too ya know...smile.  Or a system resource deal...
    
    You might not want to hear this, but you should upgrade to V1.4, there
    were some holes in V1.3 that I believe were fixed... I know there were
    memory leaks once upon a time, could be the culprit...
    
    Remember, if you get brave and upgrade, you'll have to change the CMA
    libraries if you chose to upgrade VMS as well...  See previous notes, I
    think with my name in them about CMA$....whatever.
    
    You might want to check in the Rdb notes conference and ensure the
    version of Rdb you are using doesn't have any gotchas... You might have
    to call Oracle, but they've got all the Rdb gurus...
    
    Try creating a new domain, almost forgot, EXPORTING is domain specific,
    isn't it?  Create a new domain, try just 1 router or hub, or whatever,
    submit a batch job with just that domain and Rdb file in it and see if
    you see the same issues...
    
    Of course, you've got the background processes setup properly, yes?  Do
    a few, DIR/SINC *.LOG's to see if you are seeing failures with your
    batch jobs... you might have a SUBMIT/RESTART statement in your batch
    job and perhaps, periodically you're losing your batch job and they are
    getting resubmitted... remember, EXPORTing is extrememly resource
    intensive...  There were lots of bugs and don't know that they ever got
    identified, let alone fixed... 
    
    Now that I've totally confused/perplexed you... good luck!  Remember,
    we started the REPORTER with datatrieve/CDD and the EXPORTER with Rdb,
    we abandoned the REPORTER and there's been no development work done on
    it in 2-3 years... how *GOOD* could it possibly be... 8^)
    
    /Mic
6437.2More informationSNOC01::MISNETWORKMCC=My Constant ConfusionTue Apr 09 1996 09:11100
Hi,

I am trying to help Fred solve this mystery. Just to restate the problem, we 
are seeing holes in the data as it is stored in the RDB database. For example 
if we have collected one sample every hour in a day, we would expect to see 24 
samples in the database. Instead we might only get 10 samples for a particular 
entitity. So for entity A, on a Monday we might miss data that should be there 
from 09:00 to 13:00, while entity B has data for the whole day. Looking at the 
same entities for data on the Tuesday, we find that both entities have data for 
each hour of the day.

So here are some answers for .1

Yes, we are using OpenVMS platform.

After checking the exporter logs (MCC_EXPORTER_BACKGROUND.LOG;*), found the 
3 different errors in some of the logs -

$ ver = f$verify(0)
 SGK: Error attempting to close socket 5
 Close
: mount device busy
%MCC-E-QREADERR, error reading queue file.
  NETMAN       job terminated at  4-APR-1996 05:18:07.44

  Accounting information:
  Buffered I/O count:         1721898         Peak working set size:   10244
  Direct I/O count:            997548         Peak page file size:    109233
  Page faults:                4303070         Mounted volumes:             0
  Charged CPU time:           0 06:38:15.56   Elapsed time:     6 18:18:59.02
 

$ ver = f$verify(0)
%MCC-E-QREADERR, error reading queue file.
  NETMAN       job terminated at 28-MAR-1996 02:22:04.82

  Accounting information:
  Buffered I/O count:          199618         Peak working set size:   10244
  Direct I/O count:            128841         Peak page file size:     75213
  Page faults:                 485115         Mounted volumes:             0
  Charged CPU time:           0 00:43:30.16   Elapsed time:     0 19:25:41.81


 $ ver = f$verify(0)
 SGK: Error attempting to close socket 5
 Close
: bad file number
 
Your guess is as good as mine as to what these mean.

The RDB files are definately growing. We might miss a couple of hours data on
a Monday say, and have a full days data on the Tuesday. This thing is not 
stopping totally, just occasionally, and then restarting. 

SHOW ERROR gives no clues. Disk space is not a problem. Queues setup ok -

AHY1V1>sh que/bat/all *export* /full
Batch queue EXPORT$BATCH, available, on AHY1V1::
  /BASE_PRIORITY=3 /JOB_LIMIT=4 /OWNER=[NETMAN] /PROTECTION=(S:E,O:D,G:RW,W)

  Entry  Jobname         Username             Status
  -----  -------         --------             ------
    559  MCC_EXPORTER_BACKGROUND
                         NETMAN               Executing
         Submitted  4-APR-1996 18:24 /KEEP
         /PARAM=("DISK$MCC:[REPORTS]MCC_DB.RDB","30") /NOPRINT /PRIORITY=100
         /RESTART=EXPORT$BATCH /WSEXTENT=10240
         File: _AHY1V1$DKA0:[SYSCOMMON.SYSMGR]MCC_EXPORTER_BACKGROUND.COM;9

AHY1V1>sh lo disk$mcc
   "DISK$MCC" = "AHY1V1$DKA100:" (LNM$SYSTEM_TABLE)
AHY1V1>sh dev d

Device             Device           Error    Volume         Free  Trans Mnt
 Name              Status           Count     Label        Blocks Count Cnt
AHY1V1$DKA0:       Mounted              0  VAXVMSRL054     300378   491   1
AHY1V1$DKA100:     Mounted              0  MCC             312045   177   1
AHY1V1$DKA200:     Mounted              0  DATA              7578    10   1
DAD0:              Online               0

Exporting too much stuff, well that might be a possibility as we are exporting 
on CISCO interfaces (all attributes by default). Not sure if we can reduce the 
scope for this. 

We are not exporting on bridges, we are exporting on SNMP devices, spefically 
CISCOs ( and also Chipcom Hubs).

V1.4 upgrade, well normally I would have also suggested this, but in this case 
we are avoiding any upgrades as this site may well be disappearing after June, 
so we don't want to invest the time of upgrading. 

Seeing the EXPORTER is teeming with bugs, we may have to look at upgrading. 
Would be good to get this going as it is.

Cheers,
Louis


    
6437.3try Historian to find problemRANGER::SAMSTue Apr 23 1996 17:0824
	Hi, several years ago I was a member of MCC development team 
and I worked on Exporter. A lot of details are already gone from my
memory but I'll share with you what I still remember. Hope it will
help to solve your problem.

From what I remember I doubt that Exporter background process dies
periodically (although it could be the case). Most likely the missing 
records in the RDB are due to "no response" from entity or to the fact
that the system is overloaded. Exporter writes into RDB several time
stamps which can give you an idea if the system is too busy to handle 
all exporting requests. If a number of exporting requests is too big
you could try to break them into different databases (current time 
Exporter works on database basis NOT on domain).
The first thing I would recommend is to use Historian just to verify
that you are getting responses from all entities being exported.
You should set up recordings with the same schedule as exportings
and later use past time "show" command to view data.    
Generally speaking, in term of system resources Historian is match
cheaper that Exporter. So you may try firstly to record data using
Historian and later export them using "past time" export.

  Hope this helps, Sam
                                                                               
6437.4PATCH from IPMTGIDDAY::HONSCHIf in doubt blame MurphyFri May 03 1996 06:097
    This problem was logged with IPMT and they have come through with a
    patch that could solve the problem. I will appy this and report here.
    IPMT also suggested that if the patch did not solve the problem then we
    should upgrade to V1.4.
    Thanks to all for your input and suggestions.
    
    Fred
6437.5Patches are V1.4 FilesGIDDAY::HONSCHIf in doubt blame MurphyFri May 10 1996 03:245
    Well I applied the patch files and that completly stopped exporting!.
    It seems that the files are for V1.4!. Back to the drawing board or
    upgrade to V1.4
     
    Fred
6437.6Still not the correct PatchGIDDAY::HONSCHIf in doubt blame MurphyThu May 23 1996 03:225
    Have now had two versions of V1.4 patch files from IPMT to try and of
    course they don't work with V1.3!!. Will try for the hopefully correct
    patch and report?.
    
    Fred
6437.73RD Patch to try, X fingers!GIDDAY::HONSCHIf in doubt blame MurphyWed May 29 1996 03:464
    IPMT have given me another patch to try. Will install and test and
    report here!.
    
    Fred
6437.8PATCH WORKEDGIDDAY::HONSCHIf in doubt blame MurphyTue Aug 13 1996 04:444
    Patch worked but holes still evident. Will consider upgrade to 1.4
    when/if contract lasts.
    
    Fred