[Search for users] [Overall Top Noters] [List of all Conferences] [Download this site]

Conference iosg::all-in-1_v30

Title:*OLD* ALL-IN-1 (tm) Support Conference
Notice:Closed - See Note 4331.l to move to IOSG::ALL-IN-1
Moderator:IOSG::PYE
Created:Thu Jan 30 1992
Last Modified:Tue Jan 23 1996
Last Successful Update:Fri Jun 06 1997
Number of topics:4343
Total number of notes:18308

2768.0. "DAF file growing inordinately" by GIDDAY::BURT (Chele Burt - CSC Sydney, DTN 7355693) Fri May 28 1993 03:44

Hello and greetings,

A customer has reported a problem with his ALL-IN-1 V2.4 DAF file - it has 
grown to nearly 20000 blocks - his DOCDB is is less than a thousand blocks.

I had a stare at his DAF - there are a heck of a lot of duplicate (or certainly
what appear to be duplicate entries. Some records have no duplicates, some have 
a dozen or more.

Any ideas what could be causing this? ANALY/RMS showed no errors, TRU showed 
no errors, TRM scheduled for tonight.

The only way I can think of alleviating this problem is by physically editing 
the file, deleting the duplicates out, then converting the file? Is there an 
easier way (it's kinda tedious)?

Thanks & regards.

Chele
T.RTitleUserPersonal
Name
DateLines
2768.1Editing ALL-IN-1 Data files... hmmm...FLEX7::ALLINGHAM_PDPermenantly Peaking!Fri May 28 1993 09:2019
    I certainly wouldn't go near the editor until I was absolutely sure
    that those 'duplicate' reocords weren't there for a good reason!
    
    Have you had a look inside the file cabinet itself from an ALL-IN-1
    point of view - you might find documents with 30 attachments each or
    something like that?  I doubt very much that they are there by
    accident.
    
    As a rule (2.4) there are no such things as duplicate records in
    ALL-IN-1 data files, the space around bytes 64 - 72 is probably
    different for each record. (Continuation marks...)
    
    Unless you really know what you are doing, TPUing an ALL-IN-1 datafile
    to remove 'rubbish' is like descaling a warthogs dentures with a cotton
    bud - not for the faint hearted!  Just *maybe* TRM will turn something
    up but a close look at the documents themselves is likely to show more.
    Regards,
    
    Peter.                 
2768.2IOSG::PYEGraham - ALL-IN-1 Sorcerer's ApprenticeFri May 28 1993 12:086
    Agreed!
    
    Are you (.0) sure they are duplicate records, and not continuation
    records for messages with large distribution lists?
    
    Graham
2768.3DAF that won't dieGIDDAY::BURTChele Burt - CSC Sydney, DTN 7355693Mon Jun 07 1993 06:4116
Hello again,

OK, I wimped out on the advice of the Masters - no editing was done.
Took an axe to the OUTBOX instead. TRM & TRU made no difference 
- probably 1/3 rd the documents were in the OUTBOX. When these were deleted 
& the wastebasket emptied, TRU & TRM were done again. File size barely changed.

The most irritating/peculiar thing about the file is that it reportedly 
changed dramatically in size from one day to the next.

If anyone wants to have a stare at the file, it's on 
MSOTIS::DKA300:[BURT.PUBLIC.ALL-IN-1]DAF.DAT & DOCDB.DAT

This one isn't fun.

Chele_with_Grumble_mode_on
2768.4Unshare?FORTY2::ASHGrahame Ash @REOMon Jun 07 1993 17:3119
Hi Chele,

Good to see you're keeping your pecker up (hope I'm allowed to say that)!

Well, fwiw, I've had a look, and very nice it looks too. Firstly, the 
duplicates seem OK - they're correctly-marked continutaion records (as for 
messages with many addressees).

The strange thing is that this is the PDAF, and so all of these mail messages 
are in the user's private area - filenames of [.DOCn]z*.WPL. And all the ones 
I checked are in the READ folder. (As the user has approx 1100 msgs in READ, 
this was not a surprise!).

This is not normal - READ messages should be shared. So, is there an UNSHARE 
tool running on the site? I imagine that an overnight run of a tool which 
unshares all messages with usage-count =1 could significantly increase the 
PDAF of a user who doesn't delete stuff . . .

grahame
2768.5GIDDAY::BURTChele Burt - CSC Sydney, DTN 7355693Tue Jun 08 1993 07:4016
Hello Grahame,

Re UNSHARE, the customer was packed up and whacked onto one system from 
another (the customer partook (?) of the Transfer user facility within the 
illustrius product), so I guess that explains why so many documents are his 
very own (rather than shared)

But.. it really is a dirty great big file. Other than filing the messages & 
attachments, is there any way of dumping the distribution lists (seems 
unlikely, but I'll ask any way)

Re your greeting. Could be tricky - I'm chromosomally disadvantaged.

Thanks again,

Chele
2768.6You never said what the actual problem wasFORTY2::ASHGrahame Ash @REOTue Jun 08 1993 10:3610
Well, I'm a bit lost now - what's the problem? OK, the file's big - it's
supposed to be, it's got a lot of info in it!

Has the user complained that performance is worse? (If he has, he could always 
try clearing out his READ and OUTBOX folders!).

If there really is a problem, then one obvious fix is to re-share all of the 
messages (CAB SHARE) and make the SDAF over-sized instead!

g
2768.7Some serious distribution lists!IOSG::CHINNICKgone walkaboutTue Jun 08 1993 16:5560
    
    Hi 'Chele...
    
    I had a quick look at the PDAF and it is just very very big because
    there are 1137 documents but 5163 actual records.
    
    DAF format files have multiple physical RMS records to represent each
    document because document attributes can be variable in length and
    number.
    
    Normally, these extra physical records hold things like attached
    document pointers and TO or CC addresses.
    
    Some of the documents on file have up 800 addressees like:
    
    	[.DOC7]ZUIGQGHB9.TXT 	{READ                          022437}
    
    
    To some calculations:
    
    	This means that the DAF needs to be big:
    
    		 5000 records x 2000 bytes ~= 5000 x 4 blocks = 20k blocks
    
    	Now this is simplistic because some records will be smaller, but
    	add in the RMS index data and you soon get up there.
    
    
    Soooo, as Grahame says... either SHARE the documents or kill the address
    lists.
    
    To do these, you can produce a script which runs down the DOCDB, checks
    the DAPOINTER field to make sure its "P" (PDAF) and perhaps check its
    SENT or READ in the MAIL_STATUS.
    
    Then kill off the addressees  (perhaps it should check the count of
    them to just hit the really big ones) and then use CAB DELETE_ATTRIBUTE
    to get rid of them all. Alternatively, just use CAB SHARE_DOCUMENT to
    put them back to the SDAF [and most importantly on someone elses disk
    quota!].
    
    Then, using CONVERT or reorganizing the user cabinet will shrink the
    PDAF.
    
    If symptoms persist, seek medical advice!
    
    Paul.
    
    P.S> Just out of curiosity [and for the benefit of other noters]...
    
    	 Is there some reason why you prefer to be called Chele (which I
	 gather is short for Michele?) ? 
    
    	 It seems to have caused quite some confusion as to gender and
         origin for those not in the know. ;-) Or "light entertainment" -
         depending upon how you look at it! 
    
         I guess 7 or 5 letters is better than 4 anyway. Most people seem
         to use 4 when referring to me! :-)
    
2768.8GIDDAY::BURTChele Burt - CSC Sydney, DTN 7355693Wed Jun 09 1993 01:3335
Hello all,

The main problem is that the user, for some reason, "suddenly" had his DAF 
expand in size. It is entirely possibly that some system distribution lists 
performed a Topsy act "just grew". The user's concern was that he was unable 
to use ALL-IN-1 due to quota restrictions.
It's going to be put back to the user to 
1.  Check his READ Index and select anything he wants to keep.
2.  Update those Selections to a folder called KEEP
3.  Exit the READ index, then enter it again & select and delete all entries
4.  Empty wastebasket
5.  RFF from KEEP to READ

MIS to do TRM & TRU

Dull, isn't it?


Thanks all,

Chele


PS.

'lo Paul. 
Chele is pronounced Chele, not Shelly, and not CHEELE.
MOST confusing in an ULTRIXy conversation.
 "C Shell" 
 "See what?"
(I've been using the name off & on since I was 7 - it's MINE)

Regards from,
Me, Chele.    

2768.9My contributionSCOTTC::MARSHALLSpitfire Drivers Do It ToplessWed Jun 09 1993 12:2316
>> problem is that the user, for some reason, "suddenly" had his DAF
>> expand in size

As already mentioned, this would have happened when the user's account was
transferred.  All their READ and SENT messages would have been unshared, and
moved back into the PDAF.  In itself, this is not a problem or any cause for
concern, it's just the way it works (in V2.4; it's better in V3.0)

>> user's concern was that he was unable 
>> to use ALL-IN-1 due to quota restrictions

Again, as already mentioned, user should (ask system manager to) increase
the disk quota, or CAB SHARE all the READ and SENT messages, or write a script
to remove long addressee lists, or delete some documents, or ...

Scott
2768.10What's in a nameIOSG::CHINNICKgone walkaboutWed Jun 09 1993 12:5047
    
    OK Chele...
    
    sounds like the way to go. Can't see how it happened 'suddenly' unless
    something UNSHAREd all the docs (like the TRANSFER USER?).
    
    Just one last point for your future reference on DAF files...
    
    The have a realy nasty format to try to understand from just looking at
    them but when I was last in Support about 3 years ago I knocked
    together a little tool called "CABFIX".
    
    CABFIX is only available to selected specialists (i.e. CSC's) because
    it can all too easily do more damage than good.
    
    One useful feature of it though is that it has a /DUMP switch which can
    let you examine files in DAF format which can help you understand what
    is on file. Unfortunately, a V3 version isn't generally available yet
    but the V2.4 format is certainly close enough.
    
    It can show you why all those records are as big as they are and also
    counts the records on file. Very nitty gritty level, but it can
    sometimes be very useful. The tool does a few other things also like
    report structure errors and can even repair some. [Perhaps someone in
    the CSC there who you can talk to has the kit and doc for it?]
    
    Cheers now,
    
    Paul.
    
    P.S. Yes - I gathered your name was like "Shell Oil" but it is amusing to
    here some people calling it "Cheeley" but then when you see some of the
    British place names like:
    
    	Bicester		Pronounced	"Bister"
    	Tocester				"Toaster"
    	Cirencester				"Siren-sester"
    	Leicester				"Lester"
    	Beaconsfield				"Beckonsfield"
    	Aldeburgh				"Alderburra"
    	etc. etc.
        [No doubt "Andrew.D.Wicks" could cite some good Welsh names too.]
    
    I can well understand their confusion. :-) Who knows what they make of
    Wagga-Wagga, Nuriootpa and Cunamulla not to mention the ones I can't
    even spell [in Victoria]?? Try asking for a Coonawarra Cab Sav!
    
2768.11waiting for customerGIDDAY::BURTChele Burt - CSC Sydney, DTN 7355693Thu Jun 17 1993 04:2616
Hiya Paul,

Sorry for the delay responding, the customer is "terribly busy". Probably 
won't take an ax to his index 'til mid July.

So you're the CABFIX man. hmmm

Thanks for all your help,

Chele
 
PS
(S.S. who has a British accent, amuses me no end when he attempts to pronounce 
"Murwillumbah" - snicker snicker)


2768.12Yeah... you got a problem with that!?IOSG::CHINNICKgone walkaboutThu Jun 17 1993 10:3629
    
    OK Chele...		we'll leave you to it.
    
    >So you're the CABFIX man. hmmm
    
    Is there a problem here? :-)
    
    I did write it 'cause we needed it. Unfortunately, having been
    elsewhere for 3 years (since I'm but a `humble' contractor [pronounced by
    some here as "scumbag" :-)]), no work has been done to update it for V3 of
    ALL-IN-1.
    
    If I get time... or if my boss decrees then this might happen. But like
    you, the "if I have time" agenda is getting pretty damn long!
    
    I also confess to producing SSTRAP, RECOVER and our latest and greatest
    tool VMTAG. It was me Your Honour! I know I shouldn't have done it but
    you don't think of these things at the time. ;-)
    
    
    Paul.
    
    P.S. When I worked in TSC in Sydney I had a boss from Murwillumbah. I'm
    South Australian and even I had trouble pronouncing it! I must admit
    that after 5 years in Europe - I've got a "pommie" accent - worst of
    both worlds really! :-)
    
    
    
2768.13TINNIE::SETHIAhhhh (-: an upside down smile from OZThu Jun 17 1993 12:049
    Hi Paul,
    
    Murwillumbah
    
    You see I can say it without a problem :-)!!
    
    Regards,
    
    Sunil