[Search for users] [Overall Top Noters] [List of all Conferences] [Download this site]

Conference forty2::x500

Title:X.500 Directory Services
Notice:Sprt: FORTY2::X500_SUPPORT, Kits: 216.*, try dir/titl=OFFICIAL
Moderator:FORTY2::PULLEN
Created:Tue Jan 30 1990
Last Modified:Thu Jun 05 1997
Last Successful Update:Fri Jun 06 1997
Number of topics:1016
Total number of notes:4299

987.0. "performance problems using rfc822mailbox as an index" by GOBUCS::COOLEY (Megan and Michelle's Daddy) Tue Mar 18 1997 17:33

Hi,

I'm testing an 150,000 entry V3.0 X.500 implementation running Digital Unix on
an AlphaServer 2000 4/275 with 384K of memory.

I've added an INDEX FOR EQUALITY MATCH on some additional attributes including
rfc822Mailbox.

I'm using XDSU which uses XDS calls to basically do the following:

     Loop for tens of thousands of entries
        search where rfcmailbox=<some value>
        if (not found) modify <dn> add attributes rfc822mailbox, + some others
        else log error (RFC822mailbox not unique)
     End loop

I've noticed that searches on rfc822Mailbox are running quite slow (several
seconds) as opposed to other indexed attributes which take less than a second.
So, when updating tens of thousands of entries, this is taking a while...

The only difference I see is that rfc822mailbox is an iA5StringSyntax attribute
and this uses caseIgnoreIA5StringMatch instead of caseIgnoreStringMatch.

Is this explainable?
Is there a performance problem when using indexed iA5StringSyntax attributes?

Regards,
Warren
T.RTitleUserPersonal
Name
DateLines
987.1FORTY2::PULLENJulian PullenTue Mar 18 1997 17:484
Are you using the new schema. On unix you have to delete dsa to
a snapshot file and create the DSA from the snapshot file.

	Julian
987.2GOBUCS::COOLEYMegan and Michelle&#039;s DaddyTue Mar 18 1997 17:535
    Julian,
    
    Thanks for the reminder, but that has been done.
    
    Warren
987.3a-109.tunnel.crl.dec.com::FORTY2::PALKAAndrew Palka Altavista DirectoryWed Mar 19 1997 09:305
I can't think of any reason why ia5 syntax should be significantly
different from printable string. The matching rules are almost
identical.

Andrew
987.4FORTY2::PULLENJulian PullenWed Mar 19 1997 14:525
What type of match are you using a match of
a substring match starting with "*<string>"
can not use the index.

	Julian
987.5More informationGOBUCS::COOLEYMegan and Michelle&#039;s DaddyWed Mar 19 1997 17:3424
    Some more information.
    
    I can use dxim to see the problem:
    
    I do dxim select where rfc822mailbox=asdfasjdlfj
    and it takes 5-6 seconds to come back with no matches. (even when repeated)
    I do dxim select where surname=asdlkfjad
    and it comes back immediately with no matches.
    
    (I positive rfc822mailbox is indexed, because it was taking even longer
     and doing a select on another non-index attribute is still taking longer).
    
    There are currently 146,000 entries all of which have a surname.
    Only about 20,000 entries have a value for rfc822mailbox.
    All the rfc822mailbox entries end in "@lmco.com".  Is that Significant?
    
    The other thing I'm seeing is the DSA process doesn't seem to want to
    page in all it's data space.  The vsize of DSA is 320M but the rssize
    doesn't seem to want to get bigger than about 200M (seen using ps).  
    Is there a way to make sure the DSA process gets paged in and stays there?
    I'm now up to 512M of physical memory.
    
    Regards,
    Warren
987.6Problem solved - rfc822mailbox wasn't indexed afterallGOBUCS::COOLEYMegan and Michelle&#039;s DaddyThu Mar 20 1997 17:2020
Hi,

I decided to delete my previous note since it was written too late at night :-)
I apoligize if anyone thought it was offensive.

Anyway, the problem has been solved!!!

It turned out to be that rfc822mailbox was in fact not indexed because the
customized *.sc file which contained the Index clause was included FIRST in
dxd_schema.sc.  When I moved it to be included LAST, recompiled the schema,
disabled, deleted to snapshot, created from snapshot, enabled - access to
rfc822mailbox is fast.  My program, which was accessing records based on
this attribute is now taking 9 minutes instead of 15 hours.

Please QAR the dxd_sc compiler program to include a warning message when
this happens (or fix it to be indexed) instead of ignoring the index request.
If you would like me to mail this request to FORTY2_SUPPORT, let me know.

Regards,
Warren