[Search for users] [Overall Top Noters] [List of all Conferences] [Download this site]

Conference turris::digital_unix

Title:DIGITAL UNIX(FORMERLY KNOWN AS DEC OSF/1)
Notice:Welcome to the Digital UNIX Conference
Moderator:SMURF::DENHAM
Created:Thu Mar 16 1995
Last Modified:Fri Jun 06 1997
Last Successful Update:Fri Jun 06 1997
Number of topics:10068
Total number of notes:35879

9469.0. "Performance Issue - 'Page in' very high" by HYDRA::BRYANT () Thu Apr 10 1997 19:46

InterMail from Software.com has been ported from Sun to DU 4.0b.  Load testing
is showing that the software runs four times slower on DU.  This is a 4100 with
4 processors, 1 GB RAM, dual HSZ40's, 50 GB storage, .....

InterMail appears to be spawning around 50 processes some which are writing ~7K
byte files to disk.

Performance analysis from VmUBC reports 660 MB of RAM free and a tremendous
amount of page ins occuring at the same time.  Have already tweaked numerous
system parameters with no luck.

Has anyone seen this performance issue on 4.0b?  Reasons behind it? 
Suggestions? 

Thanks.
Pat Bryant
Software Partners Engineering

T.RTitleUserPersonal
Name
DateLines
9469.1KITCHE::schottEric R. Schott USG Product ManagementThu Apr 10 1997 22:027
Hi

 You might try running sys_check so we can see more
tuning parameters...

  http://www-unix.zk3.dec.com/tuning/tools/sys_check/sys_check.html

9469.2InterMail performance problemUSCTR1::16.178.0.161::BENGHUIThu Apr 10 1997 22:4924
>InterMail from Software.com has been ported from Sun to DU 4.0b.  Load 
>testing is showing that the software runs four times slower on DU.
>This is a 4100 with 4 processors, 1 GB RAM, dual HSZ40's, 50 GB storage, 
.....

Cool machine.

>InterMail appears to be spawning around 50 processes some which are writing 
>~7K byte files to disk.

Just a guess... I suspect you are using Advfs. Advfs seems to be
slightly slower than ufs for small i/o. Is i/o a problem in
the setup ?

>Thanks.
>Pat Bryant
>Software Partners Engineering

I wonder if you can keep me update on the porting process ?
I have a potential customer here that is waiting to try 
InterMail.

Thanks.

9469.3Some speculationsWIBBIN::NOYCEPulling weeds, pickin' stonesFri Apr 11 1997 09:0810
Does the Sun machine also experience a tremendous amount of
page ins?  Could it be that some data that ought to be BSS
is actually in the DATA portion of the image, so that it
needs to be read off disk instead of created from zeroed
pages?  Or could it be that some data that ought to be shared
(RDATA) is actually non-shared (DATA)?  Does the application
carefully organize things for SPARC's page size (4KB?) and
need to redo it for Alpha's (8KB)?

Can we assume the CPU's are nearly idle?
9469.4More informationHYDRA::NEWMANChuck Newman, 508/467-5499 (DTN 297), MRO1-3/F26Fri Apr 11 1997 10:1624
Re:  .2
� Just a guess... I suspect you are using Advfs. Advfs seems to be
� slightly slower than ufs for small i/o. Is i/o a problem in
� the setup ?

Yes, they do use ADVfs.  Suspecting that ADVfs would have relatively high
overhead for file system functions, I suggested that they try ufs.  They
were reluctant to make this change unless they had a high level of confidence
that it would help (gobs of RZ29s on two HSC40s).  In their tests, the disk
activity can be characterized as write once, read never (WORN :-) ).  That
is, they are basically just storing gazillions of MAIL messages to disk.

One individual here suggested that they use open/mmap/close rather than
open/write/close in order to eliminate a buffer copy -- does that make sense?


Re:  .3
� Can we assume the CPU's are nearly idle?

They said that CPU utilization swings back and forth between ~15% and ~85%.

The low CPU usage levels coincide with high paging levels.

								-- Chuck Newman
9469.5Yes, their using AdvfsHYDRA::BRYANTFri Apr 11 1997 10:2116
When page in activity is high, CPU utilization is low and vice versa.
I'm looking for something blantingly wrong here.  If switching to UFS is only
going to give a slight performance benefit, then that's not enough.

I'll gather information via sys_check and get some information about the Sun
machine as well.

This identical performance issue is occuring on a machine that is running
Software.com's older product, Post.office, only after the system was upgraded to
4.0x.  That machine is also running Advfs.  I don't see any reported performance
problems running Advfs on 4.0.  Maybe it makes sense to remove it from the
picture?




9469.6KITCHE::schottEric R. Schott USG Product ManagementFri Apr 11 1997 15:0916
Hi

 I would gather the data...advfs does not cause the paging described.

I think we are looking for another problem...some sys_check output would
let us check the system parameters...you need to findout what the
application is trying to do in parallel.

Don't mess with advfs yet.

Please ensure you have installed latest OS patches, and hardware firmware is
up to rev also.

regards

Eric
9469.7sys_check output on fluid.mro.dec.com /pub/software.comHYDRA::BRYANTFri Apr 11 1997 15:130
9469.8KITCHE::schottEric R. Schott USG Product ManagementFri Apr 11 1997 15:244
Hi

How about a URL?

9469.9It'sHYDRA::BRYANTFri Apr 11 1997 15:411
http://fluid.mro.dec.com/www-spe/www-mroctr/pb/swcom.html
9469.10Your .html files are no good!VIRGIN::SUTTERWho are you ??? - I'm BATMAN !!!Mon Apr 14 1997 06:408
Your sys_c?.html files contain some strange (MIME?) characters (=3D and the 
sort). 

Make sure you place clean HTML files !!

Regards, 

Arnold
9469.11COL01::LINNARTZTue Apr 15 1997 06:5720
    Pat,
    any chance that Solrais is using threaded smtp version. It's just a
    guess, but I would think that they are all waiting on a select. 
    check for voluntary and involuntary context switches of this 
    processes (nivcsw nvcws) and at the faults (majflt, minflt,pagein)
    you can also look directly at the structure by using dbx -k /vmunix,
    assign to pid and look at uu_tru structure via 
    p *thread->stack->uthread->np_uthread (per thread) and 
    p *thread->stack->uthread->utask (per task, look for uu_[?]ru)
    
    an easy way to check for vnode backuped data (read, shared) as Web
    mentioned is to switch to eager swap mode. anonymous backed memory
    (malloc, stack, data) gets saved in the swap area, but as you've got 
    no real pageout, i would guess the machine gets busy due to different 
    wakeups of the smtp proc's.
    
    if it's not due to the previous, and it's due to disk I/O, I would
    increase the ubcmaxpercent.
    
    Pit 
9469.12COL01::LINNARTZTue Apr 15 1997 07:057
    oops, sorry sent to soon,
    also look that the socket and network parameters are sized ok,
    there are some tuning papers around (a couple reagrding webserver,
    but the underlying protocols are the same, and explained in quite 
    detail)
    
    Pit