[Search for users] [Overall Top Noters] [List of all Conferences] [Download this site]

Conference turris::digital_unix

Title:DIGITAL UNIX(FORMERLY KNOWN AS DEC OSF/1)
Notice:Welcome to the Digital UNIX Conference
Moderator:SMURF::DENHAM
Created:Thu Mar 16 1995
Last Modified:Fri Jun 06 1997
Last Successful Update:Fri Jun 06 1997
Number of topics:10068
Total number of notes:35879

8987.0. "xtaso and memory allocation" by RHETT::HALETKY () Thu Feb 27 1997 13:58

    Hello,
    
    
    We have a cusotmer who claims that -xtaso and -xtaso_short eats up 2x
    the memory they predict.
    
    This is on v4.0b of OSF/1.
    
    
    #include <stdio.h>
    
    #define BLOCK_SIZE  1024 * 4
    
    int main (int argc, char *argv[])
    {
        char *pointer;
        long count = 0;
    
        do {
            count++;
            pointer = (char *) malloc(BLOCK_SIZE);
        } while (pointer != NULL);
    
        printf("Failed malloc on iteration %d\n", count);
        printf("Max memory allocated = %d bytes\n", count * BLOCK_SIZE);
        exit (0);
    }
    
    Once I unlimit the above code it will crash my machine as I only have
    256MBs of Swap. Yet the customer claims that after allocating 256MBs of
    memory it will actually hog 512MBs of memory. 
    
    We will attempt to reproduce this on a larger machine but that will be
    hard to find.
    
    #/bin/csh -f
    cc -taso -xtaso mem_eater.c -o mem_eater.xtaso
    cc mem_eater.c -o mem_eater.norm
    
    echo "mem_eater.xtaso" > mem_eater.results
    mem_eater.xtaso >& /tmp/junk
    cat /tmp/junk >> mem_eater.results
    
    echo "##################################" >> mem_eater.results
    
    echo "mem_eater.norm" >> mem_eater.results
    mem_eater.norm >& /tmp/junk
    
    
    The above is the execution. I did note that the failure in allocation
    occurs exactly at the same location. I would think that the one
    compiled with -taso would occur 2x slower than the one without taso.
    
    Failed malloc on iteration 32511
    Max memory allocated = 133165056 bytes
    Failed malloc on iteration 32511
    Max memory allocated = 133165056 bytes
    
    You would think the first one would fail at around iter 64K.
    
    Thoughts, Feedback?
    
    Best regards,
    Ed Haletky
    Digital CSC
T.RTitleUserPersonal
Name
DateLines
8987.1QUARRY::nethCraig NethThu Feb 27 1997 14:4237
I'm not sure where to start here.

The first thing is that -xtaso doesn't do _anything_ for this test program.
All that -xtaso does is enable the '#pragma pointer_size' pragma.   Since
that isn't specified, there is absolutely no difference between compiling
this program -xtaso and without it.   -xtaso_short will make all the pointers
in this program (all one of them) 32 bits, but you don't appear to be using
that.   

Even if you were, I find it curious that the customer appears to think that 
-xtaso_short would somehow change the amount of memory requested by this 
program.    -xtaso_short (if used properly, see below) will change the size of 
exactly *one* object in this program - 'pointer'.   It will NOT change the 
amount of memory allocated by the malloc; malloc simply allocates the number 
of bytes  requested.    In this program, that's a constant.  
-xtaso_short (if it were specified) doesn't have any effect on the value of 
BLOCK_SIZE.

Now, linking with -taso does have some impact on memory allocation, but
the impact is not on the size of the requests, but rather the amount of
address space available to the heap.   The heap normally fits between the
end of the data segment of the program and 'below' the shared libraries; this
space is 'relatively' small in a -taso application.  (about 750Mb, give or take)
If they need more space, they can change the -T and -D values that the 
application is linked with to put more distance between the .so's and the end 
of the data segment.

It should be kept in mind that every malloc request results in additional heap 
space being needed for 'overhead'.    So you shouldn't expect to be able to 
allocate every byte possible - malloc needs some to keep track of what you're 
doing.

FWIW, I don't see any 'header file protection' in the test program;  the
header files all describe 64-bit interfaces but -xtaso_short will make
every pointer 32 bits, which will lead to problems.   We're working on
making this protection 'automatic' in a future release but for the moment
you have to do things yourself.   See the programmers guide for more details.
8987.2example of -T and -D ?KERNEL::SMITHTue Mar 11 1997 05:297
    
    Can you explain or give an example of how to use the -T and -D options?
    
    I haven't used any of this before.
    
    Thanks
    Ewan Smith - UK CSC
8987.3QUARRY::nethCraig NethTue Mar 11 1997 10:5313
 

cc -taso mem.c -T 10000000 -D 12000000

This will move the text and data segments 'down' 0x2000000 from
their default -taso assignments of 12000000 and 14000000, which will add 
32Mb of heap space.

You can move them further 'down' but you need to be careful to leave enough
room for the stack, which starts 'below' the -T number and grows down.



8987.4is this documented in the DU docs?HYDRA::DONSBACHJeff Donsbach, Software Partner Engineering, DTN 297-6862Tue Mar 11 1997 13:577
    
    where in the Digital Unix documentation is the program segment layout
    documented for both -taso and non -taso executables (along with
    there defaults)?
    
    -Jeff D.
    
8987.5QUARRY::nethCraig NethTue Mar 11 1997 15:1011
The 'ULTRIX to DEC OSF/1 Migration Guide", Chapter 7, has lots of
information about -taso, including a memory map and the default -T and
-D values.

The ld man page documents the 'non-taso' -T and -D values.

I don't know of any existing place that documents the non-taso memory map.

This is an area that could probably use improvement.   A QAR would be helpful 
in getting this addressed.   (I don't suspect many people would think
to look in the porting guide for -taso information).
8987.6SMURF::RICKABAUGHMike Rickabaugh Quo flamma est?Wed Mar 12 1997 12:304
For the default non-taso map, see Figure 7-7 "Layout of ZMAGIC Files"
in the Guide to Assembly Language Programming.

-mike