[Search for users] [Overall Top Noters] [List of all Conferences] [Download this site]

Conference clt::cma

Title:	DECthreads Conference

Moderator:	PTHRED::MARYSTEON

Created:	Mon May 14 1990
Last Modified:	Fri Jun 06 1997
Last Successful Update:	Fri Jun 06 1997
Number of topics:	1553
Total number of notes:	9541

1544.0. "Pthread_create afterm 500MB alloc fails." by RHETT::HALETKY () Wed May 14 1997 14:24

    
    Hello,
    
    Allocated 598723244 bytes of memory
    Allocated 24 bytes of memory
    pthread_Create failed. Status is 12
    Insufficient memory
    Allocated 50 * 1048576 - 50 * 1MBytes of memory
    
    
    That is the output of the following program. It does not make anysense
    why the pthread_create would fail. Please assist or explain.
    
    /**********************************************************************
    * This test file uses straight pthreads. The logic is :
    *   Malloc 598723244 Bytes of memory
    *   Malloc 24 bytes of memory
    *   create thread (Using pthreads API) fails with "Insufficient
    Memory".
    *   allocate 50 MBytes of memory successfully!)
    *
    *   This test file results in a failure.
    *
    * Allocated 598723244 bytes of memory
    * Allocated 24 bytes of memory
    * Status is 12
    * Allocated 50 * 1048576 - 50 * 1MBytes of memory
    **********************************************************************/
    
    #include <stdio.h>
    #include <pthread.h>
    
    #include <stdlib.h>
    #include <stdio.h>
    #include <errno.h>
    
    typedef int SOCKET;
    
    void *vTransThread(SOCKET *);
    
    void *vTransThread(SOCKET *pstSocketInfo)
    {
        printf ("Inside of the created thread\n");
    
        pthread_exit(0);
    }
    
    
    void main(int argc, char* argv[])
    {
        SOCKET              *pstSockInfo = NULL;
        int                 nStatus=0;
        void                *pMemory, *pMemory1, *pMemory2;
        pthread_t           stThreadStruct;
    
        pMemory = malloc(598723244); // Allocated 570.9869804382 MBytes
    
        if (pMemory == NULL)
    
    
        {
            printf ("Failed to allocated 598723244 bytes of memory\n");
            exit (-1);
        }
        else
            printf ("Allocated 598723244 bytes of memory\n");
    
        pMemory1 = malloc (24);  // Allocate another 24 bytes just for
    giggles...
    
        if (pMemory1 == NULL)
        {
            printf ("Failed to allocate 24 bytes of memory\n");
            exit(-1);
        }
            printf ("Allocated 24 bytes of memory\n");
    
        if (pthread_create(&stThreadStruct,
                            NULL,
                            (void *)vTransThread,
                            (void *)pstSockInfo) != 0)
        {
        {
            nStatus = errno;
            printf("pthread_Create failed. Status is %d\n",nStatus);
    
            if(nStatus == EAGAIN)
            {
                printf("Lacks resources\n");
            }
            if(nStatus == EINVAL)
            {
                printf("Invalid parameter\n");
            }
            if(nStatus == ENOMEM)
            {
                printf("Insufficient memory\n");
            }
            if(nStatus == EPERM)
            {
                printf("No permissions\n");
            }
        }
        else
        {
            if (pthread_join((pthread_t)stThreadStruct, NULL) != 0)
                printf ("ERROR - pthread_join with thread failed with error
    = %d\n",
     errno);
        }
    
        pMemory2 = malloc (50 * 1048576);
        if (pMemory2 == NULL)
        {
            printf ("Failed to allocated 50 * 1048576 - 50 * 1MBytes of
    memory\n");
            exit (-1);
        }
        else
            printf ("Allocated 50 * 1048576 - 50 * 1MBytes of memory\n");
    
        free (pMemory);
        free (pMemory1);
        free (pMemory2);
    }

T.R	Title	User	Personal Name	Date	Lines
1544.1	Can you provide the OS version where you are having the problem?	PTHRED::PORTANTE	Peter Portante, DTN 381-2261, (603)881-2261, MS ZKO2-3/Q18	`Wed May 14 1997 15:30`	6
	Ed, Can you provide the OS version where you are having the problem? -Peter
1544.2	And the compile command line?	WTFN::SCALES	Despair is appropriate and inevitable.	`Wed May 14 1997 16:39`	6
	.1> Can you provide the OS version where you are having the problem? And would you post the command line you used to build the program? Webb
1544.3	Version and compile line	RHETT::HALETKY		`Thu May 15 1997 10:34`	6
	OSF version: v4.0b Command line: cc -pthread c.c -o c -ed
1544.4	More info	RHETT::HALETKY		`Thu May 15 1997 10:37`	4
	v4.0a is where we have been able to reproduce it due to the memory requirements. -ed
1544.5	See 1469.*	EDSCLU::GARROD	IBM Interconnect Engineering	`Thu May 15 1997 12:53`	4
	See note 1469.*. We ran into exactly the same issue. Looks like Digital UNIX V4.x has some rather aggressively low memory limits. Dave
1544.6	Faulty diagnosis	WTFN::SCALES	Despair is appropriate and inevitable.	`Thu May 15 1997 14:44`	29
	.5> Looks like Digital UNIX V4.x has some rather aggressively low memory limits. Dave G., I think Ed's point is that after the pthreads failure, which appears to be due to lack of memory, his program is able to allocate another 50Mb of memory. .0> if (pthread_create(&stThreadStruct, .0> NULL, .0> (void )vTransThread, .0> (void )pstSockInfo) != 0) .0> { .0> { .0> nStatus = errno; .0> printf("pthread_Create failed. Status is %d\n",nStatus); Ed, in the standard Pthreads interface (which you have selected by specifying the -pthread switch), the error status code is returned by immediate function value and NOT via errno. So, yes it would seem that your call to pthread_create() is failing, but we don't know the actual reason. .3> OSF version: v4.0b .4> v4.0a is where we have been able to reproduce it due to the memory .4> requirements. You aren't actually building on V4.0b and running on V4.0a, are you? I would assume that that is not supported, and it might not work! Webb
1544.7	Built on same OS, Why failure then?	RHETT::HALETKY		`Fri May 16 1997 17:01`	6
	Hello, No, everything was built on v4.0a. Hrm, so why is there a failure? Perhaps there is a way to find this out? Ed
1544.8	OK, so it has nothing to do with the OS.	WTFN::SCALES	Despair is appropriate and inevitable.	`Fri May 16 1997 17:18`	16
	[Ed, you had entered .7 as a new topic; I took the liberty of making it a reply to this topic, as I presume you had intended.] .7> Hrm, so why is there a failure? It's kind of hard for us to tell, based on what you've posted. .7> Perhaps there is a way to find this out? I'd suggest modifying your test program so that it prints out the actual error status -- that is, the return value from pthread_create(), instead of whatever happens to be in errno at the time. Webb
1544.9	Is this a bug?	RHETT::HALETKY		`Mon May 19 1997 19:10`	25
	Hello, I did as you suggested and got the following output: ./rp Allocated 598723244 bytes of memory Allocated 24 bytes of memory pthread_Create failed. Status is 12 Insufficient memory Allocated 50 * 1048576 - 50 * 1MBytes of memory Which implies that the return code is 12. Now here is how I see it. I have provided the example and if /you/ do not know what is going on, how can I know. You know much more about threads than I do. Some assistance would be appreciated to explain this behavior. If it can not be explained then it must be a bug. Best regards, Ed Haletky Digital CSC
1544.10	works for me...	MSKRAT::ANKAN	Idle words waste time	`Tue May 20 1997 00:33`	16
	Then again, when I run it on my crumme DEC 3000/64MB/686MB swap I get: Allocated 598723244 bytes of memory Allocated 24 bytes of memory Inside of the created thread Allocated 50 * 1048576 - 50 * 1MBytes of memory ...so you have some kind of a tuning problem. I have: maxusers=1024 mapentries=2048 vpagemax=131072 Oh, this is 4.0-464, and I use lazy swap.
1544.11		DCETHD::BUTENHOF	Dave Butenhof, DECthreads	`Tue May 20 1997 08:48`	11
	Yes, the big difference between the memory allocated when you create a thread, and the additional 50Mb you allocate later, is that creating a stack implies creating a guard page -- which requires changing the protection of a page, and therefore splitting of a VM map region. That means that your problem is not "available space" but rather "available map entries" -- either mapentries or vpagemax is too low. Unfortunately, there is no useful way to distinguish various "memory allocation errors" with standard error codes. /dave
1544.12		MSKRAT::ANKAN	Idle words waste time	`Tue May 20 1997 10:50`	6
	Well, couldn't one be a bit proactive, and call mprotect() or some such after the thread creation failed. This would tell you if you're low on vpagemax...