[Search for users] [Overall Top Noters] [List of all Conferences] [Download this site]

Conference turris::c_plus_plus

Title:C++
Notice:Read 1.* and use keywords (e.g. SHOW KEY/FULL KIT_CXX_VAX_VMS)
Moderator:DECCXX::AMARTIN
Created:Fri Nov 06 1987
Last Modified:Thu Jun 05 1997
Last Successful Update:Fri Jun 06 1997
Number of topics:3604
Total number of notes:18242

3498.0. "setenv leaking memory on Digital UNIX ?" by HERO::COUNOTTE () Tue Mar 18 1997 06:53

I'm running a complex program which does this :

Run 24x7 a week,
Starts child processes every 10 minutes.


Before starting a child process using a fork/exec combination, we
set three environment variables using setenv().

It appears that the setenv() call is leaking memory each time it
is called ! Is there a work-around ?

If I use putenv() will it fix the problem ?


Using third degree to detect memory leak, I get the following
results by running the program during 30 seconds, which starts
two child processes :

------------------------------------------------------------------------
Searching for new leaks in heap after program exit


A total of 135 bytes in 6 leaks were found:

49 bytes in 2 leaks (including 2 super leaks) created at:
    malloc                         mm_server
    __setenv                       mm_server
    MMS_Entity::start_scan(const char*) mm_server, mms_entity.cxx, line 328
    MMS_LocalServer::req_get_scan(VSocket*, NetList*)
                                       mm_server, mms_localserver.cxx, line 703
    start_scan                     mm_server, mms_schedul.cxx, line 241
    cma__thread_base               mm_server

46 bytes in 2 leaks (including 2 super leaks) created at:
    malloc                         mm_server
    __setenv                       mm_server
    MMS_Entity::start_scan(const char*) mm_server, mms_entity.cxx, line 326
    MMS_LocalServer::req_get_scan(VSocket*, NetList*)
                                       mm_server, mms_localserver.cxx, line 703
    start_scan                     mm_server, mms_schedul.cxx, line 241
    cma__thread_base               mm_server

40 bytes in 2 leaks (including 2 super leaks) created at:
    malloc                         mm_server
    __setenv                       mm_server
    MMS_Entity::start_scan(const char*) mm_server, mms_entity.cxx, line 327
    MMS_LocalServer::req_get_scan(VSocket*, NetList*)
                                       mm_server, mms_localserver.cxx, line 703
    start_scan                     mm_server, mms_schedul.cxx, line 241
    cma__thread_base               mm_server

------------------------------------------------------------------------
------------------------------------------------------------------------
                memory layout at program exit
                           heap     351392 bytes [0x140032000-0x140087ca0]
                          stack      20272 bytes [0x11fffb0d0-0x120000000]
                 mm_server data     203616 bytes [0x140000000-0x140031b60]
                 mm_server text    2621440 bytes [0x120000000-0x120280000]




Then after 90 seconds, starting three child processes :

------------------------------------------------------------------------
Searching for new leaks in heap after program exit


A total of 204 bytes in 9 leaks were found:

75 bytes in 3 leaks (including 3 super leaks) created at:
    malloc                         mm_server
    __setenv                       mm_server
    MMS_Entity::start_scan(const char*) mm_server, mms_entity.cxx, line 328
    MMS_LocalServer::req_get_scan(VSocket*, NetList*)
                                       mm_server, mms_localserver.cxx, line 703
    start_scan                     mm_server, mms_schedul.cxx, line 241
    cma__thread_base               mm_server

69 bytes in 3 leaks (including 3 super leaks) created at:
    malloc                         mm_server
    __setenv                       mm_server
    MMS_Entity::start_scan(const char*) mm_server, mms_entity.cxx, line 326
    MMS_LocalServer::req_get_scan(VSocket*, NetList*)
                                       mm_server, mms_localserver.cxx, line 703
    start_scan                     mm_server, mms_schedul.cxx, line 241
    cma__thread_base               mm_server

60 bytes in 3 leaks (including 3 super leaks) created at:
    malloc                         mm_server
    __setenv                       mm_server
    MMS_Entity::start_scan(const char*) mm_server, mms_entity.cxx, line 327
    MMS_LocalServer::req_get_scan(VSocket*, NetList*)
                                       mm_server, mms_localserver.cxx, line 703
    start_scan                     mm_server, mms_schedul.cxx, line 241
    cma__thread_base               mm_server

------------------------------------------------------------------------
------------------------------------------------------------------------
                memory layout at program exit
                           heap     408736 bytes [0x140032000-0x140095ca0]
                          stack      20272 bytes [0x11fffb0d0-0x120000000]
                 mm_server data     203616 bytes [0x140000000-0x140031b60]
                 mm_server text    2621440 bytes [0x120000000-0x120280000]
T.RTitleUserPersonal
Name
DateLines
3498.1Using unsetenv doesn't fix the leakHERO::COUNOTTETue Mar 18 1997 06:5511
By the way,

after the child process is started, we call unsetenv
to remove these variables.

Just in case :-).

Thanks in advance for your help.

Cedric.
3498.2DECC::FOLTANTue Mar 18 1997 09:247
To my knowledge I have not heard of other customers reporting 
problems with setenv.  My advice is to cross post this to the 
Digital UNIX notes conference or report it as an OSF_QAR.
Login to GORGE or use http://webster.zk3.dec.com/webqar.

Lois Foltan
DEC C++ Development
3498.3NPSS::GLASERSteve Glaser DTN 226-7212 LKG1-2/W6 (G17)Tue Mar 18 1997 09:2522
    setenv has ALWAYS leaked memory on almost every Unix platform.
    
    The environment is initially set up by the operating system's exec
    mechanism.  setenv creates a copy.
    
    There is nothing in C that prevents you getting a pointer to an
    enviromnent variable and keeping it for a LONG time.  Thus setenv can't
    release the old copy since there might be a pointer to something in it
    (even if setenv could distinguish between the environment created at
    the base of the stack by exec and the copies created by earlier setenv
    calls it shouldn't release the copies).
    
    unsetenv creates yet another copy and just makes things worse.  Don't
    use it if you're going to do another setenv anyway.
    
    Unix programs also don't have to use setenv anyway.  Whatever pointer
    value is in the global variable _environ is used for getenv or exec. 
    During exec, the new address space gets a copy of the environment at
    the base of the new stack so the exact addresses pointed to by _environ
    don't matter.
    
    Steveg
3498.4NPSS::GLASERSteve Glaser DTN 226-7212 LKG1-2/W6 (G17)Tue Mar 18 1997 09:2911
    Suggestion to .0
    
    Do the setenv after the fork but before the exec.  That way you get a
    memory leak in the about to be destroyed address space where it doesn't
    matter.
    
    This is the traditional "solution".
    
    It also keeps the enviroment of the parent process unchanged.
    
    Steveg
3498.5VAXCPU::michaudJeff Michaud - ObjectBrokerTue Mar 18 1997 10:456
>     Unix programs also don't have to use setenv anyway.  Whatever pointer
>     value is in the global variable _environ is used for getenv or exec. 

	And if you use execve or execle then you pass in an array of
	your own that will be used as the environment, instead of
	using the array pointed to by the global variable environ.
3498.6putenv(3) ?DECC::SULLIVANJeff SullivanTue Mar 18 1997 11:1581
Would using putenv (see man putenv(3)) be useful in this case. That system call
actually changes the user's environment. This point is not really described in
the man page, so I reported OSF_QAR 50146 (see below).

-Jeff


<<<<< Entered by WEB/QAR [sullivan] on - Fri Dec  6 16:52:03 1996 >>>>>

 The putenv(3) man page does not specify that changing the string
 will change the environment. Although this behavior is similar to
 that of other UNIX implementations, this is a subtle point that
 needs to be described.

 The man page may lead one to believe that a copy of the string is
 stored in the environ after the putenv call, when it actually just
 stores a pointer to the original string. Changing the original string
 contents will therefore change the environment.

 Below is a simple program that demonstrates that case:

 % cat putenv_test.c

 #include <stdlib.h>
 #include <stdio.h>

 void print_env(char *);

 main() {
   char tmpstring[20];

   print_env("HOME set by default, TEMPENV not set:");

   strcpy(tmpstring, "HOME=/usr/newhome");
   putenv(tmpstring);
   print_env("Reset HOME, TEMPENV not set:");

   strcpy(tmpstring, "TEMPENV=tempenv");
   putenv(tmpstring);
   print_env("Set TEMPENV:");
 }

 void print_env(char *str) {
   char *envptr;

   printf("\n%s\n", str);

   envptr = getenv("HOME");
   if (envptr)
     printf("HOME=%s\n", envptr);
   else
     printf("HOME not set\n");

   envptr = getenv("TEMPENV");
   if (envptr) 
     printf("TEMPENV=%s\n", envptr);
   else
     printf("TEMPENV not set\n");
 }

 % cc putenv_test.c; a.out

 HOME set by default, TEMPENV not set:
 HOME=/home/jps
 TEMPENV not set

 Reset HOME, TEMPENV not set:
 HOME=/usr/newhome
 TEMPENV not set

 Set TEMPENV:
 HOME not set
 TEMPENV=tempenv


 This misunderstanding was the cause of a problem reported
 against /usr/sbon/dop today by me.



 <<<<<< End of WEB/QAR enter >>>>>>
3498.7Solution to setenv leak...HERO::COUNOTTEWed Mar 19 1997 05:0273
Hi all,

thanks for all your useful replies... Here follows a solution to the
setenv memory leak problem. Just replace it by the following by
defining the following in your include files :

#define getenv(s1) osf_getenv(s1)
#define setenv(s1,s2) osf_setenv(s1,s2)
#define unsetenv(s1) osf_setenv(s1,NULL)

Then in a misc.cxx (attached at the bottom) file.

As you can see, nothing very complex, just using the fact that putenv will
always loose the old address when overriding an env.var. .

Though, you may still get memory fragmentation...

To improve the getenv behavior, one should override its definition to
return NULL when the env.var. is empty.


int osf_getenv( char *name )
{
	env = getenv( name );
	if ( env && !*env )
		return NULL;
	return env;
}


int osf_setenv( char *name, char *value )
{
  char *buffer, *t, *tmp;

        if ( !name )
                return -1;

	//
	//	Keep old env var if any.
	//
        t = getenv( name );
	if ( t )
		t -= strlen(name) + 1;

	//
	//	Allocate new buffer...
	//
        buffer = new char[ strlen(name) + ((value)?strlen(value):0) + 2 ];

	//
	//	Build env buffer
	//
	tmp = buffer;
	while( *name )
		*tmp++ = *name++;
	*tmp++ = '=';
	if ( value )
		while( *value )
			*tmp++ = *value++;

	//
	//	Set environment variable
	//
        putenv( buffer );

	//
	//	Delete old obsolete buffer
	//
        if ( t )
                delete [] t;
}

3498.8When you need to do it this way...HERO::COUNOTTEWed Mar 19 1997 05:4710
By the way (again !),


I needed to define the environment variables before I perform the fork,
simply because the fork is in a specific class used to start process
on UNIX, VMS and NT...

Regards,
Cedric.
3498.9NPSS::GLASERSteve Glaser DTN 226-7212 LKG1-2/W6 (G17)Wed Mar 19 1997 12:596
    Te code in .7 will fail big time if the environment variable exists
    when the program starts up.
    
    In that case, the delete at the end will try to free something that was
    not allocated by malloc and will corrupt other parts of the address
    space (at least nearby environment variables).
3498.10So true !HERO::COUNOTTEThu Mar 20 1997 04:5714
Re .9

That's true, but that was not much of my concern, because I only set new environment
variable. Well, maybe not in the future...

Damned ! If I need to, do you have a suggestion so that I don't free it the first time ?

The only thing I can see so far, is to keep a global list of environment variable 
created by the program... What a pain !


Regards,
Cedric.
3498.11NPSS::GLASERSteve Glaser DTN 226-7212 LKG1-2/W6 (G17)Thu Mar 20 1997 13:2817
    On most Unix systems the environment is created in a contiguous region
    at the base of the stack.  If you walked the environment and captured
    the address range involved, you could use address comparison to decide
    whether to do the delete[].
    
    If you're careful, this code could be portable as long as you assume
    that the environment is contiguous in address space (in particular
    nothing returned by new will ever be in the address range).  If
    something other than your code did a putenv before you got there, this
    assumption would not be true.
    
    Alternatively, you could create a duplicate copy of the envirnment to
    start and ignore the other one.  That way everything would be under
    your control and any delete calls would be fine since you allocted the
    memory in question.
    
    Steveg