Conference 7.286::atarist

    Well, I played with this, and here's what I found.
    
    I could Malloc up to around 220 (the number seemed to vary across
    system resets) chunks of memory without giving any back.  I could
    Malloc until I was blue in the face as as long as the total number
    of chunks didn't exceed this number--that is, there is a limit on
    the total number of outstanding memory blocks, and not on the number
    of times a process can call Malloc (as the MWC docs seem to insinuate).
    
    What this is saying to me is that there are somewhere around 256
    memory blocks availible for the whole system.  If I wrote a DA that
    sucked up 200 blocks of memory, then my other program would probably
    only be able Malloc 20 blocks.
    
    Does this conflict with anyone else's experience?
    
    Steph

    Since the response to my last question was overwhelming, I will
    ask another question.
    
    I noticed in a comp.sys.atarist message that someone (Moshe Braner,
    if it really matters) mentioned that DAs shouldn't call Malloc once
    they are running.
    
    My question is ``Why not?''
    
    Specifically, my DA will want to, when it gets control, Malloc all
    availible memory for use as a buffer.  Then, when the user exits,
    it will give it all back.
    
    This sounds kosher to me, as I suspect the reason for not allowing
    DA's to Malloc is that if they keep the memory, the main application
    will not be able to return memory in the order in which it was
    Malloc'd.
    
    Does anybody know differently?
    
    Thanks,
       Steph
    

    
    As far as I know there's no technical, in terms of bugs, reason.
    But DA are loaded at boot, when loaded there are executed. This
    isn't noticed because after initialization the DA's all wait with
    EVNT_.... calls. 
    But they are running. So if you grab all available memory during
    initialization there will be none left for other applications. 
    I read also somewhere that AES will only load accesoires if there
    will be at least 128 Kb free afterwards for other application. So
    maybe this might give you also problems. A trick to overcome the
    last is to call an EVNT_TIMER and wait till everything is over and
    then grab memory.
    
    I used this last trick for "auto" starting a GEM application. I
    say GEM with a DA to do a double click 3 sec. after boot. At that
    time desktop is visible, AES initialized. You only have to put the
    program under the mousepointer, wich is in center of the screen,
    save it in DESKTOP.INF and here you go. Every time you boot with
    that DA your wordproccesor will automaticly start.
                                          
    If you experiment a little I would like to hear the results.
                                                      
    Renato
    

    Actually, I'm not planning to grab the memory at initialization,
    but rather after I get an event.
    
    I'm going to be using it as a buffer for incoming or outgoing MIDI
    data, so I will get the memory, do the transfer, and then release
    the memory.
    
    I figure that if I take ALL the memory, then no other applications
    can take any, and I am therefore guaranteed that the chunk which
    I want to use and return will be the last chunk Malloced.
    
    I have to get my algorithms and interface worked out, and then I'll
    work on the DA problem, though.
    
    Let you know what happens.
    
       Steph

>    Actually, I'm not planning to grab the memory at initialization,
>    but rather after I get an event.
> 
>    I'm going to be using it as a buffer for incoming or outgoing MIDI
>    data, so I will get the memory, do the transfer, and then release
>    the memory.

This may be a disappointment, but in a DA, you can only use the Malloc
function in the beginning of the program and even only before you issue
any GEM-call at all ! I wanted to do something simular for my
application, but this didn't work. What I suggest you do is allocate a
reasonable buffer at the start of your program and write your 'own
Malloc' to use this pool.
   
Hope this will help you on the way !

Cor

    When you say ``It didn't work'' what was the symptom?
    
    Did you try anything lower-level than malloc?  For example, I was
    considering examining the free list (or whatever the heck it is
    called) and just using a chunk of unallocated memory without explicitly
    allocating it.
    
    I can't imagine that there is no way around this problem in my
    situation.
    
    Thanks,
      Steph

    Re: DAs Mallocing.
    
    I've tried this, and it works fine.  My DA does a Malloc(-1), performs
    its transfer, and then does a Free when complete.  No memory seems
    to get lost, and I haven't found a program which crashes as a result
    of my unethical behaviour.
    
    Steph

   Dear Fellow Atari Enthusiasts & Programmers,
   
   If I may continue the malloc() discussion a bit further and specific...
    
   Here's the problem:  I'm using MWC to compile these routines (actually
   larger copies of them; i.e., there is more code and this portion is
   the ONLY portion of concern here) and all is well.  Assume that
   everything is accounted for because the whole program DOES work.  The
   curious thing is this:  If I do *NOT* exercise this portion of code,
   the problem does not occur and it is this:  Upon program exit, all of
   GEM seems to be quite sluggish.  Then, when I attempt to manipulate a
   window (do ANYthing with the mouse, really), it takes a few times, and
   then the environment gets out of "molasses mode".  This behaviour is a
   REAL puzzler!  If the HISTORY function has been used, then "nohist()"
   will attempt to clear out that memory.  As the comment says, this is
   done at program exit... because you shouldn't rely on the user to
   clean up after himself.  Other portions of code that use  malloc()
   do *NOT* cause the same behaviour upon program exit!
   
   The only GOOD suggestion I've recieved so far, is to get the code and
   create my own  malloc()  and  free()  to be SURE things are working
   right!  Is this a GEM problem, a MWC problem, or my own?

   The following code fragments illustrate WHERE  malloc()  is being
   called.  I tried changing "nohist" to release memory in ASCending
   order, instead of descending, and it made very little difference.
   Below,  SAVE()  is called to fill  HIST[].  Then, the NOHIST() is
   called at program exit (unless the user clears it).  The display of
   HIST[] works just fine and isn't relevant here.
   
   Any ideas, suggestions or comments are greatly appreciated!
   
   Many thanks for your time!
   
   Sincerely,
   
   Bruce
   
   /*********************************************************************
    * CLEAR_HIST ::: Clear the history buffer
    */
   clear_hist()
   {
      if (hc>0) {
         nohist();
         hc = 0;
         form_alert(1,"[0][ |History Buffer cleared!| ][ All Gone ]");
      }
      else form_alert(1,"[2][ |No Data in History Buffer!| ][ REALLY? ]");
   }
   
   nohist() {		/* Setup this way so we can call from MAIN at exit */
         for (i=hc; i>0; i--) free(hist[i-1]);
   }
   
   /************************************************************************
    * SAVE ::: Save filename, starting sector, error and some text to HIST[n]
    *		Compresses leading spaces & removes bogus characters
    *		Insert error text if err<0
    */
   save(ssec,err,data)
   int ssec;
   long err;
   char *data;
   {
      int c=0,p,n;
      char temp[80],buff[50];
   
         n = strlen(data);			/* Find first text char */
   
         if (hc<500) {
   	 for (i=0; i<50; i++) temp[i]='\0';
   	 p = strlen(filenm) - 3;		/* Strip off "device:\" */
   	 for (i=0; i<p; i++) temp[i]=filenm[i+3];
   	 for (n=i; n<12; n++) strcat(temp," ");
   	 sprintf(buff,"   %5d  %3D  ", ssec,err);
   	 strcat(temp,buff);
   
   	 if (err<0)				/* Translate error */
   	    sprintf(buff,"\033b1>>>>>  %s\033b3",errcod[err*-1]);
   	 else {
   	    for (i=0; i<n; i++)
   	       if (data[i]>=' ') break;
   	    for (n=p=i; n<i+50; n++) {
   	       if (data[p]<' ') data[p]=' ';
   	       while (data[p]==' ' && data[p+1]==' ') p++;
   	       buff[c++]=data[p++];
   	    }
   	 }
   	 strncat(temp,buff,50);
   	 hist[hc] = malloc(strlen(temp)+1);
   	 strcpy(hist[hc++],temp);
         }
   }
   

>      char temp[80],buff[50];
If "filename" can contain any arbitrary path (as apposed to just the file
name) then "temp[]" should realy be 117 (directories can be up to 8 deep)
plus the maximum length of your formatted error messages (which I presume is
50 judging by your strncat() call).

>   	 hist[hc] = malloc(strlen(temp)+1);
>   	 strcpy(hist[hc++],temp);
The return value of malloc() should be checked to insure that really were
allocated a chunk of memory. malloc() returns 0L if memory was not allocated.

I don't see anything wrong with the malloc() and free() useage. MWC supposedly
does not use the system Malloc() call so you should not be seeing problems
caused by bugs in the routine.

I occasionaly have seen programs which make mouse button click response be
real slow for a short time after they have run. Though I have no idea what
could cause that, it is as if the program tapped into the vector which handles
mouse clicks and forgot to restore the vector.

	Ray

    Ray,
       Thanks for the tips on making the program 'smarter'!  As for the
    mouse clicks, I do *not* intercept any vectors.  I use the standard GEM
    calls; graf_mouse(M_ON,0L), etc and the  v_hidemouse/showmouse (I think)
    so, maybe I should just exercise the mouse a bit (with calls) before
    exiting the program.  If there are any other ideas out there, please
    contribute... look for a "Lost GEMs" note to be posted soon!
    
    Bruce

    
    I am trying to diagnose a memory allocation bug that manifests itself when
    developing software using Sozobon C.
    
    The problem occurs after several runs of a compile/assemble/link job
    run with BATCHMON, the batch monitor supplied in the kit. In one
    particular case it failed consistently at the 13th job, but it varies
    according to what is being compiled. After a re-boot, the failed job
    will run OK. That seems to clear Sozobon of responsibility.
    
    The symptom is that the compile will be successful, but BATCHMON
    complains of "problems running JAS.TTP". BATCHMON doesn't trap the
    error, but blunders on only to fail the loader, LD, with the same
    problem.
    
    Trying the same job using a DOS shell, gives the same symptom. That seems
    to eliminate BATCHMON as the culprit.
    
    I doctored the DOS shell to record how much free memory there is before
    and after running any command and to tell me if these differed. Sure
    enough a large chunk of memory goes missing when the problem occurs.
    
    The Sozobon C compiler, HCC, uses the Dlibs12 library memory allocation
    functions. The comments in the sources hint that the algorithm used in
    Dlibs12 is designed to overcome a GEMDOS Malloc problem. Topic 303.*
    discusses a similar-sounding problem, but isn't conclusive)
    
    It looks like a problem with Dlibs12 malloc and/or GEMDOS Malloc.
    
    I tried writing a test harness to investigate further, but
    a) I can't reproduce the symptom
    b) Things get confused by 'hidden' mallocs e.g. triggered by using printf
    c) I don't (yet) have enough familiarity with 68000 machine code or
    adequate debugging tools to be able to debug at low level.
    
    Q1 Is this a known problem in Sozobon C?
    
    Q2 What happens when a program terminates - should memory allocated to
    it by GEMDOS be de-allocated automatically? (i.e. even if the program or
    Dlibs12 didn't de-allocate properly, should GEMDOS tidy up after it?)
    
    Q3 The GEMDOS Malloc problem - can any one tell me *exactly* what the
    problem is? What symptoms it exhibits ? Is it cumulative across
    different program runs?
    
    Any help or hints would be much appreciated.
    
    btw: I have double-checked that I am not *really* running short of
    memory!
    
    Brian fed_up_with_re-booting Gilbert

This does not sound like the Malloc() bug. It does sound like one of the tools
you are using (or your program) is not dealloc()'ing memory before exiting.

The OS does NOT dealloc() memory for you. If you malloc() memory then you must
dealloc().

	Ray

    Thanks.
    
    I am still trying to get a consistently reproducible set of symptoms to
    work on. Whenever I investigate a possible cause, the symptoms disappear!
    
    All constructive suggestions invited.
    
    Brian

    
    RE:.12
    
    I have to take issue with you, Ray.
    
    Test programs show that GEMDOS *does* tidy up after an application.
    
    I wrote two versions: one to use Dlibs malloc() and a second to use
    GEMDOS Malloc().
    
    In both cases, the program repeatedly requests memory in 32K chunks
    until there is none left, and exits without deallocating any memory.
    After running the test program the original amount of free memory was
    still intact in both cases.
    
    Examination of the sources of Dlibs and the SOZOBON start-up code shows
    no evidence of any memory tidy-up code, so it must be GEMDOS doing the job.
    
    Of course, this only proves INTENT; it may not work correctly in every
    case, so it could still be the cause of the problem.
    
    Brian

Yesm I stand corrected. Please except my appologies. I recently read some old
articles from someone at Atari about Malloc which did state that upon program
exit TOS does deallocate memory and close files for the program.

I still most definately have seen programs which "eat" memory, though at the
moment I don't know how or why (I thought I knw :-).

	Ray

    OK, let's assume the problem is in the application.
    
    How might it eat memory, yet beat the GEMDOS tidy-up?
    
    Hypothesis:-
    
    The documentation on the GETMPB GEMDOS call states that unallocated
    memory is chained in a linked list.
    
    If, through an application bug, a header in the unallocated chain was
    overwritten with 00's, the chain would prematurely end and memory could 
    'disappear' from view.
    
    To test this out I'd need to be able to check the integrity of the
    GEMDOS memory management data structures before and after running the
    suspect application.
    
    Now back to an earlier theme: GETMPB should give me access to said
    data structures, but doesn't work properly.
    
    A note in OLD_ATARIST discussed this point, and quoted some addresses
    to use, but unfortunately these appear to be GEMDOS version dependent.
    Does anyone know what the values are for later GEMDOS releases?
    
    Brian
    

    This is what I found in German magazines about the Malloc-bug in TOS
    upto version 1.2 (Blitter-TOS):
    
    >OK, let's assume the problem is in the application.
    >How might it eat memory, yet beat the GEMDOS tidy-up?
    Gemdos seems to be able to combine freed memory only if it's freed in
    the reverse order (like a stack) it was allocated. If you don't free it
    yourself, Gemdos frees all memory for an exiting process in this way.
    But if you free allocated memory in a mixed order, they are chopped
    forever (or until reset, whatever comes first).
    Could anyone be so kind as to test this hypothesis? I'm not able to,
    because I'm using TOS 1.4 :-)
    
    >Now back to an earlier theme: GETMPB should give me access to said
    >data structures, but doesn't work properly.
    Wrong. The docs say something like this, but what really happens is:
    The data structures are initialised and GETMPB delivers a pointer to
    these initialised structure. It should better been named INITMPB or
    something like that.
    
    >A note in OLD_ATARIST discussed this point, and quoted some addresses
    >to use, but unfortunately these appear to be GEMDOS version dependent.
    >Does anyone know what the values are for later GEMDOS releases?
    Since TOS 1.2 the OS-Header (address of that animal living at _sysbase)
    contains a pointer to the memory list.
    
    Jens

    re .-1
    
    Thanks for your response Jens. All contributions are welcome, I have
    run out of inspiration!
    
    Unfortunately, some of the information does not appear to be accurate
    - no reflection on you Jens, I understand you are relaying information
    from another source.
    
    On the three points you raise:-
    
    1) I wrote a test program which requested 32K chunks of memory until
    all is used up. The program then frees every alternate block from the
    list and exits. This should leave memory well and truly fragmented if
    GEMDOS cannot recombine blocks not released in reverse order.
    
    After exit, the memory is all intact.
    
    2) GETMPB - can you/anyone explain exactly in what circumstances you
    would use it?
    
    3) On my system (GEMDOS V1.2, 22-Apr-1987) _sysbase (at 0x4F2) contains
    0xFC0000 i.e points to the start of the ROM.
    The header of the ROM does not contain anything of relevance to Malloc.
    
    Brian

    re .-1
    
    I'm sorry for not being able to give more info. My quotations were from
    memory, so I'll try to find them again at home and see for any
    differences.
    
    >2) GETMPB - can you/anyone explain exactly in what circumstances you
    >would use it?
    As being a user: never. This function should be called exactly once
    during the initialization of the OS, and that is what is done by
    GEMDOS. Afterwards this function should be called after restarts only,
    that is whenever the OS initialzes itself again.
    
    >3) On my system (GEMDOS V1.2, 22-Apr-1987) _sysbase (at 0x4F2) contains
    >0xFC0000 i.e points to the start of the ROM.
    This is the address of a struct _os_header, which contains a pointer to
    the list of memory chunks GEMDOS knows about. This pointer is one of
    the three new ones introduced with V1.2, but documented since 1.4 only.
    I'll look up more info about that at home.
    
    Jens
    

    

    
    After much single-stepping of the paths through GEMDOS, I have
    discovered that the address of the Malloc data structure is 0x7E8E.
    NB this is version-specific - mine is a UK Version 1.2 ROM system.
    The data structures it points to are as described for the return values
    of the GETMPB BIOS call.
    
    I wrote a new command for my DOS shell so that I can list out the free
    and allocated chains before and after the bug occurs. (Once the bug 
    happens you can't load another program or return to the desktop because
    there is virtually no free memory.)
    
    When the bug occurs the start of allocated chain shows three memory blocks
    left allocated to the now-terminated SOZOBON assembler. Two of the blocks
    are small (44 and 500-ish bytes) and the third is 500K+ bytes.
    
    By the way, it now looks like the problem is associated with
    termination of the assembler, not loading it as I previously thought. I
    am still assuming that because the problem is not repeatable, it isn't
    a bug in JAS itself.
    
    Brian

    
    RE .11 and several others (memory leakage with SOZOBON C):
    
    I upgraded to TOS 1.4 several weeks ago. The memory problem has gone
    away.
    
    Brian.