On Tue, 29 Jan 2008 12:47:27 +0100, jacob navia wrote:

> 1:
> It is not possible to check EVERY malloc result within complex software.

Says who? There are a limited number of places memory is allocated, thus
a limited number of places checks need to be put in.

> 3:
> A solution like the one proposed by Mr McLean (aborting) is not possible
> for software quality reasons. The program must decide if it is possible
> to just abort() or not.

Oh, this solution is always *possible*, but I'd be very much annoyed with
any developer stupid enough to adopt this as a default strategy in any
application I actually use.


> Solution:
>
> 1) At program start, allocate a big buffer that is not used elsewhere in
> the program.

Note that in at least one app, I do exactly this - and I *expect* that
the allocation will fail on a pretty regular basis. Of course, if I'm
stupid enough to be using something like Malcolm's xmalloc, it's too
late, the app has already bombed by the time I would normally just detect
the NULL and reduce the size of the request.

'Sides, it's a messy approach, one which ties up resources you don't
really need, resources best left alone for other things. Just because it
makes *your* app easier to write doesn't mean it's a good idea - ponder a
word processor grabbing 128MB for document space, with no documents
loaded, on an otherwise busy machine.

1:
It is not possible to check EVERY malloc result within complex software.

2:
The reasonable solution (use a garbage collector) is not possible for
whatever reasons.

3:
A solution like the one proposed by Mr McLean (aborting) is not
possible for software quality reasons. The program must decide
if it is possible to just abort() or not.

Solution:

1) At program start, allocate a big buffer that is not used
elsewhere in the program. This big buffer will be freed when
a memory exhaustion situation arises, to give enough memory
to the error reporting routines to close files, or otherwise
do housekeeping chores.

2) xmalloc()

static int (*mallocfailedHandler)(int);
void *xmalloc(size_t nbytes)
{
restart:
void *r = malloc(nbytes);
if (r)
return r;
// Memory exhaustion situation.
// Release some memory to the malloc/free system.
if (BigUnusedBuffer)
free(BigUnusedBuffer);
BigUnusedBuffer = NULL;
if (mallocfailedHandler == NULL) {
// The handler has not been set. This means
// this application does not care about this
// situation. We exit.
fprintf(stderr,
"Allocation failure of %u bytes\n",
nbytes);
fprintf(stderr,"Program exit\n");
exit(EXIT_FAILURE);
}
// The malloc handler has been set. Call it.
if (mallocfailedHandler(nbytes)) {
goto restart;
}
// The handler failed to solve the problem.
// Exit without any messages.
exit(EXIT_FAILURE);
}

4:
Using the above solution the application can abort if needed, or
make a long jump to a recovery point, where the program can continue.

The recovery handler is supposed to free memory, and reallocate the
BigUnusedBuffer, that has been set to NULL;


--
jacob navia
jacob at jacob point remcomp point fr
logiciels/informatique
http://www.cs.virginia.edu/~lcc-win32

In article <fnn3og$o40$1@aioe.org>, jacob navia <jacob@nospam.org> wrote:

>It is not possible to check EVERY malloc result within complex software.

I assume you're just setting up a scenario here rather than making
this claim yourself.

>1) At program start, allocate a big buffer that is not used
>elsewhere in the program. This big buffer will be freed when
>a memory exhaustion situation arises, to give enough memory
>to the error reporting routines to close files, or otherwise
>do housekeeping chores.

This is not guaranteed to with all malloc() implementations.
Some use different areas of memory for different sized blocks, and may
not have code to re-purpose the freed block. Or they may free the
memory back to the OS, in which case it may get gobbled up by
something else before they have the chance to re-use it.

-- Richard
--
:wq

On Jan 29, 1:47 pm, jacob navia <ja...@nospam.com> wrote:
> 1:
> It is not possible to check EVERY malloc result within complex software.
It is, with proper design it becomes easier too.

> 3:
> A solution like the one proposed by Mr McLean (aborting) is not
> possible for software quality reasons. The program must decide
> if it is possible to just abort() or not.
That was not a solution.

> Solution:
>
> 1) At program start, allocate a big buffer that is not used
> elsewhere in the program. This big buffer will be freed when
> a memory exhaustion situation arises, to give enough memory
> to the error reporting routines to close files, or otherwise
> do housekeeping chores.
That does not guarantee anything, it depends on the OS, and the
implementation of free/malloc.
Also it makes your program slower, and what if the big buffer
allocation fails?
Would you exit because your program failed to allocate resources it
would not use?
What if no allocation fails? The buffer is only a waste of resources.

> 2) xmalloc()
>
> static int (*mallocfailedHandler)(int);
> void *xmalloc(size_t nbytes)
> {
> restart:
> void *r = malloc(nbytes);
> if (r)
> return r;
> // Memory exhaustion situation.
> // Release some memory to the malloc/free system.
> if (BigUnusedBuffer)
> free(BigUnusedBuffer);
free it anyway, free(NULL); does nothing.

> BigUnusedBuffer = NULL;
> if (mallocfailedHandler == NULL) {
> // The handler has not been set. This means
> // this application does not care about this
> // situation. We exit.
> fprintf(stderr,
> "Allocation failure of %u bytes\n",
> nbytes);
Undefined behavior, you pass size_t where a variadic function expects
unsigned int.

> fprintf(stderr,"Program exit\n");
> exit(EXIT_FAILURE);
> }
What about previous allocations that have been done with xmalloc?
memory leak.

> // The malloc handler has been set. Call it.
> if (mallocfailedHandler(nbytes)) {
> goto restart;
> }
> // The handler failed to solve the problem.
> // Exit without any messages.
> exit(EXIT_FAILURE);
Memory leak because you don't give the user a chance to free previous
allocations

> The recovery handler is supposed to free memory, and reallocate the
> BigUnusedBuffer, that has been set to NULL;
So if the allocation for BigUnusedBuffer succeeds but the allocation
after the callback fails, we will enter a loop of free-ing/allocating
the big buffer, great.

mr Jacob, I suggest you read a book or two on program structure &
designing.

On 29 Jan 2008 at 11:47, jacob navia wrote:
> 1:
> It is not possible to check EVERY malloc result within complex software.
>
> 2:
> The reasonable solution (use a garbage collector) is not possible for
> whatever reasons.
>
> 3:
> A solution like the one proposed by Mr McLean (aborting) is not
> possible for software quality reasons. The program must decide
> if it is possible to just abort() or not.
>
> Solution:
>
> 1) At program start, allocate a big buffer that is not used
> elsewhere in the program. This big buffer will be freed when
> a memory exhaustion situation arises, to give enough memory
> to the error reporting routines to close files, or otherwise
> do housekeeping chores.
>
> 2) xmalloc()

Why xmalloc? There's already a mechanism for dealing with such problems,
viz signals.

#include <signal.h>

void alloc_failure_handle(int x)
{
fputs("Dis baby gon crash and burn!\n", stderr);
exit(EXIT_FAILURE);
/* or insert your free-buffer-and-clean-up stuff if you prefer */
}

main()
{
signal(SIGSEGV, alloc_failure_handle);
/* ... */
}

"jacob navia" <jacob@nospam.com> wrote in message
news:fnn3og$o40$1@aioe.org...
> 1:
> It is not possible to check EVERY malloc result within complex software.
>

<snip>

>
> 4:
> Using the above solution the application can abort if needed, or
> make a long jump to a recovery point, where the program can continue.
>
> The recovery handler is supposed to free memory, and reallocate the
> BigUnusedBuffer, that has been set to NULL;
>

5:
add exception handling (ie, as a library feature, or, possibly as a compiler
extension).

this way, we can throw, and see if the app has any nifty ideas...
I propose implementing something combining both signal handling and
unwinding features.

sza=(1<<31)-1;
while(1)
{
i=exBegin();
if(!i)
{
//try...
pa=malloc_throw(sza);
}else if(i==EX_NO_MEM)
{
//catch...
sza=(sza/3)*2;
continue;
}else
{
//cleanup...
exRelay(i); //relay to next handler
}
break;
}


or something...


(ok, nevermind that this idea is stupid, as directly handling the NULL
pointer is simpler).

sza=(1<<31)-1;
pa=malloc(sza);
while(!pa)
{
sza=(sza/3)*2;
pa=malloc(sza);
}


>
> --
> jacob navia
> jacob at jacob point remcomp point fr
> logiciels/informatique
> http://www.cs.virginia.edu/~lcc-win32

On Jan 29, 2:26 pm, vipps...@gmail.com wrote:
> On Jan 29, 1:47 pm, jacob navia <ja...@nospam.com> wrote:> 1:
> > It is not possible to check EVERY malloc result within complex software.
> mr Jacob, I suggest you read a book or two on program structure &
> designing.

Here's a solution I propose:

Have a linked list for each allocation done and three functions,
--
xmalloc_start(); /* initializes a global linked list for future
allocations */
if(atexit(xmalloc_end)) /* handle error */
/* ... */
p = xmalloc(123, 0); /* false = 0, if the allocation fails, xmalloc
exits. */
s = xmalloc(123, 1); /* true = !0, if the allocation fails, xmalloc
returns NULL */
--

In article <slrnfpu71u.u2u.nospam@nospam.invalid>,
CJ <cj@nospam.invalid> wrote:

>Why xmalloc? There's already a mechanism for dealing with such problems,
>viz signals.

> signal(SIGSEGV, alloc_failure_handle);

An ignored malloc() failure is not guaranteed to lead to a SIGSEGV.
It may just result in memory corruption.

-- Richard

--
:wq

jacob navia <jacob@nospam.com> wrote:

> 1:
> It is not possible to check EVERY malloc result within complex software.

Bollocks from the start. Well done.

Richard

CJ wrote:

> On 29 Jan 2008 at 11:47, jacob navia wrote:
>> 1:
>> It is not possible to check EVERY malloc result within complex
>> software.
>>
>> 2:
>> The reasonable solution (use a garbage collector) is not possible for
>> whatever reasons.
>>
>> 3:
>> A solution like the one proposed by Mr McLean (aborting) is not
>> possible for software quality reasons. The program must decide
>> if it is possible to just abort() or not.
>>
>> Solution:
>>
>> 1) At program start, allocate a big buffer that is not used
>> elsewhere in the program. This big buffer will be freed when
>> a memory exhaustion situation arises, to give enough memory
>> to the error reporting routines to close files, or otherwise
>> do housekeeping chores.
>>
>> 2) xmalloc()
>
> Why xmalloc? There's already a mechanism for dealing with such
> problems, viz signals.
>
> #include <signal.h>
>
> void alloc_failure_handle(int x)
> {
> fputs("Dis baby gon crash and burn!\n", stderr);

The Standard library functions are not guaranteed to be reentrant and
you cannot safely call them from a signal handler.

> exit(EXIT_FAILURE);
> /* or insert your free-buffer-and-clean-up stuff if you prefer */
> }
>
> main()
> {
> signal(SIGSEGV, alloc_failure_handle);
> /* ... */
> }

The state of a program after it has handled a SIGSEGV is undefined as
per the Standard.

IMHO, it's far better to handle out-of-memory conditions within normal
code rather than invoking all the complexity and fragility of signals.

Richard Tobin wrote:

> In article <fnn3og$o40$1@aioe.org>, jacob navia <jacob@nospam.org>
> wrote:
>
>>It is not possible to check EVERY malloc result within complex
>>software.
>
> I assume you're just setting up a scenario here rather than making
> this claim yourself.
>
>>1) At program start, allocate a big buffer that is not used
>>elsewhere in the program. This big buffer will be freed when
>>a memory exhaustion situation arises, to give enough memory
>>to the error reporting routines to close files, or otherwise
>>do housekeeping chores.
>
> This is not guaranteed to with all malloc() implementations.
> Some use different areas of memory for different sized blocks, and may
> not have code to re-purpose the freed block. Or they may free the
> memory back to the OS, in which case it may get gobbled up by
> something else before they have the chance to re-use it.

Yes. A better strategy is to use this buffer directly. No need to invoke
all the uncertainty involved in freeing and allocating again.

"jacob navia" <jacob@nospam.com> wrote in message
> 3:
> A solution like the one proposed by Mr McLean (aborting) is not
> possible for software quality reasons. The program must decide
> if it is possible to just abort() or not.
>
xmalloc() calls a caller-defined emergency function. It can do several
things, one of which is to simply exit. Don't confuse the requirement to
return the memory requested or (inherently) abort with the requirement to
abort.

--
Free games and programming goodies.
http://www.personal.leeds.ac.uk/~bgy1mm

jacob navia wrote:

>A solution for the allocation failures problem

You may have noticed this is somewhat controversial here. Doubt you will get
much agreement. It's one of those things where everyone has their own ideas.

--
Bart

jacob navia wrote:
> 1:
> It is not possible to check EVERY malloc result within complex software.

Why not? It's no harder than checking every fopen() for
success -- or is that another impossibility?

> 2:
> The reasonable solution (use a garbage collector) is not possible for
> whatever reasons.

Garbage collection (when it works) eases the memory management
problem by relieving the programmer of the need to call free().
But collecting all the garbage does not imply that every malloc()
will succeed! In fact, a collector that cannot relocate the non-
garbage is likely to increase fragmentation and produce allocation
failures when a free()-as-soon-as-possible strategy would not.

> 3:
> A solution like the one proposed by Mr McLean (aborting) is not
> possible for software quality reasons. The program must decide
> if it is possible to just abort() or not.
>
> Solution:
>
> 1) At program start, allocate a big buffer that is not used
> elsewhere in the program. This big buffer will be freed when
> a memory exhaustion situation arises, to give enough memory
> to the error reporting routines to close files, or otherwise
> do housekeeping chores.

This is a reasonable thing to try, and has been tried
often. The hard part is choosing a value for "big:" too little
and there's not enough for the cleanup activity, too much and
you provoke allocation failures that don't need to happen.

> 4:
> Using the above solution the application can abort if needed, or
> make a long jump to a recovery point, where the program can continue.

The problems of longjmp() are well understood. It's possible
to use it effectively, but the programmers who write functions
"between" the setjmp() and the longjmp() must be constantly aware
that they might not get a chance to clean up:

char *this = dup(getenv("THIS"));
char *that = dup(getenv("THAT"));
printf ("THIS = %s, THAT = %s\n", this, that);
free (this);
free (that);

See the memory leak? If the second dup() eventually calls
longjmp() and returns to an ancestor of this code, the
memory allocated to `this' is never freed.

It's possible to work around this problem: various people
have put together macro packages that imitate a try/finally
discipline, for example. But the language itself gives little
, and the compilers won't warn if somebody forgets (for
example, when a function that originally didn't need cleanup
acquires such a need during maintenance). Error recovery
based on longjmp() is do-able, but difficult.

--
Eric.Sosman@sun.com

jacob navia <jacob@nospam.com> writes:
> 1:
> It is not possible to check EVERY malloc result within complex software.
[...]

It's not only possible, it's required.

--
Keith Thompson (The_Other_Keith) <kst-u@mib.org>
Nokia
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

"Richard Bos" <rlb@hoekstra-uitgeverij.nl> schreef in bericht
news:479f1fc3.2785306889@news.xs4all.nl...
> jacob navia <jacob@nospam.com> wrote:
>
>> 1:
>> It is not possible to check EVERY malloc result within complex software.
>
> Bollocks from the start. Well done.
>
> Richard

Think he meant it's not possible to HANDLE every malloc failure. Indeed its
pretty easy putting an if in there, but not so easy to actually gracefully
handle it.

"Serve Laurijssen" <ni@hao.com> writes:
> "Richard Bos" <rlb@hoekstra-uitgeverij.nl> schreef in bericht
> news:479f1fc3.2785306889@news.xs4all.nl...
>> jacob navia <jacob@nospam.com> wrote:
>>
>>> 1:
>>> It is not possible to check EVERY malloc result within complex software.
>>
>> Bollocks from the start. Well done.
>
> Think he meant it's not possible to HANDLE every malloc
> failure. Indeed its pretty easy putting an if in there, but not so
> easy to actually gracefully handle it.

Yeah, programming is hard.

--
Keith Thompson (The_Other_Keith) <kst-u@mib.org>
Nokia
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

Serve Laurijssen <ni@hao.com> wrote:

> "Richard Bos" <rlb@hoekstra-uitgeverij.nl> schreef in bericht
> news:479f1fc3.2785306889@news.xs4all.nl...
> > jacob navia <jacob@nospam.com> wrote:
> >
> >> 1:
> >> It is not possible to check EVERY malloc result within complex software.
> >
> > Bollocks from the start. Well done.
> >
> > Richard

> Think he meant it's not possible to HANDLE every malloc failure. Indeed its
> pretty easy putting an if in there, but not so easy to actually gracefully
> handle it.

If you write software not intending to recover from a malloc failure, then,
yes, upon inspection your code will prove to be very difficult to modify to
handle such recovery.

But, that's beside the point. If at outset you write your code intending to
handle recovery, its not difficult at all. I don't remember (and granted I
can't remember what I wrote when I first began programming in C) ever being
in a situation where I found it difficult to recover or unwind from a path
because of a failed malloc call. Of course, I have developed a very
structured, almost rote method for writing software which suits me. But I
did so by necessity, because from very early on I never accepted the premise
that memory failure could or should be ignored.

I originally learned to program with Perl, then JavaScript (Navigator 3.0).
Subsequently, to me memory management was a _feature_. I found it much more
satisfying to have the capacity (if not perfectly realized or utilized) to
write more resilient programs.

When I don't wish to exercise that feature, and if C is otherwise not
particularly suited to a task, I use another language.

Aside from glib, I can't off-hand think of any widely used Free Software C
library which doesn't check and propogate memory allocation failures.

"William Ahern" <william@wilbur.25thandClement.com> wrote in message
> But, that's beside the point. If at outset you write your code intending
> to
> handle recovery, its not difficult at all. I don't remember (and granted I
> can't remember what I wrote when I first began programming in C) ever
> being in a situation where I found it difficult to recover or unwind from
> a
> path because of a failed malloc call. Of course, I have developed a very
> structured, almost rote method for writing software which suits me. But I
> did so by necessity, because from very early on I never accepted the
> premise that memory failure could or should be ignored.
>
I wrote xmalloc() for Baby X, which is a minimalist X windows toolkit.
The problem is that the whole IDE is strung together with function pointers,
so there is no high level to propagate to. Most of the allocations are for
tiny structures associated with windows, anyway. It is also not clear what
you should do if unable to put up a window. My solution is to nag the user
for more memory until he gives it or kills the application.

--
Free games and programming goodies.
http://www.personal.leeds.ac.uk/~bgy1mm

On Tue, 29 Jan 2008 12:47:27 +0100, jacob navia <jacob@nospam.com>
wrote in comp.lang.c:

> 1:
> It is not possible to check EVERY malloc result within complex software.

Yes, it is. Poor programmers might not wish to do so, and might
produce arguments defending their choice, but that does not change the
fact that it is possible.

--
Jack Klein
Home: http://JK-Technology.Com
FAQs for
comp.lang.c http://c-faq.com/
comp.lang.c++ http://www.parashift.com/c++-faq-lite/
alt.comp.lang.learn.c-c++
http://www.club.cc.cmu.edu/~ajo/docs/FAQ-acllc.html

jacob navia wrote:
> 1:
> It is not possible to check EVERY malloc result within complex software.
>

Pardon me.I haven't programmed for a few years but I was involved in
some fairly complex software development. Why is it not possible to
check every malloc result?

On Tue, 29 Jan 2008 12:47:27 +0100,
jacob navia <jacob@nospam.com> wrote:
> 1:
> It is not possible to check EVERY malloc result within complex software.

Why not?

Martien
--
|
Martien Verbruggen | System Administration is a dirty job, but
| someone said I had to do it.
|

ediebur@rcn.com wrote:
> jacob navia wrote:
>
>> 1:
>> It is not possible to check EVERY malloc result within complex
>> software.
>
> Pardon me.I haven't programmed for a few years but I was involved
> in some fairly complex software development. Why is it not
> possible to check every malloc result?

Don't believe everything you find written here.

--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]: <http://cbfalconer.home.att.net>
Try the download section.


--
Posted via a free Usenet account from http://www.teranews.com

jacob navia wrote:
> 1:
> It is not possible to check EVERY malloc result within complex software.

I disagree. It is possible. You just have to decide you want to sweat
the details bad enough and do the hard work.

Regards,
Stan Milam
--
Life simultaneously, and paradoxically, makes indurate and pliable the soul.

[snips]

On Tue, 29 Jan 2008 10:00:39 -0800, William Ahern wrote:

> But, that's beside the point. If at outset you write your code intending
> to handle recovery, its not difficult at all. I don't remember (and
> granted I can't remember what I wrote when I first began programming in
> C) ever being in a situation where I found it difficult to recover or
> unwind from a path because of a failed malloc call. Of course, I have
> developed a very structured, almost rote method for writing software
> which suits me. But I did so by necessity, because from very early on I
> never accepted the premise that memory failure could or should be
> ignored.

I think that's the key to it. I keep hearing how it's so hard to check
such things, the code is messy, yadda yadda yadda, yet it's not. Why,
then, do some of us do this and find it relatively straightforward, while
some don't?

I suspect it's because we developed habits - design habits - which
incorporate such checking from the outset, while others don't. We, by
sheer habit, write functions such that they are capable of reporting
failure to their callers and, also by sheer habit, write callers which
check for those errors.

If we were to write code where the called function _could_ fail, but the
caller didn't bother to check for it, we would be faced with three
choices:

1) Just accept our code will fail in unpredictable ways
2) Try to go back and find every place such things can fail and bolt on
the error handling code after the fact
3) Do something like xmalloc - if the allocation fails, just abort

I suspect this is exactly what's happening - that some folks are writing
the code *without* the error handling, then going back in to add it on
later. Obviously, this will mean it's nigh-on impossible to ensure, in a
complex application, that every case is checked. Obviously, it's going
to make things ugly. Obviously, it's going to be a whole heck of a lot
of work.

I don't know that this is what they're doing, but it sounds like it. It
would explain why they think something like xmalloc makes sense, why they
think that "checking every allocation" is so hard to ensure.

It really is just a matter of design, but even simpler, just a matter of
habit. If you're writing code which opens a file or allocates memory, or
some similar operation which can fail in a manner which you can both
detect and deal with, add the code to check it. Now.

It's a habit one can develop, and once it becomes habit, it is very
little effort indeed to ensure you do, in fact, check such things - each
and every time.