Subject: RE: [hackers-il] Malloc Checking [was Re: Re: Another Interesting Case Study: Permutations of a range of characters ]

From: Omer Musaev <[email-masked]>
Date: Wed, 16 Jan 2002 21:42:52 +0200

> -----Original Message-----
> From: Adi Stav [mailto:[email-masked]]
> Sent: Wednesday, January 16, 2002 10:49 PM
> To: [email-masked]
> Subject: Re: [hackers-il] Malloc Checking [was Re: Re: Another
> Interesting Case Study: Permutations of a range of characters ]
> 
> 
> On Wed, Jan 16, 2002 at 05:23:10PM +0200, Nadav Har'El wrote:
> > On Wed, Jan 16, 2002, Adi Stav wrote about "Re: 
> [hackers-il] Malloc Checking [was Re: Re: Another Interesting 
> Case Study: Permutations of a range of characters ]":
> > > either succeed or segfaults. The limit appears to be 8MB 
> per stack 
> > > frame, regardless of whether it comes from alloca() or 
> the compiler.
> > 
> > When we're talking about a single-threaded process (on 
> Linux), this 8MB (on
> > Linux 2.4) is simply the ulimit (see ulimit -s) your 
> process has set. You can
> > theoretically increase this limit to fill up whatever 
> virtual memory you have.
> 
> Ahh, so that's why it's so reproducible. I didn't think of checking 
> ulimit -- I assumed that since I didn't limit it, everything was 
> unlimited. This proves my point though -- even in the most obvious
> of cases, alloca() failed to fail.

Segfault _is_ a failure. Ask every person next cube to you.
Function that fails to tell me that it could not accomplish its task
is not worth considering in 12-year-uptime environments. 
API that pretends that it can not fail ( which is possible only 
when you had proved that a. you have no side effects, and b. you 
have sufficent resources for the function to complete ), but fails 
to deliver what it asked to do - is an utter lie. Lies are good for
marketing people ( "yes, of course, Mercury supports BSD _AND_ OS/360" )
but not for code.
 
> Kernel environment is so different that I'd expect every code ported
> to it to go at least some auditing. Intensive stack use will also 
> give trouble to e.g. embedded or special systems. alloca() is not a
> very portable function. But if you know your code is going to run on
> a UNIX/NT type system, it can be useful.

I can not agree more. Actually I can. Porting to Linux kernel will 
require rewrite, not auditing. 
For example - you can not do malloc on 50Mbs in real memory 
while in module. You can not realloc - this will make some hard time
porting a well-behaved stable code written in C for Solaris into 
Linux kernel. 

But this is not the point. The point is to use stack wisely. 
Recursion ( with continuations ) is a wise use of stack, 
while allocate is not. 
And again, function that can not fail, but has a side effect 
is an utter lie, and shall be removed from memories of upcoming 
generations (tm) ;-)

---

From: Adi Stav <[email-masked]>
Date: Thu, 17 Jan 2002 13:06:08 +0200

> is not worth considering in 12-year-uptime environments. 

You miss my point. alloca(), as a function, does not fail. What fails 
is the entire stack, and you never had guarantees about it to start 
with. From the process's point of view, the entire 8MB has been
preallocated at exec() time and it is available to use. It might run 
out, sure, and then you get a segfault. But it might run out because
of recursion, or use of large objects in functions. alloca( 4MB ) has
/exactly/ the same reliability as "char buf[ 4MB ]". Both will give 
you 4MB on the stack and both will segfault if there is not enough 
stack left. Stack is a very critical resource, nothing can work 
without it; you cannot run any functions, with many configurations 
you can't even have signal handlers. If alloca() returned NULL when 
it runs out of stack then it would give you reliability above and 
beyond what the system normally provides. Think of alloca() as a 
compiler extension and not as an API.

That's how I see it from a philosophical point of view. Practically, 
you can always say that alloca() encourages abuse of the stack and
large allocations from what is essentially a limited resource. But 
so are large arrays and large structs, or recursion. A large alloca() 
might finish almost all of the 8MB stack and let the next function 
call cause a segfault. Or the stack might be nearly finished for 
other reasons and the following alloca() fails. alloa()==NULL would 
not help in either case. If you trust yourself to check every 
alloca() return value, you can just invest the effort in making sure 
that every argument to alloca() (and, indeed, to any array 
declaration) is sane.

> marketing people ( "yes, of course, Mercury supports BSD _AND_ OS/360" )

Here I agree. What annoys me about BSD/glibc alloca(), though, is not 
that it claims not to be able to fail, but that it claims to return
NULL, which I don't see it doing. The Solaris manpage is much more
truthful in this regard, although it, too, is way too implicit and 
noncommitting to be considered completely honest.

> Recursion ( with continuations ) is a wise use of stack, 

What is recursion with continuations and why is it safer?