abort on error - is this being addressed?

[Marc Glisse]

On Sat, 28 Aug 2010, Vincent Lefevre wrote:

> On 2010-08-26 11:38:10 +0200, Marc Glisse wrote:
>> First, notice that any OS that overcommits memory (like Linux unless
>> you use some option that is not recommended) makes the point moot.
>
> Linux no longer blindly overcommits, so that if you have insanely
> large computations, malloc will fail.

Unless you use the very strictest mode (something linux is not optimized 
for, but with enough swap it should work all right, although the 
documentation itself says that not everything is accounted for), linux 
still overcommits. Wildly unrealistic mallocs will fail, but that's not 
the gmp way, it will usually only malloc things whose size is the same 
order of magnitude (say within a factor 10) as something already malloced, 
so you can easily fall between the safe zone and the wild zone, 
especially if there are other processes around.

I am not saying that testing the return of malloc is useless, but in an 
out of memory situation, you only have chances that this will be detected 
by a malloc failure, not a guarantee. The improvements to linux may have 
increased this probability to make it good enough for some scenarios.


On Sat, 28 Aug 2010, Joerg Arndt wrote:

> Even if the kernel overcommits (and one cannot change that)
> there is a "pretty safe" strategy:
> malloc all memory you'll need and write to it (to force
> getting actual RAM), then use this memory as a pool.

I assume you mean to reserve all that memory when the application starts 
(not replace malloc with calloc). Looks like java ;-) That's a very 
realistic option, but it seems like a shame to reserve all the memory like 
this, when you have no idea how much you will need. And you still need to 
decide how you handle that failure to allocate, now that you can detect it 
:-)

Note that having a separate process to do the computations (one whose 
possible crash is handled) seems like a common enough strategy, despite 
all the obvious disadvantages.

-- 
Marc Glisse