abort on error - is this being addressed?
marc.glisse at inria.fr
Sat Aug 28 11:17:44 CEST 2010
On Sat, 28 Aug 2010, Vincent Lefevre wrote:
> On 2010-08-26 11:38:10 +0200, Marc Glisse wrote:
>> First, notice that any OS that overcommits memory (like Linux unless
>> you use some option that is not recommended) makes the point moot.
> Linux no longer blindly overcommits, so that if you have insanely
> large computations, malloc will fail.
Unless you use the very strictest mode (something linux is not optimized
for, but with enough swap it should work all right, although the
documentation itself says that not everything is accounted for), linux
still overcommits. Wildly unrealistic mallocs will fail, but that's not
the gmp way, it will usually only malloc things whose size is the same
order of magnitude (say within a factor 10) as something already malloced,
so you can easily fall between the safe zone and the wild zone,
especially if there are other processes around.
I am not saying that testing the return of malloc is useless, but in an
out of memory situation, you only have chances that this will be detected
by a malloc failure, not a guarantee. The improvements to linux may have
increased this probability to make it good enough for some scenarios.
On Sat, 28 Aug 2010, Joerg Arndt wrote:
> Even if the kernel overcommits (and one cannot change that)
> there is a "pretty safe" strategy:
> malloc all memory you'll need and write to it (to force
> getting actual RAM), then use this memory as a pool.
I assume you mean to reserve all that memory when the application starts
(not replace malloc with calloc). Looks like java ;-) That's a very
realistic option, but it seems like a shame to reserve all the memory like
this, when you have no idea how much you will need. And you still need to
decide how you handle that failure to allocate, now that you can detect it
Note that having a separate process to do the computations (one whose
possible crash is handled) seems like a common enough strategy, despite
all the obvious disadvantages.
More information about the gmp-devel