integer overflow yields incorrect results and buffer overflow on 64-bit machines

[Vincent Lefevre]

On 2008-02-25 20:01:22 +0100, Torbjorn Granlund wrote:
> This is a known behaviour, GMP does not define any exception
> processing.

The fact that GMP does not define any exception processing (in the
sense, just abort in case implementation or system limits are reached)
was known to me, but not the fact that GMP could yield incorrect
results and behave erratically just because some internal limits
(unknown to the user) are reached.

> Reporting it as a bug, in particular to Debian, is utterly
> pointless, and I am sure you are fully aware of that.

Well, at least the bug is recorded somewhere.

> One might say that the slogan "Arithmetic without limtations" no
> longer holds, since some big tin 64-bit machines could perhaps have
> enough memory to make this GMP limitation reachable.  But we're
> talking of more than 128 Gibyte memory here.

Note that's 16 GB here (but see below).

> Unfortunately, changing the _mp_size and _mp_alloc fields to long
> cannot be done without breaking compatibility (the size of the type
> should not change).

Perhaps I wasn't very clear. I didn't ask to change _mp_size and
_mp_alloc fields to long (at least for the time being), in particular
because to do really useful computations with such large values, one
may need some factor higher the integer sizes (several hundreds of
GBs?). The problem here is that one can silently get incorrect results
and I also fear about the security[*] (but I really don't know every
context in which GMP is used and with what limits). So, what I was in
mind, was just to detect such integer overflows, and e.g. abort when
they occur. If for some reason, this can't be done (because it would
be noticeably slower, for instance), then the limits should clearly be
documented.

Now, concerning this bug, I think that a test in _mpz_realloc and
mpz_realloc2 would be sufficient (note that INT_MIN should be
forbidden too as the code sometimes take the opposite).

[*] To give an example, GMP can be used in PHP. In some contexts, the
webmaster may choose not to set any computational limit (e.g. in case
of a dedicated server). So, DoS could be accepted, but letting a way
to hijack the server would be unacceptable.

-- 
Vincent Lefèvre <vincent at vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)