integer overflow in mpn/get_d.c from GMP 5.1.2
nisse at lysator.liu.se
Sat Sep 21 09:58:06 CEST 2013
Vincent Lefevre <vincent at vinc17.net> writes:
> I don't know what led to the "wrong code" here. But for instance,
> here's what a compiler would be allowed to do. The test is:
Thanks for explaining. But I'm still not quite following you.
> if (UNLIKELY ((unsigned long) (GMP_NUMB_BITS * size)
> > (unsigned long) (LONG_MAX - exp)))
> [...] Since exp <= LONG_MAX, the compiler knows that LONG_MAX - exp is
> also nonnegative and fits in a long.
Here, for non-negativity, we're also making the critical assumption that
the compiler's global optimization infers that the function is never
called with exp < 0, right?
> Hence a wrong result if exp is negative.
And then the generated code breaks when, in fact, the function *is*
called with exp < 0. That means that either the compiler's inference is
buggy. Or that the GMP bug is elsewhere, with the negative value of exp
resulting from some *earlier* computation with undefined behaviour.
Hmm, or ar you saying that the compiler's assumption exp >= 0 comes from
*local* inference? "If the caller provides a negative exp, then we get
undefined behaviour in this expression. Hence, let's just pretend that
the caller would never do such a thing".
Makes a bit sense in it's own way, and I guess that's the only way
moving around the cast can make a difference.
Niels Möller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.
More information about the gmp-bugs