overflow in mpz_sizeinbase

Joe Weening jweening at ccrwest.org
Thu Apr 21 02:25:22 CEST 2005

mpz/sizeinbase.c calls:

  MPN_SIZEINBASE (result, PTR(x), ABSIZ(x), base);

This causes the "size" argument of MPN_SIZEINBASE to be of type "int",
and the computation

        __totbits = (size) * GMP_NUMB_BITS - (__cnt - GMP_NAIL_BITS);   \

in gmp-impl.h overflows when (size) * GMP_NUMB_BITS is 2^31 or more,
even though __totbits is large enough.  Casting size to mp_size_t at
an appropriate point should fix this, I would think.

				Joe Weening

More information about the gmp-devel mailing list