[BUG/PATCH] GMP 6.3.0 Mini-GMP: allocation metadata overflow above INT_MAX limbs

[Niels Möller]

Nad Ka <french.cryptonad at gmail.com> writes:

> Mini-GMP computes requested allocation sizes in mp_size_t, but stores
> _mp_alloc in int. On the tested LP64 systems, INT_MAX+1 limbs is
> representable in mp_size_t but narrows _mp_alloc to INT_MIN. The issue is
> reproducible through mpz_init2, mpz_realloc2, and mpz_limbs_modify. A later
> free callback can then receive an incorrect size derived from the corrupted
> allocation metadata.

I think the small patch itself looks quite reasonable (but please
include small patches inline or as text/plain attachments). However,
note that

1. mini-gmp is not intended to be used with really large bignums.

2. Checking for overflow when converting from mp_size_t to int is not
   sufficient for detecting all cases of overflow. On platforms where
   mp_size_t and mp_bitcnt_t also are 32-bit, the conversion is no
   problem but arithmetic on those types may overflow. E.g.,
   mini-gmp.c:mpz_mul includes the line

     mpz_init2 (t, (un + vn) * GMP_LIMB_BITS);

   which will overflow when both un and vn get close to the maximum
   value of mp_size_t.

Regards,
/Niels

-- 
Niels Möller. PGP key CB4962D070D77D7FCB8BA36271D8F1FF368C6677.
Internet email is subject to wholesale government surveillance.