Latest commit introduces undefined behavior in hgcd2.c

Torbjörn Granlund tg at gmplib.org
Wed Sep 18 19:20:41 UTC 2019


Guido Vranken <guidovranken at gmail.com> writes:

  My bignum fuzzer running at OSS-Fuzz came up with this:

  hgcd2.c:223:42: runtime error: shift exponent 64 is too large for
  64-bit type 'mp_limb_t' (aka 'unsigned long')
  #0 0x76a4db in div2 /src/libgmp/mpn/hgcd2.c:223:42
  #1 0x769684 in __gmpn_hgcd2 /src/libgmp/mpn/hgcd2.c:372:18
  #2 0x74ac55 in __gmpn_gcd /src/libgmp/mpn/gcd.c:200:11
  #3 0x73c209 in __gmpz_gcd /src/libgmp/mpz/gcd.c

  Introduced in commit https://gmplib.org/repo/gmp/rev/f044264e2fe9

I think it is a false positive.  The result of the shifted value is
masked when the shift count is not in range.

(We got the same false positive from our nightly testing using gcc's
sanitized-something command-line option.)

-- 
Torbjörn
Please encrypt, key id 0xC8601622


More information about the gmp-bugs mailing list