Latest commit introduces undefined behavior in hgcd2.c

Vincent Lefevre vincent at
Wed Sep 18 19:33:37 UTC 2019

On 2019-09-18 21:20:41 +0200, Torbjorn Granlund wrote:
> Guido Vranken <guidovranken at> writes:
>   My bignum fuzzer running at OSS-Fuzz came up with this:
>   hgcd2.c:223:42: runtime error: shift exponent 64 is too large for
>   64-bit type 'mp_limb_t' (aka 'unsigned long')
>   #0 0x76a4db in div2 /src/libgmp/mpn/hgcd2.c:223:42
>   #1 0x769684 in __gmpn_hgcd2 /src/libgmp/mpn/hgcd2.c:372:18
>   #2 0x74ac55 in __gmpn_gcd /src/libgmp/mpn/gcd.c:200:11
>   #3 0x73c209 in __gmpz_gcd /src/libgmp/mpz/gcd.c
>   Introduced in commit
> I think it is a false positive.  The result of the shifted value is
> masked when the shift count is not in range.

If the shift count is not in range, the behavior is undefined,
whatever you do with the "result". Having such undefined behavior
is a bug, as the compiler can do some optimization, such as
considering this as dead code. This is different from the case
where you have implementation-defined behavior.

Vincent Lefèvre <vincent at> - Web: <>
100% accessible validated (X)HTML - Blog: <>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

More information about the gmp-bugs mailing list