Unsigned integer overflow in `toom_eval_pm2.c`
Andrew Teylu
andrewvaughanj at gmail.com
Sun Sep 3 19:17:12 CEST 2023
Hi,
I'm working with gmp-6.3.0 compiled with clang 16 and its
`-fsanitize=integer` flag.
When I run `multiply.c` from gmpbench [https://gmplib.org/gmpbench],
I'm seeing an unsigned integer overflow in `toom_eval_pm2.c` on this
line:
```
neg ^= ((k & 1) - 1)
```
The values we're normally getting are:
* neg (before): 0
* k: 6
* neg (after): -1
I fully appreciate that unsigned integer overflow is implementation
defined, but I am not sure if this is the intended behaviour of
`mpn_toom_eval_pm2` or not.
Apologies for the noise if this is intended behaviour.
Cheers,
Andrew
More information about the gmp-bugs
mailing list