Unsigned integer overflow in `toom_eval_pm2.c`

Andrew Teylu andrewvaughanj at gmail.com
Sun Sep 3 19:17:12 CEST 2023


Hi,

I'm working with gmp-6.3.0 compiled with clang 16 and its
`-fsanitize=integer` flag.

When I run `multiply.c` from gmpbench [https://gmplib.org/gmpbench],
I'm seeing an unsigned integer overflow in `toom_eval_pm2.c` on this
line:

```
neg ^= ((k & 1) - 1)
```

The values we're normally getting are:

     * neg (before): 0

     * k: 6

     * neg (after): -1

I fully appreciate that unsigned integer overflow is implementation
defined, but I am not sure if this is the intended behaviour of
`mpn_toom_eval_pm2` or not.

Apologies for the noise if this is intended behaviour.

Cheers,

Andrew


More information about the gmp-bugs mailing list