Unsigned integer overflow in `toom_eval_pm2.c`

Torbjörn Granlund tg at gmplib.org
Sun Sep 3 20:16:51 CEST 2023


Andrew Teylu <andrewvaughanj at gmail.com> writes:

  When I run `multiply.c` from gmpbench [https://gmplib.org/gmpbench],
  I'm seeing an unsigned integer overflow in `toom_eval_pm2.c` on this
  line:

  neg ^= ((k & 1) - 1)

  I fully appreciate that unsigned integer overflow is implementation
  defined, but I am not sure if this is the intended behaviour of
  `mpn_toom_eval_pm2` or not.

In C, unsigned arithmetic is completely defined as computing mod 2^k,
where k is the bit size of the corresponding type.

I am not sure the arithmetic on unsigned types is what clang is unhappy
about, though.  Perhaps it dislikes the xor with "neg", which is a
signed variable.

Arithmetic on signed types as well as assignments between signed and
unsigned is not well-defined for certain operand ranges.

This is not a real problem with any compiler I am aware of.  Of course,
an evil compiler is free to make demons fly out our noses whenever we
trigger "undefined" signed integer operations.

I don't think it is worth our time trying to placate clang and its ever
growing world of warnings.  We would end up with utterly messy code
which nobody could read.


-- 
Torbjörn
Please encrypt, key id 0xC8601622


More information about the gmp-bugs mailing list