Likely GMP bug

Torbjörn Granlund tg at gmplib.org
Fri May 25 12:27:33 UTC 2018


nisse at lysator.liu.se (Niels Möller) writes:

  And code is essentially 

        count_trailing_zeros (c, t);
        ulimb >>= (c + 1);

  The intention is to shift right to get rid of both trailing zero bits,
  and the redundant least significant one bit.

  That fails with undefined behavior if by chance t == 2^31, so that c ==
  31.

And ubsan complains about exactly that.

  I don't see how that can happen, though, since ulimb, vlimb < 2^31
  through out the loop, and t = (ulimb - vlimb) mod 2^32.

The setting GMP_CHECK_RANDOMIZE=140064609456624 seems to trigger it, but
that could of course also be bugs in the compiler.

  And I also wonder why USE_ZEROTAB is set to 0 here.

That might be good as count_trailing_zeros is usually fast.

-- 
Torbjörn
Please encrypt, key id 0xC8601622


More information about the gmp-bugs mailing list