Unsigned integer overflow in `toom_eval_pm2.c`

Niels Möller nisse at lysator.liu.se
Mon Sep 4 10:13:53 CEST 2023


marco.bodrato at tutanota.com writes:

> Would a comment "It returns 0 or ~0, depending on the sign of the result"
> added to all the mpn_toom_eval_ functions suffice?

Yes. It would be nice if there were some reasonable place to document
that only once, but don't know where that could be.

> From the value in {0,1} we can get mask = -value, and from the mask in
> {0, ~0} we can get value = mask & 1. Of course we can switch to a
> different convention, if someone feels it's important. I don't.

If it's actually used as a mask in several places, I'm happy to leave as
is.

Regards,
/Niels

-- 
Niels Möller. PGP key CB4962D070D77D7FCB8BA36271D8F1FF368C6677.
Internet email is subject to wholesale government surveillance.


More information about the gmp-bugs mailing list