Unsigned integer overflow in `toom_eval_pm2.c`
Niels Möller
nisse at lysator.liu.se
Mon Sep 4 10:13:53 CEST 2023
marco.bodrato at tutanota.com writes:
> Would a comment "It returns 0 or ~0, depending on the sign of the result"
> added to all the mpn_toom_eval_ functions suffice?
Yes. It would be nice if there were some reasonable place to document
that only once, but don't know where that could be.
> From the value in {0,1} we can get mask = -value, and from the mask in
> {0, ~0} we can get value = mask & 1. Of course we can switch to a
> different convention, if someone feels it's important. I don't.
If it's actually used as a mask in several places, I'm happy to leave as
is.
Regards,
/Niels
--
Niels Möller. PGP key CB4962D070D77D7FCB8BA36271D8F1FF368C6677.
Internet email is subject to wholesale government surveillance.
More information about the gmp-bugs
mailing list