Unsigned integer overflow in `toom_eval_pm2.c`

[Niels Möller]

marco.bodrato at tutanota.com writes:

> Yes, unsigned is the best choice, it used to store a positive or
> negative number, but now it actually is a mask: 0 or ~0.
>
> I attach a possible patch.

Makes sense, I think.

As Vincent suggested, it would be good to document somewhere what the
convention is (values 0 or ~0). And maybe change to 0 and 1 convention,
since that fits better with assigning it from the return value from
mpn_sub_n, and it seems generally more consistent with how we handle
boolean values elsewhere. But may need further changes, like to
abs_sub_add_n, also noted by Vincent.

Regards,
/Niels

-- 
Niels Möller. PGP key CB4962D070D77D7FCB8BA36271D8F1FF368C6677.
Internet email is subject to wholesale government surveillance.