Overflow in mpz_cmp

[Niels Möller]

Marco Bodrato <bodrato at mail.dm.unipi.it> writes:

> Ciao,
>
> Il 2020-02-10 18:25 Guillaume Melquiond ha scritto:
>> When the operand sizes do not match, the mpz_cmp function function just
>> returns the difference of the signed sizes. Unfortunately, this
>> difference might not fit inside the "int" return type, when the numbers
>> are of opposite sign.
>
> In mini-gmp we defined a macro:
> #define GMP_CMP(a,b) (((a) > (b)) - ((a) < (b)))
>
> We may use the same idea here too. I mean something like the following:
>
> diff -r f5601c2a8b11 mpz/cmp.c
> --- a/mpz/cmp.c	Sun Feb 09 16:16:19 2020 +0100
> +++ b/mpz/cmp.c	Tue Feb 11 14:20:39 2020 +0100
> @@ -35,15 +35,15 @@
>  int
>  mpz_cmp (mpz_srcptr u, mpz_srcptr v) __GMP_NOTHROW
>  {
> -  mp_size_t  usize, vsize, dsize, asize;
> +  mp_size_t  usize, vsize, asize;
>    mp_srcptr  up, vp;
>    int        cmp;
>
>    usize = SIZ(u);
>    vsize = SIZ(v);
> -  dsize = usize - vsize;
> -  if (dsize != 0)
> -    return dsize;
> +  cmp = (usize > vsize) - (usize < vsize);
> +  if (cmp != 0)
> +    return cmp;

I would be tempted to keep it simple,

  if (usize != vsize)
    return (usize > vsize) ? 1 : -1;

It's not clear to me if this is worth micro optimizing, and ensure we
get only a single branch.

Regardsa,
/Niels

-- 
Niels Möller. PGP-encrypted email is preferred. Keyid 368C6677.
Internet email is subject to wholesale government surveillance.