Overflow in mpz_cmp

[Marc Glisse]

On Tue, 11 Feb 2020, Niels Möller wrote:

> Marco Bodrato <bodrato at mail.dm.unipi.it> writes:
>
>> Ciao,
>>
>> Il 2020-02-10 18:25 Guillaume Melquiond ha scritto:
>>> When the operand sizes do not match, the mpz_cmp function function just
>>> returns the difference of the signed sizes. Unfortunately, this
>>> difference might not fit inside the "int" return type, when the numbers
>>> are of opposite sign.
>>
>> In mini-gmp we defined a macro:
>> #define GMP_CMP(a,b) (((a) > (b)) - ((a) < (b)))
>>
>> We may use the same idea here too. I mean something like the following:
>>
>> diff -r f5601c2a8b11 mpz/cmp.c
>> --- a/mpz/cmp.c	Sun Feb 09 16:16:19 2020 +0100
>> +++ b/mpz/cmp.c	Tue Feb 11 14:20:39 2020 +0100
>> @@ -35,15 +35,15 @@
>>  int
>>  mpz_cmp (mpz_srcptr u, mpz_srcptr v) __GMP_NOTHROW
>>  {
>> -  mp_size_t  usize, vsize, dsize, asize;
>> +  mp_size_t  usize, vsize, asize;
>>    mp_srcptr  up, vp;
>>    int        cmp;
>>
>>    usize = SIZ(u);
>>    vsize = SIZ(v);
>> -  dsize = usize - vsize;
>> -  if (dsize != 0)
>> -    return dsize;
>> +  cmp = (usize > vsize) - (usize < vsize);
>> +  if (cmp != 0)
>> +    return cmp;
>
> I would be tempted to keep it simple,
>
>  if (usize != vsize)
>    return (usize > vsize) ? 1 : -1;
>
> It's not clear to me if this is worth micro optimizing, and ensure we
> get only a single branch.

On x86_64, both gcc and clang optimize (usize > vsize) ? 1 : -1 to 2 * 
(usize > vsize) - 1 (as a single LEA for gcc, 2 ADD for llvm). So the 
generated code may be just as good with the simple code.

-- 
Marc Glisse