mpn_mul_fft type overflow issue

[Torbjorn Granlund]

Mark Sofroniou <marks at wolfram.com> writes:

  Thanks. I wasn't completely sure what the right type was in all cases.
  Most of the changes are to use mp_size_t instead of int - these are
  the important ones. There are a couple (related to the variables K2 and K3)
  that change unsigned int to mp_limb_t. I also changed mpn_mul_fft_lcm
  to mp_limb_t and that doesn't look exactly right - size_t doesn't
  look right there - maybe a bit counter type (not sure what you
  use for that other than int)?
  
We introduced a type mp_bitcnt_t with GMP 5.0.

  One of the reasons for sending this for discussion was that I wanted
  to know if you are amenable to these sorts of changes and whether
  it is worth me investing more time on them. I think that it would be
  nice to not have plain C uses of int/long and unsigned int/long internally
  in the mpn functions and to replace them with scalable types.
  
I think we do that almost everywhere.  Aside from mpn/generic/mul_fft.c,
I believe mpn/generic/perfpow.c also needs a typing overhaul.  I think
most other functions will be alright.

(We actually use 32-bit computations on some asm code implementing O(n²)
algorithms.  The thinking is that (2³²)² will lead to non-termination
anyway.)

  There are things like:
  
    int sqr = (n == m && nl == ml);
  
  that would maybe be clearer as a boolean type.
  
Let's leave that for a different patch.  (You may notice that the
conditions for sqr vary with file, something that we might want to fix
too.)

-- 
Torbjörn