Reporting a gmp bug

[Niels Möller]

jy l <linjy0410 at gmail.com> writes:

> It seems like in `mpz_nextprime` this line (
> https://gmplib.org/repo/gmp/file/tip/mpz/nextprime.c#l204), when `n` is
> very large, it doesn't restrict the value of `odds_in_composite_sieve`
> which leads to the `alloca` below crash and might cause more buffer
> overflow.

I agree the array size odds_in_composite_sieve should have an upper
bound here (and if we expect a very large sieve to be useful, it should
be allocated with TMP_ALLOC_TYPE, which falls back to heap allocation
for large sizes).

I'm afraid I don't understand the comment

    /* Corresponds to a merit 14 prime_gap, which is rare. */
    odds_in_composite_sieve = 5 * nbits;

Thanks for reporting.

Regards,
/Niels

-- 
Niels Möller. PGP key CB4962D070D77D7FCB8BA36271D8F1FF368C6677.
Internet email is subject to wholesale government surveillance.