Patch: support callback on allocation overflow instead of calling abort()

[Niels Möller]

Colin Caine <cmcaine at gmail.com> writes:

> Sorry, this patch was posted to this list last month, but I didn't see it
> in the archives. See here for related discussion:
> https://gmplib.org/list-archives/gmp-bugs/2020-September/004865.html

I asked some questions (some off list), and haven't seen satisfactory
answers. The thing is, it seems fairly safe to use the propsed callback
to write a friendly error message, backtrace of involved julia
functions, etc, and then exit the process.

But my impression is that julia may also longjmp out and continue
execution, and to me, that seems very brittle. You can have memory
leaks, you may leave GMP data structures in an inconsistent state.

To make any progress with adding a callback like this, we'd need to
either:

1. Find out and document how to longjmp out from the callback safely.

2. Agree and document that when GMP invokes this callback, GMP state
   should be considered invalid. The process must not make any further
   calls to any GMP functions.

And in the latter case, would that be satisfactory for Julia's use?

There are other approaches to avoid these crashes, e.g., I think the
recent emacs integration uses a (configurable) limit on bignum size, and
will raise an emacs exception long before hitting GMP's limits, and the
emacs process can go on running with no issues.

Regards,
/Niels

-- 
Niels Möller. PGP-encrypted email is preferred. Keyid 368C6677.
Internet email is subject to wholesale government surveillance.