error handling

[David M. Warme]

 > Will this container hold only the mpz_t structs, or also the limb arrays?

The container would contain a pointer to a dynamically
allocated array of mpX_t structs (where X \in {z, q, f}),
each of which points to their own limb array as usual.

There are actually several distinct issues lurking about
here.  It may help crystallize the discussion to identify
them and to perhaps consider each separately, even
though a common mechanism might address all or
several of the issues:

     1. Providing more flexible methods for reporting of
        errors and exceptional conditions (results that are
        mathematically undefined, out of memory, etc.).

     2. Providing these in a manner that preserves as
        much of the present API/ABI as possible (preferably
        all of it).

     3. Providing these in a manner that does not grow
        the size of the existing GMP objects.

     4. Proper cleanup during the handling of exceptional
        conditions (e.g., freeing of locally allocated memory,
        whether held by mpX_t objects or otherwise).

     5. Additional state information that should be
        associated more-or-less directly with a set of
        mpX_t objects, such as memory lifetime/usage hints.

Although the typical cryptographic applications may only
use a few GMP objects and can easily afford for an
mpz_t to triple in size, my application uses arrays of mpX_t's,
(or their MPFR equivalents) by the tens/hundreds of millions
or more.  Profligate expansion of the base objects (especially
for features that may only be used rarely) is unwarranted.  I
would lobby strongly to keep the base objects as they are
and find some other mechanism for providing flexible
reporting of exceptional conditions.

Also, most of the cryptographic algorithms occupy very
finitely bounded space, and have few (if any) exceptional
conditions that cannot be tested for outside of GMP (e.g.,
modulus < 2).  Stack overflow is certainly still one possibility
if the application gobbles most of it and invokes GMP right
at the brink.  Arguably, even this could also be tested before
calling GMP (although one could envision an OS that unmaps
excess stack virtual pages, making them available at one
moment and gone the next).

My suggestion for a container to hold a homogeneous
array of mpX_t objects was motivated by the desire to
amortize the storage cost for the proposed extensions
over a large number of mpX_t objects.  The main problem
with this is that none of the existing GMP functions currently
receive any of these "containers" as input parameters.

So how do you make the proposed container visible to the
API?  I saw two main approaches worth considering:

     a. Statically.  Define new API functions along the lines
        of mp_{get|set}_memory_functions().

     b. Dynamically (as in dynamically scoped).

The dynamically scoped approach provides a more direct
solution for the scenario where several libraries that use
GMP can establish their own custom handling of exceptional
conditions, while still inter-operating smoothly with each
other.  It also provides a better solution in multi-threaded
applications, since each thread has its own handler context.

Now that the proposed extension data reside in a separate
object (whether managed in a global/static or dynamically
scoped manner), the final step is to notice that it is no longer
essential to retain the notion of being a container for arrays
of mpX_t objects.  (Note, however, that with the dynamically
scoped method this can also provide a handy means for
cleaning up temporaries just before doing a longjmp().)

Please let me know if any of the GMP developers would find
it helpful to have more concrete examples that spell this out
in more detail.  I could provide code snippets, box-and-arrow
drawings (or both) if these would help you to fully understand
this proposal, together with its advantages and drawbacks.

Item 5 above (memory lifetime/usage hints, etc.) are a
different matter.  These cannot be completely divorced
from the collection of mpX_t objects that they govern.
The notion of a "container" for a large collection of mpX_t
objects is essential here.  This container could indeed be a
"memory pool", although a sophisticated memory manager
might want to know more about the memory that GMP is
requesting than the current interfaces allow.  For example,
the memory manager might want to handle limb arrays
differently from other objects.  I will submit a concrete
proposal in a separate E-mail.

David

On 12/19/2014 03:28 AM, Niels Möller wrote:
> "David M. Warme" <David at Warme.net> writes:
>
>> Design a new "container" object that holds a single,
>> homogeneous array of GMP numbers.
> Will this container hold only the mpz_t structs, or also the limb
> arrays?
>
>> Also, ask yourself the following question: if you ignore
>> C++ wrappings of solitary GMP numbers (where applications
>> would instead allocate arrays of individually C++ wrapper'ed
>> GMP numbers), how many applications out there actually do
>> lots of GMP arithmetic *without* resorting to using arrays of
>> GMP numbers?  Probably a few, but I'd bet that the most
>> GMP-intensive applications out there have most of their GMP
>> number instances residing within such homogeneous arrays.
> At least in cryptographic applications, I think arrays of numbers are
> rare. Typical cases are objects with one or a few bignums, and functions
> with one or a few temporaries.
>
> Regards,
> /Niels
>