error handling

[David M. Warme]

Consider another approach, which makes more sense in
applications that use arrays of GMP numbers:

Design a new "container" object that holds a single,
homogeneous array of GMP numbers.  One could
construct, e.g., a container of 7000 mpq's, which would
be mpq_init'd for you automatically.  You can then ask
for a pointer to the K-th member of this array.  This
container can have additional data in it:

     - Local overrides for memory alloc/realloc/free, and
       any other exception handling you might like.

     - Hints about lifetime and/or usage patterns for the
       contained numbers.

     - Garbage collector tracing info.

     - Thread usage info (e.g., single-thread only,
       single-writer/multiple-reader threads, etc.)

The cost of these fields are amortized over all elements
of the array.  If you really want to do this on individual
GMP numbers, you could allocate containers having arrays
of one number in them.

Each thread would have a thread-local pointer variable
containing a chain of all such containers currently
allocated on the stack.  (A single global variable suffices
for single-threaded configurations.)  Searches for handler
functions proceeds down the chain.  Fairly simple code
similar to the TMP_DECL, TMP_MARK, TMP_ALLOC, and
TMP_FREE mechanism (used inside of GMP) could be
provided for use within applications to manage this
context chain.  Applications that use longjmp() would
have to perform special updates to the chain before
invoking longjmp(), or alternatively, GMP could provide
a special wrapper routine around longjmp() the performs
the necessary stack unwinding before calling longjmp().

Dynamically allocated containers would of course persist
until explicitly freed, and would not be linked into this
per-thread chain.

Note -- the objects residing within a per-thread context
chain of this sort need not even contain any GMP
numbers.  One could conceivably have several different
types of (type-tagged) objects appearing within the
chain, each providing its own functionality.  The point is
that this provides a place to hang context-sensitive
extensions of the type you are proposing, in a place
where GMP can get at them without altering its existing
API much, if at all.

This gives each GMP-client library an opportunity to
do things "their way" in a manner that stands a pretty
good chance of providing smooth interoperability with
each other and with the main application.

The one place where these various parties would need to
avoid collision would be the "type-tags" used in the objects
along this chain.  These could be 8 or 16-bit integers
allocated dynamically from a central authority API
that simply bumps a protected counter.

I have suggested this before (a separate "container"
object holding a homogeneous array of GMP numbers),
for the purpose of providing reliable garbage collector
tracing.  It can also provide local hooks for error handling,
and anything else that you don't want to tack onto every
single GMP number on planet earth.

Those who really want the benefits of this mechanism
would be willing to create such a container in the local
stack frame, initialize local pointer variables to point to
individual array elements, and then use these pointer
variables as if they were separate mpz_t, mpq_t, or mpf_t
objects, as is currently done.

Also, ask yourself the following question: if you ignore
C++ wrappings of solitary GMP numbers (where applications
would instead allocate arrays of individually C++ wrapper'ed
GMP numbers), how many applications out there actually do
lots of GMP arithmetic *without* resorting to using arrays of
GMP numbers?  Probably a few, but I'd bet that the most
GMP-intensive applications out there have most of their GMP
number instances residing within such homogeneous arrays.

For the C++ case, one could have derived versions of the
wrapper classes that operate on pointers to GMP objects
instead of containing a single embedded GMP number.  These
could be instantiated on individual array elements within
the container object (that can also be C++ wrapper'ed).
One could also have an "iterator" object for the container
that also provided the interfaces necessary to appear like
a standard C++ wrapped GMP number, but that would
really be a reference into the array that can be re-positioned
using next(), prev() and seek() sorts of operations.

Perhaps this sort of stack chain mechanism really belongs
in glibc, where its interaction with routines such as longjmp()
can be managed better, and where several standard
type-tags and object formats can be defined.

Please consider other options before stealing bits inside of
every single GMP number in the universe, or making the
base numbers larger in size.

David Warme


On 12/17/2014 08:49 AM, Torbjörn Granlund wrote:
> [Moved thread from gmp-devel.]
>
> I'd like to think of error handling also in a C perspective, and
> consider a few more problems at the same time.
>
> Which sources of exceptions do we have currently?
>
> 1 We divide by 0 to generate a SIGFPE.  I think we do that for division
>    by zero as well as some other mathematically undefined operation.
>
> 2 We detect overflow of _mp_size.
>
> 3 Allocation problems.
>
> 4 Stack overflow leading to SIGSEGV.  (I've recently trimmed the stack
>    usage to stay within a few hundred KiB, but some threaded environment
>    give tiny default stacks.)
>
> 5 ASSERT_ALWAYS
>
> 6 More that I have forgotten.
>
> I saw the suggestion to invoke a user-defined function instead of the
> explicitly detected errors (i.e., 1, 2, 3, 5, perhaps 6).  That is an
> idea worth considering.  We have function pointers to for allocation,
> reallocation, freeing; this would be one more such global pointer.
>
> The error handler function would then use the knowledge of the
> allocation functions to clean up the memory state, and probably longjump
> to to deallocate stack.
>
> GMP might be able to clean up TMP_* memory, since that is a kind of
> stack.
>
> I am not too fond of these global pointers.  It would be better design
> to refer the memory handling and error handling functions from each GMP
> user variable, akin to object oriented languages' vtables.  Except that
> this would make these small structures 3 times larger.
>
> There are real scenarios where one would want different sets of
> allocation functions.  E.g., many libraries use GMP.  Some of them have
> their orn error reporting mechanisms, and memory handles.  But then the
> user might use GMP directly, or she might from the same program use
> several libraries which use GMP.  The current GMP memory allocation
> mechanism is not suitable here.
>
> We have another problem with the GMP structures.  On 64-bit machines,
> the 32-bit _mp_size field is starting to hurt for some applications, and
> with that also the _mp_alloc field.
>
> We might therefore consider changing these structures in an incompatible
> way.  We culd then sneak in a field for choosing 'handler functions'.
> Perhaps like this:
>
>          signed long _mp_size     : 46;
>          unsigned long _mp_alloc  : 14;  // 8-bit mantissa, 6-bit exponent
>          unsigned long _mp_handler:  4;
>          mp_limb_t _mp_d;        // 64
>
> The trickery with field sizes keeps the structure at 128 bits.  There
> will be a cost for it, though.
>
> The _mp_handler field chooses one of 16 handler groups.  Each such group
> will have function pointers for allocation, reallocation, freeing, and
> error reporting.  I'd say that 16 groups will be sufficient for any use.
>
> The _mp_alloc alloc field is a home-brew float.  Allocations <= 255 will
> have have field values 1..255 which require no extra processing.
>
> An alternative _mp_alloc trickery would define the current allocation in
> terms of the _mp_size field, saying how much unused space there is.
> Such a field wouldn't need to be very large, but then there would be
> situation where GMP would need to realloce when an operand decreases a
> lot in size.
>
> To avoid any of this trickery, we could change the structure to be 192
> bits.  That would cost a lot in cache load for applications which
> e.g. use arrays of GMP numbers.  Like this:
>
>          signed long _mp_size     : 64;
>          unsigned long _mp_alloc  : 60;
>          unsigned long _mp_handler:  4;
>          mp_limb_t _mp_d;        // 64
>
> (These sizes are hard to overflow, except for those with lots of time
> and who can afford 8 Eibyte of memory.)
>