Additional memory handler features.

David M. Warme David at Warme.net
Sun Jan 4 08:45:04 UTC 2015


> >And then there's some *potential* to use them to implement better recovery
> > on allocation failure, as has been discussed recently, but no one is doing
> > that for real as far as I know.
>
> Er, people have been using the custom allocators for ...

Two classic "cleanup" solutions regarding GMP:

1. Make __tmp_marker be a structure:

		struct __tmp_mark_record {
			struct __tmp_reentrant_t *	root;
			struct __tmp_mark_record *	prev;
		};

    TMP_DECL becomes this:

		struct __tmp_mark_record	__tmp_marker;
		struct __tmp_mark_record **	__tmp_mark_root_addr;

    TMP_MARK would have to do this:

		/* Expensive ref to thread-local var.  Do only once. */
		__tmp_mark_root_addr = &__tmp_mark_root;
		__tmp_marker.root = 0;
		__tmp_marker.prev = *__tmp_mark_root_addr;
		*__tmp_mark_root_addr = &__tmp_marker;

    and TMP_FREE would have to do one additional step:

		*__tmp_mark_root_addr = __tmp_marker.prev;

    where __tmp_mark_root is a thread-local variable that can be queried
    by a new GMP function during cleanup.  Single-threaded configurations
    could just use a plain static variable.

    This could be coded in a way that makes the "prev" member of
    __tmp_mark_record (and the code that manipulates it) be optional,
    controlled by a --disable-foo switch to configure.

2. Have GMP tell the allocator which blocks are "temporary".  Armed with
    this knowledge (together with the stack pointer address at the time that
    alloc() is called) provides all the info necessary to identify all
    off-stack heap-allocated temporaries that must be freed to cleanup all
    aborted GMP stack frames.

> > To make it possible to really stay in full control of allocation with
> > GMP, another option is to push forward with the itch/scratch-ifying of
> > the mpn functions, and then stick to using mpn.

Excellent suggestion.  Thank you!

I've worked through some of the details on this, and there are some
glitches.  My app is itself a library that uses GMP objects at the API it
presents to the caller / application.

Assume my library only uses mpn.  mpn only allocates "temporary limb arrays",
(e.g., FFT/Toom-Cook workspaces) so one might assume that every invocation
of my memory handling functions is a "temp limb" request and I can use
door number 2 cleanup methods.  Unfortunately, even if the end-user's app
does not establish its own memory handlers, it would still invoke my own
memory handlers both from mpn and from the mpx_t layers as it uses GMP
functions, thereby breaking the assumptions needed to make it work correctly.

Another solution is to make a customized copy of GMP (or at least the
mpn portions of GMP) with renamed symbols, packaged as a separate library
in order to comply with LGPL.  The custom copy would have its own memory
function pointers, and care would be needed when copying data across my
API boundary (between two different copies of GMP).  The trick is to
accomplish this with few (and therefore contributable) changes to GMP
itself.

I will certainly explore this option.

By the way, my previous suggestion for "dynamically scoped" exception
handlers looks exactly like door number 1 above.  Different types of
objects, different root pointer, and the record probably resides in
the stackframe of a user function, not a GMP function.  Add the new
"record" structure and two new functions to gmp.h:

	struct gmp_exception_handler_record {
		gmp_exception_handler_func_t *		handler;
		struct gmp_exception_handler_record *	prev;
	};

	void gmp_push_exception_handler (volatile gmp_exception_record * p, gmp_exception_handler_func_t * h)
	{
		p -> handler	= h;
		p -> prev	= thread_local_exception_root;
		thread_local_exception_root = p;
	}

	void gmp_pop_exception_handler (volatile gmp_exception_record * p)
	{
		thead_local_exception_root = p -> prev;
	}
...
	bool user_exception_handler (...) { ... }

	void user_function ()
	{
	volatile struct gmp_exception_handler_record	local_record;
		...
		gmp_push_exception_handler (&local_record, user_exception_handler);

		/* Code protected by user_exception_handler. */

		gmp_pop_exception_handler (&local_record);
	}

Once a handler is established, the user can also directly poke different
handler functions into local_record.handler on-the-fly (assuming that
pointers are stored atomically, i.e., one instruction, not two or more).
Exception handlers could do this also, providing a state-transition
behavior for successive and/or nested exceptions.

There may still be some embedded CPU's that need 2 or more instructions
to load/store a pointer.  If so, then care must be exercised when updating
the `thread_local_exception_root' variable in a truly atomic manner
(e.g., with interrupts disabled).

This solution is far better than adding a handler pointer to every
mpx_t object.  Also better than a single, static handler pointer, which
can be tricky to share among different libraries.

Cost is zero except when exceptions are signaled, or handler state is
being altered.  It allows a mechanism where each handler can return a
flag indicating whether the exception was handled or not.  If not, then
the exception is propagated to successively higher handlers.  A static
handler pointer can be the handler of last resort.

Cleanup can also be implemented as a special "unwind-protect" exception,
raised into each aborted stack frame's handler just before doing longjmp().

This more general application might require placing this mechanism into
glibc and standardizing the interfaces more widely.

David



More information about the gmp-devel mailing list