Additional memory handler features.

[Niels Möller]

"David M. Warme" <David at Warme.net> writes:

> I think the use case has already been made, at least in general terms.

A *concrete* use case would help. I'm pretty sure the involved tradeoffs
are non-trivial. To make progress, please help us understanding the
tradeoffs, and please write a bit more concisely and focused. I think
we're all doing gmp work in our spare time, and it's very time consuming
to digest and respond to a big wall of text.

>> I believe this design was mistake, and it is under
>> reconsideration. See
>> https://gmplib.org/list-archives/gmp-devel/2012-January/002161.html
>
> I disagree.  For custom memory allocation systems such as I am working with,

It would probably aid the discussion if you could briefly spell out a
bit more of the concrete context in which your working.

> requiring that the exact same size be passed to all three functions (for a
> given block) is both a reasonable requirement and can also be *usefully*
> exploited, e.g., for performance benefits.

Do you have any examples of libraries with custom allocation functions
which you think think gets this right?

My gut reaction is that it's trivial to store the length in the
allocation header. Sure, it's some waste to duplicate this information
both in the memory allocator's own structures and in, e.g., the mpz_t
object. So this boils down to a tradeoff between simplicity and
performance. I'm no expert in writing memory allocators, but I'd also
expect that if you want to implement realloc so that a resize from size
n to size n-1 is cheap, the allocator probably needs its own size field
for the recording the "real" size anyway, since it may differ from the
size requested by the application.

> In my opinion, since it is apparently either inefficient or technically
> infeasible for mpz_sizeinbase() to always return the exact answer, then I would
> abandon its use within mpz_out_str (and its cohorts), and instead write the ASCII
> data into a linked-list of temporary structures, each containing a fixed-sized
> array of characters, and a "next" pointer.

It's cheap to compute up front how much storage is needed, with an error
of at most one byte. I doubt that using a linked list of storage buffers
can ever be the right solution under those circumstances.

On second reading of my old message on this, I think a better way to fix
this bug, assuming we decide to keep the current allocation interface,
is to bite the bullet and let mpz_out_str realloc the area in the case
that it initially allocated a byte too much. And hope that (i) such a
reallocation is a cheap operation, and (ii) it happens too rarely matter
even if it the realloc implies a copy operation.

>> I agree a special stack allocator could have a place here (in
>> particular, if for some reason running with a very small execution
>> stack). And it could be done internally in gmp.
>
> Is GMP providing a custom memory allocation facility or not?

If it is beneficial in general to use a stack for these allocations (I'm
thinking of a largish memory area with only start address, size, and
stackpointer needed to keep track of the allocation state), then it
makes a lot of sense for GMP to implement TMP_ALLOC like that, and get
the large block using the application's allocation function.

Which doesn't rule out further customization specifically of the
TMP_ALLOC facility, if needed.

Regards,
/Niels

-- 
Niels Möller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.