major formatted output function bug with %c and the value 0

[Vincent Lefevre]

On 2023-12-03 20:19:10 +0100, Vincent Lefevre wrote:
> With GMP 6.3.0, the formatted output functions do not handle %c
> with the value 0 correctly. For gmp_sprintf, the return value is
> incorrect.

In printf/sprintffuns.c, function gmp_sprintf_format(), I suppose that

  vsprintf (buf, fmt, ap);
  ret = strlen (buf);

should actually be something like

  ret = vsprintf (buf, fmt, ap);
  if (ret < 0)
    ret = 0;

to avoid issues due to non-terminating null characters (not tested).

But I'm wondering how an error is possibly detected. The comment is:

/* If vsprintf returns -1 then pass it upwards.  It doesn't matter that
   "*bufp" is ruined in this case, since gmp_doprint will bail out
   immediately anyway.  */

> For gmp_asprintf and gmp_vasprintf, this is either a buffer overflow
> (according to the GMP manual: "The block will be the size of the
> string and null-terminator.") or, in case this is an error in the
> GMP manual, possible memory corruption when freeing the allocated
> memory, if the custom memory allocation function cares about the
> size parameter.

This should be checked and the GMP manual should be fixed.

> 
> Testcase for gmp_sprintf:
> 
> ------------------------------------------------------------
> #include <stdio.h>
> #include <gmp.h>
> 
> static void test (int flag)
> {
>   char s[3] = { 1, 1, 1 };
>   int r;
> 
>   r = (flag ? sprintf : gmp_sprintf) (s, "%c", 0);
>   printf ("%4s: r = %d, s = { %d %d %d }\n",
>           flag ? "libc" : "gmp", r, s[0], s[1], s[2]);
> }
> 
> int main (void)
> {
>   test (0);
>   test (1);
>   return 0;
> }
> ------------------------------------------------------------
> 
> which currently gives:
> 
>  gmp: r = 0, s = { 0 0 1 }
> libc: r = 1, s = { 0 0 1 }
> 
> MPFR has various issues concerning %c with the value 0, but an
> attempt to fix them fails due to
> 
>   length = gmp_vasprintf (...);
> [...]
>   mpfr_free_str (s);
> 
> which is similar to GMP's tests/misc/t-printf.c file, which contains
> 
>   got_len = gmp_vasprintf (&got, fmt, ap);
> [...]
>   (*__gmp_free_func) (got, strlen(got)+1);
> 
> But replacing
> 
>   mpfr_free_str (s);
> 
> by
> 
>   mpfr_free_func (s, length + 1);
> 
> i.e. using the return value length instead of strlen(s), also fails.
> I suppose that this is related to the incorrect return value.

Well, the failure was due to another mpfr_free_func, no longer this
one (I've now completely fixed MPFR, I think).

-- 
Vincent Lefèvre <vincent at vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)