0.1387778780781445675529539585113525390625e31

John P. Hartmann jphartmann at gmail.com
Tue Feb 17 15:56:55 CET 2009


With version 4.2.4 (and also older versions) the subject number is
converted to this string of hex digits:

 00000011 8427b3b4 a05bc8a8 a4de8459 86800000 00000000 00000000

The number is then rounded to a 112-bit number (a L floating constant
in IBM parlance).  The truncation point is between the 6 and 8.  IBM's
High Level Assembler truncates the last digit to 6 (for reasons it
does not divulge), while with gmp the last digit becomes 7, because of
rounding.

IBM's HLASM uses an IBM-proprietary infinite precision facility.

One of the two has to be wrong.  To nail this down, I'd like to follow
the various stages of the conversion to see what carries run how far.
Howto?

I tried doing the conversion to a variable that had ten million digits
precision, but the result was the same (as it should be).

Thanks,

   j.

Test case:

#include <stdio.h>
#include "gmp.h"

/* Forward declarations:                                             */
/* End of forward declarations.                                      */

int
main(int argc, char ** argv)
{
   mpf_t accum;
   int i;
   unsigned int n;

   printf("version %s %d %d %d \n", gmp_version, __GNU_MP_VERSION,
__GNU_MP_VERSION_MINOR, __GNU_MP_VERSION_PATCHLEVEL);
   mpf_init2(accum, 112);
   mpf_set_str(accum, "0.1387778780781445675529539585113525390625e31",
10);              /* Get value */
   for (i=accum->_mp_size-1; 0<=i; i--)
   {
      n=accum->_mp_d[i];
      printf(" %08x", n);
   }
   printf("\n");
   return 0;
}


More information about the gmp-discuss mailing list