mpf_eq revision

Kevin Ryde user42 at zip.com.au
Sun Mar 21 22:59:13 CET 2004

I'm looking at doing something about the long-standing 100000 vs
011111 business in mpf_eq, by coming up with a definition that can
make such values equal.  Proposed new words for the manual below.

There's a bit of a choice between whether values say like 15 and 16
should be considered equal to 4 bits or 5.  It's sort of 4 if you look
at the 15 and sort of 5 if you look at the 16.

I can't see it'll make much difference either way in practice, so only
needs to follow some tolerably sensible rule.  A relative difference
seems pretty clean to me, unless anyone has a better idea.

Incidentally, what I've put for op3==0 is a departure from the way the
code currently works.  Right now under op3==0 you must have the same
exponent in limbs, but any data.  This makes some some nearby values
equal (like 1.0 and 2.0) but others not (0.5 and 1.0).

I'm assuming this is either an oversight or arises only because op3==0
was never meant to be used.  Either way I think we can consider
ourselves free to tweak it, and I'm inclined to think op3==0 may as
well mean any two values equal.

- Function: int mpf_eq (mpf_t OP1, mpf_t OP2, unsigned long int op3)
Return non-zero if the first OP3 bits of OP1 and OP2 are equal, or
return zero if not.  Ie. test if OP1 and OP2 are approximately
equal.

Values like `1.00000...' and `0.11111...' are considered equal if
there's OP3 zeros in the first and OP3 ones in the latter, since
their difference is within OP3 bits precision.

If OP1 and OP2 have different signs, or if one is zero and the
other non-zero, then they are considered not equal.  The only
exception to this is when OP3 is zero, in which case any values at
all are considered equal.

The test can be viewed in terms of the relative difference between
OP1 and OP2.  `mpf_eq' returns true (for non-zero OP3) when

abs(OP1-OP2)       1
------------ <= -------
max(OP1,OP2)    1/2^OP3

The "max" in the denominator expresses the way `1000...' and
`0111...' are handled.  For example values 16 and 15 (`10000' and
`1111') are considered equal to 4 bits.