Quality for binary distributions
tg at gmplib.org
Sun Jun 21 12:43:39 UTC 2015
nisse at lysator.liu.se (Niels Möller) writes:
Here's a common scenario where make check may fail to detect problems.
Say a distributor compiles gmp, runs make check, and builds some type of
binary package, which is distributed to users. For supported platforms
(i.e., x86), fat builds are typically used. Now, the user has a
different cpu flavour, so may exersize different code than make check on
the machine where gmp was built.
As far as I understand, it would typically affect only assembly code
(which is supposedly well tested and not subject to compiler bugs). But
if the fat machinery selects between assembly and C implementation (does
it ever do that?), the user could also see compiler bugs not visible on
the build machine.
Can we do anything about this? Like, distribute some gmp checking tools
to end users?
What got me thinking about this was a gnutls bug report about occasional
problems with bad rsa signatures (which could be a bug in gnutls,
nettle, gmp, or the hardware). It would be good to be able to check if
the problem is below or above gmp. See
On the nettle side, I ought to add code to check validy of all generated
signatures before passing them on, since an error in the computation may
leak the secret key.
At https://gmplib.org/devel/GMPng.html there is an action item which
Make GMP_CPU_TYPE fat CPU selection standard for a fat build (but perhaps
rename it to something more specific, GMP_FAT_CPU_TYPE_SELECT). Motive:
It is marked as green, but it is not yet checked in. I'll make it red again.
It can surely happen that some "fat function" are handled by asm on one CPU
and in C in another CPU.
The main miscompilation issue is with -march and -mtune options. Even clang
doesn't unconditionally miscompile GMP (except on some non-x86 platforms);
but every tested release of it except one miscompiles for some such option.
While certainly possible, it is quite unlikely that we hit a compiler bug
through the fat mechanism.
1. It will be rare that we'll hit C code at run-time through the fat vector.
2. That snippet of C code needs to be miscompiled.
So, how can we handle this? The GMP_CPU_TYPE trickery, allowing override of
the cpuid instruction makes that almost possible. We also need to provide a
mechanism for looping over all alternative CPUs, and then get people to use
it. But then there will be failures, "Illegal Instruction" traps, for CPUs
which are newer than the one of the tester. (We could try to detect that to
avoid such false alarms, but this is getting very complex...)
We talked about blacklisting broken compilers (read: clang) recently. I
started working on it, but then realised that say clang 3.6 on one GNU/Linux
dist is not the same as on another. For instance, on one Ubuntu install here
(ivyubu64v1504.gmplib.org), most clang releases seem OK, on other dists the
same releases miscompile for several subarchs. What to do? We could compile
from official source in order to find the presumably buggiest version, then
blacklist that version and its offspring. Messy, and a fairly blunt tool
against compiler bugginess.
The most effective protection for end users is that we find miscompiled code
snippets and make these into GMP configure tests. Unfortunately, this is a
lot of work to maintain.
Please encrypt, key id 0xC8601622
More information about the gmp-devel