Invalid operand size in inline assembler instruction on x86-64 with LTO
chortos at inbox.lv
Mon Oct 31 21:12:27 UTC 2016
GMP 6.1.1 (and probably earlier versions) uses a wrong constraint in the x86 inline assembler code for the MPN_IORD_U macro that blows up (fails to assemble) on x86-64 when compiled using GCC 6 with link-time optimization enabled. The problem occurs when a compile-time constant does not fit in a 32-bit immediate operand, so my guess is that it is specifically triggered by constant propagation across source files.
The code currently uses a constraint that allows any compile-time constant as an immediate operand, but x86-64 supports only 32-bit immediate values (even in 64-bit instructions), whereas the value MPN_IORD_U uses with this constraint may be longer than that. GCC, at least since version 3.1.1, actually provides a constraint just for such an occasion, which allows a compile-time constant as an immediate operand if and only if its value fits in 32 bits. In fact, GCC provides two such constraints: one for signed values (for sign-extending instructions) and one for unsigned values (for zero-extending instructions). MPN_IORD_U is used only with sign-extending instructions (ADD and SUB), so the fix is easy: simply replace the currently used constraint with the signed 32-bit immediate constraint.
I have not investigated whether the same bug appears in other places. GMP currently builds successfully for me with the below patch applied.
Here is a patch:
diff --git a/gmp-impl.h b/gmp-impl.h
index 6987581..dcc233b 100644
@@ -2687,7 +2687,7 @@ __GMP_DECLSPEC mp_bitcnt_t mpn_remove (mp_ptr, mp_size_t *, mp_srcptr, mp_size_t
ASM_L(done) ":\n" \
: "=r" (__ptr_dummy) \
: "0" (ptr), \
- "ri" ((mp_limb_t) (incr)), "n" (sizeof(mp_limb_t)) \
+ "re" ((mp_limb_t) (incr)), "n" (sizeof(mp_limb_t)) \
: "memory"); \
} while (0)
More information about the gmp-bugs