make check errors on recent MacOS versions appear to be due to new default linker in XCode

[Mark Mentovai]

Hans Åberg wrote:

> Of what importance is -flat_namespace to GMP? —Specifically, does it help
> performance?
>

It’s not important to gmp.

-flat_namespace changes loader semantics away from the traditional Apple
two-level namespace behavior. Some might argue that this is actually
desirable, because it enables more ELF-like behavior that the program may
be expecting. However, opting for these semantics can carry a load-time
performance penalty, and depending on the circumstance, even a correctness
penalty. If the program doesn’t actually need ELF-like flat namespace
semantics, and gmp doesn’t as far as I can see, -flat_namespace can only be
detrimental. Additionally, there are newer performance-oriented linker and
loader features such as what Apple calls “chained fixups”, directed toward
reducing module size and load time, that are unavailable when certain
legacy options like -flat_namespace are in effect.

-flat_namespace is coming in via libtool, but is also unimportant to
libtool itself. It’s only used there to configure allow_undefined_flag (
https://www.gnu.org/software/libtool/manual/html_node/libtool-script-contents.html#index-allow_005fundefined_005fflag)
to use -undefined suppress. -undefined suppress was the only way to achieve
this behavior on very old Mac OS X versions (<10.3), but it required the
flat namespace. Targeting newer macOS versions, -undefined dynamic_lookup
is a better option to achieve this, precisely because it doesn’t require
-flat_namespace. libtool erroneously treated recent macOS versions (≥11) as
Mac OS X 10.0 for this purpose, which is how -flat_namespace accidentally
reappeared, despite having become unnecessary in 2005.

As a separate matter, I don’t think that allow_undefined_flag is important
to gmp either, but libtool seems committed to using it by default if it’s
able to, never mind the fact that it’s not the traditional linker behavior
on any modern platform. Like -flat_namespace, -undefined dynamic_lookup is
one of those legacy ld options that’s incompatible with the “chained
fixups” size/speed optimization I described above. libtool’s continued use
of -undefined dynamic_lookup has already prompted them to take
https://git.savannah.gnu.org/cgit/libtool.git/commit/m4/libtool.m4?id=f0507df8a780e3fda3eef05bce2c74ec8d032c8e
as part of a “warnings whack-a-mole” exercise. It’s concerning that these
years of accumulation have created a situation where libtool, by default
and on behalf of its users, declines optimizations that would otherwise be
available “for free”. It’s possible for programs like gmp that use libtool
as part of their build system to overcome this by opting out of the use of
libtool’s allow_undefined_flag, which can be done by specifying
-no-undefined to libtool --mode=link (
https://www.gnu.org/software/libtool/manual/html_node/Link-mode.html#:~:text=%2dno%2dundefined).
In gmp’s automake-based build, this could go into libgmp_la_LDFLAGS and
libgmpxx_la_LDFLAGS, probably without incident:

diff -r ca451d583385 Makefile.am
--- a/Makefile.am Wed May 15 20:51:11 2024 +0200
+++ b/Makefile.am Thu Oct 03 13:01:07 2024 -0400
@@ -291,7 +291,7 @@
   $(MPN_OBJECTS) @mpn_objs_in_libgmp@ \
   $(PRINTF_OBJECTS)  $(SCANF_OBJECTS) $(RANDOM_OBJECTS)
 libgmp_la_LIBADD = $(libgmp_la_DEPENDENCIES)
-libgmp_la_LDFLAGS = $(GMP_LDFLAGS) $(LIBGMP_LDFLAGS) \
+libgmp_la_LDFLAGS = $(GMP_LDFLAGS) $(LIBGMP_LDFLAGS) -no-undefined \
   -version-info $(LIBGMP_LT_CURRENT):$(LIBGMP_LT_REVISION):$(LIBGMP_LT_AGE)


@@ -305,7 +305,7 @@
 libgmpxx_la_SOURCES = cxx/dummy.cc
 libgmpxx_la_DEPENDENCIES = $(CXX_OBJECTS) libgmp.la
 libgmpxx_la_LIBADD = $(libgmpxx_la_DEPENDENCIES)
-libgmpxx_la_LDFLAGS = $(GMP_LDFLAGS) $(LIBGMPXX_LDFLAGS) \
+libgmpxx_la_LDFLAGS = $(GMP_LDFLAGS) $(LIBGMPXX_LDFLAGS) -no-undefined \
   -version-info
$(LIBGMPXX_LT_CURRENT):$(LIBGMPXX_LT_REVISION):$(LIBGMPXX_LT_AGE)



Incidentally, this patch would also independently solve the problem of
-flat_namespace appearing while linking libgmp, and would thus also fix the
“make check” test failures.