GMP license problem, anyone?

[Bill Hart]

>We switched from LGPL 2.1 to LGPL 3 between GMP 4.2.1 and 4.2.2.

>Did that cause any compatibility concerns for a project you know
>about, or are directly involved in?  Have the problems been solved, or
>are they still relevant?

>[Please reply to the list, not just to me.]

Hi TG,

I appreciate this question and the openness of discussing it on this
list. I am working under the assumption that this really originated
from you and that you wouldn't bring it up unless the matter was
worrying you.

Whenever I or my colleagues discuss licensing at conferences, we have
the rule that everyone present must have drunk some alcohol. So I hope
you will have a beer before reading the following.

Unfortunately, I can confirm that the license change caused a serious
issue for my project FLINT. I may have mentioned the project before,
it is a medium sized (~50,000 lines of tighly coded C) library for
doing polynomial arithmetic and many other number theoretical
calculations, and is (currently) built on top of GMP. It is currently
a standard package in the SAGE computer algebra system and that
project is my main customer (though I am aware of other people using
it). It is as far as I am aware, uniformly faster than all other
packages, whether open or closed source, for doing various
computations in univariate polynomial arithmetic. Naturally, a lot of
that speed can be achieved due to the excellent assembly code in GMP,
though it also benefits greatly from other assembly patches which have
not been merged into the GMP project.

Some months ago now, at a conference, the license issue was brought
up. Because SAGE makes use of numerous packages, some of which are GPL
v2+ it cannot change to GPL v3+ and numerous members of the SAGE
community appear to be against doing that on principal anyway. I have
to admit I am also such a person. FLINT will always remain GPL v2+ and
so cannot ever use future versions of GMP if they remain LGLP v3+.
This leaves me with few options.

At the conference, many people aired their opinions. Paul Zimmermann
was an invited speaker and he spoke strongly in favour of *NOT*
forking GMP, as this would split future efforts over two projects
instead of one. I strongly respect Paul's opinion and recognise his
outstanding contributions to GMP and other projects over the years.

For some months it remained the opinion of myself and various
colleagues that we should not fork GMP, though we had a long list of
reasons to do so, related to the direction we wanted to take our
respective projects. However, since that time a decision was
deliberately made to maintain our own version of GMP and to release
what is essentially a fork, under a different name.

I need to point out firstly that what I say now represents my own
opinions entirely and is not meant to represent the opinons of my
colleagues or be said in any way on their behalf. I also will not name
our new project, as I do not wish to appear as though I am trying to
gain support for it on the GMP list.

I should also point out that the work we are doing on our own version
of GMP (which I will call X to save typing) and the work I do on
FLINT, is in many respects insignificant in comparison to the much
larger businesses which you deal with.

Although we have applied for various grants related to our work and
already have some corporate sponsorship, our intended audience is
somewhat different, financially speaking. The main difference is that
our projects are *GPL* v2+ not LGPL. We felt that the LGPL was hurting
our project objectives as much as v3+ (I'll leave the reasons unsaid,
as I know you already realise what they are, and sympathise fully, as
various statements on your website make plain - and I completely
understand why GMP has to be LGPL).

At any rate, my point is that the type of sponsorship we are
attracting (and targetting) is likely to be very different to the sort
of sponsorship that GMP gets, and we are in no position to offer
financial support to people working on the official GMP project, so in
some senses, what I have to say has emotive value only, not financial
value.

The aims of our new project are also quite different to GMP. Here is a
list of things we have done or are nearly done doing:

1) Merge Pierrick Gaudry's AMD64 patches.
2) Merge Jason Martin's Core 2 Patches.
3) Make GMP build natively on Microsoft Windows machines (with Brian
Gladman's extensive contributions and Michael Abshoff as release
manager, we are nearly there I think).
4) Get rid of M4 as a macro language, especially for the x86 assembler code.
5) Convert the x86 assembler code to Intel format from gas format and
use Yasm as the x86 assembler (hopefully support will be added to Yasm
in the future for other chips).
6) Make the build system recognise more modern chips (e.g. Core 2). I
realise GMP 4.2.2 already does this, but we have had to do it all
again.
7) Support for other platforms as first class citizens, which I am
afraid I have forgotten the names of: I think it is OSX (XCODE) and
certain Sun platforms.
8) Fully public svn repository and fully open development (GPL affords
us this possibility).

Here is what we have planned for the future.

6) Get rid of autoconf as a build system and switch to a python based
system (I am now very skeptical this can be done - I just spent about
three months wasting my time trying to work through all the issues
involved in getting some of the above to work and I now appreciate how
ridiculously much autoconf is doing for us).
7) Full support for parallel processing. In particular we are writing
a whole new module using OpenMPI for operation on multicore and
multiprocessor machines.
8) Better support for (single and multidimensional) arrays of
integers. The next generation FLINT fmpz integer type will be
incorporated as a new integer type in X. This will require some
rewriting of the existing mpz layer, but will operate alongside mpz,
not replace it. It gives a noticeable performance improvement for
integers of at most 62 bits and for arrays of integers (e.g.
polynomials or matrices). I have written proof of principle code for
this and demonstrated it on an LLL application. But I have not yet
incorporated any of the code directly into X.
9) Preconditioned arithmetic.
10) Incorporate the FLINT FFT code for large integer multiplication.
It performs about as well as the new FFT code of Paul Zimmermann and
colleagues. I'd really like to see his FFT in GMP 5 and our FFT in X.

Anyhow, now that you see what we are doing, back to the licensing
issue. Clearly FLINT was badly hurt by the license change. It means I
can't use GMP v4.2.2 or above for FLINT. I am therefore forced to add
support for whatever I need directly to my own version of GMP *based
on GMP 4.2.1*. This means duplicating all the effort you and other GMP
contributors are making (with the possible exception of the existing
patches for things like fast GCD and division, which are on the GMP
website). This is not how open source is supposed to be!!

As a result of all this effort, no work has occurred on FLINT for a
number of months as I tried to get X up to a standard suitable for use
in FLINT. In particular, GMP 4.2.1 doesn't recognise Core 2 chips and
understanding the build system may be a trivial thing for someone like
you who has spent years working on it, but it took a lot of effort for
me (there are still some things I don't understand all that well).

But is there realistically anything you can do about this? Now that
you have turned your copyright over the the FSF, can you change the
license on that code? You don't have control over the licensing any
more.

Obviously I would very much like X to remain fully GMP compatible in
the future, especially when GMP 5 comes out, since there's some really
nice stuff going into GMP. But what options do I have? I am going to
have to redesign and reimplement all that stuff, without looking at
the code of the people contributing to GMP. And this is open source!?

It would be wonderful if we could actually include improvements to GMP
in X.  Given the completely different direction we are taking X, it
would be lovely if both projects were able to benefit each other
somehow. The one difficulty I see there though is that I am definitely
not turning over my copyright to the FSF, and I certainly won't
contemplate making any of my code LGPL anything, for at least one year
after it has been released under the GPL. There are contributors to X
who may not ever wish to LGPL their code, as it never suits their
goals. But I personally don't want my code improved upon and sold back
to me. (There will be some contributions by others to X that will be
LGPL'd, I suppose).

In answer to your other question, yes I am aware of other projects for
whom the GMP licensing issue, v3+ vs v2.1+ was a problem. But I don't
wish to speak on their behalf.

I'm really interested to hear what your thoughts are on these issues.

Bill.