m4-not-needed

[Jay K]

 > Subject: Re: m4-not-needed


I am confused by the quoting, but my response should still be sensible.

 > I think you have different PATHs in your batch job
 > and 'live', and thus find different flex commands.


This is not the cause of the overall problem.
  i.e. it is not my case, and I fully understand and have seen the
  problem in question. Should I explain it again? The problem
  has many steps so my explanations have perhaps been unclear.


The problem occurs with just one flex on the system.
I suspect Emmanuel's differences were due to signal(sigpipe, sigign)
being in force or not, but I don't know.
That *is* why most people don't have a problem with this -- running
gmp/configure from within sh doesn't have the problem, from with within
Python does. Python is where the signal(sigpipe, sigign) comes.



>   I see that the mercurial code now leaves the variable M4 untouched in
>   case it already exists, which seems more reasonable than overriding it
>   with bogus content unconditionally.

That is slightly better, as it allows a workaround in the caller of M4=m4.
But still, caller should not have to set M4.

> flex is run from within gmp's configure. I assume it's possibly needed for the
> calc demo (?). The only thing configure needs to know is the name of the
> output file, e.g. whether it's lex.yy.c or something else.
> 
> Since flex is from from gmp's configure, the changes to $M4 which are
> made by the configure script propagate to the children processes of
> course, including flex.


The fact that the M4 change is "temporary" does not make the current code good/ok/great.
  In fact, sort of, it makes it worse -- since flex is being configured one way
  and then run another way.


I understand the goal is only to find the output file name.
And even with that slight goal, the code is messed up. Setting M4 messes it up.



Here is a full explanation again:


gmp/configure wants to find the flex output file name.
As part of normal runs and as part of gmp/configure, flex does run m4.
  A little surprising, but true. Flex uses m4.


The logic flex uses to find m4 is that if the $M4 variable is set, it uses it.
If the M4 variable is not set, it uses plain "m4" (presumably subject
to exec*p* $PATH search).


Flex creates the lex.yy.c file.
And then it runs m4 ($M4).
When M4=m4-not-needed, running m4 fails, and a signal is generated.
As I recall, sigpipe.


Normally, flex then immediately dies because of this signal, this signal being unhandled.
That leaves the lex.yy.c file present, for gmp/configure to notice, and gmp/configure moves along.


However under Python, there is signal(SIGPIPE, SIGIGN), so the run of m4-not-needed
doesn't cause flex to immediately die. It moves along. But I guess it still hits some error,
such as maybe reading the m4 pipe producing no output. Eventually deleting the lex.yy.c file
as part of some cleanup. And then gmp/configure can't find a file to determine the filename.


As well, it should be clear, if you don't gmp/configure "none", then M4=m4, and flex "works"
in a more normal way -- running m4 works, no signal.



The fix is that the M4 environment variable should never be touched.


It is unfortunate that $M4 is used by flex.
But as such, gmp/configure should not touch it.
For gmp/configure, M4 is meant to just be an internal temporary variable.
It does not mean to pass it on down to flex.
In fact, I think it doesn't export it. That it is exported might be due to
running as part of gcc/configure. Or, perhaps, as part of the presumably little
used feature of getting flex to use it.


It is also unfortunate, I guess, that Python does signal(SIGPIPE, SIGIGN).
   It is ok in general for Python, but it should first get the signal handler and
   restore it for fork/exec. I think.


At this point, it might be a compatibility problem in Python.
Python is popular and people run build automation under it.


gmp/configure can be fixed in various ways.
The easiest is probably to use a more unique variable name for its internal purposes.
Such as GMP_CONFIGURE_M4.


Another is maybe avoid any variable at all.
Or to reorder the checks such that M4 is set after running flex.


I understand the problem is a little hard to understand (many steps) and a little hard to reproduce, but
hopefully enough of us has been clear enough to be convincing.


I will gladly explain it again if anything is unclear. I will even solidify some of the
details if needed (such as which signal is involved). I will gladly test
proposed patches. I will gladly provide patches if they will likely be accepted.
Please specify which of:
  rename the M4 variable 
  use M4 after running flex 
  don't set any variable 
is desired.


(I suspect others would volunteer as well: Nightstrike and Emmanuel?)



Thanks,
 - Jay