Opened 2 months ago

Closed 2 months ago

#8764 closed bug (fixed)

Usage of `sed' in GHC build system (Solaris build failure).

Reported by: kgardas Owned by:
Priority: normal Milestone: 7.8.1
Component: Build System Version: 7.8.1-rc1
Keywords: Cc:
Operating System: Solaris Architecture: x86
Type of failure: Building GHC failed Difficulty: Unknown
Test Case: Blocked By:
Blocking: Related Tickets:

Description

Hello,
I've found that (1) GHC's configure attempts to find the GNU sed and once found it's saved into the SED makefile variable. But (2) a lot of GHC's makefiles are using straight 'sed' instead of '$(SED)', i.e found GNU sed.
The reason for this bugreport is that on Solaris, the provided sed fails on parsing sed expression in rules-builddependencies.mk. I've solved this issue by fixing 'sed' to '$(SED)' there (patch attached), but the general question is if this should not be fixed everywhere.
Thanks!
Karel

Attachments (3)

0002-use-detected-GNU-sed-on-expression-where-Solaris-fai.patch (1.4 KB) - added by kgardas 2 months ago.
build-dependencies.mk (6.8 KB) - added by maeder 2 months ago.
replaced ;-usage by -e arguments
0002-fix-sed-expression-in-build-dependencies-rules-to-wo.patch (2.2 KB) - added by kgardas 2 months ago.
Christian provided patch in git patch

Download all attachments as: .zip

Change History (19)

comment:1 Changed 2 months ago by kgardas

  • Status changed from new to patch

comment:2 Changed 2 months ago by maeder

Don't we have an SED expert that can turn this sed expression into a backward compatible one? Who is the author of this sed expression? It is the only one that requires GNU SED!

comment:3 Changed 2 months ago by hvr

...maybe it's easier to express that operation in terms of awk?

For convenience, here's a copy of sed script in question:

/hs$$$$/ p                                      ;
/hs$$$$/ s/o /hi /g                             ;
/hs$$$$/ s/:/ : %hi: %o /                       ;
/hs$$$$/ s/^/$$$$(eval $$$$(call hi-rule,/      ;
/hs$$$$/ s/$$$$/))/                             ;
/hs-boot$$$$/ p                                 ;
/hs-boot$$$$/ s/o-boot /hi-boot /g              ;
/hs-boot$$$$/ s/:/ : %hi-boot: %o-boot /        ;
/hs-boot$$$$/ s/^/$$$$(eval $$$$(call hi-rule,/ ;
/hs-boot$$$$/ s/$$$$/))/'

comment:4 Changed 2 months ago by maeder

awk is currently not used for building (see attached log file of #8781) and maybe awk is not always GNU awk, too. (But GNU awk is detected by configure.)

In any case, I think, this sed script should be more comprehensible for the open source community.
Maybe splitting this into several sed calls would do the same job.

comment:5 Changed 2 months ago by kgardas

IMHO it would be the best option to replace the sed expression in question with something more POSIX compatible, i.e. something which at least pass thorough non-GNU seds distributed by let say FreeBSD and Solaris.
Unfortunately as a long time Solaris user, I have to admit that current state of GHC build system is absolutely cryptic for me and certainly I'm neither sed expert so I'm not able to distil at all what this make/sed combination should do at the end. Otherwise I would provide (or at least try) the fix/patch myself.
So if anybody from developers here can help with explanation what exactly is done or even better how exactly is that done, then perhaps me or Christian will be able to fix the expression and test on our Solarises...
Thanks!

comment:6 Changed 2 months ago by kgardas

  • Milestone set to 7.8.1
  • Version changed from 7.9 to 7.8.1-rc1

comment:7 follow-up: Changed 2 months ago by maeder

I believe, the following works, but still need to test it.

        sed -e '/hs$$$$/ p' -e '/hs$$$$/ s/o /hi /g' \                                                      
            -e '/hs$$$$/ s/:/ : %hi: %o /'                       \                                                      
            -e '/hs$$$$/ s/^/$$$$(eval $$$$(call hi-rule,/'      \
            -e '/hs$$$$/ s/$$$$/))/'                             \
            -e '/hs-boot$$$$/ p' -e '/hs-boot$$$$/ s/o-boot /hi-boot /g' \
            -e '/hs-boot$$$$/ s/:/ : %hi-boot: %o-boot /'        \
            -e '/hs-boot$$$$/ s/^/$$$$(eval $$$$(call hi-rule,/' \
            -e '/hs-boot$$$$/ s/$$$$/))/'                        \
            $$@.tmp2 > $$@

comment:8 in reply to: ↑ 7 Changed 2 months ago by kgardas

Replying to maeder:

I believe, the following works, but still need to test it.

        sed -e '/hs$$$$/ p' -e '/hs$$$$/ s/o /hi /g' \                                                      
            -e '/hs$$$$/ s/:/ : %hi: %o /'                       \                                                      
            -e '/hs$$$$/ s/^/$$$$(eval $$$$(call hi-rule,/'      \
            -e '/hs$$$$/ s/$$$$/))/'                             \
            -e '/hs-boot$$$$/ p' -e '/hs-boot$$$$/ s/o-boot /hi-boot /g' \
            -e '/hs-boot$$$$/ s/:/ : %hi-boot: %o-boot /'        \
            -e '/hs-boot$$$$/ s/^/$$$$(eval $$$$(call hi-rule,/' \
            -e '/hs-boot$$$$/ s/$$$$/))/'                        \
            $$@.tmp2 > $$@

This is great! It works here on both Solaris 11.1 and Solaris 10 update 8! If it also works on other platforms this would be the best solution of this issue IMHO.
Thanks for providing it! BTW: Are you able to provide this patch as a Git patch and attach to this ticket? If not, I may do this for you...

comment:9 Changed 2 months ago by maeder

Did you test with a non-GNU sed under Solaris? It worked for me under linux. Under Mac OSX I got a (possibly unrelated) failure:

 "inplace/bin/ghc-stage1" -hisuf hi -osuf  o -hcsuf hc -static  -H32m -O    -package-name base-4.7.0.0 -hide-all-packages
 -i -ilibraries/base/. -ilibraries/base/dist-install/build -ilibraries/base/dist-install/build/autogen -Ilibraries/base/
dist-install/build -Ilibraries/base/dist-install/build/autogen -Ilibraries/base/include   -optP-DOPTIMISE_INTEGER_GCD_LC
M -optP-include -optPlibraries/base/dist-install/build/autogen/cabal_macros.h -package ghc-prim-0.3.1.0 -package integer
-gmp-0.5.1.0 -package rts-1.0 -package-name base -XHaskell2010 -O2  -no-user-package-db -rtsopts      -odir libraries/ba
se/dist-install/build -hidir libraries/base/dist-install/build -stubdir libraries/base/dist-install/build -split-objs -d
ynamic-too -c libraries/base/./Control/Applicative.hs -o libraries/base/dist-install/build/Control/Applicative.o -dyno l
ibraries/base/dist-install/build/Control/Applicative.dyn_o
ghc-stage1: panic! (the 'impossible' happened)
  (GHC version 7.8.20140130 for x86_64-apple-darwin):
        tcIfaceGlobal (local): not found:
    base:Control.Applicative.$fApplicativeWrappedMonad{v ri}
    [(0c7, Identifier ‛base:Control.Applicative.<*>{v 0c7}’),

and under Solaris I got:

 gmake[4]: Entering directory `/local/home/maeder/haskell/ghc-7.8.20140130-src/libraries/integer-gmp/gmp/gmpbuild/mpn'
/bin/sh ../libtool --tag=CC   --mode=compile /opt/csw/bin/gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I.. -D__GMP_WITHIN_GMP -I.
. -DOPERATION_`echo fib_table | sed 's/_$//'`    -m32 -O2 -pedantic -fomit-frame-pointer -mtune=i386 -march=i386 -c -o f
ib_table.lo fib_table.c
../libtool: bad substitution

I suspect that gmp can not be compiled without GNU sed.

comment:10 follow-up: Changed 2 months ago by kgardas

The MacOSX looks indeed unrelated. I've been hit by it also during the weekend and it was caused probably by some object files lying around from previous and different configured build. See #8374

Solaris issue is interesting, but I'm not able to duplicate it here. So yes, I've tested your patch on both Solaris 11.1 and Solaris 10 and both are able to compile GHC HEAD well.
Please keep in mind that I'm minimalistic here, so I configure standard Single Unix Spec. v 3 PATH here (/usr/xpg6/bin:/usr/xpg4/bin:/usr/bin:/usr/ucb....) and I'm running w/o any CSW provided software! My LD_LIBRARY_PATH variable is undefined too. The build is configured with just

$ ./configure

and runs fine. Please try to duplicate your issue using my GHC 7.6.3 distro from here: https://app.box.com/s/2nk7ffnr33lpg7b3mxeh -- unset any PATH pointing to /opt/csw. And build with:

$ ./configure
$ /opt/csw/bin/gmake

BTW: You will probably need to reinstall cabal/alex/happy to be also CSW libraries independent, but this is possible with my bindist provided above.

Changed 2 months ago by maeder

replaced ;-usage by -e arguments

comment:11 Changed 2 months ago by maeder

my second test on OSX went through on a clean tree! (and I could reproduce the crash in the old tree.)

comment:12 Changed 2 months ago by maeder

Also building under Solaris went through! (I've got no explanation for the above libtool problem.)

comment:13 in reply to: ↑ 10 Changed 2 months ago by maeder

Replying to kgardas:

[...] using my GHC 7.6.3 distro from here: https://app.box.com/s/2nk7ffnr33lpg7b3mxeh [...]

I propose to make this binary-dist available via
http://www.haskell.org/ghc/download_ghc_7_6_3#binaries

Changed 2 months ago by kgardas

Christian provided patch in git patch

comment:14 Changed 2 months ago by Austin Seipp <austin@…>

In e638acb6a825559fbd93bdaf8ae2704535c157f1/ghc:

fix sed expression in build dependencies rules to work well with non-GNU sed (fixes #8764)

The patch is provided by Christian Maeder <Christian.Maeder@dfki.de>

Signed-off-by: Karel Gardas <karel.gardas@centrum.cz>
Signed-off-by: Austin Seipp <austin@well-typed.com>

comment:15 Changed 2 months ago by thoughtpolice

  • Status changed from patch to merge

comment:16 Changed 2 months ago by thoughtpolice

  • Resolution set to fixed
  • Status changed from merge to closed
Note: See TracTickets for help on using tickets.