`make sdist-ghc` broken

I noticed this because it resulted in the PPA builds of GHC HEAD failing; I was able to hunt this down to bdd2d286 which updated libraries/Cabal;

Basically, the following invocation (where ghc/alex/happy/etc. can only be found in /opt/ghc-stage0-toolchain/bin, and nowhere else; so this relies on configure to capture their absolute locations; however, this is not what causes the failure below)

./boot

PATH=/opt/ghc-stage0-toolchain/bin:$PATH ./configure

make sdist-ghc

results in

...
test ! -d testsuite || make -C testsuite distclean
"rm" -rf sdistprep/ghc/ghc-8.3.20171109/libraries/tarballs/  
"rm" -rf sdistprep/ghc/ghc-8.3.20171109/libraries/stamp/  
"rm" -rf sdistprep/ghc/ghc-8.3.20171109/compiler/stage[123]  
"rm" -f sdistprep/ghc/ghc-8.3.20171109/mk/build.mk  
for i in parallel random primitive vector dph; do rm -rf sdistprep/ghc/ghc-8.3.20171109/libraries/$i/; done
cd sdistprep/ghc/ghc-8.3.20171109 && "/usr/bin/find" mk rules docs distrib bindisttest libffi includes utils docs rts compiler ghc driver libraries libffi-tarballs iserv \( -name .git -o -name "autom4te*" -o -name "*~" -o -name "\#*" -o -name ".\#*" -o -name "log" -o -name "*-SAVE" -o -name "*.orig" -o -name "*.rej" \) -print | "xargs"  "rm" -rf
"cp" compiler/stage2/build/CmmLex.hs sdistprep/ghc/ghc-8.3.20171109/compiler/cmm
mv sdistprep/ghc/ghc-8.3.20171109/compiler/cmm/CmmLex.x sdistprep/ghc/ghc-8.3.20171109/compiler/cmm/CmmLex.x.source
"/opt/ghc-stage0-toolchain/bin/happy"  -agc --strict    compiler/cmm/CmmParse.y -o compiler/stage2/build/CmmParse.hs
"cp" compiler/stage2/build/CmmParse.hs sdistprep/ghc/ghc-8.3.20171109/compiler/cmm
mv sdistprep/ghc/ghc-8.3.20171109/compiler/cmm/CmmParse.y sdistprep/ghc/ghc-8.3.20171109/compiler/cmm/CmmParse.y.source
"/opt/ghc-stage0-toolchain/bin/alex"  -g --latin1   compiler/parser/Lexer.x -o compiler/stage2/build/Lexer.hs
"cp" compiler/stage2/build/Lexer.hs sdistprep/ghc/ghc-8.3.20171109/compiler/parser
mv sdistprep/ghc/ghc-8.3.20171109/compiler/parser/Lexer.x sdistprep/ghc/ghc-8.3.20171109/compiler/parser/Lexer.x.source
"/opt/ghc-stage0-toolchain/bin/happy"  -agc --strict    compiler/parser/Parser.y -o compiler/stage2/build/Parser.hs
"cp" compiler/stage2/build/Parser.hs sdistprep/ghc/ghc-8.3.20171109/compiler/parser
mv sdistprep/ghc/ghc-8.3.20171109/compiler/parser/Parser.y sdistprep/ghc/ghc-8.3.20171109/compiler/parser/Parser.y.source
"inplace/bin/mkdirhier" utils/hpc/dist-install/build//.
"/opt/ghc-stage0-toolchain/bin/happy"  -agc --strict    utils/hpc/./HpcParser.y -o utils/hpc/dist-install/build/HpcParser.hs
unused terminals: 1
"cp" utils/hpc/dist-install/build/HpcParser.hs sdistprep/ghc/ghc-8.3.20171109/utils/hpc/
mv sdistprep/ghc/ghc-8.3.20171109/utils/hpc//HpcParser.y sdistprep/ghc/ghc-8.3.20171109/utils/hpc//HpcParser.y.source
"inplace/bin/mkdirhier" utils/genprimopcode/dist/build//.
"/opt/ghc-stage0-toolchain/bin/alex"  -g    utils/genprimopcode/./Lexer.x -o utils/genprimopcode/dist/build/Lexer.hs
"cp" utils/genprimopcode/dist/build/Lexer.hs sdistprep/ghc/ghc-8.3.20171109/utils/genprimopcode/
mv sdistprep/ghc/ghc-8.3.20171109/utils/genprimopcode//Lexer.x sdistprep/ghc/ghc-8.3.20171109/utils/genprimopcode//Lexer.x.source
"/opt/ghc-stage0-toolchain/bin/happy"  -agc --strict    utils/genprimopcode/./Parser.y -o utils/genprimopcode/dist/build/Parser.hs
"cp" utils/genprimopcode/dist/build/Parser.hs sdistprep/ghc/ghc-8.3.20171109/utils/genprimopcode/
mv sdistprep/ghc/ghc-8.3.20171109/utils/genprimopcode//Parser.y sdistprep/ghc/ghc-8.3.20171109/utils/genprimopcode//Parser.y.source
make[1]: *** No rule to make target 'libraries/Cabal/Cabal/dist-install/build/Distribution/Parsec/Lexer.hs', needed by 'sdist_libraries/Cabal/Cabal_dist-install_Lexer'.  Stop.
Makefile:161: recipe for target 'sdist-ghc' failed
make: *** [sdist-ghc] Error 2

Trac metadata

Trac field	Value
Version	8.3
Type	Bug
TypeOfFailure	OtherFailure
Priority	high
Resolution	Unresolved
Component	Build System
Test case
Differential revisions
BlockedBy
Related
Blocking
CC
Operating system
Architecture

changed weight to 7

added Tbug Trac import labels

I can reproduce the issue on a fresh checkout without overriding $PATH; just ./boot; ./configure; make sdist-ghc is enough to trigger this exact error.

https://github.com/haskell/cabal/issues/4633 and https://github.com/haskell/cabal/commit/5e4f4d588 suggest that a pre-generated Lexer.hs is now committed to the Cabal repository, and Lexer.x has been moved outside the main source tree, the intention seems to be that alex is called manually, and that automated builds use the committed Lexer.hs directly.

For now, simply skipping the explicit Alex step in ghc.mk seems to work fine, but as soon as Cabal adopts a more proper solution to the issue, we will probably have to cater for that and change things on our end as well.

changed milestone to %8.4.1

We'll need this for 8.4.1.

mentioned in commit eb86e867

closed

I have verified that sdist-ghc works.

Trac metadata

Trac field	Value
Resolution	Unresolved → ResolvedFixed

The new Hadrian build system has been merged. Relabeling the tickets concerning the legacy make build system to prevent confusion.

Trac metadata

Trac field	Value
Component	Build System → Build System (make)

added Phigh label