Plugins/dynamic loading subtly broken (it seems)

[thoughtpolice]

Imported from #8276, which was originally about a Haddock bug (that was different in nature but appeared similarly):

I still experience this panic on GHC 7.8.2 with Windows x86_64:

$ ghci
GHCi, version 7.8.2: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
λ> import StaticFlags
λ> import Data.IORef
λ> readIORef v_opt_C_ready
Loading package array-0.5.0.0 ... linking ... done.
Loading package deepseq-1.3.0.2 ... linking ... done.
Loading package bytestring-0.10.4.0 ... linking ... done.
Loading package containers-0.5.5.1 ... linking ... done.
Loading package Win32-2.3.0.2 ... linking ... done.
Loading package filepath-1.3.0.2 ... linking ... done.
Loading package old-locale-1.0.0.6 ... linking ... done.
Loading package time-1.4.2 ... linking ... done.
Loading package directory-1.2.1.0 ... linking ... done.
Loading package pretty-1.1.1.1 ... linking ... done.
Loading package process-1.2.0.0 ... linking ... done.
Loading package Cabal-1.18.1.3 ... linking ... done.
Loading package binary-0.7.1.0 ... linking ... done.
Loading package bin-package-db-0.0.0.0 ... linking ... done.
Loading package hoopl-3.10.0.1 ... linking ... done.
Loading package hpc-0.6.0.1 ... linking ... done.
Loading package template-haskell ... linking ... done.
Loading package transformers-0.3.0.0 ... linking ... done.
Loading package ghc-7.8.2 ... linking ... done.
False
λ> staticFlags
ghc.exe: panic! (the 'impossible' happened)
  (GHC version 7.8.2 for x86_64-unknown-mingw32):
        Static flags have not been initialised!
        Please call GHC.parseStaticFlags early enough.

Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

But not with GHC 7.8.2 on Linux x86_64:

$ ghci
GHCi, version 7.8.2: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
λ> import StaticFlags
λ> import Data.IORef
λ> readIORef v_opt_C_ready
Loading package array-0.5.0.0 ... linking ... done.
Loading package deepseq-1.3.0.2 ... linking ... done.
Loading package bytestring-0.10.4.0 ... linking ... done.
Loading package containers-0.5.5.1 ... linking ... done.
Loading package filepath-1.3.0.2 ... linking ... done.
Loading package old-locale-1.0.0.6 ... linking ... done.
Loading package time-1.4.2 ... linking ... done.
Loading package directory-1.2.1.0 ... linking ... done.
Loading package pretty-1.1.1.1 ... linking ... done.
Loading package process-1.2.0.0 ... linking ... done.
Loading package Cabal-1.18.1.3 ... linking ... done.
Loading package binary-0.7.1.0 ... linking ... done.
Loading package bin-package-db-0.0.0.0 ... linking ... done.
Loading package hoopl-3.10.0.1 ... linking ... done.
Loading package hpc-0.6.0.1 ... linking ... done.
Loading package template-haskell ... linking ... done.
Loading package transformers-0.3.0.0 ... linking ... done.
Loading package ghc-7.8.2 ... linking ... done.
True
λ> staticFlags
[]

This has caused problems for me when running programs such as ​HERMIT on Windows, since at some point, parseStaticFlagsFull seems to be called during program execution. A workaround is to call initStaticOpts manually somewhere in the program, but I feel like that shouldn't be necessary.

The one above was on Windows. Also reported:

Hello,

A similar panic also happens when running Liquid Haskell ​http://goto.ucsd.edu/~rjhala/liquid/haskell/blog/about/ on a file that uses the Units of Measure plugin ​https://github.com/adamgundry/uom-plugin

liquid: liquid: panic! (the 'impossible' happened)
  (GHC version 7.10.1 for x86_64-unknown-linux):
	Static flags have not been initialised!
        Please call GHC.parseStaticFlags early enough.

[darchon]

A similar panic also happens when you run doctest ​http://hackage.haskell.org/package/doctest on a file that uses a type-checker plugin ​https://github.com/christiaanb/ghc-typelits-natnormalise/blob/master/tests/Tests.hs

doctest: doctest: panic! (the 'impossible' happened)
  (GHC version 7.10.1 for x86_64-apple-darwin):
	Static flags have not been initialised!
        Please call GHC.parseStaticFlags early enough.

[ezyang]

I'd be much appreciative if you could try these examples with a recent GHC HEAD that includes ad6d6a76eeeb9e33a96054f18c1306e9ebafa652.

I have also noticed that calling the static flags initializing functions sometimes doesn't fix this sort of issue, especially in GHCi-style situations.

[darchon]

Replying to ezyang:

I'd be much appreciative if you could try these examples with a recent GHC HEAD that includes ad6d6a76eeeb9e33a96054f18c1306e9ebafa652.

I have also noticed that calling the static flags initializing functions sometimes doesn't fix this sort of issue, especially in GHCi-style situations.

I tried, but I'm still getting the same error. In the end, I had to add this workaround in my own code: ​https://github.com/christiaanb/ghc-typelits-natnormalise/commit/afc4f2538081b46439e26e1a2bc6b7a5c3751781

[mjmrotek]

I'm getting a similar panic on Windows 8 x86_64 (latest MinGHC) from GHC 7.10.1 when compiling the UoM Plugin (​https://github.com/adamgundry/uom-plugin):

[11 of 13] Compiling Data.UnitsOfMeasure.Convert ( src\Data\UnitsOfMeasure\Convert.hs dist\build\Data\UnitsOfMeasure\Convert.o ) ghc.exe: panic! (the 'impossible' happened)

(GHC version 7.10.1 for x86_64-unknown-mingw32):

Static flags have not been initiailised! Please call GHC.parseStaticFlags early enough.

[ezyang]

I can't reproduce this on Linux (but I can on Windows), reclassifying as a Windows bug.

[mjmrotek]

I can confirm that my second bugs (the ghc.exe bug) occured only on Windows, but the first (the liquid haskell + uom plugin bug. minor, because it's very situational) bug happened on Linux.

[thoughtpolice]

Moving to 7.10.3 milestone - if you think this is an error (or a showstopping bug), please remilestone it to 7.10.2 and let us know why.

[thomie]

The test plugins/T10294 is failing on Windows with the same error as above:

ghc-stage2.exe: panic! (the 'impossible' happened)
  (GHC version 7.11.20150713 for x86_64-unknown-mingw32):
	Static flags have not been initialised!
        Please call GHC.parseStaticFlags early enough.

[nomeata]

Also on travis. Shall we mark the test case as known_broken on this bug?

[nomeata]

BTW, this could be due to DYNAMIC_GHC_PROGRAMS = NO, couldn’t it?

[Joachim Breitner <mail@…>]

In 4ee658a02ccc6d3aa0b6a0a5f2f5934a593f1356/ghc:

Mark test case for #10294 expect_broken on #10301

as it is broken on Travis, and in #10301 others have reported the same
error.

[thomie]

Replying to nomeata:

Also on travis. Shall we mark the test case as known_broken on this bug?

Well, then Phabricator's build will fail.

[Joachim Breitner <mail@…>]

In 8e6a50339a4a61d4f2cbec645c78abc85098a294/ghc:

Mark test case for #10294 conditionally expect_broken on #10301

the hypothesis is that it only breaks with `DYNAMIC_GHC_PROGRAMS = NO`,
so use `unless(have_dynamic(),expect_broken(10301))` to not break the
Phabricator build.

[Thomas Miedema <thomasmiedema@…>]

In b1063b1b64989749292d156b189eb64a73fb329a/ghc:

Testsuite: mark T10294 conditionally expect_broken on #10301

Fix 8e6a50339a4a61d4f2cbec645c78abc85098a294.

[thomie]

Not Windows only. DYNAMIC_GHC_PROGRAMS=NO is needed to trigger the bug.

[bgamari]

Will not be fixed for 7.10.3.

[Thomas Miedema <thomasmiedema@…>]

In bc436f9e/ghc:

Testsuite: mark frontend01 conditionally expect_broken on #10301

This should fix validate on Travis.

[thomie]

frontend01 is failing in the same way when DYNAMIC_GHC_PROGRAMS=NO.

[ezyang]

So, is this not what reinitializeGlobals fixes? I can't easily test for frontend01 because reinitializeGlobals is only implemented for the Core monad? Are they unrelated?

[adamgundry]

I suspect the fix might indeed be to implement the equivalent of reinitializeGlobals for the TcPluginM and GHC monads.

[bgamari]

T10294 seems to pass on Travis pretty reliably since c3c702441137dc8f7ee0dd5ac313be96d625459a. Strangely enough I've not seen it pass locally nor on Phabricator.

[bgamari]

Ahh, I see. .travis.yml configures the build with DYNAMIC_GHC_PROGRAMS = NO, which is the only case where we expected T10294 to fail. Consequently it seems that we can mark this as fixed. Hooray!

[Ben Gamari <ben@…>]

In 0cad52d/ghc:

testsuite: Mark T10294 as fixed

It seems that c3c702441137dc8f7ee0dd5ac313be96d625459a resolved #10301.
It took a while to notice this since it only broke when tested against a
statically linked GHC, a configuration which Harbormaster doesn't test.

Test Plan: Validate

Reviewers: angerman, austin

Subscribers: thomie, nomeata

Differential Revision: https://phabricator.haskell.org/D2856

GHC Trac Issues: #10294, #10301