Opened 4 years ago

Closed 6 months ago

Last modified 3 months ago

#8025 closed bug (fixed)

-fno-code and Template Haskell are incompatible

Reported by: mojojojo Owned by:
Priority: normal Milestone: 8.4.1
Component: Compiler (Type checker) Version: 7.6.3
Keywords: Cc: gershomb
Operating System: Unknown/Multiple Architecture: x86
Type of failure: Incorrect warning at compile-time Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s): Phab:D3441
Wiki Page:

Description (last modified by bgamari)

Building the included project with

cabal configure && cabal build --ghc-options="-fforce-recomp -Wall -fno-code"

results in the following typechecker error:

ByteCodeLink.lookupCE
During interactive linking, GHCi couldn't find the following symbol:
  supabugzm0zi1zi0_A_makeLenseszq_closure
This may be due to you not asking GHCi to load extra object files,
archives or DLLs needed by your current session.  Restart GHCi, specifying
the missing library using the -L/path/to/object/dir and -lmissinglibname
flags, or simply by naming the relevant files on the GHCi command line.
Alternatively, this link failure might indicate a bug in GHCi.
If you suspect the latter, please send a bug report to:
  glasgow-haskell-bugs@haskell.org

The problem here is that GHC goes looking for the code of modules needed for TemplateHaskell splices, which was not produced due to -fno-code.

Attachments (1)

supabug.7z (583 bytes) - added by mojojojo 4 years ago.
archived project

Download all attachments as: .zip

Change History (18)

Changed 4 years ago by mojojojo

Attachment: supabug.7z added

archived project

comment:1 Changed 3 years ago by dmcclean

difficulty: Unknown

I have a similar error on 7.8.3 on Windows. I can't see anything remarkable about the function it complains about. Calls to it seem to work.

comment:2 Changed 3 years ago by GregWeber

This issue is being seen while running: cabal install yesod-core https://github.com/yesodweb/yesod/issues/721

I am using ghc 7.8.3 on Ubuntu linux from this Dockerfile: zsol/haskell-platform-2014.2.0.0

comment:3 Changed 3 years ago by thomie

Operating System: MacOS XUnknown/Multiple
Status: newinfoneeded
$ sudo apt-get install p7zip
$ 7zr x supabug.7z

Does anyone have an example that doesn't depend on lens or yesod.

comment:4 Changed 2 years ago by tonyday567

I found a similar bug with this same command here:

https://github.com/mikeizbicki/subhask/commit/c98d2b5f1184153820867d83e4fba44118b97bc2

Error:

<no location info>:
    
ByteCodeLink.lookupCE
During interactive linking, GHCi couldn't find the following symbol:
  subhazuAF2UqVq5DNs9kS0u9qf2pe_SubHaskziTemplateHaskellziMutable_mkMutable_closure
This may be due to you not asking GHCi to load extra object files,
archives or DLLs needed by your current session.  Restart GHCi, specifying
the missing library using the -L/path/to/object/dir and -lmissinglibname
flags, or simply by naming the relevant files on the GHCi command line.
Alternatively, this link failure might indicate a bug in GHCi.
If you suspect the latter, please send a bug report to:
  glasgow-haskell-bugs@haskell.org

As far as I can tell, it doesn't involve Lens. ghc --interactive and cabal build both work fine, so the bug is isolated to the -fno-code flag.

comment:5 Changed 2 years ago by rwbarton

Status: infoneedednew

(How) is this supposed to work at all? I mean, if you want to be able to run a TH splice, you need to at some point generate code for the values used in the splice, right? Is the TH runner supposed to recompile the necessary module(s) with the bytecode generator?

A minimal reproducer is simply

-- A.hs
{-# LANGUAGE TemplateHaskell #-}
module A where
a = [|3|]

-- B.hs
{-# LANGUAGE TemplateHaskell #-}
module B where
import A
x = $(a)

-- command: ghc -fno-code B

It might be possible to diagnose the situation and produce a better error message, but making this actually work seems solidly in feature request territory to me.

comment:6 Changed 21 months ago by gershomb

Cc: gershomb added

comment:7 Changed 8 months ago by duog

I have not contributed to GHC before, and I would like to work on resolving this issue. Both haddock and flycheck should be able to use -fno-code to do a lot less work. Currently, haddock essentially turns off no-code when any modules use template haskell and flycheck uses no-link to work around both this issue and #10600.

I think this can be resolved by having ghc itself enable codegen on home modules depended on by modules that use template haskell.

I have a patch here https://github.com/duog/ghc/tree/wip-nocode-th which I just rebased onto your master. Note that the haddock submodule points to my github fork.

downsweep is changed to do an additional pass over the modules, targetting any ModSummaries transitively depended on by a module that has LangExt.TemplateHaskell enabled. Those targeted modules have hscTarget changed from HscNothing to the default target of the platform.

There are two shell scripts in the root of my fork which boot and run an ad-hoc benchmark of cabal haddock on the wreq library. This library uses template haskell to generate lenses, and imports home modules from modules that use template haskell. However, most modules are not imported from modules that use template haskell. With this patch, cabal haddock runs in about half the time.

At least two points that need to be addressed:

  • Should this behaviour require an additional flag? It doesn't seem good for -fno-code to sometimes generate code.
  • Is the default target of the platform the correct choice?

I would require some guidance to get this into a mergable state, if you are interested in taking this fix please let me know the next step. I am happy to use your phabricator if that is appropriate.

comment:8 Changed 8 months ago by bgamari

Hi Duog, thanks for your work on this and sorry for the delayed response; I unfortunately missed the notification for this. It would be great if you could put this up on Phabricator. See Phabricator to get started and don't hesitate to reach out to me via email or IRC if you need help.

comment:9 Changed 8 months ago by bgamari

Duog, your approach seems like a very interesting compromise that I had not considered. Great work!

There is the question of which HscTarget we want to use here. Arguably HscInterpreted is more appropriate as it is faster to generate, faster to link, and doesn't unexpected leave object files lying around. However, it unfortunately lacks support for unboxed tuples and sums, which may break some (I suspect quite small) fraction of programs. It would be interesting to characterize the difference.

comment:10 Changed 8 months ago by bgamari

Description: modified (diff)
Status: newpatch
Summary: "During interactive linking, GHCi couldn't find the following symbol" typechecker error with TemplateHaskell involved-fno-code and Template Haskell are incompatible

comment:11 Changed 8 months ago by bgamari

Duog, I've posted the relevant pieces of your branch to Phabricator as Phab:D3427. Feel free to "commandeer" this Differential using the Actions menu at the bottom on the page. Once you have done so you can update the Diff using arc diff --update D3427. Do let me know if I can be of assistance.

comment:12 Changed 8 months ago by duog

Thanks Ben, I've just posted a diff moments before I saw this. I have abandoned your D3427 in favour of D3441. I see that you have removed my bash scripts. I'll remove them from D3441, let me know if you'd like me to make them available elsewhere. My github fork will not be a very reliable place to get them as it's likely to get rebased a lot.

comment:13 Changed 8 months ago by bgamari

duog, if you could push the test scripts (or perhaps just the original patches) to a GitHub branch and include a link in the Diff description that would be great.

comment:14 Changed 6 months ago by Ben Gamari <ben@…>

In 53c78be0/ghc:

Compile modules that are needed by template haskell, even with -fno-code.

This patch relates to Trac #8025

The goal here is to enable typechecking of packages that contain some
template haskell. Prior to this patch, compilation of a package with
-fno-code would fail if any functions in the package were called from
within a splice.

downsweep is changed to do an additional pass over the modules,
targetting any ModSummaries transitively depended on by a module that
has LangExt.TemplateHaskell enabled. Those targeted modules have
hscTarget changed from HscNothing to the default target of the platform.

There is a small change to the prevailing_target logic to enable this.

A simple test is added.

I have benchmarked with and without a patched haddock
(available:https://github.com/duog/haddock/tree/wip-no-explicit-th-compi
lation).  Running cabal haddock on the wreq package results in a 25%
speedup on my machine:

time output from patched cabal haddock:

real    0m5.780s
user    0m5.304s
sys     0m0.496s
time output from unpatched cabal haddock:

real    0m7.712s
user    0m6.888s
sys     0m0.736s

Reviewers: austin, bgamari, ezyang

Reviewed By: bgamari

Subscribers: bgamari, DanielG, rwbarton, thomie

GHC Trac Issues: #8025

Differential Revision: https://phabricator.haskell.org/D3441

comment:15 Changed 6 months ago by bgamari

Differential Rev(s): Phab:D3441
Milestone: 8.4.1
Resolution: fixed
Status: patchclosed

Thanks Duog!

comment:16 Changed 3 months ago by Saizan

This is great, but in some cases generating code for all the imported modules is too much.

Would it make sense to have {-# OPTIONS_GHC -fobject-code #-} change the HscTarget for the module, so that authors can fine-tune for which modules code needs to be generated? (together the old behavior of -fno-code, I mean)

Currently having that pragma seems to have no effect.

comment:17 Changed 3 months ago by duog

I agree that having the ability to override the current behaviour would be useful, but I don't think that overloading -fobject-code to do it is right.

I would suggest that we need an "-fno-code-auto" flag.

Currently, when we are deciding which modules to codegen in GhcMake.enableCodegenForTH, we take all transitive dependencies of modules that use -XTemplateHaskell.

We could instead take all transitive dependencies (where those dependencies don't disable -fno-code-auto) of modules that use -XTemplateHaskell.

I think it would make sense to disable -fno-code-auto by default if we improved the error message to suggest it be enabled. One could just add {-# OPTIONS_GHC -fno-code-auto #-} to the modules that need to be codegenned.

Separately, a flag -fno-code-target={default,interpreted,asm(?),llvm(?)} should perhaps be added to determine the method of codegen. This would enable us to use interpreted by default, again provided that errors from, for example, unboxed tuples, suggested the user change it.

Note: See TracTickets for help on using tickets.