Opened 5 months ago

Last modified 4 days ago

#13604 new bug

ghci no longer loads dynamic .o files by default if they were built with -O

Reported by: George Owned by:
Priority: high Milestone: 8.2.2
Component: Compiler Version: 8.2.1-rc1
Keywords: RecompilationCheck Cc: ezyang
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: Runtime performance bug Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s): Phab:D3514
Wiki Page:

Description

In 8.2.1-rc1 loading a file compiled with -O2 into ghci results in ghci recompiling the file into interpreted byte code. In 8.0.2 it simply loads the compiled object file.

8.2.1

 ghc -dynamic -O2 eh2.hs
[1 of 1] Compiling Main             ( eh2.hs, eh2.o )
Linking eh2 ...
bash-3.2$ ghci -ignore-dot-ghci
GHCi, version 8.2.0.20170404: http://www.haskell.org/ghc/  :? for help
Prelude> :load eh2
[1 of 1] Compiling Main             ( eh2.hs, interpreted ) [flags changed]
Ok, modules loaded: Main.

8.0.2

 ghc --version
The Glorious Glasgow Haskell Compilation System, version 8.0.2
bash-3.2$ pwd
/Users/gcolpitts/haskell
bash-3.2$ ghc -dynamic -O2 eh2.hs
[1 of 1] Compiling Main             ( eh2.hs, eh2.o )
Linking eh2 ...
bash-3.2$ ghci -ignore-dot-ghci
GHCi, version 8.0.2: http://www.haskell.org/ghc/  :? for help
Prelude> :load eh2
Ok, modules loaded: Main (eh2.o).

Change History (24)

comment:1 Changed 5 months ago by George

This was on a Mac with 8.2.1-rc1 compiled from source. Setting OS and architecture to Unknown/Multiple as I don't see any reason why this would be Mac specific. See also #13002

Last edited 5 months ago by George (previous) (diff)

comment:2 Changed 5 months ago by George

Version: 8.0.18.2.1-rc1

comment:3 Changed 5 months ago by George

Summary: regression in ghc 8.2.1-rc1 (8.2.0.2017040)regression in ghc 8.2.1-rc1 (8.2.0.20170404)

comment:4 Changed 5 months ago by RyanGlScott

I wonder if this bug has the same underlying cause as #13500? The [flags changed] part makes me suspect so.

comment:5 Changed 5 months ago by RyanGlScott

Ah, never mind. I tried using GHC HEAD with Phab:D3398 (the proposed fix for #13500) applied, but this bug was still present.

comment:6 Changed 5 months ago by RyanGlScott

Cc: ezyang added

I've identified commit 818760d68c0e5e4479a4f64fc863303ff5f23a3a (Fix #10923 by fingerprinting optimization level) as the culprit. cc'ing ezyang, the author of that commit.

comment:7 Changed 5 months ago by ezyang

Makes sense: GHCi asks for no optimization, but the object files are optimized, so GHCi has no choice but to reinterpret the files.

It would be an easy matter to revert this patch, but if we also want to keep #10923 fixed, we will need to have a discussion about what the intended semantics here are.

comment:8 Changed 5 months ago by George

Suppose I wanted ghci to load optimized files, specifically -O2, how would I do that? See #13002 as mentioned above. Assuming I could specify it then ghci would simply load the compiled file, right?

Currently there is no way to have a file compiled with -O2 loaded or compiled and loaded into ghci, right? That is a deficency and a regression. right?

Last edited 5 months ago by George (previous) (diff)

comment:9 Changed 5 months ago by ezyang

In principle, one ought to be able to pass -O2 to GHCi to make this happen. However, it turns out you also must pass -fobject-code, at which the desired behavior is seen.

ezyang@sabre:~$ ghc-8.2 -O2 -c A.hs
ezyang@sabre:~$ ghc-8.2 -O2 -c A.hs -dynamic
ezyang@sabre:~$ ghc-8.2 --interactive -O2

when making flags consistent: warning:
    -O conflicts with --interactive; -O ignored.
GHCi, version 8.2.0.20170413: http://www.haskell.org/ghc/  :? for help
Prelude> :load A.hs
[1 of 1] Compiling A                ( A.hs, interpreted ) [flags changed]
Ok, modules loaded: A.
*A> 
Leaving GHCi.
ezyang@sabre:~$ ghc-8.2 --interactive -O2 -fobject-code
GHCi, version 8.2.0.20170413: http://www.haskell.org/ghc/  :? for help
Prelude> :load A.hs
Ok, modules loaded: A (A.o).
Prelude A> 

comment:10 Changed 5 months ago by George

Thanks, that answers my question, you can do it as you described. Also you can specify those options in your .ghci file and that works as would be expected.

So the question seems to be, if you have an object file compiled with flags different than the ghci flags (possibly the default ones) should ghci load it (as it did in 8.0.2) or should it compile the source into interpreted byte code and load that (as it does now)

I think I prefer the old behavior, if you want interpreted byte code, remove the object file otherwise load will load your object file as is. However if we want to change this behavior to what it is currently I could live with that as long as the documentation is clear about the change and the current behavior.

comment:11 Changed 5 months ago by ezyang

So, the motivating principle for the change in behavior for ghc --make is that, if you run a ghc command, the end result should always be the same as if you had run GHC on a clean project. This means that, yes, if the optimization level changed, you better recompile your files.

I don't think this is necessarily what GHCi users are looking for. Without -fobject-code, optimization level doesn't matter at all and I can definitely see an argument for the semantics, "Please ignore my flags and use the on-disk compiled products as much as possible, so long as they accurately reflect the source code of my program." This interpretation is supported by the fact that -O flag doesn't do anything right now. (But, it is probable that -fobject-code should shunt us back into the --make style semantics.)

comment:12 Changed 5 months ago by simonpj

Whether or not it is the "right" behaviour, it would be good if the user manual documented the behaviour and the underlying principles. And describes how to work around it if you want something different.

comment:13 Changed 5 months ago by ezyang

Differential Rev(s): Phab:D3514

I don't think a proper fix can make it for 8.2, so I've put up a revert for review.

comment:14 Changed 5 months ago by rwbarton

Summary: regression in ghc 8.2.1-rc1 (8.2.0.20170404)ghci no longer loads dynamic .o files by default if they were built with -O

comment:15 Changed 5 months ago by bgamari

Frankly, I've been surprised by the converse of this bug: GHC doesn't recompile when I request different optimization options. I can see the argument for a "go fast" mode, which uses any existing build products to avoid recompilation if at all possible, but it's not clear to me that this should be the default behavior.

comment:16 Changed 5 months ago by George

I think I will be happy with whatever people come up with here. I filed the bug originally as I was surprised by the change in behavior. As Simon wrote above: it would be good if the user manual documented the behaviour and the underlying principles. And describes how to work around it if you want something different.

As Ben wrote above ideally the final fix will also address the converse of this bug.

comment:17 Changed 8 weeks ago by George

Milestone: 8.4.1

By 8.4.1, as Simon says in comment 12: it would be good if [at least] the user manual documented the behaviour and the underlying principles. And describes how to work around it if you want something different.

comment:18 Changed 7 weeks ago by elaforge

I just ran into this issue myself. The main thing I have to add is to note that this issue also affects -fhpc, so whatever solution should take that into account too. I took a look through FlagChecker.hs for other candidates, but didn't see anything else.

A simple fix would be for ghci to note that -fhpc or -O are being ignored, but still include them in its flag hash when it does the recompilation check. I didn't notice that suggestion in the discussion above.

comment:19 Changed 7 weeks ago by elaforge

By the way, if the fix is really put off until 8.4.1, it means I'll have to skip a whole major version! This issue is a show stopper for my program, and I'd expect for any program that uses the GHC API for a REPL (maybe not that many, but still, it's a compelling GHC feature for me). It also makes interactive testing really slow, which for me makes it a show stopper for my style of development as well.

If people like the "ghci allows but ignores -fhpc and -O" idea, I'd be happy to give the implementation a shot.

comment:20 Changed 7 weeks ago by George

I like the "ghci allows but ignores -fhpc and -O" idea and I'd like to see this in 8.2.2. As my original summary wrote this is a regression and I'd like to see it resolved sooner rather than later. However see also https://ghc.haskell.org/trac/ghc/ticket/13002, not sure if "ghci allows but ignores -fhpc and -O" means that 13002 would not get resolved.

Last edited 7 weeks ago by George (previous) (diff)

comment:21 Changed 7 weeks ago by bgamari

Component: GHCiCompiler
Milestone: 8.4.18.2.2
Priority: normalhigh

We can fix the checker for 8.2.2.

comment:22 Changed 4 days ago by bgamari

Keywords: RecompilationCheck added

comment:23 Changed 4 days ago by bgamari

elaforge, George, can you describe precisely what you would propose that you help with this? I'm not sure teaching ghci to ignore -O and -fhpc is a great idea since there may be users that want to use these flags from within an interactive session.

comment:24 Changed 4 days ago by George

I'll settle for what Simon suggested: " if the user manual documented the behaviour and the underlying principles. And describes how to work around it if you want something different." Whatever the solution, I'd like to be able to specify the optimization level of compilation, e.g. -O or -02 in a .ghci file as well as by an argument to ghci, so that it will take effect when I compile inside of emacs. i.e. in other words I'd like 13002 fixed too, if possible. The use case here is working with optimized compiled code in emacs/ghci, making changes and measuring the performance; thus you want to be able to compile those changes in emacs at a given optimization level.

I think elaforge may want something more specific but I'll let him speak for himself.

Note: See TracTickets for help on using tickets.