Panic on generating Core code

changed weight to 5

This appears to be caused by the presence of the literal 1 in certain typing contexts. It crashes when I place the following in the position of 1 in the above test:

0
2
3 [...]
(-1)
(fromIntegral 1)
[1,2]
(Just 1)
(toInteger 5)
(1 :: Integer)
(1 :: Rational)

It does NOT crash when replaced with:

1e3
0x45
(1 :: Int)
"foo"
'a'
("foo" :: Integer)

Attached file log (download).

result of ghc -v10 -fext-core Panic.hs &> log

As the attached log notes, the panic happens after a warning about attempting to delete a non-existent .s file:

*** Deleting temp files:
Deleting: /tmp/ghc10588_0/ghc10588_0.s
Warning: deleting non-existent /tmp/ghc10588_0/ghc10588_0.s
*** Deleting temp dirs:
Deleting: /tmp/ghc10588_0
ghc: panic! (the 'impossible' happened)
  (GHC version 7.4.1 for x86_64-unknown-linux):
        MkExternalCore died: make_lit

Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

(I should note that after GHC exits, the directory /tmp/ghc10588_0 is indeed deleted.)

Whenever we extend Core we have to invent new External Core syntax to match. And we have not done that with type-level literals.

External Core needs love. Does anyone feel like being the source of that love? We'll gladly help with advice etc.

Simon

Ah, I see. Is this where we should be looking?: http://hackage.haskell.org/trac/ghc/browser/compiler/coreSyn/PprExternalCore.lhs

Trac metadata

Trac field	Value
Component	Compiler → External Core
CC	→ jameshfisher@gmail.com
Operating system	Linux → Unknown/Multiple
Architecture	x86_64 (amd64) → Unknown/Multiple

changed milestone to %7.6.1

Replying to [ticket:5844#comment:58287 simonpj]:

Whenever we extend Core we have to invent new External Core syntax to match. And we have not done that with type-level literals.

I think in fact it's the new Integer literals, not type-level literals, causing the reported problem here.

Replying to [ticket:5844#comment:58292 JamesFisher]:

Ah, I see. Is this where we should be looking?: http://hackage.haskell.org/trac/ghc/browser/compiler/coreSyn/PprExternalCore.lhs

Yes.

Short answer Yes... but there is a longer and more interesting answer. External Core is designed to round-trip, thus:

External Core is a data type defined in coreSyn/ExternalCore.
Core is converted to the External Core data type by coreSyn/MkExternalCore
The External Core data type is printed by PprExternalCore into concrete syntax.
The modified External Core can be parsed by parser/ParserCore. However, it is not parsed into the External Core data type, but rather into "Iface Core" defined in iface/IfaceSyn.
The Iface Core data type (thus parsed) can be converted to Core by iface/TcIface.

What is Iface Core? Iface Core is a data type that GHC uses every time you compile a module. During compilation, GHC converts Core into Iface Core, and then serialises Iface Core into the "M.hi" interface file, in a binary format. This is done by iface/BinIface. Then, when GHC wants to read an interface file, it de-serialises M.hi into Iface Core, and then converts Iface Core into Core with iface/TcIface. What this means is that Iface Core gets plenty of love: it is on our critical path.

Why does "External Core" do the round trip via the External Core data type on the way our, but via Iface Core data type on the way in? This mis-match just an aretefact of an earlier era: when we first implemented External Core, there was no Iface Core.

I think the Right Thing to do is to complete the change that we have already started:

Abandon the Exgternal Core data type entirely.
The conversion from Core to Iface Core is already done (in iface/MkIface), because GHC uses it every time it compiles a moudle.
Change the pretty-printer for External Core to pretty-print the Iface Core rarther than External Core. In fact there already is a pretty-printer for Iface Core, in iface/LoadIface.pprModIface, although it might require minor modification.

The bottom line is that we'd be abandoning the External Core data type (though not its concrete syntax), and switching entirely to Iface Core, which is much better maintained. Less code, more robust to change; what's not to like?

Now, while I say that there is a pretty printer for Iface Core already, it is designed mainly for humans; it is used when you say ghc --show-iface M.hi, and for no other purpose. The External Core concrete syntax, on the other hand, is designed to be parseable as well, so that the round-trip mechanism (print Exgternal Core, modify it, and read it back in) works right. But I think it would be quite acceptable to modify the Iface Core pretty-printer to use External Core syntax, because the Iface Core pretty printer has an output-only role in GHC. (If you do this, I suggest you put the new pretty-printer in a module of its own, perhaps iface/PprIface.)

If this was done, the maintenance burden of External Core, if/when we add new features to Core, would be reduced to

Designing new concrete (ASCII) syntax for the new features
Writing the pretty-printer and parser for this new syntax

I do think that the concrete syntax of External Core should be described by a human-readable BNF grammar, not only by a Happy parser. We have a very old Latex document describing External Core (including is syntax); I'd like to see it become a full part of the documentation, perhaps by convrting it to SGML and making it a full part of the GHC user manual.

This isn't a big task, and it's one that is well separated from the rest of GHC. Would anyone like to take it on? I'd be happy to advise if so.

I wonder who uses External Core?

Does my attached drawing summarize the situation accurately?

Replying to [ticket:5844#comment:58339 simonpj]:

We have a very old Latex document describing External Core (including is syntax); I'd like to see it become a full part of the documentation, perhaps by convrting it to SGML and making it a full part of the GHC user manual.

Is this the document you mean?

I'm a bit confused as to what the "official" GHC documentation is. There's the Developer wiki, Haskell wiki, and the user guide. Where would the External Core document go if converted?

(I ask because the user manual seems slightly strange place to document compiler internals. Perhaps not though.) Where is the source repository for the user manual?

Ignore me; found it. Would it help if was to convert ext-core/core.tex into, say users_guide/core.xml?

Yes, your diagram is right, except that the arrow from .hi to Iface Core should be labelled iface/BinIface as well; BinIface does deserialisation as well a serialisation.

And rather that "External Iface" I might say "the output of ghc --show-iface or something like that.

Documenting in the user manual is slightly odd; but only slightly. After all, it's simply documenting what ghc -ext-core does; in particular, defining the language it produces into the .hcr file. The implementation diagram could form a useful part of the Commentary: http://hackage.haskell.org/trac/ghc/wiki/Commentary.

Yes converting core.tex to core.xml would be a great start.

Simon

assigned to @trac-JamesFisher

I'm working on that conversion as of now.

I'm not experienced in ticketing systems so I'm not sure what the usual way of stating that I'm doing this is. For the moment I'm assigning this ticket to me.

Attached file core.png (download).

Schematic of Core organization, current and proposed

Attached file external_core.xml (download).

To be included as docs/users_guide/external_core.xml

I've attached my DocBook conversion of the External Core LaTeX document. It validates as a DocBook chapter, and builds OK with dblatex, but I can't work out how to build the User's Guide, so someone should check it builds.

Attached file 0001-New-User-s-Guide-document-docs-users_guide-external_.patch (download).

Patch to include previously attached XML file

Please let me know if the attached patch is wrong in any way; I have as much experience of VC as of ticketing systems. I attempted to follow <http://hackage.haskell.org/trac/ghc/wiki/WorkingConventions/FixingBugs>.

mentioned in commit 8c0196b4

Replying to [ticket:5844#comment:58339 simonpj]:

I wonder who uses External Core?

The results are in! Mouse-over for complete answer text.

Poll hosted on Haskellers.com. Advertised on Reddit and on Haskell Café.

unassigned @trac-JamesFisher

Thanks, I've added your external_core.xml to the user guide, and removed the old doc.

Thanks for doing this James. We'll leave the ticket open for the larger question of refactoring the External Core handling as per my long comment above.

Replying to [ticket:5844#comment:59646 simonpj]:

Thanks for doing this James.

No problem.

We'll leave the ticket open for the larger question of refactoring the External Core handling as per my long comment above.

IMO the next task would be to update the External Core document to represent the current code. Or would you consider this wasted effort if we're then changing things again?

Whatever the next task is, is the convention that I open a new ticket for it, or just to continue this discussion?

The two seems pretty independent to me: the doc is about the concrete syntax and how to drive it; the implementation will change but that should not radically change the external syntax.

I don't have a strong opinion about new vs old tickets. I'd stick to this one for now.

changed milestone to %7.6.2

changed milestone to %7.10.1

Moving to 7.10.1.

closed

External Core has been removed in commit 5bf22f06. Also see this mailinglist discussion: https://www.haskell.org/pipermail/ghc-devs/2014-April/004791.html.

Trac metadata

Trac field	Value
Resolution	Unresolved → ResolvedWon'tFix

added compiler crash label

added Pnormal label

Panic on generating Core code

Child items ...

Activity

Trac field	Value
Version	7.4.1
Type	Bug
TypeOfFailure	OtherFailure
Priority	normal
Resolution	Unresolved
Component	Compiler
Test case
Differential revisions
BlockedBy
Related
Blocking
CC
Operating system	Linux
Architecture	x86_64 (amd64)