Opened 11 years ago

Last modified 10 days ago

#1290 new bug

ghc runs preprocessor too much

Reported by: guest Owned by:
Priority: normal Milestone:
Component: Driver Version: 6.7
Keywords: cpp Cc: dfeuer, lelf
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: Compile-time performance bug Test Case:
Blocked By: #14553 Blocking:
Related Tickets: Differential Rev(s):
Wiki Page:

Description

When an external preprocessor has been specified with -F -pgmF, and then using ghc --make, ghc runs the preprocessor on every .hs file, even those for which compilation is skipped. This seem unnecessary and it slows down recompilation significantly when you have many files.

-- Lennart

Change History (17)

comment:1 Changed 11 years ago by simonmar

I don't see how to avoid this easily: we need to parse the imports of every source file in order to construct the dependency tree, and we can't parse the imports without preprocessing the source file first.

Perhaps keeping a cache file containing the imports for each source file with the last-modified date of the source?

comment:2 Changed 11 years ago by guest

Yes, you're right. It's difficult to fix. It would take a cache to avoid, which is a pain.

comment:3 Changed 11 years ago by igloo

Milestone: _|_

comment:4 Changed 10 years ago by simonmar

Architecture: UnknownUnknown/Multiple

comment:5 Changed 10 years ago by simonmar

Operating System: UnknownUnknown/Multiple

comment:6 Changed 5 years ago by morabbin

Type of failure: None/Unknown

Still difficult ...

comment:7 Changed 3 years ago by ezyang

Milestone: 7.12.1

What if this information was cached in the interface file? Essentially, interface files should contain enough information for us to construct a ModSummary, and they are as recent as the source file we can just use this directly. (And we're going to read the interface file anyway shortly thereafter do check for recompilation avoidance.)

comment:8 Changed 3 years ago by thoughtpolice

Milestone: 7.12.18.0.1

Milestone renamed

comment:9 Changed 2 years ago by ezyang

This is related to #618, as if ghc --make remembered dependency information it would not need to preprocess *stable* modules.

comment:10 Changed 2 years ago by thomie

Milestone: 8.0.1

comment:11 Changed 12 months ago by bgamari

Type of failure: None/UnknownCompile-time performance bug

comment:12 Changed 5 months ago by dfeuer

I think this is essentially impossible to fix. Along with the import dependencies, there are also the preprocessing dependencies. If I have

module Blah where
#include Foo.h

then I need to preprocess Blah.hs any time Foo.h changes. But that means we need to know that Blah.hs depends on Foo.h. Without detailed knowledge of the preprocessor language, I don't see any way to determine this. I'm very tempted to close this ticket as invalid, but let's see if anyone disagrees.

comment:13 Changed 5 months ago by dfeuer

Cc: dfeuer added

comment:14 Changed 5 months ago by dfeuer

bgamari points out that if something like https://ghc.haskell.org/trac/ghc/wiki/Proposal/NativeCpp goes in, we may get the information we need about preprocessor dependencies for CPP.

comment:15 Changed 5 months ago by dfeuer

Blocked By: 14553 added

comment:16 Changed 5 months ago by dfeuer

Keywords: cpp added

comment:17 Changed 10 days ago by lelf

Cc: lelf added
Note: See TracTickets for help on using tickets.