ghc runs preprocessor too much

[guest]

When an external preprocessor has been specified with -F -pgmF, and then using ghc --make, ghc runs the preprocessor on every .hs file, even those for which compilation is skipped. This seem unnecessary and it slows down recompilation significantly when you have many files.

-- Lennart

[simonmar]

I don't see how to avoid this easily: we need to parse the imports of every source file in order to construct the dependency tree, and we can't parse the imports without preprocessing the source file first.

Perhaps keeping a cache file containing the imports for each source file with the last-modified date of the source?

[guest]

Yes, you're right. It's difficult to fix. It would take a cache to avoid, which is a pain.

[morabbin]

Still difficult ...

[ezyang]

What if this information was cached in the interface file? Essentially, interface files should contain enough information for us to construct a ModSummary, and they are as recent as the source file we can just use this directly. (And we're going to read the interface file anyway shortly thereafter do check for recompilation avoidance.)

[thoughtpolice]

Milestone renamed

[ezyang]

This is related to #618, as if ghc --make remembered dependency information it would not need to preprocess *stable* modules.

[dfeuer]

I think this is essentially impossible to fix. Along with the import dependencies, there are also the preprocessing dependencies. If I have

module Blah where
#include Foo.h

then I need to preprocess Blah.hs any time Foo.h changes. But that means we need to know that Blah.hs depends on Foo.h. Without detailed knowledge of the preprocessor language, I don't see any way to determine this. I'm very tempted to close this ticket as invalid, but let's see if anyone disagrees.

[dfeuer]

bgamari points out that if something like ​https://ghc.haskell.org/trac/ghc/wiki/Proposal/NativeCpp goes in, we may get the information we need about preprocessor dependencies for CPP.