CPP pragma ignored if top comments and Opt_KeepRawTokenStream

HaRe sets Opt_KeepRawTokenStream to be able to round trip the source code.

If we have a module starting

{-

A normal comment, to check if we can still pick up the CPP directive after it.

-}
{-# LANGUAGE FlexibleInstances #-}
{-# LANGUAGE CPP #-}
-- Check that we can parse a file which requires CPP
module BCpp where
bob :: Int -> Int -> Int
#if __GLASGOW_HASKELL__ > 704
bob x y = x + y
#else
bob x y = x + y * 2
#endif

then the call to load LoadAllTargets via the GHC API fails with

SourceError (lexical error at character 'i' which is the normal error when #if is hit and CPP is not enabled.

If Opt_KeepRawTokenStream is not set it loads without problems.

Also, files using CPP but not having a top comment load properly too.

changed milestone to %8.0.1

changed weight to 5

added Tbug Trac import labels

changed the description

It seems the simplified parser in HeaderInfo.getOptions' needs to to skip over any vanilla comments seen and keep going.

Alternatively Opt_KeepRawTokenStream must be unset in HeaderInfo.getOptions before calling getToks.

I suspect the latter will be the simplest.

A workaround (as now in HaRe) is to only set the Opt_KeepRawTokenStream flag in the cached DynFlags in the ModSummary before invoking parseModule

tweakModSummaryDynFlags :: GHC.ModSummary -> GHC.ModSummary
tweakModSummaryDynFlags ms =
  let df = GHC.ms_hspp_opts ms
  in ms { GHC.ms_hspp_opts = GHC.gopt_set df GHC.Opt_KeepRawTokenStream }

Trac metadata

Trac field	Value
Component	Compiler → Compiler (Parser)

Trac metadata

Trac field	Value
Differential revisions	- → D1444

assigned to @mpickering and unassigned @alanz

mentioned in commit 5d6133be

closed

Merged.

Trac metadata

Trac field	Value
Resolution	Unresolved → ResolvedFixed

added Pnormal label