T12227 is failing on ghc-8.0

[ezyang]

Fresh validate from 5eab189b329344630f76b8751c1289ce480ca46b on x86_64 Linux:

=====> T12227(normal) 1 of 1 [0, 0, 0] 
cd ./perf/compiler &&  "/home/hs01/ezyang/ghc-validate/inplace/test   spaces/ghc-stage2" -c T12227.hs -fforce-recomp -dno-debug-output -fshow-warning-groups -no-user-package-db -rtsopts -fno-warn-tabs -fno-warn-missed-specialisations -fno-ghci-history  -O2 -ddump-hi -ddump-to-file +RTS -M1G +RTS -V0 -tT12227.comp.stats --machine-readable -RTS > T12227.comp.stderr 2>&1
bytes allocated value is too high:
    Expected    T12227(normal) bytes allocated: 1822822016 +/-5%
    Lower bound T12227(normal) bytes allocated: 1731680915 
    Upper bound T12227(normal) bytes allocated: 1913963117 
    Actual      T12227(normal) bytes allocated: 9256914200 
    Deviation   T12227(normal) bytes allocated:      407.8 %
*** unexpected stat test failure for T12227(normal)

That's a pretty awful regression, if it is real.

[simonpj]

Sylvain, this your patch. Any ideas?

[bgamari]

The patch the Edward cites is not the first that shows this failure. I've seen this is a few other cases and have been meaning to track it down. Last time I looked at this I determined that 2756af87aebee769ffca959adc4b9dc607a49fdb was the first failing commit, although ran out of time before I could replicate it.

[hsyl20]

I don't think my patch caused this failure: T12227 doesn't even use the FFI.

2756af87aebee769ffca959adc4b9dc607a49fdb is the patch adding the failing test in ghc-8.0 branch. If it is the first failing commit as @bgamari says, we should check patches from master that have not been cherry-picked into the ghc-8.0 branch (a priori committed in master before 89a8be71a3715c948cebcb19ac81f84da0e6270e if it validates).

[bgamari]

Indeed the patch I cited in comment:2 is the patch that adds the failing test. Sadly I don't seeing any relevant patches to compiler/util/Pretty.hs missing from ghc-8.0. Tracking this one down may be a bit tricky. Thankfully the difference is quite large, so comparing ticky profiles from ghc-8.0 and master around the time should be illuminating.

[bgamari]

According to ticky, it looks like the primary differences between master (namely 2cb8cc26df6af431d30b6964710ea2d859ca2bcd) and ghc-8.0 (namely d84a824cebe94711528727bcb587bfc369ec36a2, which is admittedly quite far removed chronologically from the cherry-picked commit) are indeed in Pretty,

| Change        | alloc A   | alloc B    | name                                                             |      
|---------------|-----------|------------|------------------------------------------------------------------|      
| +3474432240.0 | 262333536 | 3736765776 | beside (ghc:Pretty)                                              |      
| +951367008.0  | 221431440 | 1172798448 | $waboveNest (ghc:Pretty)                                         |      
| +140163848.0  | 91135776  | 231299624  |  (ghc:Coercion.coercionKind_go)                                  |      
| +118356184.0  | 63778576  | 182134760  | $wsplitAtList (ghc:Util.)                                        |      
| +111367360.0  | 640       | 111368000  | $w$cput_6 (ghc:Binary.)                                          |      
| +94517880.0   | 18616520  | 113134400  | $wunpack (ghc:Encoding)                                          |      
| +85860960.0   | 13610200  | 99471160   | $windent (ghc:Pretty)                                            |      
| +45853128.0   | 13206856  | 59059984   | $s$wget1 (ghc:Pretty)                                            |      
| +31953400.0   | 8224616   | 40178016   | $wsep1 (ghc:Pretty.)                                             |      
| +29843944.0   | 8127648   | 37971592   | $wpprName (ghc:Name.)                                            |      
| +29057288.0   | 20964736  | 50022024   | $wsubstTyWith (ghc:TyCoRep.)                                     |      
| +22840864.0   | 11732016  | 34572880   | $wtyCoFVsOfType (ghc:TyCoRep.)                                   |      
| +22640112.0   | 17493648  | 40133760   |  (ghc:TyCon.expandSynTyCon_maybe)                                |      
| +22354288.0   | 5551864   | 27906152   | $wlay2 (ghc:Pretty)                                              |      
| +19673016.0   | 14129160  | 33802176   |  (ghc:Unify.liftCoMatch)                                         |      
| +16728760.0   | 9231400   | 25960160   |  (ghc:Unify.ty_co_match)                                         |      
| +15683720.0   | 9053800   | 24737520   | $wcoercionKindRole (ghc:Coercion.)                               |      
| +15627024.0   | 3566736   | 19193760   |  (ghc:IfaceType.toIfaceTcArgs_go)                                |      
| +15499856.0   | 3545152   | 19045008   | $wget (ghc:Pretty)                                               |      
| +15113200.0   | 5327048   | 20440248   |  (ghc:IfaceType.toIfaceType)                                     |      
| +14813696.0   | 3417088   | 18230784   | $wppr_iface_tc_app (ghc:IfaceType)                               |      
| +14507168.0   | 2747264   | 17254432   |  (ghc:Name.pprModulePrefix)                                      |      
| +13158528.0   | 2774064   | 15932592   | $wxs1 (ghc:Pretty)                                               |      
| +12767416.0   | 5934000   | 18701416   |  (ghc:OptCoercion.opt_co4)                                       |      
| +11420416.0   | 2612848   | 14033264   | $wlay1 (ghc:Pretty)                                              |      
| +11400448.0   | 7529920   | 18930368   | $ssubst_ty (ghc:TyCoRep.composeTCvSubst_)                        |      
| +10744600.0   | 79480     | 10824080   | $w$cput_7 (ghc:IfaceType.)                                       |      
| +10555272.0   | 3780088   | 14335360   |  (ghc:IfaceType.toIfaceTcArgs)                                   |      
| +10380952.0   | 6663952   | 17044904   | isAxiom_maybe (ghc:OptCoercion)                                  |      
...

Working on why this is the case will require some more effort.

[bgamari]

Unfortunately this will need to be resolved in 8.0.3, if at all.

[bgamari]

At this point it is rather unlikely that there will be an 8.0.3. Re-milestoning.

[bgamari]

Given that 8.2.1-rc1 is imminent, I'm bumping these off to the 8.4

[bgamari]

This ticket won't be resolved in 8.4; remilestoning for 8.6. Do holler if you are affected by this or would otherwise like to work on it.

[bgamari]

Sadly there has been no motion on this; demilestoning. Do ping if this sounds interesting to you, however.