Opened 4 years ago

Last modified 4 months ago

#5987 new bug

Too many symbols in ghc package DLL

Reported by: igloo Owned by:
Priority: normal Milestone:
Component: Compiler Version: 7.5
Keywords: Cc: bill@…, george.colpitts@…, tkn.akio@…, shelarcy@…
Operating System: Windows Architecture: Unknown/Multiple
Type of failure: None/Unknown Test Case:
Blocked By: Blocking: #5355
Related Tickets: Differential Rev(s):
Wiki Page:

Description

In HEAD, the libHSghc DLL currently contains about 75000 symbols, but DLLs can only have about 65000 symbols. This means that when you try to run anything that uses the libHSghc DLL, references get resolved incorrectly and the GCer segfaults.

Max ran into this before, and temporarily fixed it with:

commit dd391759500ab2b6abd23d502ade7ff8946c780f
Author: Max Bolingbroke <batterseapower@hotmail.com>
Date:   Wed Jul 6 18:01:02 2011 +0100

    Don't export the _info symbol for the data constructor worker bindings

but the number of symbols has since crept up.

Three possible solutions:

  • Split the ghc package up into ghc-base, ghc-codegen, ghc-nativegen, etc.
  • Export fewer things. We'd first have to do this at the Haskell level (i.e. stop exposing some modules in the .cabal file; this may upset GHC API users), and then thread this info through so that we can tell the linker which symbols we do/don't want it to expose. Unfortunately, re-exports at the Haskell level mean that it isn't as simple as just not exporting anything from modules that aren't exposed.
  • Automatically split the package into 2 DLLs. The tricky bit here is that we'd need dependencies to go only in one direction, i.e. if libHSghc1.dll uses symbols from libHSghc2.dll, then libHSghc2.dll doesn't use any symbols from libHSghc1.dll. Ideally Cabal would have a way to do this automatically, and the GHC build system would use the split that it calculates. However, i don't think Cabal has enough info to do this currently.

It would also be good if we can detect that building a DLL has gone wrong, and give an error.

Change History (53)

comment:1 Changed 4 years ago by batterseapower

I reported a bug to binutils over 5 months ago asking them to make "ld" error out when this happens, but there is no indication that anyone has even read the bug. I even provided a patch, so this is a bit disappointing :-(

The bug is at http://sourceware.org/bugzilla/show_bug.cgi?id=12969

comment:2 Changed 4 years ago by igloo

Simon Marlow says:

There is one problem that I know of with breaking up the compiler into separate packages: GHC will do too much recompilation when things change, which will make development somewhat painful. I discovered this when making changes in hoopl: a small change in hoopl causes recompilation of nearly every module in GHC. The reason is the way that GHC records dependencies across packages: rather than the fine-grained dependencies we get within a package, a dependency on a package module records only the ABI hash of the whole module.

comment:3 Changed 4 years ago by simonpj

I mis-read Max's comment. I thought he was saying that we (the GHC crowd) have been ignoring the ticket. But actually he's saying that he offered a patch to the binutils folk, but they have ignored it so far. Does anyone know anyone who who might help un-glue the binutils process?

Simon

comment:4 Changed 4 years ago by igloo

Note that the binutils patch doesn't actually fix the problem: It just means that the GHC build will fail, rather than silently making a bogus DLL.

comment:5 Changed 4 years ago by igloo

  • Owner set to igloo

comment:6 Changed 4 years ago by igloo@…

commit 6c29fad6a1ab20e8cfed5ad4af406a821ee74291

Author: Ian Lynagh <igloo@earth.li>
Date:   Fri Jun 29 17:47:32 2012 +0100

    Don't build the ghc DLL on Windows; works around trac #5987
    
    We also don't build DLLs for the dph-lifted-* packages as they depend
    on ghc.

 compiler/ghc.mk        |    8 ++++++++
 ghc.mk                 |   12 ++++++++++++
 rules/build-package.mk |    2 +-
 3 files changed, 21 insertions(+), 1 deletions(-)

comment:7 Changed 4 years ago by igloo@…

commit 5ddc841a547d2d37428280b04c071a1d4b83e10d

Author: Ian Lynagh <igloo@earth.li>
Date:   Fri Jun 29 01:33:17 2012 +0100

    On Windows, detect if DLLs have too many symbols; trac #5987
    
    The test isn't very pretty; it involves trawling through the
    objdump -p output. I couldn't find an easier way, unfortuantely.

 configure.ac               |    7 +++++++
 mk/config.mk.in            |    1 +
 rules/build-package-way.mk |    3 +++
 3 files changed, 11 insertions(+), 0 deletions(-)

comment:8 Changed 4 years ago by igloo

  • Milestone changed from 7.6.1 to 7.8.1

This isn't fixed, but (a) we now detect it if it happens, and (b) we no longer build a broken ghc package DLL (because we don't build the DLL at all).

So we still need a proper fix, but this will do for now.

comment:9 Changed 4 years ago by igloo

  • Owner igloo deleted

comment:10 Changed 4 years ago by marlowsd@…

commit 99fd2469fba1a38b2a65b4694f337d92e559df01

Merge: d260d91... 0d19922...
Author: Simon Marlow <marlowsd@gmail.com>
Date:   Wed Jul 4 10:34:48 2012 +0100

    Merge remote-tracking branch 'origin/master' into newcg
    
    * origin/master: (756 commits)
      don't crash if argv[0] == NULL (#7037)
      -package P was loading all versions of P in GHCi (#7030)
      Add a Note, copying text from #2437
      improve the --help docs a bit (#7008)
      Copy Data.HashTable's hashString into our Util module
      Build fix
      Build fixes
      Parse error: suggest brackets and indentation.
      Don't build the ghc DLL on Windows; works around trac #5987
      On Windows, detect if DLLs have too many symbols; trac #5987
      Add some more Integer rules; fixes #6111
      Fix PA dfun construction with silent superclass args
      Add silent superclass parameters to the vectoriser
      Add silent superclass parameters (again)
      Mention Generic1 in the user's guide
      Make the GHC API a little more powerful.
      tweak llvm version warning message
      New version of the patch for #5461.
      Fix Word64ToInteger conversion rule.
      Implemented feature request on reconfigurable pretty-printing in GHCi (#5461)
      ...
    
    Conflicts:
    	compiler/basicTypes/UniqSupply.lhs
    	compiler/cmm/CmmBuildInfoTables.hs
    	compiler/cmm/CmmLint.hs
    	compiler/cmm/CmmOpt.hs
    	compiler/cmm/CmmPipeline.hs
    	compiler/cmm/CmmStackLayout.hs
    	compiler/cmm/MkGraph.hs
    	compiler/cmm/OldPprCmm.hs
    	compiler/codeGen/CodeGen.lhs
    	compiler/codeGen/StgCmm.hs
    	compiler/codeGen/StgCmmBind.hs
    	compiler/codeGen/StgCmmLayout.hs
    	compiler/codeGen/StgCmmUtils.hs
    	compiler/main/CodeOutput.lhs
    	compiler/main/HscMain.hs
    	compiler/nativeGen/AsmCodeGen.lhs
    	compiler/simplStg/SimplStg.lhs

 .authorspellings                                   |   29 -
 .darcs-boring                                      |  256 ---
 .gitignore                                         |    1 +
 README                                             |   11 +-
 aclocal.m4                                         |   96 +-
 bindisttest/Makefile                               |    4 +-
 bindisttest/ghc.mk                                 |    4 +-
 compiler/HsVersions.h                              |   13 +-
 compiler/basicTypes/BasicTypes.lhs                 |   44 +-
 compiler/basicTypes/DataCon.lhs                    |   33 +-
 compiler/basicTypes/Demand.lhs                     |    3 -
 compiler/basicTypes/Id.lhs                         |   21 +-
 compiler/basicTypes/IdInfo.lhs                     |   17 +-
 compiler/basicTypes/Literal.lhs                    |   56 +-
 compiler/basicTypes/MkId.lhs                       |   74 +-
 compiler/basicTypes/MkId.lhs-boot                  |    3 +
 compiler/basicTypes/Name.lhs                       |    5 +-
 compiler/basicTypes/NameEnv.lhs                    |   32 +-
 compiler/basicTypes/OccName.lhs                    |   13 +-
 compiler/basicTypes/RdrName.lhs                    |   46 +-
 compiler/basicTypes/SrcLoc.lhs                     |  279 ++--
 compiler/basicTypes/UniqSupply.lhs                 |    9 +-
 compiler/basicTypes/Unique.lhs                     |   37 +-
 compiler/basicTypes/Var.lhs                        |   40 +-
 compiler/cmm/Bitmap.hs                             |    2 +-
 compiler/cmm/CLabel.hs                             |  117 +-
 compiler/cmm/Cmm.hs                                |   10 +-
 compiler/cmm/CmmBuildInfoTables.hs                 |   38 +-
 compiler/cmm/CmmCvt.hs                             |    2 +-
 compiler/cmm/CmmExpr.hs                            |    4 +
 compiler/cmm/CmmInfo.hs                            |    4 +-
 compiler/cmm/CmmLint.hs                            |   71 +-
 compiler/cmm/CmmMachOp.hs                          |    9 +-
 compiler/cmm/CmmOpt.hs                             |   31 +-
 compiler/cmm/CmmParse.y                            |   10 +-
 compiler/cmm/CmmPipeline.hs                        |   22 +-
 compiler/cmm/CmmProcPoint.hs                       |    2 +-
 compiler/cmm/CmmStackLayout.hs                     |  591 +++++
 compiler/cmm/CmmType.hs                            |    6 +
 compiler/cmm/CmmUtils.hs                           |    7 +-
 compiler/cmm/OldCmm.hs                             |    9 +-
 compiler/cmm/OldCmmLint.hs                         |   70 +-
 compiler/cmm/OldCmmUtils.hs                        |    1 +
 compiler/cmm/OldPprCmm.hs                          |   99 +-
 compiler/cmm/OptimizationFuel.hs                   |    1 +
 compiler/cmm/PprC.hs                               |   76 +-
 compiler/cmm/PprCmm.hs                             |  109 +-
 compiler/cmm/PprCmmDecl.hs                         |   97 +-
 compiler/cmm/PprCmmExpr.hs                         |  117 +-
 compiler/codeGen/CgBindery.lhs                     |   27 +-
 compiler/codeGen/CgCase.lhs                        |    1 -
 compiler/codeGen/CgClosure.lhs                     |   29 +-
 compiler/codeGen/CgCon.lhs                         |   11 +-
 compiler/codeGen/CgExpr.lhs                        |    2 +-
 compiler/codeGen/CgForeignCall.hs                  |    8 +-
 compiler/codeGen/CgInfoTbls.hs                     |    5 +-
 compiler/codeGen/CgMonad.lhs                       |    1 +
 compiler/codeGen/CgParallel.hs                     |   69 +-
 compiler/codeGen/CgPrimOp.hs                       |  186 ++-
 compiler/codeGen/CgProf.hs                         |    3 +-
 compiler/codeGen/CgStackery.lhs                    |    3 +-
 compiler/codeGen/CgTailCall.lhs                    |    1 +
 compiler/codeGen/CgTicky.hs                        |   13 +-
 compiler/codeGen/CgUtils.hs                        |   13 +-
 compiler/codeGen/ClosureInfo.lhs                   |   50 +-
 compiler/codeGen/CodeGen.lhs                       |    3 +-
 compiler/codeGen/StgCmm.hs                         |    5 +-
 compiler/codeGen/StgCmmBind.hs                     |   47 +-
 compiler/codeGen/StgCmmClosure.hs                  |   68 +-
 compiler/codeGen/StgCmmCon.hs                      |    4 +-
 compiler/codeGen/StgCmmEnv.hs                      |    1 -
 compiler/codeGen/StgCmmExpr.hs                     |    2 +-
 compiler/codeGen/StgCmmForeign.hs                  |    7 +-
 compiler/codeGen/StgCmmGran.hs                     |   57 +-
 compiler/codeGen/StgCmmHeap.hs                     |   10 +-
 compiler/codeGen/StgCmmLayout.hs                   |   27 +-
 compiler/codeGen/StgCmmMonad.hs                    |   12 +-
 compiler/codeGen/StgCmmPrim.hs                     |   17 +-
 compiler/codeGen/StgCmmProf.hs                     |    3 +-
 compiler/codeGen/StgCmmTicky.hs                    |   19 +-
 compiler/codeGen/StgCmmUtils.hs                    |   14 +-
 compiler/coreSyn/CoreArity.lhs                     |   10 +-
 compiler/coreSyn/CoreFVs.lhs                       |   20 +-
 compiler/coreSyn/CoreLint.lhs                      |  531 ++---
 compiler/coreSyn/CorePrep.lhs                      |  481 ++--
 compiler/coreSyn/CoreSubst.lhs                     |    5 +-
 compiler/coreSyn/CoreSyn.lhs                       |   92 +-
 compiler/coreSyn/CoreTidy.lhs                      |    4 +-
 compiler/coreSyn/CoreUnfold.lhs                    |  165 +-
 compiler/coreSyn/CoreUtils.lhs                     |  275 ++-
 compiler/coreSyn/MkCore.lhs                        |   54 +-
 compiler/coreSyn/MkExternalCore.lhs                |  199 +-
 compiler/coreSyn/PprCore.lhs                       |  261 ++--
 compiler/coreSyn/TrieMap.lhs                       |  137 +-
 compiler/deSugar/Coverage.lhs                      |  510 +++--
 compiler/deSugar/Desugar.lhs                       |    9 +-
 compiler/deSugar/DsArrows.lhs                      |    4 +-
 compiler/deSugar/DsBinds.lhs                       |  208 ++-
 compiler/deSugar/DsCCall.lhs                       |   13 +-
 compiler/deSugar/DsExpr.lhs                        |   63 +-
 compiler/deSugar/DsForeign.lhs                     |  136 +-
 compiler/deSugar/DsListComp.lhs                    |   50 +-
 compiler/deSugar/DsMeta.hs                         | 1005 +++++----
 compiler/deSugar/DsMonad.lhs                       |   19 +-
 compiler/deSugar/DsUtils.lhs                       |    4 +-
 compiler/deSugar/Match.lhs                         |    2 +-
 compiler/deSugar/MatchCon.lhs                      |   17 +-
 compiler/ghc.cabal.in                              |   42 +-
 compiler/ghc.mk                                    |   28 +-
 compiler/ghci/ByteCodeAsm.lhs                      |  599 +++---
 compiler/ghci/ByteCodeGen.lhs                      |  276 ++-
 compiler/ghci/ByteCodeItbls.lhs                    |   31 +-
 compiler/ghci/ByteCodeLink.lhs                     |   14 +-
 compiler/ghci/Debugger.hs                          |   11 +-
 compiler/ghci/DebuggerUtils.hs                     |    4 +-
 compiler/ghci/LibFFI.hsc                           |    4 +-
 compiler/ghci/Linker.lhs                           |  705 +++---
 compiler/ghci/ObjLink.lhs                          |   41 +-
 compiler/ghci/RtClosureInspect.hs                  |  180 +-
 compiler/hsSyn/Convert.lhs                         |  353 ++--
 compiler/hsSyn/HsBinds.lhs                         |   44 +-
 compiler/hsSyn/HsDecls.lhs                         |  419 ++--
 compiler/hsSyn/HsExpr.lhs                          |   46 +-
 compiler/hsSyn/HsImpExp.lhs                        |   19 +-
 compiler/hsSyn/HsPat.lhs                           |   11 +-
 compiler/hsSyn/HsSyn.lhs                           |    3 +-
 compiler/hsSyn/HsTypes.lhs                         |  406 ++--
 compiler/hsSyn/HsUtils.lhs                         |   94 +-
 compiler/iface/BinIface.hs                         |  169 +-
 compiler/iface/BuildTyCl.lhs                       |   15 +-
 compiler/iface/FlagChecker.hs                      |   46 +-
 compiler/iface/IfaceEnv.lhs                        |   25 +-
 compiler/iface/IfaceSyn.lhs                        |   39 +-
 compiler/iface/IfaceType.lhs                       |  154 +-
 compiler/iface/LoadIface.lhs                       |   15 +-
 compiler/iface/MkIface.lhs                         |  254 ++-
 compiler/iface/TcIface.lhs                         |  198 ++-
 compiler/llvmGen/Llvm.hs                           |    1 -
 compiler/llvmGen/Llvm/PpLlvm.hs                    |  106 +-
 compiler/llvmGen/LlvmCodeGen.hs                    |   41 +-
 compiler/llvmGen/LlvmCodeGen/Base.hs               |   16 +-
 compiler/llvmGen/LlvmCodeGen/CodeGen.hs            |   57 +-
 compiler/llvmGen/LlvmCodeGen/Ppr.hs                |   13 +-
 compiler/main/Annotations.hs                       |  107 +
 compiler/main/Annotations.lhs                      |  108 -
 compiler/main/CmdLineParser.hs                     |    4 +-
 compiler/main/CodeOutput.lhs                       |   13 +-
 compiler/main/Constants.lhs                        |    4 +
 compiler/main/DriverMkDepend.hs                    |   10 +-
 compiler/main/DriverPipeline.hs                    |  106 +-
 compiler/main/DynFlags.hs                          |  412 +++-
 compiler/main/DynFlags.hs-boot                     |   13 +
 compiler/main/DynamicLoading.hs                    |   21 +-
 compiler/main/ErrUtils.lhs                         |  119 +-
 compiler/main/ErrUtils.lhs-boot                    |    1 +
 compiler/main/GHC.hs                               |  192 ++-
 compiler/main/GhcMake.hs                           | 1306 ++++++------
 compiler/main/HeaderInfo.hs                        |   78 +-
 compiler/main/HscMain.hs                           |  325 ++--
 compiler/main/HscStats.hs                          |  160 ++
 compiler/main/HscStats.lhs                         |   12 +-
 compiler/main/HscTypes.lhs                         |  105 +-
 compiler/main/InteractiveEval.hs                   |   99 +-
 compiler/main/PackageConfig.hs                     |   50 +-
 compiler/main/Packages.lhs                         |  437 ++--
 compiler/main/StaticFlagParser.hs                  |   38 +-
 compiler/main/StaticFlags.hs                       |   68 +-
 compiler/main/SysTools.lhs                         |   25 +-
 compiler/main/TidyPgm.lhs                          |  749 ++++---
 compiler/nativeGen/AsmCodeGen.lhs                  |  647 +++---
 compiler/nativeGen/PIC.hs                          |   93 +-
 compiler/nativeGen/PPC/CodeGen.hs                  |   33 +-
 compiler/nativeGen/PPC/Ppr.hs                      |  567 +++---
 compiler/nativeGen/PPC/Regs.hs                     |   14 +-
 compiler/nativeGen/PprBase.hs                      |   17 -
 compiler/nativeGen/RegAlloc/Graph/Main.hs          |  609 +++---
 compiler/nativeGen/RegAlloc/Graph/SpillCost.hs     |   10 +-
 compiler/nativeGen/RegAlloc/Graph/Stats.hs         |   30 +-
 compiler/nativeGen/RegAlloc/Graph/TrivColorable.hs |   56 +-
 compiler/nativeGen/RegAlloc/Linear/FreeRegs.hs     |   14 +-
 compiler/nativeGen/RegAlloc/Linear/Main.hs         |   44 +-
 compiler/nativeGen/RegAlloc/Liveness.hs            |   68 +-
 compiler/nativeGen/SPARC/CodeGen.hs                |  511 ++++-
 compiler/nativeGen/SPARC/CodeGen/CCall.hs          |  343 ---
 compiler/nativeGen/SPARC/CodeGen/CondCode.hs       |    7 +-
 compiler/nativeGen/SPARC/CodeGen/Gen64.hs          |    3 +-
 compiler/nativeGen/SPARC/CodeGen/Sanity.hs         |   10 +-
 compiler/nativeGen/SPARC/Imm.hs                    |    5 +-
 compiler/nativeGen/SPARC/Ppr.hs                    |   94 +-
 compiler/nativeGen/TargetReg.hs                    |   70 +-
 compiler/nativeGen/X86/CodeGen.hs                  |  458 +++--
 compiler/nativeGen/X86/Instr.hs                    |    3 +
 compiler/nativeGen/X86/Ppr.hs                      |  155 +-
 compiler/nativeGen/X86/Regs.hs                     |   43 +-
 compiler/parser/Lexer.x                            |   75 +-
 compiler/parser/Parser.y.pp                        |  143 +-
 compiler/parser/ParserCore.y                       |   34 +-
 compiler/parser/RdrHsSyn.lhs                       |  330 ++--
 compiler/prelude/ForeignCall.lhs                   |   56 +-
 compiler/prelude/PrelNames.lhs                     |  136 +-
 compiler/prelude/PrelRules.lhs                     |  212 ++-
 compiler/prelude/PrimOp.lhs                        |    3 -
 compiler/prelude/PrimOp.lhs-boot                   |    7 +
 compiler/prelude/TysPrim.lhs                       |  114 +-
 compiler/prelude/TysWiredIn.lhs                    |   88 +-
 compiler/prelude/primops.txt.pp                    |   39 +
 compiler/profiling/CostCentre.lhs                  |    6 +-
 compiler/profiling/ProfInit.hs                     |    9 +-
 compiler/profiling/SCCfinal.lhs                    |    1 +
 compiler/rename/RnBinds.lhs                        |   80 +-
 compiler/rename/RnEnv.lhs                          |  470 +++--
 compiler/rename/RnExpr.lhs                         |   80 +-
 compiler/rename/RnHsSyn.lhs                        |  159 --
 compiler/rename/RnNames.lhs                        |  194 +--
 compiler/rename/RnPat.lhs                          |   41 +-
 compiler/rename/RnSource.lhs                       |  563 +++---
 compiler/rename/RnTypes.lhs                        |  652 ++++--
 compiler/simplCore/CSE.lhs                         |   30 +-
 compiler/simplCore/CoreMonad.lhs                   |   61 +-
 compiler/simplCore/FloatIn.lhs                     |   18 +-
 compiler/simplCore/OccurAnal.lhs                   |  116 +-
 compiler/simplCore/SetLevels.lhs                   |   41 +-
 compiler/simplCore/SimplCore.lhs                   |   92 +-
 compiler/simplCore/SimplEnv.lhs                    |    1 +
 compiler/simplCore/SimplMonad.lhs                  |    3 +-
 compiler/simplCore/SimplUtils.lhs                  |  234 +--
 compiler/simplCore/Simplify.lhs                    |  131 +-
 compiler/simplStg/SRT.lhs                          |    4 +-
 compiler/simplStg/SimplStg.lhs                     |   14 +-
 compiler/simplStg/UnariseStg.lhs                   |  167 ++
 compiler/specialise/Rules.lhs                      |  584 +++---
 compiler/specialise/SpecConstr.lhs                 |   13 +-
 compiler/specialise/Specialise.lhs                 | 1070 +++++-----
 compiler/stgSyn/CoreToStg.lhs                      |   41 +-
 compiler/stgSyn/StgLint.lhs                        |   71 +-
 compiler/stgSyn/StgSyn.lhs                         |   48 +-
 compiler/stranal/DmdAnal.lhs                       |   17 +-
 compiler/stranal/WorkWrap.lhs                      |   88 +-
 compiler/stranal/WwLib.lhs                         |   56 +-
 compiler/typecheck/FamInst.lhs                     |   78 +-
 compiler/typecheck/Inst.lhs                        |  104 +-
 compiler/typecheck/TcArrows.lhs                    |    4 +-
 compiler/typecheck/TcBinds.lhs                     |  485 ++---
 compiler/typecheck/TcCanonical.lhs                 | 1384 ++++++-------
 compiler/typecheck/TcClassDcl.lhs                  |  124 +-
 compiler/typecheck/TcDeriv.lhs                     |  292 ++-
 compiler/typecheck/TcEnv.lhs                       |  194 +-
 compiler/typecheck/TcErrors.lhs                    |  388 ++--
 compiler/typecheck/TcEvidence.lhs                  |  159 ++-
 compiler/typecheck/TcExpr.lhs                      |   88 +-
 compiler/typecheck/TcForeign.lhs                   |  104 +-
 compiler/typecheck/TcGenDeriv.lhs                  |   12 +-
 compiler/typecheck/TcGenGenerics.lhs               |  515 ++++-
 compiler/typecheck/TcHsSyn.lhs                     |  184 +-
 compiler/typecheck/TcHsType.lhs                    | 1528 +++++++------
 compiler/typecheck/TcInstDcls.lhs                  |  433 +++--
 compiler/typecheck/TcInteract.lhs                  | 1171 +++++-----
 compiler/typecheck/TcMType.lhs                     |  828 ++++----
 compiler/typecheck/TcMatches.lhs                   |   70 +-
 compiler/typecheck/TcPat.lhs                       |   77 +-
 compiler/typecheck/TcRnDriver.lhs                  |  231 ++-
 compiler/typecheck/TcRnMonad.lhs                   |   80 +-
 compiler/typecheck/TcRnTypes.lhs                   |  326 ++--
 compiler/typecheck/TcRules.lhs                     |  197 ++-
 compiler/typecheck/TcSMonad.lhs                    | 1493 ++++++++------
 compiler/typecheck/TcSimplify.lhs                  | 1115 ++++++----
 compiler/typecheck/TcSplice.lhs                    |  157 +-
 compiler/typecheck/TcSplice.lhs-boot               |    7 +-
 compiler/typecheck/TcTyClsDecls.lhs                |  848 ++++----
 compiler/typecheck/TcTyDecls.lhs                   |   18 +-
 compiler/typecheck/TcType.lhs                      |  131 +-
 compiler/typecheck/TcUnify.lhs                     |  237 +--
 compiler/types/Class.lhs                           |    3 -
 compiler/types/Coercion.lhs                        |   22 +-
 compiler/types/FamInstEnv.lhs                      |   12 +-
 compiler/types/FunDeps.lhs                         |   91 +-
 compiler/types/IParam.lhs                          |   41 -
 compiler/types/IParam.lhs-boot                     |   10 -
 compiler/types/InstEnv.lhs                         |   71 +-
 compiler/types/Kind.lhs                            |  240 +--
 compiler/types/OptCoercion.lhs                     |    1 +
 compiler/types/TyCon.lhs                           |  871 ++++----
 compiler/types/Type.lhs                            |  303 ++--
 compiler/types/TypeRep.lhs                         |  172 +-
 compiler/types/Unify.lhs                           |   59 +-
 compiler/utils/Binary.hs                           |    3 -
 compiler/utils/Digraph.lhs                         |   17 +-
 compiler/utils/Exception.hs                        |    6 +-
 compiler/utils/FastString.lhs                      |    5 -
 compiler/utils/GraphColor.hs                       |  597 +++---
 compiler/utils/GraphOps.hs                         |  923 ++++----
 compiler/utils/ListSetOps.lhs                      |   79 +-
 compiler/utils/MonadUtils.hs                       |   18 -
 compiler/utils/Outputable.lhs                      |  268 +--
 compiler/utils/Outputable.lhs-boot                 |    7 +
 compiler/utils/Panic.lhs                           |  197 ++-
 compiler/utils/Platform.hs                         |   14 +-
 compiler/utils/Pretty.lhs                          |    9 +-
 compiler/utils/UniqFM.lhs                          |    4 +-
 compiler/utils/Util.lhs                            |  211 +-
 compiler/vectorise/Vectorise.hs                    |   21 +-
 compiler/vectorise/Vectorise/Builtins/Base.hs      |    2 +-
 .../vectorise/Vectorise/Builtins/Initialise.hs     |    2 +-
 compiler/vectorise/Vectorise/Convert.hs            |    9 +-
 compiler/vectorise/Vectorise/Env.hs                |    9 +-
 compiler/vectorise/Vectorise/Exp.hs                |  716 +++++--
 .../vectorise/Vectorise/Generic/Description.hs     |    3 +-
 compiler/vectorise/Vectorise/Generic/PADict.hs     |   41 +-
 compiler/vectorise/Vectorise/Generic/PAMethods.hs  |   27 +-
 compiler/vectorise/Vectorise/Generic/PData.hs      |    4 +-
 compiler/vectorise/Vectorise/Monad.hs              |   20 +-
 compiler/vectorise/Vectorise/Monad/Base.hs         |   41 +-
 compiler/vectorise/Vectorise/Monad/Global.hs       |    9 +-
 compiler/vectorise/Vectorise/Monad/InstEnv.hs      |   18 +-
 compiler/vectorise/Vectorise/Type/Classify.hs      |    1 +
 compiler/vectorise/Vectorise/Type/TyConDecl.hs     |   29 +-
 compiler/vectorise/Vectorise/Type/Type.hs          |    1 +
 compiler/vectorise/Vectorise/Utils/Base.hs         |   19 +-
 compiler/vectorise/Vectorise/Utils/PADict.hs       |   74 +-
 config.guess                                       |  482 ++---
 configure.ac                                       |  156 +-
 distrib/compare/compare.hs                         |   23 +-
 docs/coding-style.html                             |   60 +-
 docs/comm/genesis/modules.html                     |    2 +-
 docs/comm/rts-libs/threaded-rts.html               |    2 +-
 docs/ext-core/Makefile                             |    3 -
 docs/ext-core/a4wide.sty                           |   39 -
 docs/ext-core/code.sty                             |   83 -
 docs/ext-core/core.bib                             |  124 -
 docs/ext-core/core.tex                             |  779 -------
 docs/ext-core/ghc.mk                               |   15 -
 docs/index.html.in                                 |   22 +-
 docs/users_guide/bugs.xml                          |   26 +-
 docs/users_guide/external_core.xml                 | 1807 +++++++++++++++
 docs/users_guide/ffi-chap.xml                      |   52 +
 docs/users_guide/flags.xml                         |  103 +-
 docs/users_guide/ghci.xml                          |  119 +-
 docs/users_guide/glasgow_exts.xml                  |  694 ++++--
 docs/users_guide/packages.xml                      |  108 +-
 docs/users_guide/phases.xml                        |   25 +-
 docs/users_guide/runghc.xml                        |    2 +-
 docs/users_guide/runtime_control.xml               |   43 +-
 docs/users_guide/ug-book.xml.in                    |    1 +
 docs/users_guide/ug-ent.xml.in                     |    2 +
 docs/users_guide/using.xml                         | 2354 +++++++++++---------
 docs/vh/vh.xml                                     |  593 +++---
 driver/ghci/ghc.mk                                 |    4 +-
 driver/utils/cwrapper.c                            |    6 +-
 ghc.mk                                             |   39 +-
 ghc/GhciMonad.hs                                   |   48 +-
 ghc/GhciTags.hs                                    |   42 +-
 ghc/InteractiveUI.hs                               |  616 ++++--
 ghc/Main.hs                                        |   19 +-
 ghc/ghc-bin.cabal.in                               |    4 +-
 ghc/ghc.mk                                         |   11 +-
 ghc/hschooks.c                                     |    5 +-
 includes/Cmm.h                                     |    8 +-
 includes/HaskellConstants.hs                       |   12 +-
 includes/MachDeps.h                                |   27 +-
 includes/Rts.h                                     |   26 +-
 includes/RtsAPI.h                                  |    2 +-
 includes/Stg.h                                     |   10 +-
 includes/ghc.mk                                    |   31 +-
 includes/mkDerivedConstants.c                      |   23 +-
 includes/rts/Constants.h                           |   50 +-
 includes/rts/EventLogFormat.h                      |   27 +-
 includes/rts/FileLock.h                            |    6 +-
 includes/rts/Flags.h                               |   12 +-
 includes/rts/Hooks.h                               |    6 +-
 includes/rts/Messages.h                            |   15 +-
 includes/rts/Threads.h                             |    4 +
 includes/rts/Types.h                               |    6 +-
 includes/rts/storage/GC.h                          |    5 +-
 includes/rts/storage/InfoTables.h                  |   22 +-
 includes/rts/storage/TSO.h                         |    2 +-
 includes/stg/DLL.h                                 |   25 +-
 includes/stg/MachRegs.h                            |  209 +-
 includes/stg/MiscClosures.h                        |    2 +
 includes/stg/Types.h                               |   29 +-
 libffi/ghc.mk                                      |   11 +-
 .../Distribution/InstalledPackageInfo/Binary.hs    |    4 +-
 libraries/gen_contents_index                       |   37 +-
 mk/config.mk.in                                    |   21 +-
 mk/tree.mk                                         |    2 +-
 mk/validate-settings.mk                            |   18 +-
 packages                                           |    3 +-
 rts/Adjustor.c                                     |  477 +++--
 rts/Capability.c                                   |   50 +-
 rts/Capability.h                                   |    6 +-
 rts/ClosureFlags.c                                 |   18 +-
 rts/Disassembler.c                                 |   18 +-
 rts/FileLock.c                                     |  144 ++
 rts/FileLock.h                                     |   15 +
 rts/GetTime.h                                      |    3 +
 rts/HeapStackCheck.cmm                             |   12 +-
 rts/Interpreter.c                                  |   48 +-
 rts/Linker.c                                       |  826 +++++++-
 rts/LinkerInternals.h                              |    6 +-
 rts/PosixSource.h                                  |    4 +
 rts/Prelude.h                                      |    8 +-
 rts/PrimOps.cmm                                    |   39 +-
 rts/Printer.c                                      |   26 +-
 rts/ProfHeap.c                                     |    2 +-
 rts/RaiseAsync.c                                   |   77 +-
 rts/RetainerProfile.c                              |    4 +-
 rts/RtsDllMain.c                                   |    4 +-
 rts/RtsDllMain.h                                   |    2 +-
 rts/RtsFlags.c                                     |   20 +-
 rts/RtsMain.c                                      |    7 +-
 rts/RtsProbes.d                                    |   37 +-
 rts/RtsStartup.c                                   |   20 +-
 rts/Schedule.c                                     |   38 +-
 rts/Stats.c                                        |  163 +-
 rts/Stats.h                                        |   19 +-
 rts/StgCRun.c                                      |   87 +-
 rts/StgMiscClosures.cmm                            |    6 +-
 rts/StgStartup.cmm                                 |   12 +-
 rts/Task.c                                         |  104 +-
 rts/Task.h                                         |   30 +-
 rts/Threads.c                                      |   54 +-
 rts/Ticky.c                                        |   26 +-
 rts/Trace.c                                        |  148 ++-
 rts/Trace.h                                        |  263 ++-
 rts/eventlog/EventLog.c                            |  201 ++-
 rts/eventlog/EventLog.h                            |   34 +
 rts/ghc.mk                                         |   23 +-
 rts/hooks/MallocFail.c                             |    2 +-
 rts/hooks/OutOfHeap.c                              |    2 +-
 rts/hooks/StackOverflow.c                          |    2 +-
 rts/package.conf.in                                |    8 +-
 rts/posix/Clock.h                                  |   35 +
 rts/posix/FileLock.c                               |  145 --
 rts/posix/FileLock.h                               |   15 -
 rts/posix/GetTime.c                                |   50 +-
 rts/posix/Itimer.c                                 |   13 +-
 rts/posix/OSMem.c                                  |    2 +-
 rts/posix/OSThreads.c                              |    1 -
 rts/posix/Select.c                                 |   81 +-
 rts/posix/Select.h                                 |    2 +-
 rts/sm/Compact.c                                   |    2 +-
 rts/sm/Evac.c                                      |    2 +-
 rts/sm/GC.c                                        |   25 +-
 rts/sm/Sanity.c                                    |   18 +-
 rts/sm/Scav.c                                      |    6 +-
 rts/sm/Storage.c                                   |  178 +-
 rts/sm/Storage.h                                   |    3 +-
 rts/win32/AwaitEvent.c                             |    4 +-
 rts/win32/GetTime.c                                |   74 +-
 rts/win32/IOManager.c                              |    2 +-
 rts/win32/OSMem.c                                  |   20 +-
 rts/win32/ThrIOManager.c                           |    6 +-
 rts/win32/libHSbase.def                            |    1 -
 rts/win32/libHSghc-prim.def                        |    1 +
 rts/win32/seh_excn.c                               |    5 +-
 rules/build-package-data.mk                        |    8 +-
 rules/build-package-way.mk                         |   15 +-
 rules/build-package.mk                             |    2 +-
 rules/build-prog.mk                                |    4 +-
 rules/c-suffix-rules.mk                            |   22 +-
 rules/cmm-suffix-rules.mk                          |   12 +-
 rules/distdir-way-opts.mk                          |   46 +-
 rules/hs-suffix-rules-srcdir.mk                    |   30 +-
 rules/hs-suffix-rules.mk                           |    8 +-
 rules/package-config.mk                            |   15 +-
 rules/pretty_commands.mk                           |   13 +
 sync-all                                           |   63 +
 utils/compare_sizes/Main.hs                        |    1 -
 utils/fingerprint/fingerprint.py                   |    2 +-
 utils/genapply/GenApply.hs                         |  650 +++---
 utils/genprimopcode/Main.hs                        |  433 ++--
 utils/genprimopcode/Parser.y                       |    1 -
 utils/genprimopcode/ParserM.hs                     |    1 -
 utils/genprimopcode/Syntax.hs                      |   20 +-
 utils/ghc-cabal/Main.hs                            |   44 +-
 utils/ghc-cabal/ghc.mk                             |    4 +-
 utils/ghc-pkg/Main.hs                              |  108 +-
 utils/ghc-pkg/ghc-pkg.wrapper                      |    2 +-
 utils/ghc-pkg/ghc.mk                               |    8 +-
 utils/ghctags/Main.hs                              |   16 +-
 utils/hp2ps/Key.c                                  |   23 +-
 utils/runghc/ghc.mk                                |    3 +
 utils/runghc/runghc.hs                             |   12 +-
 validate                                           |   27 +-
 483 files changed, 33989 insertions(+), 25880 deletions(-)

comment:11 Changed 4 years ago by thorkilnaur

The commit 5ddc841a547d2d37428280b04c071a1d4b83e10d causes tn23 (a Mac OS X builder) to fail with

configure: error: cannot find objdump in your PATH

See http://darcs.haskell.org/ghcBuilder/builders/tn23/655/8.html. Best regards Thorkil

comment:12 Changed 4 years ago by igloo

  • Blocking 3658 added

comment:13 follow-up: Changed 3 years ago by duncan

Have we looked into whether we can avoid exporting so many symbols by hiding all package-internal symbols. My intuition is that there must be lots of them that are only needed for intra-package linkage and are not needed in the external interface. The only symbols that should need to be external are the ones for things accessable via the .hi files of the exposed modules. Though if basically every module is exposed then that would not help much.

comment:14 in reply to: ↑ 13 Changed 3 years ago by simonmar

Replying to duncan:

Have we looked into whether we can avoid exporting so many symbols by hiding all package-internal symbols. My intuition is that there must be lots of them that are only needed for intra-package linkage and are not needed in the external interface. The only symbols that should need to be external are the ones for things accessable via the .hi files of the exposed modules. Though if basically every module is exposed then that would not help much.

We should do this, but it wouldn't help in the case of the GHC package: as you say, every module is exposed.

It would be good to have a per-module flag to say "this is a hidden module", and then we could give the symbols the correct linker hint so that they don't populate the .so file's symbol table.

comment:15 Changed 3 years ago by rassilon

  • Cc bill@… added

Here's a pretty nutty idea: What if we split the DLL in parts by automatically forcing the .o files into partitions and create a .def file for each partition thus breaking the import library cycle.

GHC already uses .def files to break the cycle between the RTS and the base DLLs.

Then you would "just" modify Cabal to understand partitioned DLLs. Unfortunately, this might end up increasing the total number of partitions if you're not careful because internal symbols would now need to be exported from the partitions. :(

comment:16 Changed 3 years ago by igloo@…

commit 56353e3da9d5718dfd25e25ccf61c78b25deefe8

Author: Ian Lynagh <igloo@earth.li>
Date:   Tue Mar 12 22:12:38 2013 +0100

    Finish adding support for 2 DLLs in the ghc package; fixes #5987

 ghc.mk                     |    3 ---
 rules/build-package-way.mk |   20 +++++++++++++++++---
 utils/ghc-cabal/Main.hs    |   27 +++++++++++++++++++++++++--
 3 files changed, 42 insertions(+), 8 deletions(-)

comment:17 Changed 3 years ago by igloo

  • Resolution set to fixed
  • Status changed from new to closed

Fixed. We now have

libHSghc-7.7.20130310-0-ghc7.7.20130310.dll (42123 symbols)
libHSghc-7.7.20130310-ghc7.7.20130310.dll   (47675 symbols)

comment:18 Changed 3 years ago by simonpj

Great. Are there some Wiki notes or comments in the code to explain the thinking, what the problem is, how it is solved, where the partioning is done, how you managed to factor GHC into two? Etc. This kind of global info can be hard to find.

Indeed is there a wiki page surveying the whole dynamic-linking question and how we address it? I can't find it if so.

Simon

comment:19 Changed 3 years ago by george.colpitts

For the wiki page surveying the whole dynamic-linking question I believe it is http://ghc.haskell.org/trac/ghc/wiki/DynamicGhcPrograms

comment:20 Changed 3 years ago by george.colpitts

  • Cc george.colpitts@… added

comment:21 Changed 3 years ago by awson

Now x86_64 build fails on Windows again:

Number of symbols in compiler/stage2/build/libHSghc-7.7.20130915-0-ghc7.7.20130915.dll: 65583
...
Too many symbols in DLL compiler/stage2/build/libHSghc-7.7.20130915-0-ghc7.7.20130915.dll

comment:22 Changed 3 years ago by akio

  • Cc tkn.akio@… added

comment:23 Changed 3 years ago by awson

  • Resolution fixed deleted
  • Status changed from closed to new

comment:24 Changed 3 years ago by thoughtpolice

  • Owner set to thoughtpolice

comment:25 Changed 3 years ago by tomberek

compiling master:

# Now check that the DLL doesn't have too many symbols. See trac #5987.
SYMBOLS=`c:/Haskell/ghc/inplace/mingw/bin/objdump.exe -p compiler/stage2/build/l
ibHSghc-7.7.20131018-0-ghc7.7.20131018.dll | sed -n "1,/^.Ordinal\/Name Pointer/
 D; p; /^$/ q" | tail -n +2 | wc -l`; echo "Number of symbols in compiler/stage2
/build/libHSghc-7.7.20131018-0-ghc7.7.20131018.dll: $SYMBOLS"
Number of symbols in compiler/stage2/build/libHSghc-7.7.20131018-0-ghc7.7.201310
18.dll: 70721
case `c:/Haskell/ghc/inplace/mingw/bin/objdump.exe -p compiler/stage2/build/libH
Sghc-7.7.20131018-0-ghc7.7.20131018.dll | sed -n "1,/^.Ordinal\/Name Pointer/ D;
 p; /^$/ q" | grep "\[ *0\]" | wc -l` in 1) echo DLL compiler/stage2/build/libHS
ghc-7.7.20131018-0-ghc7.7.20131018.dll OK;; 0) echo No symbols in DLL compiler/s
tage2/build/libHSghc-7.7.20131018-0-ghc7.7.20131018.dll; exit 1;; [0-9]*) echo T
oo many symbols in DLL compiler/stage2/build/libHSghc-7.7.20131018-0-ghc7.7.2013
1018.dll; c:/Haskell/ghc/inplace/mingw/bin/objdump.exe -p compiler/stage2/build/
libHSghc-7.7.20131018-0-ghc7.7.20131018.dll | sed -n "1,/^.Ordinal\/Name Pointer
/ D; p; /^$/ q" | tail; exit 1;; *) echo bad DLL compiler/stage2/build/libHSghc-
7.7.20131018-0-ghc7.7.20131018.dll; exit 1;; esac
Too many symbols in DLL compiler/stage2/build/libHSghc-7.7.20131018-0-ghc7.7.201
31018.dll
        [5176] ghczm7zi7zi20131018_Var_zdfUniquableVar_closure
        [5177] ghczm7zi7zi20131018_Var_zdfUniquableVar_info
        [5178] ghczm7zi7zi20131018_Var_zdwsetVarUnique_closure
        [5179] ghczm7zi7zi20131018_Var_zdwsetVarUnique_info
        [5180] ghczm7zi7zi20131018_Var_zdwupdateTyVarKindM_closure
        [5181] ghczm7zi7zi20131018_Var_zdwupdateTyVarKindM_info
        [5182] ghczm7zi7zi20131018_Var_zdwzdcgmapMp_closure
        [5183] ghczm7zi7zi20131018_Var_zdwzdcgmapMp_info
        [5184] setHeapSize

make[1]: *** [compiler/stage2/build/libHSghc-7.7.20131018-0-ghc7.7.20131018.dll]
 Error 1
make[1]: *** Deleting file `compiler/stage2/build/libHSghc-7.7.20131018-0-ghc7.7
.20131018.dll'
make: *** [all] Error 2

Windows 7, MSYS, ld 2.23.2

The stage1 of the compiler seems to work as expected on non-TH code. Let me know if any other information is needed.

Last edited 3 years ago by tomberek (previous) (diff)

comment:26 Changed 3 years ago by shelarcy

  • Cc shelarcy@… added

comment:27 Changed 3 years ago by Austin Seipp <austin@…>

In 9bb909bbcf280f137c7a379d05acf5bc21268e5b/ghc:

Temporarily lower unfolding threshold on Windows

This is a very temporary, very unsatisfactory hack to fix #5987
(for now.)

The included comments essentially say it all: we lower the unfolding
threshold to minimize some amount of exported symbols from the GHC
stage2 DLL. I unfortunately had to lower it quite substantially for the
dynamic stage2 build to pass.

As of this writing, the DLL split between ghc.dll and ghc-0.dll is
something like 26,000 vs 63,000 exported symbols, respectively. So we're
still quite in danger of tripping it, but I think we will be OK at this
exact moment.

Signed-off-by: Austin Seipp <austin@well-typed.com>

comment:28 Changed 3 years ago by thoughtpolice

  • Milestone changed from 7.8.1 to 7.10.1

I've temporarily fixed this in HEAD by lowering the unfolding threshold for the stage2 build on windows. We're still extremely close to the symbol limit, but it should stay OK for 7.8.

I'm leaving this highest priority and bumping to 7.10.1 - we must find a better fix.

comment:29 Changed 2 years ago by Austin Seipp <austin@…>

In 4af1e76c701a7698ebd9b5ca3fb1394dd8b56c8d/ghc:

Add Windows to NoSharedLibsPlatformList

We're punting on full -dynamic and -dynamic-too support for Windows
right now, since it's still unstable. Also, ensure "Support dynamic-too"
in `ghc --info` is set to "NO" for Cabal.

See issues #7134, #8228, and #5987

Signed-off-by: Austin Seipp <austin@well-typed.com>

comment:30 Changed 2 years ago by carter

is there some way we could do a hack to do split DLLs and allow cycles? (would probably mean some sort of indirect jump penalty in one direction of the cycle).

One extreme would be to use some sort of graph clustering software tool like METIS http://glaros.dtc.umn.edu/gkhome/metis/metis/overview

do .hi files have enough information to sort out the directed graph of imports exports? I'd be happy to help do some graph clustering experimentation on the object files (that piece isn't windows specific, so i could hack it out on my current machine)

comment:31 Changed 2 years ago by carter

alternatively, if the free version of visual studio, visual studio express, http://www.visualstudio.com/en-us/products/visual-studio-express-vs.aspx comes with the Visual Studio linker, doesn't that have some support for dealing with cyclic deps in linking?

comment:32 Changed 2 years ago by thoughtpolice

  • Milestone changed from 7.10.1 to 7.8.2

comment:33 Changed 2 years ago by thoughtpolice

  • Milestone changed from 7.8.2 to 7.8.3

comment:34 Changed 2 years ago by thoughtpolice

  • Priority changed from highest to high

comment:35 Changed 2 years ago by thoughtpolice

  • Milestone changed from 7.8.3 to 7.8.4

Moving to 7.8.4.

comment:36 Changed 22 months ago by thoughtpolice

  • Owner changed from thoughtpolice to non-existant-user

comment:37 Changed 22 months ago by thoughtpolice

  • Owner changed from non-existant-user to thoughtpolice

comment:38 Changed 21 months ago by thoughtpolice

  • Blocking 3658 removed

comment:39 Changed 20 months ago by thomie

  • Operating System changed from Unknown/Multiple to Windows

comment:40 Changed 20 months ago by thoughtpolice

  • Milestone changed from 7.8.4 to 7.10.1

Moving (in bulk) to 7.10.4

comment:41 Changed 18 months ago by thomie

  • Blocking 7389 added

comment:42 Changed 18 months ago by thomie

  • Blocking 5355 added

See also the discussion in #5292.

Last edited 18 months ago by thomie (previous) (diff)

comment:43 Changed 16 months ago by thoughtpolice

  • Milestone changed from 7.10.1 to 7.12.1

Moving to 7.12.1

comment:44 Changed 11 months ago by Phyx-

  • Blocking 7389 removed

comment:45 Changed 9 months ago by thoughtpolice

  • Milestone changed from 7.12.1 to 8.0.1

Milestone renamed

comment:46 follow-up: Changed 6 months ago by bgamari

  • Owner thoughtpolice deleted
  • Priority changed from high to normal

Demoting priority as we have disabled dynamic linking on Windows (see comment:29).

My understanding of this situation is that it is not an intrinsic limitation of PE. The relevant data structures are the .edata and .idata tables which are defined in sections 63 and 64 of the PE/COFF specification. The specification specifies that one means of referring to exported symbols is via a 16-bit ordinal. However, this is not the only way; if the lookup fails the loader will fall back to a standard name search. Indeed, Microsoft even explicitly says that ordinals are a legacy artifact and their use is discouraged in new code (this document describes the .def format, but the idea still applies),

You can use @ordinal to specify that a number, and not the function name, will go into the DLL's export table. Many Windows DLLs export ordinals to support legacy code. It was common to use ordinals in 16-bit Windows code, because it can help minimize the size of a DLL. We don’t recommend exporting functions by ordinal unless your DLL’s clients need it for legacy support. Because the .LIB file will contain the mapping between the ordinal and the function, you can use the function name as you normally would in projects that use the DLL.

For this reason I believe the problem is in fact that ld.bfd is producing DLLs with symbols exported by (or perhaps objects that are imported by) ordinal.

comment:47 Changed 5 months ago by Ben Gamari <ben@…>

In cdeefa4/ghc:

ghc.mk: Add reference to Trac #5987

comment:48 in reply to: ↑ 46 Changed 5 months ago by rassilon

Replying to bgamari:

My understanding of this situation is that it is not an intrinsic limitation of PE. The relevant data structures are the .edata and .idata tables which are defined in sections 63 and 64 of the PE/COFF specification. The specification specifies that one means of referring to exported symbols is via a 16-bit ordinal. However, this is not the only way; if the lookup fails the loader will fall back to a standard name search. Indeed, Microsoft even explicitly says that ordinals are a legacy artifact and their use is discouraged in new code (this document describes the .def format, but the idea still applies),

Your understanding doesn't match my understanding of the PE/COFF specification. Page 75 of the 02/06/2013 version of the spec which is the page where the Export Ordinal Table of the .edata section is described.

The section includes pseudo-code explaining how the SymbolRVA of a named symbol is determined:

i = Search_ExportNameTable(ExportName);

ordinal = ExportOrdinalTable[i];

SymbolRVA = ExportAddressTable[ordinal - ordinalBase];

So, the problem isn't with this pseudo-code, but the pseudo-code combined with the Export Ordinal Table being defined as:

The export ordinal table is an array of 16-bit indexes into the address table.

This is especially sad, since the Address Table Entries field of the Export Directory Table is 32 bits of data.

The text you quote below is from the syntax of .DEF files explaining why 16-bit DLLs, didn't include exported symbol string names and utilized the ordinal number as the ONLY API between DLLs. The documentation you quote is recommending usage of string symbols for providing a cross-DLL API resolution and only using the ordinal API for legacy reasons. The documentation doesn't mean to imply that ordinals are optional (based on my reading of the PE/COFF sections about .edata).

You can use @ordinal to specify that a number, and not the function name, will go into the DLL's export table. Many Windows DLLs export ordinals to support legacy code. It was common to use ordinals in 16-bit Windows code, because it can help minimize the size of a DLL. We don’t recommend exporting functions by ordinal unless your DLL’s clients need it for legacy support. Because the .LIB file will contain the mapping between the ordinal and the function, you can use the function name as you normally would in projects that use the DLL.

comment:49 follow-up: Changed 4 months ago by thomie

rassilon: you seem to know stuff. How to fix dynamic linking on Windows? Some more problems here: #4824 #5620 #7665 #8228.

comment:50 in reply to: ↑ 49 Changed 4 months ago by rassilon

Replying to thomie:

rassilon: you seem to know stuff. How to fix dynamic linking on Windows? Some more problems here: #4824 #5620 #7665 #8228.

The "easiest" way of fixing this issue is by refactoring the GHC internals in such a way as to minimize the dependencies between the partitions of the GHC base DLL. (in short, removing all need for Haskell source level cyclic imports could help a bunch here)

Either that, or come up with an automatic partitioning scheme that could handle cyclic references between the partition DLLs.

The PE/COFF limitation of only being able to export 16bits worth of APIs from a dynamic library is the biggest problem of all of these issues. #5620 would only be worth addressing when we have a solution to this issue one way or another.

Given that the root of the 16bit exported API problem stems from the PE/COFF specification, it would seem that any alternatives we come up would be conceptually equivalent to our current usages of non-dlopen(or equivalent) dynamic loading/linking that RTS/GHCi do today. In fact the code necessary to fix this issue would probably make that logic even more convoluted and difficult to understand/maintain. (Since the code would need to understand whatever additional sections/per-DLL API functions/etc.. that would allow the code to be relocated and linked properly.)

Such a scheme would have additional downsides as well:

  • It would be (by necessity) be Windows specific.
  • It would probably completely shatter the mechanism for calling Haskell functions from C/C++ either in the same DLL or another DLL.

Of the possible above approaches, I would think an automatic partitioning approach that handled cyclic references between the partitions would be the approach that had the best long term chances of successfully being maintained.

Having said that of course, even automatic partitioning is still a non trivial amount of work. Since ideally, Cabal would know about DLL partitioning as well so that when anyone referenced a Haskell package with partitions, Cabal would help (or direct) GHC to link against all of the partition DLLs of that package.

Ugh. Maybe it'll be easier once Microsoft lets us push a pull request to Window's dynamic linker to support more than 16bits of ordinals for Windows 20. ;)

comment:51 Changed 4 months ago by thomie

  • Milestone 8.0.1 deleted

comment:52 Changed 4 months ago by simonmar

I propose:

  • Do not dynamically-link GHC on Windows
  • Do not build a dynamic version of the GHC package
  • Continue to build dynamic versions of everything else
  • Use Remote GHCi (-fexternal-interpreter) to get dynamic linking in GHCi
  • Close this ticket :)

comment:53 Changed 4 months ago by rassilon

I'm not sure I completely understand your GHCi suggestion (but that is unnecessary) , but I certainly agree with the rest of your suggestions. If someone cares enough, they can fund the necessary engineering. :)

Note: See TracTickets for help on using tickets.