Changes between Version 6 and Version 7 of Plugins/ReinitializeGlobals


Ignore:
Timestamp:
Jul 5, 2013 4:08:38 PM (23 months ago)
Author:
nfrisby
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • Plugins/ReinitializeGlobals

    v6 v7  
    33== Background == 
    44 
    5 === !CoreMonad.reinitializeGlobals === 
     5=== `CoreMonad.reinitializeGlobals` === 
    66 
    7 For unfortunate reasons I don't fully understand (#5355, #5292), the host compiler and plugins have distinct copies of global variables (unless the host compiler dynamically loads libHSghc, eg on Windows; see the next section). The current workaround is `CoreMonad.reinitializeGlobals`, which every plugin is supposed to call at the beginning of its `install` routine.  This function overwrites the plugin's global variables with the corresponding values of the host compiler's. It requires a little plumbing to make this work but not much, and the plugin author sees only `reinitializeGlobals`. 
     7For unfortunate reasons I don't fully understand (#5355, #5292), the host compiler and the set of all plugins each have distinct copies of global variables (unless the host compiler dynamically loads libHSghc, eg on Windows; see the next section).  All plugins share the same copy, so there are at most two copies. The current workaround is `CoreMonad.reinitializeGlobals`, which every plugin is supposed to call at the beginning of its `install` routine.  This function overwrites the plugin's global variables with the corresponding values of the host compiler's. It requires a little plumbing to make this work but not much, and the plugin author sees only `reinitializeGlobals`. 
    88 
    99The long-term plan is to eventually totally avoid having two separate images of the ghc library and then redefine `reinitializeGlobals = return ()`. 
     
    1818}}} 
    1919 
    20 This mechanism is currently used for some !StaticFlags, some Linker state, and some !DynFlags. I just recently added (partial) support for the !FastString table. 
     20This mechanism is currently used for some `StaticFlag`s, some `Linker` state, and some `DynFlags`. I just recently added (partial) support for the `FastString` table. 
    2121 
    2222=== `DYNAMIC_GHC_PROGRAMS` === 
     
    4848=== `FastString.string_table` === 
    4949 
    50 All the !FastStrings created during compilation are memoized in a hash table. For speedy comparison, each string is associated with a unique, which is allocated linearly whenever a !FastString is created that has no corresponding entry in the hash table. This involves two pieces of global state, which are held in the same global variable. 
     50All the `FastString`s created during compilation are memoized in a hash table. For speedy comparison, each string is associated with a unique, which is allocated linearly whenever a `FastString` is created that has no corresponding entry in the hash table. This involves two pieces of global state, which are held in the same global variable. 
    5151 
    5252{{{ 
     
    6262== The Problem with `FastString.string_table` == 
    6363 
    64 During its use, the !FastString table increments the `!Int` argument. `reinitializeGlobals` alone is incapable of supporting this appropriately; it was designed to only copy global variables' values from the host compiler to the plugin, never in the opposite direction. 
     64During its use, the `FastString` table increments the `!Int` argument. `reinitializeGlobals` alone is incapable of supporting this appropriately; it was designed to only copy global variables' values from the host compiler to the plugin, never in the opposite direction. Those original global variables were global for convenience of access, not for the need to be mutable. The `FastString` table breaks the mold. 
    6565 
    6666=== Option 1: The General Solution === 
     
    8080 1) call `reinitializeGlobals` ASAP, just like now 
    8181 
    82  2) call `reverseReinitializeGlobals` at the end of the `install` routine; this new function would stash the values of the plugin's globals in a new, otherwise unused Writer output of !CoreM. 
     82 2) call `reverseReinitializeGlobals` at the end of the `install` routine; this new function would stash the values of the plugin's globals in a new, otherwise unused Writer output of `CoreM`. 
    8383 
    84  3) similarly wrap every !PluginPass contained in the result of `install` 
     84 3) similarly wrap every `PluginPass` contained in the result of `install` 
    8585 
    8686We'd also have to add some logic around the host compiler's calls to the plugin in order to copy the stashed values back into the host compiler's globals. 
     
    102102Legitimate hits: 
    103103 
    104   * these three use the GLOBAL_VAR macro and were already supported by reinitializeGlobals: !StaticFlags, !DynFlags, Linker 
     104  * these three use the GLOBAL_VAR macro and were already supported by reinitializeGlobals: `StaticFlags`, `DynFlags`, `Linker` 
    105105 
    106   * my focus: !FastString.string_table 
     106  * my focus: `FastString.string_table` 
    107107 
    108   * I don't know what these are for: Panic.interruptTargetThread, !InteractiveEval.noBreakStablePtr 
     108  * I don't know what these are for: `Panic.interruptTargetThread`, `InteractiveEval.noBreakStablePtr` 
    109109 
    110 Of all these global variables, I think only string_table needs information flow from the plugin back to the compiler. 
     110Of all these global variables, I think only `string_table` needs information flow from the plugin back to the compiler. 
    111111 
    112112=== Option 2: The Lighterweight Workaround === 
     
    116116This wouldn't affect the plugin API, but it would still require the extra logic around the compiler's calls to plugins. 
    117117 
    118 Unfortunately, the !FastString interface uses unsafePerformIO, so unless the plugin forces all of its !FastStrings before returning to the compiler, this wouldn't fix the issue. (Option 1 has this same problem.) 
     118Unfortunately, since the `FastString` interface uses unsafePerformIO, the two images' `FastString` tables may get out of synch when the evaluation of one of those thunks mutates one of the tables but not the other. Option 1 has this same problem. I'm going to call any thunk that allocates a `FastString` when its forced a "problem thunk". Since we only synchronize the two images' tables when transitioning between the compiler and a plugin, evaluation of a problem thunk by the image that did not create it is problematic. 
    119119 
    120 So the rule would be: ''if your plugin might allocate new !FastStrings, be sure to force them before returning to the compiler''. (Same for Option 1.) 
     120So the rule would be: ''Do not let the compiler force any of a plugin's thunks that allocate `FastString`s, and vice versa.'' (Same for Option 1.) 
    121121 
    122122=== Option 3: The Full Low-Level Hack === 
     
    128128=== Option 4: Do Nothing === 
    129129 
    130 For `FastString.string_table`, the lack of reverse information flow probably doesn't matter. I suspect the most common case just involves looking up !FastStrings in the table, not actually creating new ones. 
     130For `FastString.string_table`, the lack of reverse information flow probably doesn't matter. I suspect the most common case just involves looking up `FastString`s in the table, not actually creating new ones. 
    131131 
    132 Even if the plugin does create !FastStrings, it looks like the only thing that is downstream from the core2core pipeline and also is (indirectly) sensitive to a !FastString's unique is GHCI. Everything else downstream of the plugins just uses the unique associated with Names — !OccStrings are pretty much ignored after the renamer. 
     132Even if the plugin does create `FastString`s, it looks like the only thing that is downstream from the core2core pipeline and also is (indirectly) sensitive to a `FastString`'s unique is GHCI. Everything else downstream of the plugins just uses the unique associated with Names — `OccString`s are pretty much ignored after the renamer. 
    133133 
    134 So the rule would be: ''if your plugin might allocate new !FastStrings, be warned that GHCI might not work quite right''. 
     134So the rule would be: ''if your plugin might allocate new `FastString`s, be warned that GHCI might not work quite right''. 
    135135 
    136136=== Option 5: Circular `IORef`s ===