Changes between Version 3 and Version 4 of Plugins/Annotations


Ignore:
Timestamp:
Jun 30, 2008 8:25:48 AM (6 years ago)
Author:
guest
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • Plugins/Annotations

    v3 v4  
    1414 * Plugin-specific data structures 
    1515 
    16 == Possible Solution 1 == 
    17  
    18 {{{ 
    19  
    20 module Main where 
    21  
    22 {-# PLUGIN NVidia.GHC.GPU NVidiaGPUSettings { useTextureMemoryMb = 256 } #-} 
    23  
    24 {-# PLUGIN NVidia.GHC.GPU doSomethingExpensive NVidiaGPUFunctionSettings { maxStackDepth = 1024 } #-} 
    25 doSomethingExpensive :: Int -> ImpressiveResult 
    26 doSomethingExpensive = .... 
    27  
    28 }}} 
    29  
    30 In this module the user has specified one module-level annotation (NVidiaGPUSettings) and one binder-level annotation (NVidiaGPUFunctionSettings). These will be stored in a Map which is given to the plugin code so it can lookup the appropriate annotations. 
    31  
    32 A problem with this is that identifiers are somewhat unstable in Core. All non-top-level binders are destroyed and created willy-nilly by Core passes. Top-level binders are only stable if they are exported from the module. Even top level binders may vanish from an expression if e.g. we perform inlining. 
    33  
    34 This can be "solved" by marking all identifiers occuring in PLUGIN pragmas as stable (i.e. exported), and restricting such pragmas to top level identifiers only. 
    35  
    36 Another possible solution is to turn the pragmas into Notes that live on the source tree itself. However this may suffer from Notes being shuffled around by some Core passes. Additionally, this may be slightly harder to use for plugin authors. 
    37  
    38 == Possible Solution 2 == 
    39  
    40 The previous solution renamed NVidiaGPUSettings etc in an environment that only contained the imported NVidia.GHC.GPU module. This has the advantages that: 
    41  * A staging issue is solved as these pragmas are then unable to refer to identifiers in the module being compiled 
    42  * The module involved can still be compiled if the NVidia.GHC.GPU module is not available 
    43  
    44 However: 
    45  * This style of "implicit import statement" is inconsistent with Template Haskell which requires an explicit import for code that is run at compile time, even if that import is never used at runtime 
    46  * The required pragma is bloated by the fully qualified module name 
    47  * We cannot include references to the names of functions in the pragma, e.g using the Template Haskell quoting mechanism: 
    48 {{{ 
    49  
    50 module ServerApplication where 
    51  
    52 clientFunction :: (Request, Response) -> IO () 
    53 clientFunction = ... 
    54  
    55 {-# PLUGIN HVolta.TierSplitting serverFunction HV { correspondingClientFunction = 'clientFunction } #-} 
    56 serverFunction :: Request -> IO Response 
    57 serverFunction = ... 
    58  
    59 }}} 
    60  
    61 An alternative that makes the imports consistent (and makes the pragma shorter) is: 
    62  
    63 {{{ 
    64  
    65 module Main where 
    66  
    67 import NVidia.GHC.GPU 
    68  
    69 {-# PLUGIN NVidiaGPUSettings { useTextureMemoryMb = 256 } #-} 
    70  
    71 {-# PLUGIN doSomethingExpensive NVidiaGPUFunctionSettings { maxStackDepth = 1024 } #-} 
    72 doSomethingExpensive :: Int -> ImpressiveResult 
    73 doSomethingExpensive = .... 
    74  
    75 }}} 
    76  
    77  
    78 == Possible Solution 3 == 
    79  
    80 This solution has all annotations being desugared to Notes that live on the Core syntax tree. The reasons for this are as follows: 
    81  * It means annotations are basically stable at all points in the compile pipeline: you won't miss one if for example some inlining happens. This is nice and predictable for plugin authors. This is especially troublesome because some things are inlined very eagerly indeed (e.g. non-exported IDs that are referenced only once). 
    82  * It means that potentially something like SCCs could be implemented outside of GHC itself, which is very useful for plugins that perform analytics, which I suspect will be an important use case. 
    83  * Allows us to attach information to non top-level binders, which I think is essentially impossible if we require a name in the annotation (making non top-level binders stable is HARD). 
    84  
    85 Disadavantages are that: 
    86  * It becomes harder to annotate types and data constructors: a similar effect can be achieved by supplying a module level annotation that references the appropriate object by quoting its name. 
    87  * Establishing relationships between objects in an annotation may become less principled if the annotation is forcibly attached to one of the objects. 
    88  * Lack of a simple Id -> Annotation lookup facility may make plugin development harder. 
    89  * Lack of a simple Id -> Annotation correspondence essentially rules out accessing these things externally in future. 
    90  
    91 The syntax could be used in three ways. Like a traditional annotation: 
    92 {{{ 
    93  
    94 data Stuff deriving Typeable 
    95  
    96 {-# ANN foo Stuff #-} 
    97 foo x = ... 
    98  
    99 }}} 
    100  
    101 The identifier does not necessarily have to be top level as this usage is immediately transformed as follows to ensure stability: 
    102  
    103 {{{ 
    104  
    105 data Stuff deriving Typeable 
    106  
    107 foo = {-# ANN Stuff #-} \x -> ... 
    108  
    109 }}} 
    110  
    111 And this usage may also be typed directly by the user in the expression language as with SCC annotations currently. This is the "true form" of most of the annotations we will use. The only other possibility is module level annotations: 
    112  
    113 {{{ 
    114 {-# ANN Stuff #-} 
    115  
    116 module Main where 
    117  
    118 ... 
    119  
    120 }}} 
    121  
    122 All of these forms allow arbitrary expressions in "Stuff", which are renamed and typechecked almost as if they had been written inside a splice $() in Template Haskell but instead of having the thing inside the splice return Q Exp we require it returns a Typeable thing. This code will be executed during desugaring of the whole program to generate the actual value we will attach to the Core syntax tree in a note. 
    123  
    124 This treatment has a few nice effects: 
    125  * References to names of non top-level things are disallowed by Template Haskell semantics 
    126  * We get quoting of variable, constructor, type and phase names for free and in a uniform manner 
    127  * We get free checking that the Stuff does not try to execute something defined in the module being compiled 
    128  
    129 An interesting alternative might be to have a notion of an "annotation type" similar to the class name used by C# / Java, and let those types e.g. implicitly add NOINLINE semantics to what they are attached to. This should help stability of the names used in annotations at the cost of reducing optimization opportunities. 
    13016 
    13117== Other Considerations == 
     
    15642 
    15743}}} 
     44 
     45 
     46== Possible Solution 1 == 
     47 
     48{{{ 
     49 
     50module Main where 
     51 
     52{-# PLUGIN NVidia.GHC.GPU NVidiaGPUSettings { useTextureMemoryMb = 256 } #-} 
     53 
     54{-# PLUGIN NVidia.GHC.GPU doSomethingExpensive NVidiaGPUFunctionSettings { maxStackDepth = 1024 } #-} 
     55doSomethingExpensive :: Int -> ImpressiveResult 
     56doSomethingExpensive = .... 
     57 
     58}}} 
     59 
     60In this module the user has specified one module-level annotation (NVidiaGPUSettings) and one binder-level annotation (NVidiaGPUFunctionSettings). These will be stored in a Map which is given to the plugin code so it can lookup the appropriate annotations. 
     61 
     62A problem with this is that identifiers are somewhat unstable in Core. All non-top-level binders are destroyed and created willy-nilly by Core passes. Top-level binders are only stable if they are exported from the module. Even top level binders may vanish from an expression if e.g. we perform inlining. 
     63 
     64This can be "solved" by marking all identifiers occuring in PLUGIN pragmas as stable (i.e. exported), and restricting such pragmas to top level identifiers only. 
     65 
     66Another possible solution is to turn the pragmas into Notes that live on the source tree itself. However this may suffer from Notes being shuffled around by some Core passes. Additionally, this may be slightly harder to use for plugin authors. 
     67 
     68 
     69== Possible Solution 2 == 
     70 
     71The previous solution renamed NVidiaGPUSettings etc in an environment that only contained the imported NVidia.GHC.GPU module. This has the advantages that: 
     72 * A staging issue is solved as these pragmas are then unable to refer to identifiers in the module being compiled 
     73 * The module involved can still be compiled if the NVidia.GHC.GPU module is not available 
     74 
     75However: 
     76 * This style of "implicit import statement" is inconsistent with Template Haskell which requires an explicit import for code that is run at compile time, even if that import is never used at runtime 
     77 * The required pragma is bloated by the fully qualified module name 
     78 * We cannot include references to the names of functions in the pragma, e.g using the Template Haskell quoting mechanism: 
     79{{{ 
     80 
     81module ServerApplication where 
     82 
     83clientFunction :: (Request, Response) -> IO () 
     84clientFunction = ... 
     85 
     86{-# PLUGIN HVolta.TierSplitting serverFunction HV { correspondingClientFunction = 'clientFunction } #-} 
     87serverFunction :: Request -> IO Response 
     88serverFunction = ... 
     89 
     90}}} 
     91 
     92An alternative that makes the imports consistent (and makes the pragma shorter) is: 
     93 
     94{{{ 
     95 
     96module Main where 
     97 
     98import NVidia.GHC.GPU 
     99 
     100{-# PLUGIN NVidiaGPUSettings { useTextureMemoryMb = 256 } #-} 
     101 
     102{-# PLUGIN doSomethingExpensive NVidiaGPUFunctionSettings { maxStackDepth = 1024 } #-} 
     103doSomethingExpensive :: Int -> ImpressiveResult 
     104doSomethingExpensive = .... 
     105 
     106}}} 
     107 
     108 
     109== Possible Solution 3 == 
     110 
     111This solution has all annotations being desugared to Notes that live on the Core syntax tree. The reasons for this are as follows: 
     112 * It means annotations are basically stable at all points in the compile pipeline: you won't miss one if for example some inlining happens. This is nice and predictable for plugin authors. This is especially troublesome because some things are inlined very eagerly indeed (e.g. non-exported IDs that are referenced only once). 
     113 * It means that potentially something like SCCs could be implemented outside of GHC itself, which is very useful for plugins that perform analytics, which I suspect will be an important use case. 
     114 * Allows us to attach information to non top-level binders, which I think is essentially impossible if we require a name in the annotation (making non top-level binders stable is HARD). 
     115 
     116Disadavantages are that: 
     117 * It becomes harder to annotate types and data constructors: a similar effect can be achieved by supplying a module level annotation that references the appropriate object by quoting its name. 
     118 * Establishing relationships between objects in an annotation may become less principled if the annotation is forcibly attached to one of the objects. 
     119 * Lack of a simple Id -> Annotation lookup facility may make plugin development harder. 
     120 * Lack of a simple Id -> Annotation correspondence essentially rules out accessing these things externally in future. 
     121 
     122The syntax could be used in three ways. Like a traditional annotation: 
     123{{{ 
     124 
     125data Stuff deriving Typeable 
     126 
     127{-# ANN foo Stuff #-} 
     128foo x = ... 
     129 
     130}}} 
     131 
     132The identifier does not necessarily have to be top level as this usage is immediately transformed as follows to ensure stability: 
     133 
     134{{{ 
     135 
     136data Stuff deriving Typeable 
     137 
     138foo = {-# ANN Stuff #-} \x -> ... 
     139 
     140}}} 
     141 
     142And this usage may also be typed directly by the user in the expression language as with SCC annotations currently. This is the "true form" of most of the annotations we will use. The only other possibility is module level annotations: 
     143 
     144{{{ 
     145{-# ANN Stuff #-} 
     146 
     147module Main where 
     148 
     149... 
     150 
     151}}} 
     152 
     153All of these forms allow arbitrary expressions in "Stuff", which are renamed and typechecked almost as if they had been written inside a splice $() in Template Haskell but instead of having the thing inside the splice return Q Exp we require it returns a Typeable thing. This code will be executed during desugaring of the whole program to generate the actual value we will attach to the Core syntax tree in a note. 
     154 
     155This treatment has a few nice effects: 
     156 * References to names of non top-level things are disallowed by Template Haskell semantics 
     157 * We get quoting of variable, constructor, type and phase names for free and in a uniform manner 
     158 * We get free checking that the Stuff does not try to execute something defined in the module being compiled 
     159 
     160An interesting alternative might be to have a notion of an "annotation type" similar to the class name used by C# / Java, and let those types e.g. implicitly add NOINLINE semantics to what they are attached to. This should help stability of the names used in annotations at the cost of reducing optimization opportunities. 
     161 
     162 
     163== Possible Solution 4 == 
     164 
     165This is the one I'm actually going to try and implement. Annotations will look like this: 
     166 
     167{{{ 
     168{-# ANN f 1 #-} 
     169f = ... 
     170 
     171{-# ANN g Just ("foo", 1337) #-} 
     172g = ... 
     173}}} 
     174 
     175I.e. you annotate an actual identifier with an expression. We impose the further constraint that that expression has the form of an actual literal (including data constructors). In particular, general application is disallowed, so this is not kosher: 
     176 
     177{{{ 
     178{-# ANN f id 1 #-} 
     179}}} 
     180 
     181You can introduce compile time compilation by using Template Haskell as usual: 
     182 
     183{{{ 
     184{-# ANN f $(id [| 1 |]) #-} 
     185}}} 
     186 
     187We will need another notation to be able to refer to identifiers other than data constructor and function names, something like: 
     188 
     189{{{ 
     190{-# ANN_TYPE Foo Just 2 #-} 
     191data Foo = Foo 
     192}}} 
     193 
     194You may refer to the names of things in the modules being compiled, but not their implementations: 
     195{{{ 
     196data Foo = Foo ... 
     197f = ... 
     198g = ... 
     199 
     200-- OK: 
     201{-# ANN f Just ('g, ''Foo) #-} 
     202 
     203-- Not OK: 
     204{-# ANN f Just (f, Foo) #-} 
     205}}} 
     206 
     207Side note: I believe it would make sense to allow annotations to use the implementations of values in the module being compiled,: after all, I believe they can use the implementations of values in imported modules. This would filling out the relevant field with an error during compilation (for the benefit of plugins) and linking the annotation fields up to the required values after compilation. 
     208 
     209Notice that this notation does not allow annotating non-top-level names or expressions, and due to its use of an Id -> Annotation mapping may not survive inlining. These are annoying limitations but it does fit with our view that the annotations should be exportable and accessible by an Id -> Annotation mapping by other modules.