Changes between Version 3 and Version 4 of Backpack


Ignore:
Timestamp:
Jun 7, 2014 3:07:18 AM (9 months ago)
Author:
ezyang
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • Backpack

    v3 v4  
    174174 
    175175At this point in time, the design for the extended linking mechanism 
    176 is not well specified, however, it might include the following: 
    177  
    178 - Full Backpack packages can explicitly include an upstream Cabal 
    179   package with holes and explicitly fill in dependencies. This 
    180   is "full manual". 
     176is not well specified.  In Backpack cookbook below, we describe a number 
     177of common patterns where a Backpack package can be used to manually 
     178link a package in a special way. 
    181179 
    182180=== The next-generation of Cabal-Backpack packages === 
    183181 
    184 To take full advantage of Backpack features, 
     182In this model, users write their code specifically to use the Backpack 
     183module system; however, they continue to distribute their code on Hackage 
     184and may be interested in concurrently supporting users who are not 
     185interested in using Backpack. 
     186 
     187The purpose of this section is to describe what we imagine the best 
     188practices for Backpack package construction to be. 
     189 
     190(TODO: write section) 
     191 
     192== Backpack cookbook == 
     193 
     194Here are a few common tasks which show up when managing package 
     195dependencies, and how to resolve them using Backpack.  It might be a 
     196good idea to support some of these directly with nice surface syntax. 
     197Many of these recipes are based with one of the most annoying 
     198situations when using Cabal: your package fails to build because 
     199of some sort of dependency problem, and you don't really want to have 
     200to go and patch upstream to fix the problem. 
     201 
     202Note that if you are depending against tight signatures, instead of 
     203version ranges, we expect to obviate many of these measures. 
     204 
     205=== Hiding non-exported dependencies === 
     206 
     207In this situation, your application is using two separate libraries 
     208which have identically named holes for two different versions of the 
     209same upstream library.  However, one of these libraries uses the 
     210upstream library in a strictly non-exported way: informally, use of this 
     211dependency is strictly an implementation detail. 
     212 
     213In Backpack, we can arrange for a non-exported dependency by linking a 
     214hole with the appropriate implementation and then thinning the 
     215resulting module definition out.  Here is a worked example: 
     216 
     217{{{ 
     218-- Upstream library with multiple, backwards-incompatible versions 
     219package arrays-1.x-sig where 
     220    Array :: ... 
     221package arrays-1.0 where 
     222    Array = ... 
     223 
     224package arrays-2.x-sig where 
     225    Array :: ... 
     226package arrays-2.0 where 
     227    Array = ... 
     228 
     229-- Upstream (Cabal) package which is only compatible with arrays-1.x 
     230package graph where 
     231    include arrays-1.x-sig -- In Cabal, this signature is calculated automatically 
     232    Graph = ... 
     233 
     234-- (**) Package which does not export Array at all 
     235package graph-noexport-array (Graph) where 
     236    include arrays-1.0 
     237    include graph 
     238 
     239-- Application 
     240package application where 
     241    include graph-noexport-array 
     242    include arrays-2.0 
     243    Main = [ 
     244        import Array -- uses arrays-2.0 implementation 
     245    ] 
     246}}} 
     247 
     248Intuitively, the reason this works is that only one Array implementation 
     249is visible from application, so no linking (which would fail) between 
     250the array-1.0 and array-2.0 occurs.  It would also work to rename Array 
     251to another logical name, which would also prevent the linking process. 
     252 
     253{{{ 
     254-- (**) Package which exports array as a fresh name 
     255package graph-renamed-array (Graph) where 
     256    include arrays-1.0 
     257    include graph (Array as PrivateArray) 
     258}}} 
     259 
     260ezyang: Close study of this example reveals an additions to the surface 
     261language which would be quite useful.  It's easy to imagine the export 
     262list for graph-noexport-array becoming unwieldy when graph defines a lot 
     263of modules.  While it is reasonable for graph to explicitly provide a 
     264module list (current Cabal best-practice), it's less reasonable if we 
     265have to repeat the list later.  Two obvious ways of dealing with this 
     266are to allow signature-level exports/hiding.  Here, we simply collect up 
     267all of the modules mentioned in a signature and export/hide those. 
     268Hiding may be more convenient, since you have to include not only the 
     269signature of the API that was implemented, but any other types which 
     270live in other signatures which are exported. 
     271 
     272=== Injecting a backwards compatibility shim === 
     273 
     274 
    185275 
    186276------ >8 ------- 
    187277 
    188 More importantly, Backpack offers a tantalizing story for managing different versions of packages, alluded to in the paper, but not elaborated on. In Cabal, the version number range of a build-depends implicitly defines a "signature" which we depend on. There are a few ways this signature could be computed: 
    189  
    190 *   We could throw our hands up in the air and say the signature is just whatever Cabal computed. This does not lead to very reproducible builds, but it is the current status quo. 
    191  
    192 *   We could compute the “greatest common signature” for the specified version range. This signature is the widest signature for which all of the versions are compatible with. This can be used to determine if there is a buggy version range; if the greatest common signature isn’t enough to compile the package, there must exist some version that is listed as compatible, but isn’t actually. For unbounded version ranges, this can be a bit dodgy, but the PVP suggests that if you’re not unbounded over too many version numbers, you’ll be OK 
    193  
    194 *   We could further refine the greatest common signature, by finding the thinnest signature with which our package type checks with. This is the “ground truth” with regards to what our package relies on, although maintaining a signature like this could be pretty annoying, on the order of annoyance of having to maintain explicit import lists (and type signatures) for everything that you use. If this is automatically calculated, we could do away with version dependencies and just see if signatures are satisfied. This could have performance problems, and calculating this requires a bit of work. 
    195  
    196 By the way, this means that naively implementing the suggestion in the Backpack paper where modules provide signature files is quite dodgy, because the signature file is likely to contain too much “stuff”, and in any case, needs to be referred to in a way that is stable across versions. On the other hand, one could easily manage partitioning signatures into “latest and greatest” versus “well, this is pretty stable for most people”; differently, one could pin to a signature for a version, and as long as the package doesn’t break backwards compatibility that signature will work for a while. 
    197  
    198 === Interlude: Cabal-specific features === 
     278== Interlude: Cabal-specific features == 
    199279 
    200280Cabal is not just a packaging mechanism, but it also handles a number of 
     
    224304Scott thinks there is a way to automatically infer the necessary hs-boot signatures for recursive module bindings. His proposal is here: http://www.reddit.com/r/haskell/comments/1id0p7/backpack_retrofitting_haskell_with_interfaces/cb42m8l 
    225305 
    226  
    227306---- 
    228307