Changes between Version 3 and Version 4 of Backpack


Ignore:
Timestamp:
Jun 7, 2014 3:07:18 AM (15 months ago)
Author:
ezyang
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • Backpack

    v3 v4  
    174174
    175175At this point in time, the design for the extended linking mechanism
    176 is not well specified, however, it might include the following:
    177 
    178 - Full Backpack packages can explicitly include an upstream Cabal
    179   package with holes and explicitly fill in dependencies. This
    180   is "full manual".
     176is not well specified.  In Backpack cookbook below, we describe a number
     177of common patterns where a Backpack package can be used to manually
     178link a package in a special way.
    181179
    182180=== The next-generation of Cabal-Backpack packages ===
    183181
    184 To take full advantage of Backpack features,
     182In this model, users write their code specifically to use the Backpack
     183module system; however, they continue to distribute their code on Hackage
     184and may be interested in concurrently supporting users who are not
     185interested in using Backpack.
     186
     187The purpose of this section is to describe what we imagine the best
     188practices for Backpack package construction to be.
     189
     190(TODO: write section)
     191
     192== Backpack cookbook ==
     193
     194Here are a few common tasks which show up when managing package
     195dependencies, and how to resolve them using Backpack.  It might be a
     196good idea to support some of these directly with nice surface syntax.
     197Many of these recipes are based with one of the most annoying
     198situations when using Cabal: your package fails to build because
     199of some sort of dependency problem, and you don't really want to have
     200to go and patch upstream to fix the problem.
     201
     202Note that if you are depending against tight signatures, instead of
     203version ranges, we expect to obviate many of these measures.
     204
     205=== Hiding non-exported dependencies ===
     206
     207In this situation, your application is using two separate libraries
     208which have identically named holes for two different versions of the
     209same upstream library.  However, one of these libraries uses the
     210upstream library in a strictly non-exported way: informally, use of this
     211dependency is strictly an implementation detail.
     212
     213In Backpack, we can arrange for a non-exported dependency by linking a
     214hole with the appropriate implementation and then thinning the
     215resulting module definition out.  Here is a worked example:
     216
     217{{{
     218-- Upstream library with multiple, backwards-incompatible versions
     219package arrays-1.x-sig where
     220    Array :: ...
     221package arrays-1.0 where
     222    Array = ...
     223
     224package arrays-2.x-sig where
     225    Array :: ...
     226package arrays-2.0 where
     227    Array = ...
     228
     229-- Upstream (Cabal) package which is only compatible with arrays-1.x
     230package graph where
     231    include arrays-1.x-sig -- In Cabal, this signature is calculated automatically
     232    Graph = ...
     233
     234-- (**) Package which does not export Array at all
     235package graph-noexport-array (Graph) where
     236    include arrays-1.0
     237    include graph
     238
     239-- Application
     240package application where
     241    include graph-noexport-array
     242    include arrays-2.0
     243    Main = [
     244        import Array -- uses arrays-2.0 implementation
     245    ]
     246}}}
     247
     248Intuitively, the reason this works is that only one Array implementation
     249is visible from application, so no linking (which would fail) between
     250the array-1.0 and array-2.0 occurs.  It would also work to rename Array
     251to another logical name, which would also prevent the linking process.
     252
     253{{{
     254-- (**) Package which exports array as a fresh name
     255package graph-renamed-array (Graph) where
     256    include arrays-1.0
     257    include graph (Array as PrivateArray)
     258}}}
     259
     260ezyang: Close study of this example reveals an additions to the surface
     261language which would be quite useful.  It's easy to imagine the export
     262list for graph-noexport-array becoming unwieldy when graph defines a lot
     263of modules.  While it is reasonable for graph to explicitly provide a
     264module list (current Cabal best-practice), it's less reasonable if we
     265have to repeat the list later.  Two obvious ways of dealing with this
     266are to allow signature-level exports/hiding.  Here, we simply collect up
     267all of the modules mentioned in a signature and export/hide those.
     268Hiding may be more convenient, since you have to include not only the
     269signature of the API that was implemented, but any other types which
     270live in other signatures which are exported.
     271
     272=== Injecting a backwards compatibility shim ===
     273
     274
    185275
    186276------ >8 -------
    187277
    188 More importantly, Backpack offers a tantalizing story for managing different versions of packages, alluded to in the paper, but not elaborated on. In Cabal, the version number range of a build-depends implicitly defines a "signature" which we depend on. There are a few ways this signature could be computed:
    189 
    190 *   We could throw our hands up in the air and say the signature is just whatever Cabal computed. This does not lead to very reproducible builds, but it is the current status quo.
    191 
    192 *   We could compute the “greatest common signature” for the specified version range. This signature is the widest signature for which all of the versions are compatible with. This can be used to determine if there is a buggy version range; if the greatest common signature isn’t enough to compile the package, there must exist some version that is listed as compatible, but isn’t actually. For unbounded version ranges, this can be a bit dodgy, but the PVP suggests that if you’re not unbounded over too many version numbers, you’ll be OK
    193 
    194 *   We could further refine the greatest common signature, by finding the thinnest signature with which our package type checks with. This is the “ground truth” with regards to what our package relies on, although maintaining a signature like this could be pretty annoying, on the order of annoyance of having to maintain explicit import lists (and type signatures) for everything that you use. If this is automatically calculated, we could do away with version dependencies and just see if signatures are satisfied. This could have performance problems, and calculating this requires a bit of work.
    195 
    196 By the way, this means that naively implementing the suggestion in the Backpack paper where modules provide signature files is quite dodgy, because the signature file is likely to contain too much “stuff”, and in any case, needs to be referred to in a way that is stable across versions. On the other hand, one could easily manage partitioning signatures into “latest and greatest” versus “well, this is pretty stable for most people”; differently, one could pin to a signature for a version, and as long as the package doesn’t break backwards compatibility that signature will work for a while.
    197 
    198 === Interlude: Cabal-specific features ===
     278== Interlude: Cabal-specific features ==
    199279
    200280Cabal is not just a packaging mechanism, but it also handles a number of
     
    224304Scott thinks there is a way to automatically infer the necessary hs-boot signatures for recursive module bindings. His proposal is here: http://www.reddit.com/r/haskell/comments/1id0p7/backpack_retrofitting_haskell_with_interfaces/cb42m8l
    225305
    226 
    227306----
    228307