Changes between Version 3 and Version 4 of Backpack

Jun 7, 2014 3:07:18 AM (18 months ago)



  • Backpack

    v3 v4  
    175175At this point in time, the design for the extended linking mechanism
    176 is not well specified, however, it might include the following:
    178 - Full Backpack packages can explicitly include an upstream Cabal
    179   package with holes and explicitly fill in dependencies. This
    180   is "full manual".
     176is not well specified.  In Backpack cookbook below, we describe a number
     177of common patterns where a Backpack package can be used to manually
     178link a package in a special way.
    182180=== The next-generation of Cabal-Backpack packages ===
    184 To take full advantage of Backpack features,
     182In this model, users write their code specifically to use the Backpack
     183module system; however, they continue to distribute their code on Hackage
     184and may be interested in concurrently supporting users who are not
     185interested in using Backpack.
     187The purpose of this section is to describe what we imagine the best
     188practices for Backpack package construction to be.
     190(TODO: write section)
     192== Backpack cookbook ==
     194Here are a few common tasks which show up when managing package
     195dependencies, and how to resolve them using Backpack.  It might be a
     196good idea to support some of these directly with nice surface syntax.
     197Many of these recipes are based with one of the most annoying
     198situations when using Cabal: your package fails to build because
     199of some sort of dependency problem, and you don't really want to have
     200to go and patch upstream to fix the problem.
     202Note that if you are depending against tight signatures, instead of
     203version ranges, we expect to obviate many of these measures.
     205=== Hiding non-exported dependencies ===
     207In this situation, your application is using two separate libraries
     208which have identically named holes for two different versions of the
     209same upstream library.  However, one of these libraries uses the
     210upstream library in a strictly non-exported way: informally, use of this
     211dependency is strictly an implementation detail.
     213In Backpack, we can arrange for a non-exported dependency by linking a
     214hole with the appropriate implementation and then thinning the
     215resulting module definition out.  Here is a worked example:
     218-- Upstream library with multiple, backwards-incompatible versions
     219package arrays-1.x-sig where
     220    Array :: ...
     221package arrays-1.0 where
     222    Array = ...
     224package arrays-2.x-sig where
     225    Array :: ...
     226package arrays-2.0 where
     227    Array = ...
     229-- Upstream (Cabal) package which is only compatible with arrays-1.x
     230package graph where
     231    include arrays-1.x-sig -- In Cabal, this signature is calculated automatically
     232    Graph = ...
     234-- (**) Package which does not export Array at all
     235package graph-noexport-array (Graph) where
     236    include arrays-1.0
     237    include graph
     239-- Application
     240package application where
     241    include graph-noexport-array
     242    include arrays-2.0
     243    Main = [
     244        import Array -- uses arrays-2.0 implementation
     245    ]
     248Intuitively, the reason this works is that only one Array implementation
     249is visible from application, so no linking (which would fail) between
     250the array-1.0 and array-2.0 occurs.  It would also work to rename Array
     251to another logical name, which would also prevent the linking process.
     254-- (**) Package which exports array as a fresh name
     255package graph-renamed-array (Graph) where
     256    include arrays-1.0
     257    include graph (Array as PrivateArray)
     260ezyang: Close study of this example reveals an additions to the surface
     261language which would be quite useful.  It's easy to imagine the export
     262list for graph-noexport-array becoming unwieldy when graph defines a lot
     263of modules.  While it is reasonable for graph to explicitly provide a
     264module list (current Cabal best-practice), it's less reasonable if we
     265have to repeat the list later.  Two obvious ways of dealing with this
     266are to allow signature-level exports/hiding.  Here, we simply collect up
     267all of the modules mentioned in a signature and export/hide those.
     268Hiding may be more convenient, since you have to include not only the
     269signature of the API that was implemented, but any other types which
     270live in other signatures which are exported.
     272=== Injecting a backwards compatibility shim ===
    186276------ >8 -------
    188 More importantly, Backpack offers a tantalizing story for managing different versions of packages, alluded to in the paper, but not elaborated on. In Cabal, the version number range of a build-depends implicitly defines a "signature" which we depend on. There are a few ways this signature could be computed:
    190 *   We could throw our hands up in the air and say the signature is just whatever Cabal computed. This does not lead to very reproducible builds, but it is the current status quo.
    192 *   We could compute the “greatest common signature” for the specified version range. This signature is the widest signature for which all of the versions are compatible with. This can be used to determine if there is a buggy version range; if the greatest common signature isn’t enough to compile the package, there must exist some version that is listed as compatible, but isn’t actually. For unbounded version ranges, this can be a bit dodgy, but the PVP suggests that if you’re not unbounded over too many version numbers, you’ll be OK
    194 *   We could further refine the greatest common signature, by finding the thinnest signature with which our package type checks with. This is the “ground truth” with regards to what our package relies on, although maintaining a signature like this could be pretty annoying, on the order of annoyance of having to maintain explicit import lists (and type signatures) for everything that you use. If this is automatically calculated, we could do away with version dependencies and just see if signatures are satisfied. This could have performance problems, and calculating this requires a bit of work.
    196 By the way, this means that naively implementing the suggestion in the Backpack paper where modules provide signature files is quite dodgy, because the signature file is likely to contain too much “stuff”, and in any case, needs to be referred to in a way that is stable across versions. On the other hand, one could easily manage partitioning signatures into “latest and greatest” versus “well, this is pretty stable for most people”; differently, one could pin to a signature for a version, and as long as the package doesn’t break backwards compatibility that signature will work for a while.
    198 === Interlude: Cabal-specific features ===
     278== Interlude: Cabal-specific features ==
    200280Cabal is not just a packaging mechanism, but it also handles a number of
    224304Scott thinks there is a way to automatically infer the necessary hs-boot signatures for recursive module bindings. His proposal is here: