Changes between Version 3 and Version 4 of Ticket #1605


Ignore:
Timestamp:
Feb 13, 2012 9:07:08 PM (3 years ago)
Author:
gregweber
Comment:

Legend:

Unmodified
Added
Removed
Modified
  • Ticket #1605 – Description

    v3 v4  
    1 A lack of high-level data store methods is a huge weakness in haskell. 
     1A lack of a high-level data store library is a huge weakness in haskell. 
    22 
    3 Data storage is absolutely critical for web development and any program that needs to persist data or whose data processing exceeds memory. Haskell web development has languished in part because one had to choose between lower-level SQL or Happstack's experimental data store that uses only the process memory. All other widely used programming languages have multiple ORMs for proven databases. Haskell needs some better database abstraction libraries. 
     3Data storage is absolutely critical for web development or any program that needs to persist data or whose data processing exceeds memory. Haskell web development has languished in part because one had to choose between lower-level SQL or Happstack's experimental data store that uses only the process memory. All other widely used programming languages have multiple ORMs for proven databases. Haskell needs some better database abstraction libraries. 
    44 
    55The persistent library is a universal data store interface that already has PostgreSQL, Sqlite, MySQL and a MongoDB backend. Most users of the Yesod web framework are using it, and it is also being used outside of web development. With some more improvements, persistent could become a great data store interface for haskell programmers. 
    66 
    7 What improvements you ask? Certainly more backends should be written. One possibility is a pure-haskell backend, possibly using happstack's storage system- so that a haskell programmer not be required to have any database dependencies to get up and running, but would always have the option of switching to a battle-tested database later on. The current backends should be improved. Persistent has been designed with NoSQL in mind while still using SQL backends- now there is a clamoring for more SQL features. This could be satisfied with integration with a relational algebra (haskellDB or DSH) for powerful SQL capabilities. Adoption of haskellDB has been suprisingly slow in part because it hasn't provided enough perceived benefit to justify its use. With persistent we can provide a much nicer interface to cleanly map to and from Haskell data types, and have the freedom to provide an alternative query interface. Persistent can also have improvements in the realm of modularization, testing, and benchmarking. 
     7What improvements should be made you ask? More backends could be written. There is a pull request for a CouchDB backend that would take just a little time to cleanup. However I believe we are better off mostly focusing on the current supported (SQL & MongoDB) and making them better. 
    88 
    9 Michael Snoyman is willing to mentor, and there is a large community of users willing to help or give feedback. 
     9Note that we have moved Persistent in the direction of universal query interface to just a universal data store serialization interface. There are many critics of query interfaces for good reasons: we will never be able to solve all use cases. 
     10 
     11I believe future work on Persistent should continue this recent direction of allowing for raw queries. One can now write raw SQL queries but get them automatically properly serialized. The next step is to make them extraordinarily type-safe. That is, we know at compile time that the queries are valid. They reference columns correctly and they are valid database queries. There is already an experimental implementation of this for SQL called persistent-hssqlppp that checks the validity of SQL statements at compile time. 
     12 
     13Persistent's biggest limitation right now is the lack of a good technique for returning a projection of the data - we always give back a full record. Tackling this in the GSoC would be great. Another task at hand is to write a convenient Template Haskell interface for declaring a schema, which should only be a few days work because we already have all the tools in place through the Quasi-Quoted DSL. 
     14 
     15There are also some possibilities for integrating with existing Haskell backends. One would be a pure-haskell backend, possibly using happstack's storage system - so that a haskell programmer not be required to have any database dependencies to get up and running, but would always have the option of switching to a battle-tested database later on. Another interesting option is integration with HaskellDB or DSH - HaskellDB is weak because it does not automatically serialize to a Haskell record like Persistent does. 
     16 
     17Michael Snoyman and myself are willing to mentor, and there is a large community of users willing to help or give feedback.