Changes between Version 4 and Version 5 of DataParallel/WorkPlan

Jan 27, 2009 1:00:45 PM (7 years ago)



  • DataParallel/WorkPlan

    v4 v5  
    22== Work plan for implementing Data Parallel Haskell ==
    4 Major milestones:
     4=== Categories ===
    6  '''Milestone 1:''' basic SMP library (End of Feb 2007)::
    7    Implementation of the central parts of an array library of flat and segmented arrays that uses distributed types to partition work on an SMP thread gang and uses fusion to eliminate superflous join points and intermediate arrays. '''[Completed]'''
    8  '''Milestone 2:''' basic vectorisation (End of Aug 2007)::
    9   Implementation of the vectorisation transformation and basic interaction of vectorised and non-vectorised code to provide (including the results from Milestone 1) a complete path from source programs to parallel executable for simple examples.
    10  '''Milestone 2.5:''' stocktake (by ICFP)::
    11   Technical report summarising the results so far & release of a first publicly announced end-to-end NDP system.
    12  '''Milestone 3:''' larger examples (End of Feb 2008)::
    13   Optimisations and added functionality to handle larger example programs.
     6Tasks below are labelled with categories that indicate the purpose of the task:
    15 === Current work items ===
     8 ''Efficiency''::
     9   Improve scalability and/or baseline performance of generated code
     10 ''Compile time''::
     11   Improve compile times
     12 ''Ease of use''::
     13   Make the system easier or more convenient to use for end users
    17 For vectorisation:
     15=== Task assignments ===
    19  * [Roman, June/July] Implement vectorisation transformation according to the scheme in `ghc-ndp/docs/ndp/`.
    20  * [Manuel] Work with Roman and in particular implement all the iface-related code (`HscTypes.VectInfo` and friends).
     17 ''Roman''::
     18   '''Replicate''' & '''Recycling'''
     19   – status: partly implemented, but still needs serious work
    22 For the library:
     21 ''Simon''::
     22   '''Code blow up'''
     23   – status: unknown
    24  * [Gabi] Complete quicksort implementation.
     25=== Open tasks ===
    26 === Todo list for vectorisation ===
     27 1. '''Replicate''' [''Efficiency'']: Implement an extended array representation that uses an optimised representation for arrays that are the result of common forms of replication (i.e., due to free variables in lifted expressions).  The optimised representation stores the data to be replicated and the replication count(s) instead of actually replicating the data.  This also requires all functions consuming arrays to be adapted.
    28  * Implement the first version
    29  * Integeration with package ndp
    30  * Testing
     29 2. '''Recycling''' [''Efficiency'']: Use Roman's recycling optimisation (PADL'09) to avoid copying in `joinD`.
    32 === Todo list for package ndp ===
     31 3. '''Scaling''' [''Efficiency'']: Investigate the scaling problems that we are seeing with vectorised code at the moment.  ('''Replicate''' and '''Recycling''' play a role here, but it is unclear whether that's all.)
    34  * Lifted functions
    35  * Add missing functions
    36  * Fusion for segmented operations
    37  * Fusion with cost function
    38  * NUMA support
     33 4. '''Test new inliner''' [''Compile time'' & ''Efficiency'']: Retest package dph with new inliner and the simplifier changes and try to simplify the library on the basis of these new phases.
     35 5. '''Code blow up''' [''Compile time'']: GHC generates a lot of intermediate code when vectorisation is enabled, leading to excessive compilation times.  Find out whether the new inliner helped here and what else can be done to improve this situation.
     37 6. '''Conversion of vectorised representations''' [''Ease of use'']: We need other than just identity conversions between vanilla and vectorised data representations, especially `[:a:] <-> PArray a`.  This will make the system more convenient to use.