wiki:HackPar

Version 33 (modified by kevinhammond, 4 years ago) (diff)

--

Parallel Haskell Hackathon

Organisation

Dates : 10-12 December

Location : St Andrews

Participants

Vladimir Janjic (Host), Hans-Wolfgang Loidl, Kevin Hammond, Mustafa Aswad, Henrique Ferreiro, Philip W Trinder, Patrick Maier, Abyd Al Zain, Mischa Dieterle, Thomas Horstmeyer, Jost Berthold, Simon Marlow (virtual),

Source code

Important: if you can, please prepare a setup of the latest sources in advance. A darcs repository has been set up on the server in Marburg, please read here how to get a working setup: Repository Briefing.

You should contact Mischa (dieterle@mathematik...) in order to get access, or check out read-only via http (which is of course not what we want later). If you have questions or problems, edit the page or mail Jost (berthold@mathematik...)

Agenda

An Agenda Proposal is now on a separate page.

Topics to cover in the Hackathon:

  • Short overview sessions on implementation internals (KH: these should not all be on the first afternoon!):
    • GUM (Mustafa)
    • Eden (Jost)
    • Globus (Abyd)
    • Migration/Load? Balancing (Vladimir)
    • ghc HEAD developments (Simon)
    • GUM/Eden Compilation and Debugging (Vladimir?)
  • GUM-6 port
  • Merging GUM and Eden implementations
    • Packing code
    • Scheduler
    • GC interface
    • Tagging
  • Usage of new tracing infrastructure
  • Development infrastructure
    • Unified revision control (darcs)
    • Debugging infrastructure
    • Packaging
    • Setup for automatic tests
  • Parallel nofib-suite
  • Planned extensions and applications
    • Integration with GHC/SMP
    • Integrating migration
    • Globus interface
    • pre-SCSCP GAP interface
    • Micro-kernel/substrate approach to the RTS
  • Nominating people in charge of sub-projects
    • parallel nofib suite
    • Eden/GUM code maintenance
    • testing framework
  • Prize for the most awkward bug fixed in the Hackaton. Jost: I am non-competitive in this contest, since my task is to introduce them ;) HWL: Clearly you are at an disadvantage: you first have to introduce the bug; we don't have that problem;-)

Expected Outcomes

Concrete Deliverables

  • Tracing mechanism
  • Unified repository, containing Eden and GUM code
  • Common test platform - we can talk to Bodo Scholz about unibench
  • List of people in charge of sub-projects
  • Longer Term: Eden&GUM in GHC HEAD

Sub-tasks in Eden/GUM/GHC integration:

  • Merge Eden/GUM repository
  • Merge GUM into current Eden version
  • Adapt for per-CPU/Core GC
  • Integrating Vladimirs work (load balancing, migration)

Longer term system issues

  • Manycore
  • Hierarchies
  • HPC
  • Grid
  • Cloud
  • GPGPUs
  • Artcop
  • Revisit design decisions, e.g. cycles at the global level

Others

  • Update plan btw Eden/GUM and GHC-HEAD
  • Community creation
  • PhD Topic for Henrique
  • Benchmarks/Parallel? NoFib? suite - we need to be able to classify expected parallel behaviour as well as detecting e.g. space leaks
  • Launch GpH book
  • Long Term: Build Franchise
  • Agree on new Strategies module (rewrite applications)
  • Visualisation tools (which?)
  • Revisit design decisions for Eden/GUM

Discussions

SM introduced pointer tagging; it will be necessary to follow indirections, and maintain tags on exported closures we need to study evac.c looking at UNTAG_CLOSURE.

Decisions

We prioritised setting up a common repository and Eden/GUM integration. Benchmarking and packaging were seen as longer term issues.

We agreed to use the GHC ticketing system, with some specific identification for Eden/GUM issues. We agreed to use a common darcs repository for Eden/GUM.

We agreed that VJ's GUM 4.06 changes should be integrated with the combined system. The priority is to integrate thread migration.

Community Building

We need to find common ways of working to maximise our effort.

darcs for repositories

skype for communication

do we want dedicated mailing lists/IRC channels?

Who Does What

We agreed to setup hacker teams to work on different issues as follows

Repository: VJ + JB

Set up a common repository to hold GUM and Eden. Later, we should try to integrate this with the main GHC repository.

GUM/Eden Integration

  • Integrating basic GUM code into Eden version 6.13: HWL?

[IntegratingGUMProgress Progress]

  • Packing code (incl. tags): HWL+ MKA + JB

Progress?

  • Scheduler: HWL + HF + PM + PWT

Progress?

  • GC Interface + Global Addresses (with tags): HWL + VJ

Do we retain FETCHME closures or e.g. use a table

[GCProgress Progress]

  • Tracing (agree format): KH + JB + MD + TH + HWL

We need to agree a format! We want tools to work on all formats. We agreed with SM to extend the threadscope event format, adding in new event types for distributed/more detailed events.

Progress?

  • Thread Migration (packing TSOs, dealing with black holes, plus shark etc mechanisms): PWT + VJ

Progress?

Visualisation Tools: HF + JB + KH + MKA + PWT

The visualisation tools need to work with the agreed format. We may need more detailed views and/or interactive versus publication tools.

Benchmarking: HF + MD + JB + everyone

We agreed to hold a discussion on benchmarking on Friday.

Packaging and Builds

This is important in the long-term. For now we left it open until we had made progress on an integrated system.

We need to consider MPI support.