Opened 7 years ago

Last modified 4 years ago

#1009 new task

GHC Performance Index

Reported by: simonmar Owned by:
Priority: normal Milestone:
Component: Build System Version: 6.6
Keywords: Cc:
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: Difficulty: Moderate (less than a day)
Test Case: N/A Blocked By:
Blocking: Related Tickets:

Description

We want a lightweight way to detect when either (a) the compiler's performance changes, or (b) the performance of compiled code changes. This was triggerred by my noticing that the HEAD nightly builds seem somewhat slower than usual right now.

So the idea is pretty simple. We decide on a fixed set of nofib programs --intiially all of them, but we want the set to remain stable in the future -- and as part of the nightly build we time how long it takes to compile this set with a fixed set of flags (probably -O -fasm, to rule out changes in gcc). Then we time how long it takes to run all those programs, maybe 5 times each. Then we keep track of these two figures for every successful nightly build, and plot the results, giving us a nice way to track GHC's performance over time. We can go back and get some historical figures too. Of course the results are specific to a particular machine, so we'd have to keep track of results per nightly build machine or something.

So basically: some small changes to nofib to allow "just build everything" (I think even NoFibRuns?=0 might do this), and to build a fixed set of programs (the set could be built into the nofib build system, no need to make it dynamic). And some way to collect the results and plot graphs.

Change History (3)

comment:1 Changed 6 years ago by simonmar

  • Architecture changed from Unknown to Unknown/Multiple

comment:2 Changed 6 years ago by simonmar

  • Operating System changed from Unknown to Unknown/Multiple

comment:3 Changed 4 years ago by simonmar

  • Difficulty changed from Moderate (1 day) to Moderate (less than a day)
Note: See TracTickets for help on using tickets.