Opened 3 years ago

Closed 2 years ago

#11094 closed feature request (fixed)

Cost-center heap profiler should be able to emit samples to eventlog

Reported by: bgamari Owned by:
Priority: normal Milestone: 8.2.1
Component: Profiling Version: 7.10.2
Keywords: Cc: ezyang, maoe
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: Other Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s): Phab:D1722
Wiki Page:

Description

Often when looking at heap profiles I find myself wishing that I had better tools for aligning the heap behavior to various events in the lifetime of my program. The eventlog provides a nice facility for marking these events, why not record heap profile samples in the eventlog as well?

Change History (7)

comment:1 Changed 3 years ago by bgamari

Here is a proposed eventlog sample format.

Prelude

The String type below is defined to be a UTF-8 encoded NUL-terminated string.

Metadata event types

Beginning of sample stream

A single fixed-width event emitted during program start-up describing the samples that follow.

  • EVENT_HEAP_PROF_BEGIN
    • Word8: Profile ID
    • Word64: Sampling period in nanoseconds
    • Word32: Sample break-down type. One of,
      • SAMPLE_TYPE_COST_CENTER (output from -hc)
        • SAMPLE_TYPE_CLOSURE_DESCR (output from -hd)
        • SAMPLE_TYPE_RETAINER (output from -hr)
        • SAMPLE_TYPE_MODULE (output from -hm)
        • SAMPLE_TYPE_TYPE_DESCR (output from -hy)
        • SAMPLE_TYPE_BIOGRAPHY (output from -hb)
    • String: Cost centre filter
    • String: Closure description filter
    • String: Retainer filter
    • String: Module filter
    • String: Type description filter

Cost center

A variable-length packet produced once for each cost center,

  • EVENT_HEAP_PROF_COST_CENTRE
    • Word32: cost center number
    • String: label
    • String: module
    • String: source location
    • Word8: is a CAF?

Sample event types

A sample (consisting of a list of break-down classes, e.g. cost centers, and heap residency sizes), is to be encoded in the body of one or more events.

We mark the beginning of a new sample with an EVENT_HEAP_PROF_SAMPLE_BEGIN event,

  • EVENT_HEAP_PROF_SAMPLE_BEGIN
    • Word64: sample number

A heap residency census will follow. Since events may only be up to 216 bytes in length a single sample may need to be split among multiple EVENT_HEAP_PROF_SAMPLE events. The precise format of the census entries is determined by the break-down type.

Cost-center break-down

A variable-length packet encoding a heap profile sample broken down by,

  • cost-center (-hc)
  • retainer (-hr)
  • EVENT_HEAP_PROF_SAMPLE
    • Word8: Profile ID
    • Word64: heap residency in bytes
    • Word8: stack depth
    • Word32[]: cost center stack starting with inner-most (cost center numbers)

String break-down

A variable-length event encoding a heap sample broken down by,

  • type description (-hy)
  • closure description (-hd)
  • module (-hm)
  • EVENT_HEAP_PROF_SAMPLE
    • Word8: Profile ID
    • The event shall contain packed pairs of,
      • String: type description
      • Word64: heap residency in bytes

TODO Could we intern these strings as we do cost-center names?

Biography break-down

A fixed-length event encoding a biography heap sample.

  • EVENT_HEAP_PROF_SAMPLE
    • Word8: Profile ID
    • Word64: Void
    • Word64: Lag
    • Word64: Use
    • Word64: Inherent use
    • Word64: Drag
Last edited 3 years ago by bgamari (previous) (diff)

comment:2 Changed 3 years ago by bgamari

Cc: ezyang added

comment:3 Changed 3 years ago by thomie

Differential Rev(s): Phab:D1722
Status: newpatch

Also reported as #9051.

Last edited 3 years ago by thomie (previous) (diff)

comment:4 Changed 3 years ago by thomie

Component: CompilerProfiling

comment:5 Changed 3 years ago by maoe

Cc: maoe added

comment:6 Changed 3 years ago by Ben Gamari <ben@…>

In a9bc5476/ghc:

Log heap profiler samples to event log

Test Plan: Try it

Reviewers: hvr, simonmar, austin, erikd

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D1722

GHC Trac Issues: #11094

comment:7 Changed 2 years ago by bgamari

Milestone: 8.2.1
Resolution: fixed
Status: patchclosed
Note: See TracTickets for help on using tickets.