Opened 2 years ago

Closed 19 months ago

Last modified 19 months ago

#12582 closed feature request (fixed)

HSOC Eventlog live profiling

Reported by: NCrashed Owned by: NCrashed
Priority: normal Milestone: 8.2.1
Component: Runtime System Version: 8.0.1
Keywords: eventlog profile hsoc Cc: simonmar, Phyx-
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: None/Unknown Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s): Phab:D2522, Phab:D2934
Wiki Page:

Description

Hi, it is ticket for results of my Haskell Summer of Code 2016 project [1].

The goal is provide feature for redirection of events to external profilers without an application termination.

Proposed changes:

  • Changing eventlog file descriptor in runtime.
/*
 * Set custom file stream for global event log sink.
 *
 * The function overwrites previous event log file pointer. Previouss
 * sink is closed only if closePrev flag is on.
 *
 * Writing to the sink is protected by global mutex.
 *
 * The function puts header to the new sink only when emitHeader flag
 * is on. User might not want the header if it is switching to
 * already existed eventlog handle that was switched away recently.
 */
void rts_setEventLogSink(FILE *sink,
                         StgBool closePrev,
                         StgBool emitHeader);

/*
 * Get current file stream that is used for global event log sink.
 *
 * You shouldn't do anything with the pointer until
 * rts_setEventLogSink(otherFileOrNull, false) is called. After that
 * you can do anything with the file stream.
 */
FILE* rts_getEventLogSink(void);
  • Special flag -lm for RTS to store eventlog in memory chunks. User space functions to retrieve the chunks from memory.
/*
 * If RTS started with '-lm' flag then eventlog is stored in memory buffer.
 *
 * The function allows to pop chunks of the buffer. Return value of 0 means
 * that there is no any filled chunk of data.
 *
 * If the function returns nonzero value the parameter contains full chunk
 * of eventlog data with size of the returned value. Caller must free the
 * buffer, the buffer isn't referenced anywhere anymore.
 *
 * If nobody calls the function with '-lm' flag then the memory is kinda
 * to be exhausted.
 */
StgWord64 rts_getEventLogChunk(StgInt8** ptr);
  • Feature for dynamic resize of eventlog buffers.
/*
 * Reallocate inner buffers to match the new size. The size should be not
 * too small to contain at least one event.
 *
 * If RTS started with '-lm' the chunks of memory buffer is also resized.
 */
void rts_resizeEventLog(StgWord64 size);

/*
 * Return current size of eventlog buffers.
 */
StgWord64 rts_getEventLogBuffersSize(void);
  • Bindings to new features in Debug.Trace in base package.
-- | The 'setEventLogCFile' function changes current sink of the eventlog, if eventlog
-- profiling is available and enabled at runtime.
--
-- The second parameter defines whether old sink should be finalized and closed or not. 
-- Preserving it could be helpful for temporal redirection of eventlog data into not 
-- standard sink and then restoring to the default file sink.
--
-- The third parameter defines whether new header section should be emitted to the new
-- sink. Emitting header to already started eventlog streams will corrupt the structure 
-- of eventlog format.
--
-- The function is more low-level than 'setEventLogHandle' but doesn't recreate underlying
-- file descriptor and is intended to use with 'getEventLogCFile' to save and restore 
-- current sink of the eventlog.
--
-- @since 4.10.0.0
setEventLogCFile :: Ptr CFile -> Bool -> Bool -> IO ()

-- | The 'getEventLogCFile' function returns current sink of the eventlog, if eventlog
-- profiling is available and enabled at runtime.
--
-- The function is intented to be used with 'setEventLogCFile' to save and restore 
-- current sink of the eventlog.
--
-- @since 4.10.0.0
getEventLogCFile :: IO (Ptr CFile)

-- | Setting size of internal eventlog buffers. The size should be not
-- too small to contain at least one event.
--
-- If RTS started with '-lm' the chunks of memory buffer is also resized.
--
-- The larger the buffers the lesser overhead from event logging, but 
-- larger delays between data dumps.
--
-- See also: 'getEventLogChunk', 'getEventLogBufferSize'
setEventLogBufferSize :: Word -> IO ()

-- | Getting size of internal eventlog buffers.
--
-- See also: 'setEventLogBufferSize', 'getEventLogChunk'
getEventLogBufferSize :: IO Word

-- | Get next portion of the eventlog data.
--
-- If RTS started with '-lm' flag then eventlog is stored in memory buffer.
-- 
-- The function allows to pop chunks out of the buffer. Return value of Nothing 
-- means that there is no any filled chunk of data.
--
-- If the function returns nonzero value the parameter contains full chunk 
-- of eventlog data with size of the returned value. Caller must free the
-- buffer with 'free' from 'Foreign.Marshal.Alloc', the buffer isn't referenced 
-- anywhere anymore.
--
-- If nobody calls the function with '-lm' flag on then the memory is kinda
-- to be exhausted.
--
-- If '-lm' flag is off, the function returns always 'Nothing'.
--
-- See also: 'setEventLogBufferSize'
getEventLogChunk :: IO (Maybe CStringLen)

References:

  1. HSOC project description
  1. Design choices for implementation

Change History (6)

comment:1 Changed 2 years ago by NCrashed

The diffs will be uploaded to Phabricator shortly.

comment:2 Changed 2 years ago by simonmar

Thanks for opening a ticket, I'll comment on the diff.

comment:3 Changed 2 years ago by Phyx-

Cc: Phyx- added

comment:4 Changed 2 years ago by NCrashed

Differential Rev(s): Phab:D2522

comment:5 Changed 19 months ago by bgamari

Milestone: 8.2.1
Resolution: fixed
Status: newclosed

For the record, we have merged a simple variant of the original patch in Phab:D2934. I think this can be closed.

comment:6 Changed 19 months ago by bgamari

Differential Rev(s): Phab:D2522Phab:D2522, Phab:D2934
Note: See TracTickets for help on using tickets.