Opened 6 years ago

Closed 6 years ago

#3408 closed bug (fixed)

idle GC causes large CPU usage if run more frequently than 1 second

Reported by: JeremyShaw Owned by: simonmar
Priority: high Milestone: 6.12.1
Component: Runtime System Version: 6.10.4
Keywords: idle GC Cc: robgreayer@…
Operating System: Linux Architecture: Unknown/Multiple
Type of failure: Test Case:
Blocked By: Blocking:
Related Tickets: Differential Revisions:

Description

I have a web server which exhibits unusual CPU usage. When it first starts it loads a bunch of state from disk -- around 800MB. It then sits idle consuming no CPU. If I make any request to the server, it starts using 30-60% of the CPU and never goes back down -- even if the request does not interact with the state system at all.

If I disable idle GC, then the problem goes away. If I set idle GC time using, +RTS -I1, the problem goes away. If I set it to, +RTS -I0.9999999, then I see the CPU usage issue.

Setting the +RTS -I1 option is sufficient for now, but the documentation says I should report any weirdness.

I do not have a simple test case for this.

Change History (6)

comment:1 Changed 6 years ago by simonmar

  • difficulty set to Unknown
  • Milestone set to 6.12 branch
  • Owner set to simonmar

It's very odd that something is happening at the 1-second interval.

Basically we need to figure out whether this is something GHC can fix, or whether it is something special to your app. The point of the idle GC is not performance primarily, it for the detection of deadlocked threads and for running finalizers. Without the idle GC, we don't detect deadlocks until the next GC, and if the system really is deadlocked it will never GC. The problem is that in the threaded RTS we don't know whether a thread in a foreign call is about to return or whether it is blocked for ever, so we have to make a guess about when the system is idle and try to figure out whether it is deadlocked.

If you don't care about detecting deadlocks or running finalizers, then by all means disable the idle GC.

Now, the problem you're describing sounds like this: when the idle GC runs, it starts some finalizers. The finalizers then run, and create some more finalizers. A bit later, the idle GC runs again, and so on - never reaching a fixed point. If that's your problem then you either need to turn off the idle GC or restructure the app so that finalizers don't create more finalizers.

I don't understand how the 1-second parameter comes into it. Perhaps your app has a 1-second timer for something?

comment:2 Changed 6 years ago by JeremyShaw

I am not knowingly using finalizers. But my application depends on dozens of libraries from hackage, so who knows what is in there :)

I'll work on coming up with a sensible test case.

comment:3 Changed 6 years ago by robgreayer

  • Cc robgreayer@… added

Perhaps related ... I've noticed a problem running GHCi on Windows; after loading a fairly complex set of modules, the 'idle' CPU usage of GHC can be quite high. After seeing this bug report, I tried setting the idle GC time to something other than its default, (e.g. ghci.exe +RTS -I1 -RTS ...) and the 'idle' CPU usage drops to 0.

With idle GC = 0.3 (or unset), after initialization GHCi's idle CPU usage is about 2%. After loading a test case (Main module of frag-1.1.2, from hackage), it hovers at 15%-25%.

With idle GC set to 1, in both cases idle CPU usage is 0%. (This is on XP SP2, Intel Core 2 Duo CPU 2.2Ghz).

comment:4 Changed 6 years ago by simonmar

  • Milestone changed from 6.12 branch to 6.12.1
  • Priority changed from normal to high

I tried it on Linux and couldn't reproduce. Will investigate on Windows before 6.12.1.

comment:5 Changed 6 years ago by simonmar

I see the problem with GHCi 6.10.4+ on Windows. I think what is happening is this:

  • Haskeline is using ReadConsoleInput to wait for input
  • ReadConsoleInput returns non-key events
  • Windows is sending some non-key events every second or so
  • every 0.3 seconds of idle time, the RTS does another GC
  • setting -I large enough means that the RTS is never idle for long enough to trigger a GC

I'm not sure what the best fix is. I'll make the default idle time longer for now.

comment:6 Changed 6 years ago by simonmar

  • Resolution set to fixed
  • Status changed from new to closed

Fixed

Fri Sep 11 05:45:47 PDT 2009  Simon Marlow <[email protected]>
  * Fix #3408: lengthen the idle GC time to 5s for GHC/GHCi.
Note: See TracTickets for help on using tickets.