Opened 5 months ago

Closed 5 months ago

Last modified 3 weeks ago

#12912 closed bug (fixed)

IO library should not use select()

Reported by: simonmar Owned by: simonmar
Priority: normal Milestone: 8.0.2
Component: libraries/base Version: 8.0.1
Keywords: Cc:
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: None/Unknown Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s):
Wiki Page:

Description

The IO library uses select() in fdReady(), which fails in processes with more than 1024 file descriptors.

Change History (7)

comment:1 Changed 5 months ago by Ben Gamari <ben@…>

In f46369b8/ghc:

fdReady: use poll() instead of select()

select() is limited to 1024 file descriptors.  This actually blew up
in a very hard-to-debug way in our production system when using the
hinotify package.

Test Plan:
libraries/tests pass, paricularly hGetBuf001 which exercises this
code.

Reviewers: niteria, erikd, austin, hvr, bgamari

Reviewed By: bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2785

GHC Trac Issues: #12912

comment:2 Changed 5 months ago by bgamari

Resolution: fixed
Status: newclosed

comment:3 Changed 5 months ago by bgamari

comment:4 Changed 5 weeks ago by nh2

Note this change breaks the following code:

import System.IO
import System.Timeout

main = hWaitForInput stdin (5 * 1000)

See https://ghc.haskell.org/trac/ghc/ticket/8684#comment:8

Is that expected?

comment:5 Changed 3 weeks ago by bgamari

No, this is certainly not expected. 37d7c1596ee936ec6597a5c1898e1fdca7c04f77 adds a testcase to ensure that we catch this next time.

The problem appears to be that the patch assumes that all IO operations go through the IO manager, which will take care of waiting for us. However, GHC.IO.Handle.Text.hWaitForInput appears to call GHC.IO.Fd.ready directly, which then passes the timeout to fdReady. It's not entirely clear what was intended by this comment,

    // We only handle msecs == 0 on non-Windows, because this is the
    // only case we need.  Non-zero waiting is handled by the IO manager.

Jaffacake, can you clarify how you intended for this work? Where is this IO manager treatment?

Last edited 3 weeks ago by bgamari (previous) (diff)

comment:6 Changed 3 weeks ago by simonmar

I think I just missed that we also called into here from hWaitForInput, oops. We'll need to do hWaitForInput with a non-zero timeout a different way.

comment:7 Changed 3 weeks ago by bgamari

Alright, I did some grepping and I'm pretty certain that this is the only caller of ready that we need to fix.

Note: See TracTickets for help on using tickets.