Opened 7 years ago

Closed 7 years ago

Last modified 7 years ago

#4493 closed bug (invalid)

Child from forkProcess calls select/read on parent's Handle, stealing data from parent

Reported by: josh Owned by: simonmar
Priority: high Milestone: 7.0.2
Component: Runtime System Version: 6.12.1
Keywords: Cc: josh@…, joey@…
Operating System: Linux Architecture: x86_64 (amd64)
Type of failure: Incorrect result at runtime Test Case: yes
Blocked By: Blocking:
Related Tickets: Differential Rev(s):
Wiki Page:

Description

In trying out git-annex on a large repository, I ran into a strange bug where git-annex would try to operate on a filename that consisted of the beginning of one filename spliced together with the end of another filename. After staring at strace output for a while, I tracked down the problem, though not the root cause. git-annex would call pipeFrom (from MissingH's System.Cmd.Utils), which (by way of hPipeFrom) would call forkProcess to get a child process, and then call dupTo, close the original fd, and call execProcess to run the specified process. The parent process, in turn, would call fdToHandle on the other end of the pipe, and then pipeFrom would call hGetContents on that handle. In the version of git-annex I originally tested (the 0.04 release), more than one of these pipes might run at once. strace showed that on occasion, the child process created by forkProcess would call select on the file descriptor for a *previous* pipe, and read from that previous pipe, stealing data from the parent process; it would do this at some point after starting and before calling exec.

I reproduced this problem with 6.12.1 and 6.10.4, both with and without -threaded. The analysis of the strace given above came from 6.12.1 without -threaded. (Note that with -threaded, strace would fail at runtime, with "PANIC: attached pid $SOMEPID exited with 0"; see http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=603187 . So, I don't have an strace to analyze from -threaded.)

I can reliably reproduce this problem. Please let me know if I can provide any further information.

Change History (7)

comment:1 Changed 7 years ago by pgpgpg

hi josh,

i have an automated tool that might make it easier for devs to reproduce your bug. please email me at philip AT pgbovine.net if you'd like to try it out (i don't want to unnecessarily spam the bug tracker)

comment:2 Changed 7 years ago by igloo

Milestone: 7.0.2
Priority: normalhigh
Status: newinfoneeded

Please do attach a way to reproduce the bug to this ticket; thanks!

comment:3 Changed 7 years ago by joeyhess

Josh may have a better test case, but it can be reproduced by the following.

import System.Cmd.Utils
main = do
        (h, stream) <- pipeFrom "find" ["."]
        mapM (\file -> pipeFrom "ls" ["-l", file]) $ lines stream

This code is intentionally itself bad, and will leak FDs a lot, but if you run it in a directory with lots of contents, you'll see that sometimes, before dying due to too many open FDs, it runs ls with a parameter that is not a complete line output by find, due to the data stealing problem.

Here I run it in ghc's own source tree:

joey@wren:~/tmp/ghc6-6.12.1>runghc ~/testcase.hs
ls: cannot access ./.pc/alpha-tcforeign/systemTests/A/c_src/hello.c: No such
file or directory
ls: cannot access ./libraries/Cabal/tests/ow: No such file or directory
repo.hs: createPipe: resource exhausted (Too many open files)
joey@wren:~/tmp/ghc6-6.12.1>find |grep libraries/Cabal/tests/ow
joey@wren:~/tmp/ghc6-6.12.1>

comment:4 Changed 7 years ago by josh

Status: infoneedednew

comment:5 Changed 7 years ago by simonpj

Owner: set to simonmar

comment:6 Changed 7 years ago by simonmar

Resolution: invalid
Status: newclosed

This is lazy I/O. What is happening is that the filename passed to the child process is partly unevaluated (thanks to lines being nice and lazy), and when the child process demands it, it tries to read the data from the original pipe.

If you change the program to

import System.Cmd.Utils
main = do
        (h, stream) <- pipeFrom "find" ["."]
        mapM (\file -> length file `seq` pipeFrom "ls" ["-l", file]) $ lines stream

then it works.

In System.Process we always set our pipe FDs to FD_CLOEXEC which would give you a nicer error message in this case - the child would fail when trying to read the parent's pipe. I think MissingH doesn't do that (it might be a good idea to suggest making that change to John Goerzen).

comment:7 in reply to:  6 Changed 7 years ago by simonmar

Replying to simonmar:

In System.Process we always set our pipe FDs to FD_CLOEXEC which would give you a nicer error message in this case - the child would fail when trying to read the parent's pipe. I think MissingH doesn't do that (it might be a good idea to suggest making that change to John Goerzen).

I just realised that FD_CLOEXEC doesn't help here (because we read before exec'ing). What you want to do is to close all file descriptors in the child, which you can do with the close_fds option when using System.Process. I don't know if MissingH has a way to do that.

Note: See TracTickets for help on using tickets.