Opened 5 months ago

#8591 new bug

Concurrent executions of ghc-pkg can cause inconstant package.cache files

Reported by: janm Owned by:
Priority: normal Milestone:
Component: Package system Version: 7.6.3
Keywords: ghc-pkg race Cc:
Operating System: FreeBSD Architecture: Unknown/Multiple
Type of failure: Other Difficulty: Unknown
Test Case: Blocked By:
Blocking: Related Tickets:

Description

I am doing 24 way parallel builds of system images, including all packages on a system. This includes ghc and multiple ghc packages.

I am seeing intermittent dependency failure from the ghc packaging system. From examining Main.hs in ghc-pkg, I see the function withFileAtomic write to a temporary file in package.conf.d and then atomically rename on top of a target file, package.cache in the case. With parallel execution the last rename would win, leading to lost entries in package.cache.

In my case, the following things happened: ("Building" indicates a start, "Built" indicates completion, "Installing" is setup in a separate chroot'd environment and is isolated)

The FreeBSD ports system is used to drive the Haskell build system.

The process works single threaded and fails intermittently when done in parallel.

Building: devel/hs-data-default-instances-base
Building: devel/hs-data-default-instances-containers
Building: devel/hs-data-default-instances-old-locale
Built: devel/hs-dlist
Building: devel/hs-data-default-instances-dlist
Built: devel/hs-temporary
Built: jail-image-full
Installing: system-imagejail-image-full
Built: devel/hs-base64-bytestring
Built: archivers/hs-zlib
Building: security/hs-digest
Built: devel/hs-syb
Building: textproc/hs-hs-bibutils
Building: textproc/hs-pandoc-types
Built: devel/hs-utf8-string
Built: devel/hs-data-default-instances-old-locale
Built: devel/hs-data-default-instances-containers
Built: devel/hs-data-default-instances-base
Built: devel/hs-data-default-instances-dlist
Building: devel/hs-data-default
Built: devel/hs-random
Installed: system-image
lang/ghc
Installing: system-imagearchivers/hs-zlib
Installing: system-image
devel/hs-utf8-string
Installing: system-imagedevel/hs-syb
Installing: system-image
devel/hs-base64-bytestring
Installing: system-imagedevel/hs-data-default-class
Installing: system-image
devel/hs-dlist
Installing: system-imagedevel/hs-random
Installing: system-image
devel/hs-temporary
Installing: system-imagedevel/hs-extensible-exceptions
Built: devel/hs-data-default FAILED

The error from the Haskell data-default build was:

setup: At least the following dependencies are missing:
data-default-instances-base -any

Looking in the in the package.conf.d directory shows that the data-default-instances-base-0.0.1-7bdf8678f0d8637e096e397e7910f82a.conf file was present, but running "ghc-pkg list" did not show data-default-instances-base

Running /usr/local/lib/cabal/ghc-7.6.3/data-default-instances-base-0.0.1/register.sh (which was also present) caused ghc-pkg to now show data-default-instances-base.

To me this looks like a race condition between multiple instances of ghc-pkg causing the cache to become inconsistent.

Change History (0)

Note: See TracTickets for help on using tickets.