Opened 3 months ago
Concurrent executions of ghc-pkg can cause inconstant package.cache files
|Reported by:||janm||Owned by:|
|Type of failure:||Other||Difficulty:||Unknown|
|Test Case:||Blocked By:|
I am doing 24 way parallel builds of system images, including all packages on a system. This includes ghc and multiple ghc packages.
I am seeing intermittent dependency failure from the ghc packaging system. From examining Main.hs in ghc-pkg, I see the function withFileAtomic write to a temporary file in package.conf.d and then atomically rename on top of a target file, package.cache in the case. With parallel execution the last rename would win, leading to lost entries in package.cache.
In my case, the following things happened: ("Building" indicates a start, "Built" indicates completion, "Installing" is setup in a separate chroot'd environment and is isolated)
The FreeBSD ports system is used to drive the Haskell build system.
The process works single threaded and fails intermittently when done in parallel.
Built: devel/hs-data-default FAILED
The error from the Haskell data-default build was:
setup: At least the following dependencies are missing:
Looking in the in the package.conf.d directory shows that the data-default-instances-base-0.0.1-7bdf8678f0d8637e096e397e7910f82a.conf file was present, but running "ghc-pkg list" did not show data-default-instances-base
Running /usr/local/lib/cabal/ghc-7.6.3/data-default-instances-base-0.0.1/register.sh (which was also present) caused ghc-pkg to now show data-default-instances-base.
To me this looks like a race condition between multiple instances of ghc-pkg causing the cache to become inconsistent.