ghc-pkg parser eats too much memory
Parsing of spec files in ghc-pkg scales very poorly. The following script demonstrates memory consumption growth as a function of the number of tokens in ld-options (16K leads to ~6G)
#!/bin/bash # Demonstrates memory consumption behavior of ghc-pkg as a function of # the number of ld-options arguments.
for i in {10..14}; do
size=$((1 << $i)) echo $size rm -fr a.packages /usr/bin/ghc-pkg init a.packages cat > a.spec <<EOF
name: project id: project license: AllRightsReserved version: 1.0 EOF
echo -n ld-options: >> a.spec for i in $(seq 1 $size); do echo -n "x "; done >> a.spec /usr/bin/time /usr/bin/ghc-pkg --global-package-db a.packages register --force a.spec
done exit 0
# This was collected with ghc-7.6.3. 7.8.3 fares as badly. % bash a.sh 1024 Reading package info from "a.spec" ... done.
- 03user 0.00system 0:00.03elapsed 97%CPU (0avgtext+0avgdata 17848maxresident)k
0inputs+32outputs (0major+4973minor)pagefaults 0swaps 2048 Reading package info from "a.spec" ... done.
- 09user 0.01system 0:00.10elapsed 99%CPU (0avgtext+0avgdata 60872maxresident)k
0inputs+56outputs (0major+15723minor)pagefaults 0swaps 4096 Reading package info from "a.spec" ... done.
- 41user 0.07system 0:00.49elapsed 99%CPU (0avgtext+0avgdata 294340maxresident)k
0inputs+104outputs (0major+74090minor)pagefaults 0swaps 8192 Reading package info from "a.spec" ... done.
- 72user 0.30system 0:02.04elapsed 99%CPU (0avgtext+0avgdata 1168836maxresident)k
0inputs+192outputs (0major+292716minor)pagefaults 0swaps 16384 Reading package info from "a.spec" ... done.
- 11user 1.38system 0:10.51elapsed 99%CPU (0avgtext+0avgdata 5396932maxresident)k
0inputs+376outputs (0major+1349736minor)pagefaults 0swaps
Trac metadata
Trac field | Value |
---|---|
Version | 7.8.3 |
Type | Bug |
TypeOfFailure | OtherFailure |
Priority | normal |
Resolution | Unresolved |
Component | ghc-pkg |
Test case | |
Differential revisions | |
BlockedBy | |
Related | |
Blocking | |
CC | |
Operating system | |
Architecture |