#9128 closed bug (fixed)

Possible bug in strictness analyzer when where clause declared NOINLINE

Reported by: aalevy Owned by:
Priority: normal Milestone: 7.8.3
Component: Compiler Version: 7.8.2
Keywords: strictness bytestring Cc:
Operating System: Linux Architecture: x86_64 (amd64)
Type of failure: Runtime crash Test Case: simplCore/should_run/T9128
Blocked By: Blocking:
Related Tickets: Differential Revisions:

Description

I've encountered the following error message using a relatively straight forward library that wraps postgresql-simple in certain edge cases:

*** Exception: Oops!  Entered absent arg a_sYDl{v} [lid] bytestring-0.10.4.0:Data.ByteString.Internal.ByteString{tc r5T}

This happens on an invocation of a function, dbSelect under certain compilation conditions:

dbSelect :: (Model a) => Connection -> DBSelect a -> IO [a]
{-# INLINE #-}
dbSelect conn dbs = map lookupRow <$> query_ conn q
  where {-# NOINLINE #-}
    q = renderDBSelect dbs

A DBSelect is just a data-structure with different Query (wrapper around strict bytestring) fields for clauses in a SQL select query. renderDBSelect generates a single Query value from the DBSelect (by way of contructing a Blaze.Builder as an intermediate step).

When compiled with no optimizations, this works fine, no issues. With -O1, I get the error above. The errors goes away, if I compile with -fno-strictness or remove the NOINLINE *or* INLINE pragmas.

We've worked around this for now in the library by removgin the NOINLINE pragmas, but tracking this down it seems like the strictness analyzer might be falsly assuming q is never actually evaluated. For reference, this is a commit that still exhibits the bug: https://github.com/alevy/postgresql-orm/tree/93075d56ae5ffeb8f80ecc8c01436713c2656a6b

I've also attached a small test application that excercises the bug. Because of how the library sets up a scratch database, the test application requires postgres and pg_ctl to be available in the path.

Attachments (2)

test.hs (769 bytes) - added by aalevy 16 months ago.
Small test excercising the bug
test2.hs (904 bytes) - added by aalevy 16 months ago.
Small test excercising the bug without DB dependency

Download all attachments as: .zip

Change History (13)

Changed 16 months ago by aalevy

Small test excercising the bug

comment:1 Changed 16 months ago by simonpj

Can you give clear instructions for how to reproduce, starting from a fresh GHC 7.8.2. Doubtless some libraries to install? Then get some specific (non Hackage) version of postgres SQL, or something. Assume I am stupid and you won't go far wrong.

Thanks, Simon

comment:2 Changed 16 months ago by aalevy

Yes, sorry.

From a fresh install with ghc-7.8.2 and cabal (1.20 in my case, but since it's just for installing dependencies, and I don't think default optimization levels have changed recently, I suspect it doesn't matter):

  1. Prerequisites: postgresql (I'm using 9.3, but probably doesn't matter) with tools (specifically pg_ctl). Tools, for example, are not installed by default on the postgresql version that comes with OS X -- you have to install from homebrew I think. pg_ctl comes with postgresql on most distros of linux though. Unfortunately, I don't know about windows and don't have an install available to test with.
  1. Grab the unpatched version of the postgresql-orm from github:
    $ git clone https://github.com/alevy/postgresql-orm.git
    $ cd postgresql-orm
    $ git checkout 93075d56ae5ffeb8f80ecc8c01436713c2656a6b
    
  1. Install library dependencies (inside a cabal sandbox if you care about not polluting your global environment):
    $ cabal sandbox init
    $ cabal install --only-dependencies
    
  1. Copy test.hs into the project directory
  1. Compile with O1:
    $ ghc -O1 test.hs -package-db ./.cabal-sandbox/x86_64-linux-ghc-7.8.2-packages.conf.d
    
  1. Run:
$ ./test
test: Oops!  Entered absent arg a_sYDl{v} [lid] bytestring-0.10.4.0:Data.ByteString.Internal.ByteString{tc r5T}

All the machines I've tested this on are running Arch Linux, with ghc-7.8.2

Are there any steps I haven't thought to include?

Thanks! -amit

comment:3 Changed 16 months ago by aalevy

OK, I can reproduce it without a dependency on the database (which should make it easier to repro for others...)

Same instructions as before, except no need to install/setup the postgresql database server, and use the new attached (test2.hs) file instead:

  1. Grab the unpatched version of the postgresql-orm from github:
    $ git clone https://github.com/alevy/postgresql-orm.git
    $ cd postgresql-orm
    $ git checkout 93075d56ae5ffeb8f80ecc8c01436713c2656a6b
    
  1. Install library dependencies (inside a cabal sandbox if you care about not polluting your global environment):
    $ cabal sandbox init
    $ cabal install --only-dependencies
    
  1. Copy test2.hs into the project directory
  1. Compile with O1:
    $ ghc -O1 test2.hs -package-db ./.cabal-sandbox/x86_64-linux-ghc-7.8.2-packages.conf.d
    
  1. Run:
    $ ./test
    test: Oops!  Entered absent arg a_sYDl{v} [lid] bytestring-0.10.4.0:Data.ByteString.Internal.ByteString{tc r5T}
    

Changed 16 months ago by aalevy

Small test excercising the bug without DB dependency

comment:4 Changed 16 months ago by aalevy

  • Architecture changed from Unknown/Multiple to x86_64 (amd64)
  • Operating System changed from Unknown/Multiple to Linux

comment:5 Changed 15 months ago by rwbarton

Someone on #haskell reported this: http://lpaste.net/105005 which looks like it might be the same bug, but sadly they didn't include enough details to reproduce it (and my 5-minute attempt to do so by guessing the rest of the program was unsuccessful).

comment:6 follow-up: Changed 15 months ago by aalevy

@rwbarton, know who it was? It does look related. I would like to try and get info from them to help me track this down.

comment:7 in reply to: ↑ 6 Changed 15 months ago by rwbarton

Replying to aalevy:

@rwbarton, know who it was? It does look related. I would like to try and get info from them to help me track this down.

freenode Web user ticktockman (http://ircbrowse.net/browse/haskell?id=18255857&timestamp=1401773327#t1401773327). Good luck :)

comment:8 Changed 15 months ago by simonpj

I would love a smaller test case. I'm currently stalled thus

cabal install --with-ghc=/home/simonpj/5builds/HEAD-2/inplace/bin/ghc-stage2
Resolving dependencies...
[1 of 1] Compiling Main             ( /tmp/postgresql-libpq-0.9.0.1-63567/postgresql-libpq-0.9.0.1/dist/setup/setup.hs, /tmp/postgresql-libpq-0.9.0.1-63567/postgresql-libpq-0.9.0.1/dist/setup/Main.o )
Linking /tmp/postgresql-libpq-0.9.0.1-63567/postgresql-libpq-0.9.0.1/dist/setup/setup ...
Configuring postgresql-libpq-0.9.0.1...
setup: The program 'pg_config' is required but it could not be found.
Failed to install postgresql-libpq-0.9.0.1
cabal: Error: some packages failed to install:
postgresql-libpq-0.9.0.1 failed during the configure step. The exception was:
ExitFailure 1
postgresql-orm-0.3.0 depends on postgresql-libpq-0.9.0.1 which failed to
install.
postgresql-simple-0.4.2.2 depends on postgresql-libpq-0.9.0.1 which failed to
install.

I gather that pg_config is part of !PostgreSQL, which I am reluctant to install on my (shared) Linux box. (I have no clue how to even begin.) But perhaps it is not really necessary?

At the moment this looks like a somewhat serious bug (in the demand analyser) which I have no way to reproduce.

Thanks

Simon

comment:9 Changed 15 months ago by Simon Peyton Jones <simonpj@…>

In 7d9feb264a4fc3c15d1e5f88f2e7a04202ed9743/ghc:

Fix a serious, but rare, strictness analyser bug (Trac #9128)

In a special case for trivial RHSs (see DmdAnal.unpackTrivial),
I'd forgotten to include a demand for the RHS itself.
See Note [Remember to demand the function itself].

Thanks to David Terei for guiding me to the bug,
at PLDI in Edinburgh.

comment:10 Changed 15 months ago by simonpj

  • Milestone set to 7.8.3
  • Status changed from new to merge
  • Test Case set to simplCore/should_run/T9128

comment:11 Changed 15 months ago by thoughtpolice

  • Resolution set to fixed
  • Status changed from merge to closed
Note: See TracTickets for help on using tickets.