Ticket #3808: 3808.dpatch

File 3808.dpatch, 16.6 KB (added by simonmar, 5 years ago)
12 patches for repository http://code.haskell.org/~dons/code/bytestring:
3Mon Mar 22 15:39:14 GMT 2010  Simon Marlow <[email protected]>
4  * hGetContents: use hGet instead of hGetNonBlocking + hWaitForInput + hIsEOF
6  Not only is this cleaner, but it fixes a problem with read binary data
7  over a pipe, see
8    http://hackage.haskell.org/trac/ghc/ticket/3808
10  The problem is that bytestring normally works fine when the Handle is
11  not in binary mode, because it uses hGetBuf/hPutBuf which bypass the
12  encoding layer.  That is, except in this one particular case:
13  hWaitForInput might do some decoding, because it has to determine
14  whether there are any characters (not bytes) available to be read.
16  I imagine hGetNonBlocking was used due to concerns that hGet might
17  block if there is insufficient data, but that's not the case; hGet
18  returns a short read if it would otherwise block, and only blocks if
19  there is no data to read.
21Mon Mar 22 15:41:03 GMT 2010  Simon Marlow <[email protected]>
22  * update docs for hGet, hGetNonBlocking
24New patches:
26[hGetContents: use hGet instead of hGetNonBlocking + hWaitForInput + hIsEOF
27Simon Marlow <[email protected]>**20100322153914
28 Ignore-this: 43bac03fc6d7de74564f5cd608ed9f17
30 Not only is this cleaner, but it fixes a problem with read binary data
31 over a pipe, see
32   http://hackage.haskell.org/trac/ghc/ticket/3808
34 The problem is that bytestring normally works fine when the Handle is
35 not in binary mode, because it uses hGetBuf/hPutBuf which bypass the
36 encoding layer.  That is, except in this one particular case:
37 hWaitForInput might do some decoding, because it has to determine
38 whether there are any characters (not bytes) available to be read.
40 I imagine hGetNonBlocking was used due to concerns that hGet might
41 block if there is insufficient data, but that's not the case; hGet
42 returns a short read if it would otherwise block, and only blocks if
43 there is no data to read.
44] {
45hunk ./Data/ByteString/Lazy.hs 216
46 import Data.Word                (Word8)
47 import Data.Int                 (Int64)
48 import System.IO                (Handle,stdin,stdout,openBinaryFile,IOMode(..)
49-                                ,hClose,hWaitForInput,hIsEOF)
50-import System.IO.Error          (mkIOError, ioError, illegalOperationErrorType)
51+                                ,hClose)
52+import System.IO.Error          (mkIOError, illegalOperationErrorType)
53 import System.IO.Unsafe
54 #ifndef __NHC__
55 import Control.Exception        (bracket)
56hunk ./Data/ByteString/Lazy.hs 1177
57     lazyRead = unsafeInterleaveIO loop
59     loop = do
60-        c <- S.hGetNonBlocking h k
61-        --TODO: I think this should distinguish EOF from no data available
62-        -- the underlying POSIX call makes this distincion, returning either
63-        -- 0 or EAGAIN
64+        c <- S.hGet h k -- only blocks if there is no data available
65         if S.null c
66hunk ./Data/ByteString/Lazy.hs 1179
67-          then do eof <- hIsEOF h
68-                  if eof then hClose h >> return Empty
69-                         else hWaitForInput h (-1)
70-                           >> loop
71+          then hClose h >> return Empty
72           else do cs <- lazyRead
73                   return (Chunk c cs)
76[update docs for hGet, hGetNonBlocking
77Simon Marlow <[email protected]>**20100322154103
78 Ignore-this: b7b1986f438421615985a4587f97a58c
79] {
80hunk ./Data/ByteString.hs 1914
81 -- is far more efficient than reading the characters into a 'String'
82 -- and then using 'pack'. First argument is the Handle to read from,
83 -- and the second is the number of bytes to read. It returns the bytes
84--- read, up to n, or EOF.
85+-- read, up to n, or 'null' if EOF has been reached.
87+-- If there is any data to read, then 'hGet' will not block, instead
88+-- it will return whatever data is available without blocking.  It
89+-- only blocks if there is no data available to read.
90 --
91 -- 'hGet' is implemented in terms of 'hGetBuf'.
92 --
93hunk ./Data/ByteString.hs 1931
94     | i == 0    = return empty
95     | otherwise = illegalBufferSize h "hGet" i
97--- | hGetNonBlocking is identical to 'hGet', except that it will never block
98--- waiting for data to become available, instead it returns only whatever data
99--- is available.
100+-- | hGetNonBlocking is identical to 'hGet', except that it will never
101+-- block waiting for data to become available.  If there is no data
102+-- available to be read, 'hGetNonBlocking' returns 'null'.
103 --
104 hGetNonBlocking :: Handle -> Int -> IO ByteString
105 #if defined(__GLASGOW_HASKELL__)
110[Check for negative lengths in packCStringLen
111Duncan Coutts <[email protected]>**20090915234111
112 Ignore-this: a388aeeee7cd0b6df06c61d88a97588a
114[Check for negative lengths in hGet and hGetNonBlocking
115Duncan Coutts <[email protected]>**20090915234011
116 Ignore-this: c9e33f05ef3cabfe12c1d8fb509f4726
117 See ghc ticket #3514
119[Fix elemIndices and split
120Duncan Coutts <[email protected]>**20090912203950
121 Ignore-this: 69ae945462b5638c99cc1824ed329e90
122 They were using inlinePerformIO to lazily delay IO actions within the
123 scope of a withForeignPtr. Thus the ptr was still in use after the
124 ForeignPtr went out of scope. This lead to dangling pointers, incorrect
125 results and segfaults. Nasty. See ghc tickets #3486 and #3487.
126 Audited the rest of the code base for the same anit-pattern.
127 Thanks to nwn and people on the haskell-jp mailing list for reporting
128 the bug with nice test cases and also to Ian, Bertram and Don for
129 diagnosing the source of the problem.
131[Update "tested-with" list
132Duncan Coutts <[email protected]>**20090823195516
133 Ignore-this: 873e64dd5c978cc12751d3b404e3619a
134 It builds with ghc-6.4 through to 6.11 (23/08/09)
135 It builds with no warnings with ghc 6.8, 6.10 and 6.11.
137[Clean up the language extension pragmas
138Duncan Coutts <[email protected]>**20090823194920
139 Ignore-this: c7fb9daf30b639d0bee909fd396de241
140 We cannot actually specify them correctly, see ghc ticket #3457.
141 We can at least simplify things and make it clear it does not work,
142 rather than having misleading code that looks like it might work.
144[Fix import warnings
145Duncan Coutts <[email protected]>**20090823194733
146 Ignore-this: 2db4b086627d428de7add92566df9eb
147 Patches merged from ghc's fork:
148   Tue Jul  7 12:58:17 BST 2009  Ian Lynagh <[email protected]>
149     * Remove unused imports
151   Tue Jul  7 14:35:56 BST 2009  Ian Lynagh <[email protected]>
152     * Fix unused import warnings
154   Wed Jul 22 13:56:04 BST 2009  Simon Marlow <[email protected]>
155     * fix compilation with GHC 6.10
157[Update for new IO library
158Duncan Coutts <[email protected]>**20090823175302
159 Ignore-this: 539da275e4528629baa6d9f66c2f3335
160 Merge of patch from ghc's fork:
161   Thu Jun 11 15:09:37 BST 2009  Simon Marlow <[email protected]>
162     * Update for new IO library
164[Fix "cabal check" warnings
165Duncan Coutts <[email protected]>**20090823145438
166 Ignore-this: f6a17b693d9216b840bf01cf55a2fa7d
167 Merge of patch from ghc's fork:
168   Tue Aug 11 22:58:58 BST 2009  Ian Lynagh <[email protected]>
169     * Fix "Cabal check" warnings
171[Remove INLINE pragmas on recursive functions
172Duncan Coutts <[email protected]>**20090823145127
173 Ignore-this: 40b0b5e7a1c274b78231517736ab74c7
174 Merge of patch from ghc's fork:
175   Fri Dec  5 17:04:52 GMT 2008  [email protected]
176     * Remove INLINE pragmas on recursive functions
178     GHC now complains about INLINE pragmas on recursive functions,
179     and they were no-ops before in any case
181[Fix some "warn-unused-do-bind" warnings where we want to ignore the value
182Ian Lynagh <[email protected]>**20090710202441]
183[Hide the .Internal modules in the haddock docs
184Duncan Coutts <[email protected]>**20090702000819
185 This should stop users accidentally using the semi-public
186 modules while still leaving them available for extension
187 packages. Also means that references to 'ByteString' go to
188 a sensible place rather than the .Internal modules.
190[Fix docs for unsafePackAddressLen, it doesn't need null-termination
191Duncan Coutts <[email protected]>**20090629192811]
192[Make docs for fromForeignPtr point to suitable public alternatives
193Duncan Coutts <[email protected]>**20090629191336]
194[Improve the top-level documentation for the Unsafe and Internal modules
195Duncan Coutts <[email protected]>**20090629191017
196 Clarify the purpose and properties of the functions in the two modules.
197 In particular try to disuade people from using the Internal functions
198 unnecessarily.
200[Use comma separated lists in the LANGUAGE pragmas
201Duncan Coutts <[email protected]>**20090629190950
202 Haddock complains.
204[bump versoin
205Don Stewart <[email protected]>**20090108182815]
206[Add a test for the laziness of readInt and readInteger over lazy ByteStrings
207Bryan O'Sullivan <[email protected]>**20081231070009]
208[Fix strictness bugs in readInt and readInteger.
209Bryan O'Sullivan <[email protected]>**20081231064853]
210[Add TODO item to eliminate -fno-warn-orphans
211Duncan Coutts <[email protected]>**20081101190200
212 Orphan instances in libraries are not a good thing.
214[Make everything build with ghc-6.6 and 6.4
215Duncan Coutts <[email protected]>**20081101190029
216 No code changes, just cpp for new LANGUAGE pragmas
217 and not using a RULE with a funky LHS for ghc-6.4 which didn't allow it.
219[avoid import renaming errors in nhc98
220[email protected]**20081008092026]
221[Import fix for nhc98
222Duncan Coutts <[email protected]>**20081006211934]
223[Bump version to
224Duncan Coutts <[email protected]>**20081006210002]
225[Drop dependency on syb with base 4
226Duncan Coutts <[email protected]>**20081006205908]
227[fix import of Control.Exception for nhc98
228[email protected]**20081006102248]
230Duncan Coutts <[email protected]>**20081004225748]
231[Bump version to
232Duncan Coutts <[email protected]>**20081004225716]
233[Whitespace changes to the package description
234Duncan Coutts <[email protected]>**20081004223529]
235[Drop -fglasgow-exts, use LANGUAGE pragmas
236Duncan Coutts <[email protected]>**20081004223518]
237[Build with both base-3 and 4
238Duncan Coutts <[email protected]>**20081004221008]
239[Drop unrecognised and unnecessary pragma
240Duncan Coutts <[email protected]>**20081004220034]
241[stupid ghc-prim
242Don Stewart <[email protected]>**20080825214146]
243[Only a minor version bump
244Don Stewart <[email protected]>**20080825205906]
245[clarify comments on hGet and EOF
246Don Stewart <[email protected]>**20080809233325]
247[typo in comment
248Don Stewart <[email protected]>**20080716181103]
249[notes, and undo -fcpr-off
250Don Stewart <[email protected]>**20080520001839]
251[clean up flags. building on 6.9
252Don Stewart <[email protected]>**20080519215317]
253[more aggressive inlining on lazy bytestring readInt. Performance wins for sum-file
254Don Stewart <[email protected]>**20080519215149]
255[make hackage happy
256Don Stewart <[email protected]>**20080420211741]
257[Point to HPC coverage data
258Don Stewart <[email protected]>**20080420210524]
260Don Stewart <[email protected]>**20080420205553]
261[Add properties for rewrite rules, compiled sans -fhpc
262Don Stewart <[email protected]>**20080419025720
264 Following an idea of Andy Gill's, compile just some rewrite rules code
265 without -fhpc, link it into a hpc app, and get coverage / rule firing
266 data for code run by rewrite rules. Library now 93% covered by QuickCheck.
269[fix warnings
270Don Stewart <[email protected]>**20080417232307]
271[Remove old fusion mechanism. Data.ByteString.Fusion is a place holder
272Don Stewart <[email protected]>**20080417230308
273 for streams now.
275 And while we're at it, reimplement mapAccumL/R and scanl
276 directly. Remove mapIndexed, as its trivial to write with mapAccumL.
279[more tests
280Don Stewart <[email protected]>**20080417162926]
281[tune deprecation string
282Don Stewart <[email protected]>**20080417162305]
283[some more tests
284Don Stewart <[email protected]>**20080417011226]
285[pointless filter rule
286Don Stewart <[email protected]>**20080417011025]
287[more QuickCheck properties, 88% covered now
288Don Stewart <[email protected]>**20080417010205]
289[Use -fglasgow-exts to re-enable rewrite rules
290Don Stewart <[email protected]>**20080417010144]
291[Remove obscure filter (==) rewrite rule
292Don Stewart <[email protected]>**20080417010057]
293[add a cunning test for packMallocCString
294Don Stewart <[email protected]>**20080416171625]
295[typos in comment
296Don Stewart <[email protected]>**20080416161143]
297[more testing
298Don Stewart <[email protected]>**20080414065202]
299[normalise rule names
300Don Stewart <[email protected]>**20080414064537]
301[More testing. 80% of bytestring is now tested with QuickCheck
302Don Stewart <[email protected]>**20080413221600]
304Don Stewart <[email protected]>**20080413190445]
305[More coverage
306Don Stewart <[email protected]>**20080413190419]
307[better clean target
308Don Stewart <[email protected]>**20080413135043]
309[tweak makefile
310Don Stewart <[email protected]>**20080413134800]
311[properties for breakSubstring
312Don Stewart <[email protected]>**20080413134635]
313[Add breakSubstring, split a bytestring on another bytestring. More idiomatic approach than findSubstring
314Don Stewart <[email protected]>**20080413134332]
315[faster findSubstrings
316Don Stewart <[email protected]>**20080413103033]
317[more tests
318Don Stewart <[email protected]>**20080413103014]
320Don Stewart <[email protected]>**20080413102931]
321[no longer depend on array
322Don Stewart <[email protected]>**20080413102910]
323[require Instances
324Don Stewart <[email protected]>**20080413102859]
325[add cheaper hpc test target
326Don Stewart <[email protected]>**20080413090044]
328Don Stewart <[email protected]>**20080413090026]
329[use "ByteString" prefix for rules
330Don Stewart <[email protected]>**20080413090006]
331[return failures
332Don Stewart <[email protected]>**20080413085955]
333[actually fail if tests don't pass
334Don Stewart <[email protected]>**20080413084834]
335[more test coverage
336Don Stewart <[email protected]>**20080412192702]
337[improve performance of findSubstrings on small strings
338Don Stewart <[email protected]>**20080412192633]
339[Use -fno-ignore-asserts when testing
340Don Stewart <[email protected]>**20080411160639]
341[coverage for Monoid instance
342Don Stewart <[email protected]>**20080411154716]
343[reuse internal invariant test
344Don Stewart <[email protected]>**20080411154700]
345[ghc uses -XUnliftedFFITypes -XMagicHash -XUnboxedTuples -XDeriveDataTypeable
346Don Stewart <[email protected]>**20080411154632]
347[use appropriate -X flags
348Don Stewart <[email protected]>**20080411144604]
349[bump version number
350Don Stewart <[email protected]>**20080411114047]
351[note how to run testsuite with hugs
352Don Stewart <[email protected]>**20080411114030]
353[document second argument of hGet
354Don Stewart <[email protected]>**20080406034554]
355[normalise rewrite rule names
356Don Stewart <[email protected]>**20080321165326]
358Don Stewart <[email protected]>**20080321005940]
359[standardise rule names with   "ByteString" prefix
360Don Stewart <[email protected]>**20080320204043]
361[standardise rules firing with prefix "ByteString"
362Don Stewart <[email protected]>**20080320170005]
363[dead code
364Don Stewart <[email protected]>**20080320165557]
365[Document IsString ByteString, and how to enable it
366Don Stewart <[email protected]>**20080320165445]
367[remove deprecated 'join' -- people should have transitioned to intercalate in the past 12 months
368Don Stewart <[email protected]>**20080320165136]
369[more tests
370Don Stewart <[email protected]>**20080318171700]
371[Add rewrite rule for Char8, matching the Word8 one, specialising break (==x)
372Don Stewart <dons@galois.com>**20080318171256]
373[erroneos inline [1] on Char8.dropWhile
374Don Stewart <dons@galois.com>**20080318170525]
375[portably use unsafeDupablePerformIO
376Don Stewart <dons@galois.com>**20080317014444]
377[respect warnings
378Don Stewart <dons@galois.com>**20080317012323]
379[clean up some things
380Don Stewart <dons@galois.com>**20080317011200]
381[Use unsafeDupablePerformIO as a cheaper unsafePerformIO, fixes stack overflow issues in stack squeezing cases
382Don Stewart <dons@galois.com>**20080314002323]
383[Documentations typos (filterByte instead of filterChar).
385[bump version
386Don Stewart <dons@galois.com>**20080222173300]
387[some tests on Ord speed
388Don Stewart <dons@galois.com>**20080221200459]
389[Remove pessimistic INLINE on compareBytes. 5x speedup for some Ord-heavy tests
390Don Stewart <dons@galois.com>**20080221200424]
391[typo in doc. spotted by Johan Tibell
392Don Stewart <dons@galois.com>**20080112222204]
393[todo, find the memcmp length threshodl
394Don Stewart <dons@galois.com>**20080109185141]
395[bump version
396Don Stewart <dons@galois.com>**20080105050338]
397[fix laziness issue with Lazy.lines
398Don Stewart <dons@galois.com>**20080105050319]
399[instance IsString for strict and lazy bytestrings. Use -XOverloadedStrings
400Don Stewart <dons@galois.com>**20080105045950]
401[note that Lazy.lines is too strict, and sketch lazy imlementation
402Don Stewart <dons@galois.com>**20080104000246]
403[bump version
404Don Stewart <dons@galois.com>**20071220073828]
406Don Stewart <dons@galois.com>**20071220073602]
407Patch bundle hash: