Ticket #3808: 3808.dpatch

File 3808.dpatch, 16.6 KB (added by simonmar, 4 years ago)
Line 
12 patches for repository http://code.haskell.org/~dons/code/bytestring:
2
3Mon Mar 22 15:39:14 GMT 2010  Simon Marlow <marlowsd@gmail.com>
4  * hGetContents: use hGet instead of hGetNonBlocking + hWaitForInput + hIsEOF
5 
6  Not only is this cleaner, but it fixes a problem with read binary data
7  over a pipe, see
8    http://hackage.haskell.org/trac/ghc/ticket/3808
9 
10  The problem is that bytestring normally works fine when the Handle is
11  not in binary mode, because it uses hGetBuf/hPutBuf which bypass the
12  encoding layer.  That is, except in this one particular case:
13  hWaitForInput might do some decoding, because it has to determine
14  whether there are any characters (not bytes) available to be read.
15 
16  I imagine hGetNonBlocking was used due to concerns that hGet might
17  block if there is insufficient data, but that's not the case; hGet
18  returns a short read if it would otherwise block, and only blocks if
19  there is no data to read.
20
21Mon Mar 22 15:41:03 GMT 2010  Simon Marlow <marlowsd@gmail.com>
22  * update docs for hGet, hGetNonBlocking
23
24New patches:
25
26[hGetContents: use hGet instead of hGetNonBlocking + hWaitForInput + hIsEOF
27Simon Marlow <marlowsd@gmail.com>**20100322153914
28 Ignore-this: 43bac03fc6d7de74564f5cd608ed9f17
29 
30 Not only is this cleaner, but it fixes a problem with read binary data
31 over a pipe, see
32   http://hackage.haskell.org/trac/ghc/ticket/3808
33 
34 The problem is that bytestring normally works fine when the Handle is
35 not in binary mode, because it uses hGetBuf/hPutBuf which bypass the
36 encoding layer.  That is, except in this one particular case:
37 hWaitForInput might do some decoding, because it has to determine
38 whether there are any characters (not bytes) available to be read.
39 
40 I imagine hGetNonBlocking was used due to concerns that hGet might
41 block if there is insufficient data, but that's not the case; hGet
42 returns a short read if it would otherwise block, and only blocks if
43 there is no data to read.
44] {
45hunk ./Data/ByteString/Lazy.hs 216
46 import Data.Word                (Word8)
47 import Data.Int                 (Int64)
48 import System.IO                (Handle,stdin,stdout,openBinaryFile,IOMode(..)
49-                                ,hClose,hWaitForInput,hIsEOF)
50-import System.IO.Error          (mkIOError, ioError, illegalOperationErrorType)
51+                                ,hClose)
52+import System.IO.Error          (mkIOError, illegalOperationErrorType)
53 import System.IO.Unsafe
54 #ifndef __NHC__
55 import Control.Exception        (bracket)
56hunk ./Data/ByteString/Lazy.hs 1177
57     lazyRead = unsafeInterleaveIO loop
58 
59     loop = do
60-        c <- S.hGetNonBlocking h k
61-        --TODO: I think this should distinguish EOF from no data available
62-        -- the underlying POSIX call makes this distincion, returning either
63-        -- 0 or EAGAIN
64+        c <- S.hGet h k -- only blocks if there is no data available
65         if S.null c
66hunk ./Data/ByteString/Lazy.hs 1179
67-          then do eof <- hIsEOF h
68-                  if eof then hClose h >> return Empty
69-                         else hWaitForInput h (-1)
70-                           >> loop
71+          then hClose h >> return Empty
72           else do cs <- lazyRead
73                   return (Chunk c cs)
74 
75}
76[update docs for hGet, hGetNonBlocking
77Simon Marlow <marlowsd@gmail.com>**20100322154103
78 Ignore-this: b7b1986f438421615985a4587f97a58c
79] {
80hunk ./Data/ByteString.hs 1914
81 -- is far more efficient than reading the characters into a 'String'
82 -- and then using 'pack'. First argument is the Handle to read from,
83 -- and the second is the number of bytes to read. It returns the bytes
84--- read, up to n, or EOF.
85+-- read, up to n, or 'null' if EOF has been reached.
86+--
87+-- If there is any data to read, then 'hGet' will not block, instead
88+-- it will return whatever data is available without blocking.  It
89+-- only blocks if there is no data available to read.
90 --
91 -- 'hGet' is implemented in terms of 'hGetBuf'.
92 --
93hunk ./Data/ByteString.hs 1931
94     | i == 0    = return empty
95     | otherwise = illegalBufferSize h "hGet" i
96 
97--- | hGetNonBlocking is identical to 'hGet', except that it will never block
98--- waiting for data to become available, instead it returns only whatever data
99--- is available.
100+-- | hGetNonBlocking is identical to 'hGet', except that it will never
101+-- block waiting for data to become available.  If there is no data
102+-- available to be read, 'hGetNonBlocking' returns 'null'.
103 --
104 hGetNonBlocking :: Handle -> Int -> IO ByteString
105 #if defined(__GLASGOW_HASKELL__)
106}
107
108Context:
109
110[Check for negative lengths in packCStringLen
111Duncan Coutts <duncan@haskell.org>**20090915234111
112 Ignore-this: a388aeeee7cd0b6df06c61d88a97588a
113]
114[Check for negative lengths in hGet and hGetNonBlocking
115Duncan Coutts <duncan@haskell.org>**20090915234011
116 Ignore-this: c9e33f05ef3cabfe12c1d8fb509f4726
117 See ghc ticket #3514
118]
119[Fix elemIndices and split
120Duncan Coutts <duncan@haskell.org>**20090912203950
121 Ignore-this: 69ae945462b5638c99cc1824ed329e90
122 They were using inlinePerformIO to lazily delay IO actions within the
123 scope of a withForeignPtr. Thus the ptr was still in use after the
124 ForeignPtr went out of scope. This lead to dangling pointers, incorrect
125 results and segfaults. Nasty. See ghc tickets #3486 and #3487.
126 Audited the rest of the code base for the same anit-pattern.
127 Thanks to nwn and people on the haskell-jp mailing list for reporting
128 the bug with nice test cases and also to Ian, Bertram and Don for
129 diagnosing the source of the problem.
130]
131[Update "tested-with" list
132Duncan Coutts <duncan@haskell.org>**20090823195516
133 Ignore-this: 873e64dd5c978cc12751d3b404e3619a
134 It builds with ghc-6.4 through to 6.11 (23/08/09)
135 It builds with no warnings with ghc 6.8, 6.10 and 6.11.
136]
137[Clean up the language extension pragmas
138Duncan Coutts <duncan@haskell.org>**20090823194920
139 Ignore-this: c7fb9daf30b639d0bee909fd396de241
140 We cannot actually specify them correctly, see ghc ticket #3457.
141 We can at least simplify things and make it clear it does not work,
142 rather than having misleading code that looks like it might work.
143]
144[Fix import warnings
145Duncan Coutts <duncan@haskell.org>**20090823194733
146 Ignore-this: 2db4b086627d428de7add92566df9eb
147 Patches merged from ghc's fork:
148   Tue Jul  7 12:58:17 BST 2009  Ian Lynagh <igloo@earth.li>
149     * Remove unused imports
150 
151   Tue Jul  7 14:35:56 BST 2009  Ian Lynagh <igloo@earth.li>
152     * Fix unused import warnings
153 
154   Wed Jul 22 13:56:04 BST 2009  Simon Marlow <marlowsd@gmail.com>
155     * fix compilation with GHC 6.10
156]
157[Update for new IO library
158Duncan Coutts <duncan@haskell.org>**20090823175302
159 Ignore-this: 539da275e4528629baa6d9f66c2f3335
160 Merge of patch from ghc's fork:
161   Thu Jun 11 15:09:37 BST 2009  Simon Marlow <marlowsd@gmail.com>
162     * Update for new IO library
163]
164[Fix "cabal check" warnings
165Duncan Coutts <duncan@haskell.org>**20090823145438
166 Ignore-this: f6a17b693d9216b840bf01cf55a2fa7d
167 Merge of patch from ghc's fork:
168   Tue Aug 11 22:58:58 BST 2009  Ian Lynagh <igloo@earth.li>
169     * Fix "Cabal check" warnings
170]
171[Remove INLINE pragmas on recursive functions
172Duncan Coutts <duncan@haskell.org>**20090823145127
173 Ignore-this: 40b0b5e7a1c274b78231517736ab74c7
174 Merge of patch from ghc's fork:
175   Fri Dec  5 17:04:52 GMT 2008  simonpj@microsoft.com
176     * Remove INLINE pragmas on recursive functions
177 
178     GHC now complains about INLINE pragmas on recursive functions,
179     and they were no-ops before in any case
180]
181[Fix some "warn-unused-do-bind" warnings where we want to ignore the value
182Ian Lynagh <igloo@earth.li>**20090710202441]
183[Hide the .Internal modules in the haddock docs
184Duncan Coutts <duncan@haskell.org>**20090702000819
185 This should stop users accidentally using the semi-public
186 modules while still leaving them available for extension
187 packages. Also means that references to 'ByteString' go to
188 a sensible place rather than the .Internal modules.
189]
190[Fix docs for unsafePackAddressLen, it doesn't need null-termination
191Duncan Coutts <duncan@haskell.org>**20090629192811]
192[Make docs for fromForeignPtr point to suitable public alternatives
193Duncan Coutts <duncan@haskell.org>**20090629191336]
194[Improve the top-level documentation for the Unsafe and Internal modules
195Duncan Coutts <duncan@haskell.org>**20090629191017
196 Clarify the purpose and properties of the functions in the two modules.
197 In particular try to disuade people from using the Internal functions
198 unnecessarily.
199]
200[Use comma separated lists in the LANGUAGE pragmas
201Duncan Coutts <duncan@haskell.org>**20090629190950
202 Haddock complains.
203]
204[bump versoin
205Don Stewart <dons@galois.com>**20090108182815]
206[Add a test for the laziness of readInt and readInteger over lazy ByteStrings
207Bryan O'Sullivan <bos@serpentine.com>**20081231070009]
208[Fix strictness bugs in readInt and readInteger.
209Bryan O'Sullivan <bos@serpentine.com>**20081231064853]
210[Add TODO item to eliminate -fno-warn-orphans
211Duncan Coutts <duncan@haskell.org>**20081101190200
212 Orphan instances in libraries are not a good thing.
213]
214[Make everything build with ghc-6.6 and 6.4
215Duncan Coutts <duncan@haskell.org>**20081101190029
216 No code changes, just cpp for new LANGUAGE pragmas
217 and not using a RULE with a funky LHS for ghc-6.4 which didn't allow it.
218]
219[avoid import renaming errors in nhc98
220Malcolm.Wallace@cs.york.ac.uk**20081008092026]
221[Import fix for nhc98
222Duncan Coutts <duncan@haskell.org>**20081006211934]
223[Bump version to 0.9.1.4
224Duncan Coutts <duncan@haskell.org>**20081006210002]
225[Drop dependency on syb with base 4
226Duncan Coutts <duncan@haskell.org>**20081006205908]
227[fix import of Control.Exception for nhc98
228Malcolm.Wallace@cs.york.ac.uk**20081006102248]
229[TAG 0.9.1.3
230Duncan Coutts <duncan@haskell.org>**20081004225748]
231[Bump version to 0.9.1.3
232Duncan Coutts <duncan@haskell.org>**20081004225716]
233[Whitespace changes to the package description
234Duncan Coutts <duncan@haskell.org>**20081004223529]
235[Drop -fglasgow-exts, use LANGUAGE pragmas
236Duncan Coutts <duncan@haskell.org>**20081004223518]
237[Build with both base-3 and 4
238Duncan Coutts <duncan@haskell.org>**20081004221008]
239[Drop unrecognised and unnecessary pragma
240Duncan Coutts <duncan@haskell.org>**20081004220034]
241[stupid ghc-prim
242Don Stewart <dons@galois.com>**20080825214146]
243[Only a minor version bump
244Don Stewart <dons@galois.com>**20080825205906]
245[clarify comments on hGet and EOF
246Don Stewart <dons@galois.com>**20080809233325]
247[typo in comment
248Don Stewart <dons@galois.com>**20080716181103]
249[notes, and undo -fcpr-off
250Don Stewart <dons@galois.com>**20080520001839]
251[clean up flags. building on 6.9
252Don Stewart <dons@galois.com>**20080519215317]
253[more aggressive inlining on lazy bytestring readInt. Performance wins for sum-file
254Don Stewart <dons@galois.com>**20080519215149]
255[make hackage happy
256Don Stewart <dons@galois.com>**20080420211741]
257[Point to HPC coverage data
258Don Stewart <dons@galois.com>**20080420210524]
259[extra-source-files
260Don Stewart <dons@galois.com>**20080420205553]
261[Add properties for rewrite rules, compiled sans -fhpc
262Don Stewart <dons@galois.com>**20080419025720
263 
264 Following an idea of Andy Gill's, compile just some rewrite rules code
265 without -fhpc, link it into a hpc app, and get coverage / rule firing
266 data for code run by rewrite rules. Library now 93% covered by QuickCheck.
267 
268]
269[fix warnings
270Don Stewart <dons@galois.com>**20080417232307]
271[Remove old fusion mechanism. Data.ByteString.Fusion is a place holder
272Don Stewart <dons@galois.com>**20080417230308
273 for streams now.
274 
275 And while we're at it, reimplement mapAccumL/R and scanl
276 directly. Remove mapIndexed, as its trivial to write with mapAccumL.
277 
278]
279[more tests
280Don Stewart <dons@galois.com>**20080417162926]
281[tune deprecation string
282Don Stewart <dons@galois.com>**20080417162305]
283[some more tests
284Don Stewart <dons@galois.com>**20080417011226]
285[pointless filter rule
286Don Stewart <dons@galois.com>**20080417011025]
287[more QuickCheck properties, 88% covered now
288Don Stewart <dons@galois.com>**20080417010205]
289[Use -fglasgow-exts to re-enable rewrite rules
290Don Stewart <dons@galois.com>**20080417010144]
291[Remove obscure filter (==) rewrite rule
292Don Stewart <dons@galois.com>**20080417010057]
293[add a cunning test for packMallocCString
294Don Stewart <dons@galois.com>**20080416171625]
295[typos in comment
296Don Stewart <dons@galois.com>**20080416161143]
297[more testing
298Don Stewart <dons@galois.com>**20080414065202]
299[normalise rule names
300Don Stewart <dons@galois.com>**20080414064537]
301[More testing. 80% of bytestring is now tested with QuickCheck
302Don Stewart <dons@galois.com>**20080413221600]
303[tweaks
304Don Stewart <dons@galois.com>**20080413190445]
305[More coverage
306Don Stewart <dons@galois.com>**20080413190419]
307[better clean target
308Don Stewart <dons@galois.com>**20080413135043]
309[tweak makefile
310Don Stewart <dons@galois.com>**20080413134800]
311[properties for breakSubstring
312Don Stewart <dons@galois.com>**20080413134635]
313[Add breakSubstring, split a bytestring on another bytestring. More idiomatic approach than findSubstring
314Don Stewart <dons@galois.com>**20080413134332]
315[faster findSubstrings
316Don Stewart <dons@galois.com>**20080413103033]
317[more tests
318Don Stewart <dons@galois.com>**20080413103014]
319[tweaks
320Don Stewart <dons@galois.com>**20080413102931]
321[no longer depend on array
322Don Stewart <dons@galois.com>**20080413102910]
323[require Instances
324Don Stewart <dons@galois.com>**20080413102859]
325[add cheaper hpc test target
326Don Stewart <dons@galois.com>**20080413090044]
327[comments
328Don Stewart <dons@galois.com>**20080413090026]
329[use "ByteString" prefix for rules
330Don Stewart <dons@galois.com>**20080413090006]
331[return failures
332Don Stewart <dons@galois.com>**20080413085955]
333[actually fail if tests don't pass
334Don Stewart <dons@galois.com>**20080413084834]
335[more test coverage
336Don Stewart <dons@galois.com>**20080412192702]
337[improve performance of findSubstrings on small strings
338Don Stewart <dons@galois.com>**20080412192633]
339[Use -fno-ignore-asserts when testing
340Don Stewart <dons@galois.com>**20080411160639]
341[coverage for Monoid instance
342Don Stewart <dons@galois.com>**20080411154716]
343[reuse internal invariant test
344Don Stewart <dons@galois.com>**20080411154700]
345[ghc uses -XUnliftedFFITypes -XMagicHash -XUnboxedTuples -XDeriveDataTypeable
346Don Stewart <dons@galois.com>**20080411154632]
347[use appropriate -X flags
348Don Stewart <dons@galois.com>**20080411144604]
349[bump version number
350Don Stewart <dons@galois.com>**20080411114047]
351[note how to run testsuite with hugs
352Don Stewart <dons@galois.com>**20080411114030]
353[document second argument of hGet
354Don Stewart <dons@galois.com>**20080406034554]
355[normalise rewrite rule names
356Don Stewart <dons@galois.com>**20080321165326]
357[tweaks
358Don Stewart <dons@galois.com>**20080321005940]
359[standardise rule names with   "ByteString" prefix
360Don Stewart <dons@galois.com>**20080320204043]
361[standardise rules firing with prefix "ByteString"
362Don Stewart <dons@galois.com>**20080320170005]
363[dead code
364Don Stewart <dons@galois.com>**20080320165557]
365[Document IsString ByteString, and how to enable it
366Don Stewart <dons@galois.com>**20080320165445]
367[remove deprecated 'join' -- people should have transitioned to intercalate in the past 12 months
368Don Stewart <dons@galois.com>**20080320165136]
369[more tests
370Don Stewart <dons@galois.com>**20080318171700]
371[Add rewrite rule for Char8, matching the Word8 one, specialising break (==x)
372Don Stewart <dons@galois.com>**20080318171256]
373[erroneos inline [1] on Char8.dropWhile
374Don Stewart <dons@galois.com>**20080318170525]
375[portably use unsafeDupablePerformIO
376Don Stewart <dons@galois.com>**20080317014444]
377[respect warnings
378Don Stewart <dons@galois.com>**20080317012323]
379[clean up some things
380Don Stewart <dons@galois.com>**20080317011200]
381[Use unsafeDupablePerformIO as a cheaper unsafePerformIO, fixes stack overflow issues in stack squeezing cases
382Don Stewart <dons@galois.com>**20080314002323]
383[Documentations typos (filterByte instead of filterChar).
384nicolas.pouillard@gmail.com**20080311011358]
385[bump version
386Don Stewart <dons@galois.com>**20080222173300]
387[some tests on Ord speed
388Don Stewart <dons@galois.com>**20080221200459]
389[Remove pessimistic INLINE on compareBytes. 5x speedup for some Ord-heavy tests
390Don Stewart <dons@galois.com>**20080221200424]
391[typo in doc. spotted by Johan Tibell
392Don Stewart <dons@galois.com>**20080112222204]
393[todo, find the memcmp length threshodl
394Don Stewart <dons@galois.com>**20080109185141]
395[bump version
396Don Stewart <dons@galois.com>**20080105050338]
397[fix laziness issue with Lazy.lines
398Don Stewart <dons@galois.com>**20080105050319]
399[instance IsString for strict and lazy bytestrings. Use -XOverloadedStrings
400Don Stewart <dons@galois.com>**20080105045950]
401[note that Lazy.lines is too strict, and sketch lazy imlementation
402Don Stewart <dons@galois.com>**20080104000246]
403[bump version
404Don Stewart <dons@galois.com>**20071220073828]
405[TAG 0.9.0.3
406Don Stewart <dons@galois.com>**20071220073602]
407Patch bundle hash:
4089722266b6ca5842f395d3011b226eca6d5741e08