Some unicode symbols are not allow in literal characters or strings

[ertai]

main = putChar 'ₖ'

This program is rejected with following error message: lexical error in string/character literal at character '\8342'

There is at least a few other characters with the same issue, for instance this whole string should be accepted: "ₕₖₗₘₙₒₚᵣₛₜᵤᵥₓ"

A related issue is that GHCi do not let me paste these characters either.

[judahj]

GHC requires that source files be encoded in UTF-8. Can you please check whether that's the case for your program? If you're not sure or if that didn't fix the problem, can you please attach the bad program to this ticket?

For ghci: What terminal are you using (e.g. xterm, urxvt, etc.)? Also, please let us know the results of running these commands in that terminal:

echo $TERM
echo $LANG

[igloo]

It works for me:

$ hexdump -C q.hs
00000000  0a 6d 61 69 6e 20 3d 20  70 75 74 43 68 61 72 20  |.main = putChar |
00000010  27 e2 82 96 27 0a 0a                              |'...'..|
00000017
$ ghc -c q.hs
$

[ertai]

I reproduce the same file than igloo and I have the same output for hexdump.

However ghc -c q.hs yields:

q.hs:2:17:

lexical error in string/character literal at character '\8342'

(the GHC version I use is actually 7.0.3, I updated the ticket info)

echo $TERM rxvt-unicode-256color

echo $LANG en_US.UTF-8

[judahj]

I could reproduce the issue with ghc-7.0.3 and ghc-7.0.4.

I looked into this since it seemed to be affecting Haskeline too. The cause (for both problems) was that older versions of GHC support a older version of Unicode:

$ ghc-7.0.3 -e "Data.Char.generalCategory '\8342'"
NotAssigned
$ ghc-7.0.4 -e "Data.Char.generalCategory '\8342'"
NotAssigned
$ ghc-7.2.1 -e "Data.Char.generalCategory '\8342'"
ModifierLetter

So if you want to use those characters, you will probably need to upgrade to ghc-7.2.1.

[ertai]

Ok, thank you.

[igloo]

Yup, I can also reproduce it with 7.0.2 but not 7.2.1.

[joeyh]

Similarly, with ghc 8.2.2 (debian), this is not accepted:

main = putChar '🥖'

That's U+1F956 baguette. ghc says:

lexical error in string/character literal at character '\129366'

My system is fully utf-8 enabled and the original problem character works ok.

Guess this is just lag getting the unicode character tables updated. However, while it seems reasonable for ghc to not let me define a function eg

(🥖) = (</>)

since it doesn't know what kind of symbol baguette is, it seems much less reasonable to not accept any unicode inside a string.

[ulysses4ever]

I can confirm this for 8.4.3 and HEAD.

[ulysses4ever]

I renewed the Unicode tables as described ​here, and this fixed the issue. Merge?