Opened 4 years ago

Last modified 20 months ago

#4316 new feature request

Interactive "do" notation in GHCi

Reported by: mitar Owned by:
Priority: low Milestone: 7.6.2
Component: GHCi Version: 7.0.3
Keywords: Cc: mmitar@…, haskell.vivian.mcphail@…, dterei
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: None/Unknown Difficulty:
Test Case: Blocked By: #4459
Blocking: Related Tickets:

Description

Enable GHCi to run commands under StateT monad based on IO or maybe even any other monad based on IO. So that you could for example exec a state and "fall" into this monad.

This could be generalized by supporting interactive "do" notation. Currently doing something like:

(flip execStateT) state $ do

raises "Empty 'do' construct" warning, but GHCi could go into interactive mode where it would be possible to write one command after another and it would execute them.

Attachments (2)

T4316-test.patch (93.1 KB) - added by vivian 4 years ago.
Test for multiline interaction #4316
T4316.patch (199.0 KB) - added by vivian 3 years ago.
GHCi multiline command (updated to 7.1.20101104)

Download all attachments as: .zip

Change History (43)

comment:1 Changed 4 years ago by vivian

  • Cc haskell.vivian.mcphail@… added

comment:2 follow-up: Changed 4 years ago by vivian

  • Summary changed from Enable GHCi to run commands under StateT monad based on IO to Interactive "do" notation in GHCi

As far as I know, it is possible to "run commands under StateT monad based on IO."

I am interested in the "supporting interactive 'do' notation" and have changed the ticket name accordingly.

Instead of displaying "Empty 'do' construct", GHCi could provide a prompt for further input, with the prompt being the type of the 'do' monad. A blank lines signals the end of input and the entire interaction is then re-interpreted. It should be possible to nest multiple 'do's.

Prelude> flip evalStateT 1 $ do
StateT Int IO a |            i <- get
StateT Int IO a |            putStrLn $ show i
StateT Int IO a |            return $ i + 1
StateT Int IO a |
1
2
Prelude>

comment:3 in reply to: ↑ 2 ; follow-up: Changed 4 years ago by mitar

Replying to vivian:

I am interested in the "supporting interactive 'do' notation" and have changed the ticket name accordingly.

Great.

A blank lines signals the end of input and the entire interaction is then re-interpreted.

Wouldn't it be better if things would be interpreted immediately?

Like:

Prelude> flip evalStateT 1 $ do
StateT Int IO a |            i <- get
StateT Int IO a |            liftIO $ putStrLn $ show i
1
StateT Int IO a |            return $ i + 1
StateT Int IO a |
2
Prelude>

Similar to the IO monad behavior:

Prelude> i <- return 1
Prelude> putStrLn $ show i
1
Prelude> return $ i + 1
2

And I am not exactly sure empty line would be then necessary as things would be evaluated by line anyway and once there would be return/fail/throw evaluation would stop.

comment:4 in reply to: ↑ 3 ; follow-up: Changed 4 years ago by vivian

Replying to mitar:

Replying to vivian:
Wouldn't it be better if things would be interpreted immediately?

This would require GHCi to somehow know how to run arbitrary monads. This is a problem as each monad has a datatype-specific run function. Some monads return Value,State (StateT), others return Either Error Value (ErrorT), and so on.

Prelude> flip evalStateT 1 $ do
StateT Int IO a |            i <- get
StateT Int IO a |            lift $ putStrLn $ show i
1

somehow insert

flip runStateT <<<missing parameter>>> $ do

before the next line:

StateT Int IO a |            return $ i + 1
StateT Int IO a |
2
Prelude>

The difference is between "prompting for more input," which relies only on the original function call flip evalStateT, a 'syntactic' operation, and multiple evaluations, a 'semantic' operation.

comment:5 in reply to: ↑ 4 Changed 4 years ago by mitar

Replying to vivian:

This would require GHCi to somehow know how to run arbitrary monads.

Hm, no, only how to evaluate do syntax. And if command there would have a side effect (like putStrLn above) it would evaluate there and then.

Some monads return Value,State (StateT), others return Either Error Value (ErrorT), and so on.

Does it really matter what does monad return? Because this gets through evalStateT to GHCi back and GHCi just prints that.

The difference is between "prompting for more input," which relies only on the original function call flip evalStateT, a 'syntactic' operation, and multiple evaluations, a 'semantic' operation.

I agree. One is just easier to input do code, and the other is incremental evaluation. As Haskell allows this (order of evaluation of pure code is unspecified - so it can be also in order how you enter it and order of evaluation of do block is also from top to bottom and it does not matter in which monad you are) I think it should be possible.

But I am a newbie here so I will rest my case here. I just think that it would be great to have more than syntactic improvement. For syntactic part: we could then also have possibility to correct already written lines and similar, before we "commit" the call. In incremental evaluation there would not be possible to correct lines above (as they would be already evaluated) but it would be much more useful for example to test some IO with added state (what was my initial idea all about).

comment:6 follow-up: Changed 4 years ago by igloo

Not exactly the same as is being suggested, but this works in 7.0:

Prelude Control.Monad.State> :{
Prelude Control.Monad.State| flip evalStateT 1 $ do
Prelude Control.Monad.State|             i <- get
Prelude Control.Monad.State|             lift $ putStrLn $ show i
Prelude Control.Monad.State|             return $ i + 1
Prelude Control.Monad.State| :}
Loading package mtl-1.1.1.0 ... linking ... done.
1
2

comment:7 in reply to: ↑ 6 ; follow-up: Changed 4 years ago by vivian

Replying to igloo:

Not exactly the same as is being suggested, but this works in 7.0:

Prelude Control.Monad.State> :{

Personally, I could live with typing :{ before a multiline GHCi expression. Especially since (see above) I advocated the syntactic 'batch' versus semantic 'line at a time' reading.

However, I am working on libraries that provide matlab-like functionality at the !programming-language interpreter! level of GHCi. Multiline "do" statements are part of the Haskell specification (which the odd mathematical _user_ might deign to peruse), whereas this "observe formatting" GHCi meta-":" command is not standard.

There is a tension here between "users" and "programmers." The beauty of Haskell includes a succinct and elegant syntax. So much so, that I believe that we can blur the line when using GHCi between an applied mathematics environment and a programming environment.

I'd like to see Haskell used by non-programmer mathematicians.

comment:8 in reply to: ↑ 7 Changed 4 years ago by mitar

Replying to vivian:

I'd like to see Haskell used by non-programmer mathematicians.

Then we definitely need line by line execution of commands. ;-)

People who are used to imperative/procedural execution of commands will not be satisfied with just being able to enter multiple lines. They will want to see result for each of those lines.

comment:9 Changed 4 years ago by igloo

  • Milestone set to 7.2.1

comment:10 Changed 4 years ago by vivian

  • Owner set to vivian

comment:11 Changed 4 years ago by vivian

  • Status changed from new to patch

I have a working multiline do patch,

Control.Monad.State> flip evalStateT 0 $ do
Control.Monad.State| i <- get
Control.Monad.State| lift $ do
Control.Monad.State|   putStrLn "Hello World!"
Control.Monad.State|   print i
Control.Monad.State|
"Hello World!"
0
Control.Monad.State>

In order to provide a truly interactive session, the interpreter needs to know how to (i) run a monad, (ii) pass initial arguments to a monad, and (iii) take the output of running a monad and feed it to the next line.

Do we manipulate source (fragile) or manipulate HValues? But then we need to run the typechecker at runtime to make sure we make use the correct instance of overloaded (GHCi level) plumbing functions.

You also can't just evaluate "lines up to present." What happens when the last line is currently an assignment?

Control.Monad.State> flip evalStateT 11 $ do
Control.Monad.State|   f <- get

This patch does not attempt to type the partial type and use the monad type as prompt.

This patch provides multiline 'do's. Perhaps a new ticket for "interactive" 'do' could be opened if thee abovementioned points can be addressed.

comment:12 follow-up: Changed 4 years ago by simonmar

Just a thought: couldn't we use the "altnerative layout rule" to detect when a line ends without closing all of its layout contexts, and enter a multi-line mode which ends with a blank line, or when all the layout contexts have been closed?

comment:13 in reply to: ↑ 12 Changed 4 years ago by vivian

Replying to simonmar:

Just a thought: couldn't we use the "altnerative layout rule" to detect when a line ends without closing all of its layout contexts, and enter a multi-line mode which ends with a blank line, or when all the layout contexts have been closed?

I do not know how to 'hook' into the lexer to check whether a layout rule is open. Lexer.x, lexer, about line 1961, is where layout is implemented.

A statement "let foo = bar" is a valid expression at the GHCi command line, but could also be an open layout context expecting "in baz foo" to close the context on the next line.

A trailing do on the command line generates an "Empty 'do' construct" error, not an error about layout, which is why I can not see (yet) how to get the layout context information from the parser.

Perhaps simonmar could provide me with a pointer as to how to implement his suggestion?

comment:14 follow-up: Changed 4 years ago by simonmar

The basic idea is this: when we get a line of input from the user, we call the lexer in "alternative layout rule mode" to read tokens up to the end of the line, and then inspect the state of the lexer at that point. The alternative layout rule tracks layout contexts and brackets, so we can tell whether the current line is finished or not. Then we read the next line, start lexing it using the state from the previous line, and so on. If we get a blank line, finish and parse the input as usual.

I haven't thought through all the details here, but I think the idea is very cool: GHC will be able to tell automatically whether the input is complete or not by counting brackets. One disadvantage is, as you point out, a let binding would require a double-return to terminate it. So perhaps this automatic multi-line input could be optional, but I imagine people would get used to it pretty quickly.

comment:15 Changed 4 years ago by vivian

Please disregard the patches currently attached. I'll implement simonmar's suggestion.

comment:16 Changed 4 years ago by vivian

How does implementing simonmar's suggestion impact upon #3984? It seems that it renders it obsolete. Should the patch include removing the :{ }: construct?

comment:17 follow-up: Changed 4 years ago by simonmar

I don't know how well Haskeline will cope with multi-line input. Ideally you want Haskeline to be able to navigate and edit the complete multi-line expression.

Also I'm not sure how layout should behave with the prompt, because the first line is already indented: should subsequent lines be indented by the width of the prompt, or should there be a special multi-line continuation prompt?

comment:18 in reply to: ↑ 17 Changed 4 years ago by igloo

Replying to simonmar:

Also I'm not sure how layout should behave with the prompt, because the first line is already indented: should subsequent lines be indented by the width of the prompt, or should there be a special multi-line continuation prompt?

multi-line continuation prompt sounds best to me.

comment:19 in reply to: ↑ 14 ; follow-up: Changed 4 years ago by vivian

Replying to simonmar:

The basic idea is this: when we get a line of input from the user, we call the lexer in "alternative layout rule mode" to read tokens up to the end of the line, and then inspect the state of the lexer at that point.

I add Opt_AlternativeLayoutRule as a dynamic flag. I'm checking to see if the list of ALRContexts is empty and whether an opening curly brace is expected but this does not appear to be sufficient. What in the "state of the lexer" should I be checking?

comment:20 Changed 4 years ago by igloo

  • Owner vivian deleted
  • Status changed from patch to new

comment:21 Changed 4 years ago by vivian

  • Owner set to vivian

comment:22 in reply to: ↑ 19 Changed 4 years ago by simonmar

Replying to vivian:

I add Opt_AlternativeLayoutRule as a dynamic flag. I'm checking to see if the list of ALRContexts is empty and whether an opening curly brace is expected but this does not appear to be sufficient. What in the "state of the lexer" should I be checking?

I don't know. In what way is it not sufficient, can you give an example?

comment:23 Changed 4 years ago by vivian

I have a working patch that uses simonmar's idea of checking the lexer state in alternative layout mode.

There are now a couple of questions with respect to the GHCi documentation and #3984.

  • First, any open context (basically 'do' and 'let') will trigger multiline mode which is terminated by an empty (cf. blank) line. We can also define something with a 'let':
    Prelude> let foo = "Hello"
    Prelude|
    Prelude> foo
    "Hello"
    

which does not have an associated 'in.' In all these cases, an extra empty line is required. Where in the documentation should we put this feature?

  • #3984 uses :{ and }: to open and close a 'multiline' mode. I think this is now obsolete except if we want to have a 'where' clause.

comment:24 Changed 4 years ago by vivian

  • Status changed from new to patch
  • Summary changed from Interactive "do" notation in GHCi to Multiline commands in GHCi

Patch implements multiline commands in GHCi. (do, case, let).

Patch includes documentation. The testsuite patch modifies a number of GHCi tests that now require an additional blank line after case and let.

#3984 was not modified.

It would be nice to see this in 7.0.

The history is not modified as Haskeline would require modification to replace the previous n lines with one multiline command. Also, it might be better to allow the user to replay each line at a time, so that an error on one line can be changed. I suppose this depends on whether history is used to edit commands or just repeat them.

comment:25 Changed 4 years ago by judahj

You might be able to use the functionality in System.Console.Haskeline.History to merge entered commands into one multiline command. I'm not sure what would happen if a line in the history contained a newline, however.

Please feel free to add tickets for any desired functionality to http://trac.haskell.org/haskeline. Also note the following related ticket:
http://trac.haskell.org/haskeline/ticket/112

comment:26 follow-up: Changed 4 years ago by simonmar

Nice work! And the patch is quite small too.

For 7.0 I'd like to make this optional, since it does change the behaviour and may break people's scripts etc., if it turns out that people prefer multi-line mode then we can make it the default in 7.2.

Could we make it optional, enabled with :set +m for multiline-input?

comment:27 in reply to: ↑ 26 ; follow-up: Changed 4 years ago by vivian

Replying to simonmar:

Could we make it optional, enabled with :set +m for multiline-input?

Sure! maybe :set multiline as the text? (allowing unique prefixes)

Please disregard patch.

comment:28 in reply to: ↑ 27 ; follow-up: Changed 4 years ago by simonmar

Replying to vivian:

Sure! maybe :set multiline as the text? (allowing unique prefixes)

We already have a few of the single-letter options, like :set +s, so I think :set +m is the most consistent choice.

comment:29 in reply to: ↑ 28 Changed 4 years ago by vivian

Replying to simonmar:

Replying to vivian:
We already have a few of the single-letter options, like :set +s, so I think :set +m is the most consistent choice.

Of course. I was confounding :set <option> with :multiline. :set +m it is.

Changed 4 years ago by vivian

Test for multiline interaction #4316

comment:30 Changed 4 years ago by vivian

Multiline mode is now optional and set with :set +m.

The patch passes the validate script.

Please review.

comment:31 follow-up: Changed 4 years ago by mitar

Great work. But so my initial idea was lost. ;-(

I really do not understand - does everybody know exactly in advance the whole body of do command they want to execute without having to see intermediate results? Because this is why I like to use interactive GHCi, to be able to do command by command.

For example, what I have at hand is a Haskell library for robot control. It is based on StateT monad as it requires some internal state. And it would be great if I could run that StateT monad and then issue one command after the other. And see what happens. Then another command, and another...

In my case it is simply impossible to write whole body in advance and then execute it.

While, for example, in Matlab this is really a joy to do. You can your own interactive prompt for commanding your robot.

comment:32 in reply to: ↑ 31 Changed 4 years ago by vivian

Replying to mitar:

Great work. But so my initial idea was lost. ;-(

I think that your original idea is a good idea. I have implemented this first because it is an order of magnitude simpler. I personally do not yet know how to implement your idea in GHCi.

Control.Monad.State> flip evalStateT 10 $ do
Control.Monad.State| i <- get
Control.Monad.State| lift $ print i
10
Control.Monad.State| put 11
Control.Monad.State| j <- get
Control.Monad.State| lift $ print (i+j)
21
Control.Monad.State| 

I would have to know how to evaluate HValues, how to type them, and how to pass them between lines in our monads. That is, we have a polymorphic (>>=) to chain HValues. And we also have to somehow know if we need to display output (is this the IO monad? did that computation produce a side-effect? can I/should I display a value?).

This is not at all impossible, just more difficult than the current multiline commands. Please note that when I first submitted the patch I mentioned the possibility of leaving the ticket open or creating a new one.

comment:33 Changed 4 years ago by mitar

I know. You are doing a great job. Thanks. And I am sad that I do not know enough about GHCi internals to be able to help you with how to solve the interactivity problem. I just wanted to present another use case where just multiline is not enough.

Thanks for everything.

Changed 3 years ago by vivian

GHCi multiline command (updated to 7.1.20101104)

comment:34 Changed 3 years ago by vivian

Please ignore T1363.patch, wrong file - can it be removed?

T4316.patch updated to reflect recent changes, please review.

comment:35 Changed 3 years ago by simonmar

  • Owner vivian deleted
  • Status changed from patch to new
  • Summary changed from Multiline commands in GHCi to Interactive "do" notation in GHCi

Patches pushed, thanks for the contribution!

Thu Nov  4 22:13:08 PDT 2010  Vivian McPhail <haskell.vivian.mcphail@gmail.com>
  * multiline commands in GHCi #4316
  This patch adds support for multiline commands in GHCi.
  
  The first line of input is lexed.  If there is an active
  layout context once the lexer reaches the end of file, the
  user is prompted for more input.
  
  Multiline input is exited by an empty line and can be escaped 
  with a user interrupt.
  
  Multiline mode is toggled with `:set +m`

Wed Jan  5 07:45:48 PST 2011  Simon Marlow <marlowsd@gmail.com>
  * fix up multi-line GHCi patch (#4316)

Thu Jan  6 01:31:52 PST 2011  Simon Marlow <marlowsd@gmail.com>
  * fix markup

Leaving the ticket open, and changing the summary back to what it was previously.

comment:36 Changed 3 years ago by vivian

  • Blocked By 4459 added

Now that this ticket is back to mitar's original request. I think that this feature request could be implemented (creating a meta-interpreter) on the back of #4459.

By introducing bind at the interpreter level we can invoke side-effects on a line-by-line basis within any monad construct.

comment:37 follow-up: Changed 3 years ago by daniel.is.fischer

  • Version changed from 6.12.3 to 7.0.3

The test is in 7.0.3's testsuite, but the patch doesn't seem to be in:

Actual stderr output differs from expected:
--- /dev/null	2011-04-02 10:56:44.408004588 +0200
+++ ./ghci/scripts/T4316.run.stderr.normalised	2011-04-02 17:00:07.000000000 +0200
@@ -0,0 +1,20 @@
+
+<interactive>:1:22: Empty 'do' construct
+
+<interactive>:1:6:
+    No instance for (MonadState t0 IO)
+      arising from a use of `get'
+    Possible fix: add an instance declaration for (MonadState t0 IO)
+    In a stmt of an interactive GHCi command: i <- get
+
+<interactive>:1:8: Empty 'do' construct
+
+<interactive>:1:7: Not in scope: `i'
+
+<interactive>:1:1: parse error on input `in'
+
+<interactive>:1:10: parse error (possibly incorrect indentation)
+
+<interactive>:1:7: parse error on input `->'
+
+<interactive>:1:7: parse error on input `->'
*** unexpected failure for T4316(ghci)

Trying :set +m in ghci results in unknown option: 'm'.

comment:38 in reply to: ↑ 37 Changed 3 years ago by simonmar

Replying to daniel.is.fischer:

The test is in 7.0.3's testsuite, but the patch doesn't seem to be in:

Just a testsuite bug, I don't think the testsuite was fully cleaned up in the stable branch prior to the release.

comment:39 Changed 2 years ago by igloo

  • Milestone changed from 7.4.1 to 7.6.1
  • Priority changed from normal to low

comment:40 Changed 2 years ago by dterei

  • Cc dterei added

comment:41 Changed 20 months ago by igloo

  • Milestone changed from 7.6.1 to 7.6.2
Note: See TracTickets for help on using tickets.