FAQ
Hello,

Can someone suggest a lexer and parser generator like yacc to write a
compiler for a small domain specific language in golang?

Thank,
-Gerald

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Aram Hăvărneanu at Oct 21, 2014 at 9:23 pm
    Yacc. Go has yacc: http://golang.org/cmd/yacc/

    For lexer, go by hand: https://www.youtube.com/watch?v=HxaD_trXwRE

    --
    Aram Hăvărneanu

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
  • Gerald Stanje at Oct 21, 2014 at 11:07 pm
    Thanks. Are there any other options you prefer regarding the parser?

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
  • Adonovan at Oct 22, 2014 at 12:42 am

    On Tuesday, 21 October 2014 19:07:49 UTC-4, [email protected] wrote:
    Thanks. Are there any other options you prefer regarding the parser?

    A recursive descent parser is pretty straightforward. Most simple
    languages are LL(1) or close enough.
    There's a recursive descent parser for Go written in Go here you can use as
    a guide:

       http://golang.org/src/pkg/go/parser/parser.go

    Go isn't LL(1) because infinite lookahead is required to distinguish
    func(x, y, z int) from func(x, y, z), so the parseParameterList function
    has to fix things up afterwards.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
  • Andrewchamberss at Oct 22, 2014 at 3:25 am
    "Go isn't LL(1) because infinite lookahead is required to distinguish
    func(x, y, z int) from func(x, y, z)"

    It is only one token of look ahead required to see the closing brace though?
    On Wednesday, October 22, 2014 1:42:51 PM UTC+13, [email protected] wrote:
    On Tuesday, 21 October 2014 19:07:49 UTC-4, [email protected] wrote:

    Thanks. Are there any other options you prefer regarding the parser?

    A recursive descent parser is pretty straightforward. Most simple
    languages are LL(1) or close enough.
    There's a recursive descent parser for Go written in Go here you can use
    as a guide:

    http://golang.org/src/pkg/go/parser/parser.go

    Go isn't LL(1) because infinite lookahead is required to distinguish
    func(x, y, z int) from func(x, y, z), so the parseParameterList function
    has to fix things up afterwards.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
  • Nigel Tao at Oct 22, 2014 at 3:30 am

    On Wed, Oct 22, 2014 at 2:25 PM, wrote:
    "Go isn't LL(1) because infinite lookahead is required to distinguish
    func(x, y, z int) from func(x, y, z)"

    It is only one token of look ahead required to see the closing brace though?
    The ambiguity is, given only "func(x", is x a type or an argument? The
    former looks like "func(x, y, z)", the latter looks like "func(x, y, z
    T)". Resolving the ambiguity requires infinite lookahead. In
    hindsight, it might have been better to always require a type for
    every argument, so that it's "func(x T, y T, z T)" but Go 1 is what it
    is.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
  • Alan Donovan at Oct 22, 2014 at 3:37 am

    On 21 October 2014 23:30, Nigel Tao wrote:

    The ambiguity is, given only "func(x", is x a type or an argument? The
    former looks like "func(x, y, z)", the latter looks like "func(x, y, z
    T)". Resolving the ambiguity requires infinite lookahead. In
    hindsight, it might have been better to always require a type for
    every argument, so that it's "func(x T, y T, z T)" but Go 1 is what it
    is.

    Corpus delicti. Making the grammar LL(1) is not an end in itself.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
  • Andrewchamberss at Oct 22, 2014 at 3:22 am
    Writing a recursive descent parser and lexer yourself isn't that hard. You
    just need to be aware ahead of time if your language
    has any ambiguities, and how much look ahead is needed to resolve them.

    A hand written parser make it easier to generate custom error message, and
    also custom error recovery.

    That being said, Go currently has a yacc tool which works fine.
    Russ Cox has some parsers in his public repositories using it.

    Here is a C parser - https://code.google.com/p/rsc/source/browse/cc/cc.y

    On Wednesday, October 22, 2014 12:07:49 PM UTC+13, [email protected]
    wrote:
    Thanks. Are there any other options you prefer regarding the parser?
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
  • Zellyn at Oct 22, 2014 at 4:52 am
    I highly recommend Rob Pike's talk, "Lexical Scanning in Go," to, well,
    everyone: http://youtu.be/HxaD_trXwRE

    On Tuesday, October 21, 2014 2:17:58 PM UTC-7, [email protected] wrote:

    Hello,

    Can someone suggest a lexer and parser generator like yacc to write a
    compiler for a small domain specific language in golang?

    Thank,
    -Gerald
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
  • Adonovan at Oct 22, 2014 at 5:35 pm

    On Tuesday, 21 October 2014 20:02:16 UTC-4, Zellyn wrote:
    I highly recommend Rob Pike's talk, "Lexical Scanning in Go," to, well,
    everyone: http://youtu.be/HxaD_trXwRE
    I was confused by Rob's comments at 34'23" to the effect that the
    goroutines created during initialization will not run until after
    initialization is complete. There are no words to that effect in the spec,
    nor does the runtime behave that way, as these example show:

    http://play.golang.org/p/xU7eZD-MTF
    http://play.golang.org/p/vY_cwweocZ

    Indeed, it would be hard to imagine how this could be workable since it
    would in effect establish a restricted dialect of Go for all code
    potentially invoked during initialization.

    What did he mean? Was this an earlier draft of the language?

    cheers
    alan

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
  • Caleb Spare at Oct 22, 2014 at 5:39 pm
    Yes, this was an early restriction that was lifted before Go 1.
    On Wed, Oct 22, 2014 at 10:35 AM, wrote:



    On Tuesday, 21 October 2014 20:02:16 UTC-4, Zellyn wrote:

    I highly recommend Rob Pike's talk, "Lexical Scanning in Go," to, well,
    everyone: http://youtu.be/HxaD_trXwRE
    I was confused by Rob's comments at 34'23" to the effect that the
    goroutines created during initialization will not run until after
    initialization is complete. There are no words to that effect in the spec,
    nor does the runtime behave that way, as these example show:

    http://play.golang.org/p/xU7eZD-MTF
    http://play.golang.org/p/vY_cwweocZ

    Indeed, it would be hard to imagine how this could be workable since it
    would in effect establish a restricted dialect of Go for all code
    potentially invoked during initialization.

    What did he mean? Was this an earlier draft of the language?

    cheers
    alan

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
  • Andrewchamberss at Oct 22, 2014 at 10:28 pm
    Regarding robs talk, I don't necessarily see the benefit in using higher
    order functions to retain lexer state when you can just send tokens in a
    channel and never lose state. You can have a recursive lexer that can park
    state if you use channels.
    On Wednesday, October 22, 2014 1:02:16 PM UTC+13, Zellyn wrote:

    I highly recommend Rob Pike's talk, "Lexical Scanning in Go," to, well,
    everyone: http://youtu.be/HxaD_trXwRE

    On Tuesday, October 21, 2014 2:17:58 PM UTC-7, [email protected] wrote:

    Hello,

    Can someone suggest a lexer and parser generator like yacc to write a
    compiler for a small domain specific language in golang?

    Thank,
    -Gerald
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
  • Gustavo Niemeyer at Oct 23, 2014 at 11:44 am
    You cannot use a channel without goroutines, and as mentioned earlier in
    this thread you could not use goroutines during initialization back then.

    On Wed Oct 22 2014 at 8:28:12 PM wrote:

    Regarding robs talk, I don't necessarily see the benefit in using higher
    order functions to retain lexer state when you can just send tokens in a
    channel and never lose state. You can have a recursive lexer that can park
    state if you use channels.

    On Wednesday, October 22, 2014 1:02:16 PM UTC+13, Zellyn wrote:

    I highly recommend Rob Pike's talk, "Lexical Scanning in Go," to, well,
    everyone: http://youtu.be/HxaD_trXwRE

    On Tuesday, October 21, 2014 2:17:58 PM UTC-7, [email protected] wrote:

    Hello,

    Can someone suggest a lexer and parser generator like yacc to write a
    compiler for a small domain specific language in golang?

    Thank,
    -Gerald
    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
  • Andrewchamberss at Oct 23, 2014 at 9:45 pm
    Oh, sorry. I watched the talk a few months ago and didn't remember that he
    wanted to parse things before Init() finished. Its a cool technique though.
    On Friday, October 24, 2014 12:44:18 AM UTC+13, Gustavo Niemeyer wrote:


    You cannot use a channel without goroutines, and as mentioned earlier in
    this thread you could not use goroutines during initialization back then.


    On Wed Oct 22 2014 at 8:28:12 PM <[email protected] <javascript:>>
    wrote:
    Regarding robs talk, I don't necessarily see the benefit in using higher
    order functions to retain lexer state when you can just send tokens in a
    channel and never lose state. You can have a recursive lexer that can park
    state if you use channels.

    On Wednesday, October 22, 2014 1:02:16 PM UTC+13, Zellyn wrote:

    I highly recommend Rob Pike's talk, "Lexical Scanning in Go," to, well,
    everyone: http://youtu.be/HxaD_trXwRE


    On Tuesday, October 21, 2014 2:17:58 PM UTC-7, [email protected]
    wrote:
    Hello,

    Can someone suggest a lexer and parser generator like yacc to write a
    compiler for a small domain specific language in golang?

    Thank,
    -Gerald
    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to [email protected] <javascript:>.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
  • Gerald Stanje at Oct 23, 2014 at 10:29 pm
    nex looks pretty nice:
    https://crypto.stanford.edu/~blynn/nex/

    Do you know some projects where i can see the interface between parser (yacc for go) - AST and the code generator?

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
  • Gerald Stanje at Oct 23, 2014 at 10:30 pm
    nex looks pretty nice:
    https://crypto.stanford.edu/~blynn/nex/

    Do you know some projects where i can see the interface between parser (yacc for go) - AST and the code generator?

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
  • Silvan Jegen at Oct 23, 2014 at 9:09 am

    On Tuesday, October 21, 2014 11:17:58 PM UTC+2, [email protected] wrote:
    Can someone suggest a lexer and parser generator like yacc to write a
    compiler for a small domain specific language in golang?

    There is also nex as an alternative lexer that generates Go code (it seems
    to be used in bleve
    https://github.com/blevesearch/bleve/blob/master/genparser.sh).

    https://crypto.stanford.edu/~blynn/nex/


    Cheers,

    Silvan

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
  • Jan Mercl at Oct 23, 2014 at 9:16 am

    On Tue, Oct 21, 2014 at 11:17 PM, wrote:
    Can someone suggest a lexer and parser generator like yacc to write a
    compiler for a small domain specific language in golang?
    - go tool yacc[0].
    - golex[1], example [2].

       [0]: http://golang.org/cmd/yacc/
       [1]: https://github.com/cznic/golex
       [2]: https://github.com/cznic/ql/blob/master/scanner.l

    -j

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
  • Tomwilde at Oct 23, 2014 at 11:34 am

    On Tuesday, October 21, 2014 11:17:58 PM UTC+2, [email protected] wrote:
    Can someone suggest a lexer
    I'm gonna be *that* guy and say that you don't need a lexer and that it
    will take you less effort to build an LL(k) recursive-descent parser than
    learn all of yacc's nuances.

    - Tom

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedOct 21, '14 at 9:18p
activeOct 23, '14 at 10:30p
posts19
users11
websitegolang.org

People

Translate

site design / logo © 2023 Grokbase