FAQ
In a toy project I am working on, I am trying to generate Go code
automatically to cut down on the amount of boiler plate I have to write. I
was looking at the packages under go/ in the standard library, but a lot of
them seem specific to taking existing code and interpreting it rather than
the other direction. Specifically, a lot of the API depends on having a
token.FileSet, which assumes the file already exist. Since I am making it
from scratch, I don't have a file or line number or token positions.

Is there a proper way to generate code? Looking at golang/protobuf, it
seems to print out raw strings currently too. Other projects like protobuf
would likely benefit from being able to build code using standard packages.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Jan Mercl at May 7, 2016 at 7:07 pm

    On Sat, May 7, 2016 at 7:30 PM Carl Mastrangelo wrote:

    Is there a proper way to generate code?
    Go source code is just text, so the fmt package and common string
    operations is all you need. The go/format package could be used to make the
    result look good.
    --

    -j

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Carl Mastrangelo at May 7, 2016 at 7:20 pm
    That is what I ended up doing. My first attempt did use templates, but it
    was very difficult to get the formatting right. For example, getting
    indentation and new lines in the right place does work with templates, even
    with the new {{- foo -}} directives. I ended up copying protobuf's
    approach of using a tab writer, which makes for a more visually pleasing
    generated indentation, but uglier generator.

    One of the reasons I was looking at go/ast in the first place is that it is
    hard to tell if an import is used or not until the whole source is
    generated. Writing code out with fmt has problems too, since now I have
    strconv.Quote all over the place. Using "fmt" also has problems when used
    with bytes.Buffer as the temporary destination since you can't really
    "unwrite" a rune or set of bytes. Having a full syntax tree would make
    these problems easier to solve.
    On Saturday, May 7, 2016 at 12:08:17 PM UTC-7, Jan Mercl wrote:

    On Sat, May 7, 2016 at 7:30 PM Carl Mastrangelo <carl.mas...@gmail.com
    <javascript:>> wrote:
    Is there a proper way to generate code?
    Go source code is just text, so the fmt package and common string
    operations is all you need. The go/format package could be used to make the
    result look good.
    --

    -j
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Caleb Spare at May 7, 2016 at 7:36 pm
    I have a few different code gen projects and I generally use
    text/template but don't worry about formatting/indentation, just
    getting valid output. Then I run the result through go/format as Jan
    suggested (which is the library equivalent of gofmt).
    One of the reasons I was looking at go/ast in the first place is that it is hard to tell if an import is used or not until the whole source is generated.
    For generated code, I think it's reasonable not to worry about this.
    You can just add dummy usage so the code compiles.

    import (
             "bufio"
             "fmt"
             "strconv"
    )

    var (
             _ = bufio.Scanner
             _ = fmt.Sprint
             _ = strconv.Atoi
    )

    This isn't acceptable in hand-written code, but as r and others like
    to say, we hold generated code to a lower standard. (Protobuf and lots
    of other projects take this approach.)

    -Caleb

    On Sat, May 7, 2016 at 12:20 PM, Carl Mastrangelo
    wrote:
    That is what I ended up doing. My first attempt did use templates, but it
    was very difficult to get the formatting right. For example, getting
    indentation and new lines in the right place does work with templates, even
    with the new {{- foo -}} directives. I ended up copying protobuf's approach
    of using a tab writer, which makes for a more visually pleasing generated
    indentation, but uglier generator.

    One of the reasons I was looking at go/ast in the first place is that it is
    hard to tell if an import is used or not until the whole source is
    generated. Writing code out with fmt has problems too, since now I have
    strconv.Quote all over the place. Using "fmt" also has problems when used
    with bytes.Buffer as the temporary destination since you can't really
    "unwrite" a rune or set of bytes. Having a full syntax tree would make
    these problems easier to solve.
    On Saturday, May 7, 2016 at 12:08:17 PM UTC-7, Jan Mercl wrote:

    On Sat, May 7, 2016 at 7:30 PM Carl Mastrangelo <carl.mas...@gmail.com>
    wrote:
    Is there a proper way to generate code?
    Go source code is just text, so the fmt package and common string
    operations is all you need. The go/format package could be used to make the
    result look good.
    --

    -j
    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Nigel Tao at May 8, 2016 at 9:28 pm

    On Sun, May 8, 2016 at 5:20 AM, Carl Mastrangelo wrote:
    I ended up copying protobuf's approach
    of using a tab writer, which makes for a more visually pleasing generated
    indentation, but uglier generator.
    I'm not sure which protobuf you're referring to, but I wouldn't use a
    tabwriter. Instead, as Jan suggested, just let go/format auto-format
    your generated source code. For example,
    https://go.googlesource.com/image/+/master/font/basicfont/gen.go
    generates
    https://go.googlesource.com/image/+/master/font/basicfont/data.go

    One of the reasons I was looking at go/ast in the first place is that it is
    hard to tell if an import is used or not until the whole source is
    generated. Writing code out with fmt has problems too, since now I have
    strconv.Quote all over the place.
    I'm not sure what you're trying to do exactly, but you might not need
    strconv.Quote all over the place if you use the %q verb instead:
    https://play.golang.org/p/7b_pZI2rL1

    Using "fmt" also has problems when used
    with bytes.Buffer as the temporary destination since you can't really
    "unwrite" a rune or set of bytes.
    Can you give an example where you need to unwrite a rune?

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Jimmy frasche at May 8, 2016 at 4:32 am
    Using templates is the easiest way to generate code. go/ast isn't bad
    for analyzing go files, but it's hard to generate code from an AST or
    even to transform an existing AST (at least while preserving
    comments). Even if you manage to pull it off the template is going to
    be much shorter and easier to debug and read.

    On Sat, May 7, 2016 at 10:30 AM, Carl Mastrangelo
    wrote:
    In a toy project I am working on, I am trying to generate Go code
    automatically to cut down on the amount of boiler plate I have to write. I
    was looking at the packages under go/ in the standard library, but a lot of
    them seem specific to taking existing code and interpreting it rather than
    the other direction. Specifically, a lot of the API depends on having a
    token.FileSet, which assumes the file already exist. Since I am making it
    from scratch, I don't have a file or line number or token positions.

    Is there a proper way to generate code? Looking at golang/protobuf, it
    seems to print out raw strings currently too. Other projects like protobuf
    would likely benefit from being able to build code using standard packages.

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Mihai B at May 9, 2016 at 9:34 am
    templates is the only sane way(AFAIK) to go right now . I've created a
    library for the most commonly used constructs (e.g. package declaration,
      type declaration, struct literal declaration & assignment and some forms
    of function calls) to keep the code generation manageable and
    avoid redundancy but it is far from being a robust solution because the lib
    was created as a workaround rather than a holistic approach based on the
    language specifications. Initially I used go/ast as the IR which worked
    fine for simple type declarations but it proved quite challenging on more
    complex constructs so I've moved to a mix of templates (to write the source
    code) and an internal representation/types.
        Recently I've moved many parts of the "generated" code into a reflect
    based solution due the high cost of maintenance of the code generation
    tools(i.e. my library) so I would suggest you to consider a reflect based
    solution mixed with code generation(strictly only where reflect can't help)
    and if the performance is an issue move from reflect only when the API is
    stable. This will save you many weeks/months of work if the project is
    large.

    Mihai.
    On Saturday, 7 May 2016 20:30:35 UTC+3, Carl Mastrangelo wrote:

    In a toy project I am working on, I am trying to generate Go code
    automatically to cut down on the amount of boiler plate I have to write. I
    was looking at the packages under go/ in the standard library, but a lot of
    them seem specific to taking existing code and interpreting it rather than
    the other direction. Specifically, a lot of the API depends on having a
    token.FileSet, which assumes the file already exist. Since I am making it
    from scratch, I don't have a file or line number or token positions.

    Is there a proper way to generate code? Looking at golang/protobuf, it
    seems to print out raw strings currently too. Other projects like protobuf
    would likely benefit from being able to build code using standard packages.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedMay 7, '16 at 5:30p
activeMay 9, '16 at 9:34a
posts7
users6
websitegolang.org

People

Translate

site design / logo © 2021 Grokbase