FAQ
Hi!

Playing a bit with Go I found something weird, I don't know if it may be a
compiler bug. I'm using Go 1.2 and I'm seen this in ARM, linux/386 and
windows/386.

I am creating a big array (not a slice) of a struct and then accessing it
with range using the index or the value. It seems that if the struct
contains a string (even if not used) AND the struct is accessed using the
value, compilation uses a lot of memory and the program produced is really
big or even fails to compile.

Cases:
- The version that fails (binary is about 10 MB, compiler uses about 180
MB): http://play.golang.org/p/EikUFnNvqj
Increasing array size causes it to use more RAM or even to fail if no more
RAM is available.

- String removed (works): http://play.golang.org/p/PuJWOf8hUz<http://play.golang.org/p/WgMquzjtnc>

- Access by value removed (works): http://play.golang.org/p/6OH7vXKDCa<http://play.golang.org/p/kTAR0UWXA2>

Some other errors I have seen, that I don't know if they are related:

- String removed but array increased to 1000000 elements:
http://play.golang.org/p/v7i1VWOFbg
Compiler says: main.main: overflow in spadj: 0 + 1200002464 = 1200002464

-Thomas Bushnell in G+ community says:
*Interestingly, if you replace "string" with [1000]int, you get a memory
error, but a stack overflow error, which shouldn't happen for this program
at all, which should use constant stack
space. http://play.golang.org/p/_itbrJzZvN
<http://play.golang.org/p/_itbrJzZvN>*


I know that I'm not being idiomatic, it is just a toy, but I don't know if
this behaviour is expected.

Any idea?

Thanks!

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Felix at Jan 22, 2014 at 3:58 pm

    Not sure what you want to know but your first example compile and run fine
    in max OS 10.8, and the process
    use about 240mb - 300mb during runtime.

    Not that the binary been about 10mb is cause the go runtime is added to
    every binary, and go also add all
    dependencies.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Felix at Jan 22, 2014 at 4:01 pm
    Not sure what you want to know but your first example compile and run fine
    in max OS 10.8, and the process
    use about 240mb - 300mb during runtime.
    but when I can the array from 100,000 to 1,000,000, then I get that
    compile error

    `main.main: overflow in spadj: 0 + 1616003344 = 1616003344`

    Not that the binary been about 10mb is cause the go runtime is added to
    every binary, and go also add all
    dependencies.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • N. Riesco - GMail account at Jan 22, 2014 at 6:08 pm
    What I don't understand is why the following example fails when
    declaring variable s as:

          var s [100000]SS

    but it works with

          var s [100000]S

    Both types, S and SS, have the same size. The only difference is that SS
    has a string as a field.



    http://play.golang.org/p/ccB4pugdVV



    package main

    import "fmt"

    type S struct {
          a [102]int
    }

    type SS struct {
          a [100]int
          b string
    }

    func main() {
          var s [100000]SS

          sum := 0
          for _, v := range s {
              sum += v.a[50]
          }

          fmt.Println(sum)
    }



    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Ross Salas at Jan 22, 2014 at 7:00 pm
    If you remove a zero from the array s, it works. Not sure why.

    On Wed, Jan 22, 2014 at 10:08 AM, N. Riesco - GMail account wrote:

    What I don't understand is why the following example fails when declaring
    variable s as:

    var s [100000]SS

    but it works with

    var s [100000]S

    Both types, S and SS, have the same size. The only difference is that SS
    has a string as a field.



    http://play.golang.org/p/ccB4pugdVV



    package main

    import "fmt"

    type S struct {
    a [102]int
    }

    type SS struct {
    a [100]int
    b string
    }

    func main() {
    var s [100000]SS

    sum := 0
    for _, v := range s {
    sum += v.a[50]
    }

    fmt.Println(sum)
    }



    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Felix at Jan 22, 2014 at 8:59 pm
    I posted this on golang-dev so check it out
    https://groups.google.com/forum/#!topic/golang-dev/uCk_x4khKlM

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Dave Cheney at Jan 22, 2014 at 9:05 pm
    Felix, please follow rsc's advice in his response; confirm if the bug
    exists on tip, and file a bug if it does.

    On Thu, Jan 23, 2014 at 7:59 AM, Felix wrote:

    I posted this on golang-dev so check it out
    https://groups.google.com/forum/#!topic/golang-dev/uCk_x4khKlM

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • N. Riesco - GMail account at Jan 22, 2014 at 9:21 pm
    Here's a simpler example that exhibits the same problem:


    http://play.golang.org/p/UEghZR_57X

    package main

    func main() {
          const N = 10000000

          var s [N]string
          //var s [2 * N]int

          for _, v := range s {
              _ = v
          }
    }



    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • N. Riesco - GMail account at Jan 22, 2014 at 9:25 pm
    Forgot to mention, another interesting thing is the difference in size
    of the executables:

    582176 Jan 22 21:22 issue-no-string
    5580200 Jan 22 21:22 issue-string

    I'm using tip: go version devel +7326da92ff4d Mon Dec 02 09:06:41 2013
    +1100 linux/amd64

    I will check on a 32bit system later.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Dave Cheney at Jan 22, 2014 at 9:38 pm
    Thanks for the shorter repo, please add it to the issue.

    FWIW your example compiles fine with a /recent/ (dec 02 is essentially
    still go 1.2) build of tip. I did not check 386 builds or the memory
    consumption of the compilation.

    On Thu, Jan 23, 2014 at 8:25 AM, N. Riesco - GMail account wrote:

    Forgot to mention, another interesting thing is the difference in size of
    the executables:

    582176 Jan 22 21:22 issue-no-string
    5580200 Jan 22 21:22 issue-string

    I'm using tip: go version devel +7326da92ff4d Mon Dec 02 09:06:41 2013
    +1100 linux/amd64

    I will check on a 32bit system later.


    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • N. Riesco - GMail account at Jan 22, 2014 at 10:30 pm
    I've just tried in a 32bit netbook. It compiles and runs fine.

    Thus, the only way to reproduce the problem with the code I posted is on
    http://play.golang.org/

    This code http://play.golang.org/p/W4kHMC-GFJ runs out of memory,
    whereas this code http://play.golang.org/p/BrGPJCxQg6 doesn't,
    despite the size of variable s being the same in both cases.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Dave Cheney at Jan 23, 2014 at 12:18 am
    Guys, talk is cheap, put actionable details in the issue, thanks.

    On Thu, Jan 23, 2014 at 9:30 AM, N. Riesco - GMail account wrote:

    I've just tried in a 32bit netbook. It compiles and runs fine.

    Thus, the only way to reproduce the problem with the code I posted is on
    http://play.golang.org/

    This code http://play.golang.org/p/W4kHMC-GFJ runs out of memory,
    whereas this code http://play.golang.org/p/BrGPJCxQg6 doesn't,
    despite the size of variable s being the same in both cases.

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Nacho at Jan 23, 2014 at 5:19 am
    I have reproduced it in the last tip for linux/arm. I have added the
    details here: https://code.google.com/p/go/issues/detail?id=6993

    On Thu, Jan 23, 2014 at 1:18 AM, Dave Cheney wrote:

    Guys, talk is cheap, put actionable details in the issue, thanks.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Nico at Jan 23, 2014 at 7:19 am
    I don't understand why the examples I produced would fit issue 6993.

    To trigger the problem in the playground two conditions are necessary:
    * a large array of strings
    * a for-range over the values of the array

    Shall I reported a new issue?

    Note the following example with a for-range over the indices runs
    successfully in the playground (and the executable size is similar to that
    using an array of ints):

    http://play.golang.org/p/rJaMWvPTlt

    package main

    func main() {
         const N = 10000000
         var s [N]string

         for i, _ := range s {
             _ = i
         }
    }



    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Nacho at Jan 23, 2014 at 7:25 am
    Yes, it is true that you need both conditions, I'm not sure either if it is
    the same bug, but the effects (large binary, high RAM use while compiling)
    are the same. I'm assuming that the trigger is related but I don't know the
    internals.

    Dave, do you think both are related or would it be better to report it as a
    new issue?

    On Thu, Jan 23, 2014 at 8:19 AM, Nico wrote:

    I don't understand why the examples I produced would fit issue 6993.

    To trigger the problem in the playground two conditions are necessary:
    * a large array of strings
    * a for-range over the values of the array

    Shall I reported a new issue?

    Note the following example with a for-range over the indices runs
    successfully in the playground (and the executable size is similar to that
    using an array of ints):

    http://play.golang.org/p/rJaMWvPTlt

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Dave Cheney at Jan 23, 2014 at 7:36 am
    I'm pretty sure you don't the the for loop.


    On 23 Jan 2014, at 18:24, Nacho wrote:

    Yes, it is true that you need both conditions, I'm not sure either if it is the same bug, but the effects (large binary, high RAM use while compiling) are the same. I'm assuming that the trigger is related but I don't know the internals.

    Dave, do you think both are related or would it be better to report it as a new issue?

    On Thu, Jan 23, 2014 at 8:19 AM, Nico wrote:
    I don't understand why the examples I produced would fit issue 6993.

    To trigger the problem in the playground two conditions are necessary:
    * a large array of strings
    * a for-range over the values of the array

    Shall I reported a new issue?

    Note the following example with a for-range over the indices runs successfully in the playground (and the executable size is similar to that using an array of ints):

    http://play.golang.org/p/rJaMWvPTlt
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Dave Cheney at Jan 22, 2014 at 9:45 pm
    This is a different issue,
    https://code.google.com/p/go/issues/detail?id=6993

    On Thu, Jan 23, 2014 at 8:25 AM, N. Riesco - GMail account wrote:

    Forgot to mention, another interesting thing is the difference in size of
    the executables:

    582176 Jan 22 21:22 issue-no-string
    5580200 Jan 22 21:22 issue-string

    I'm using tip: go version devel +7326da92ff4d Mon Dec 02 09:06:41 2013
    +1100 linux/amd64

    I will check on a 32bit system later.


    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • N. Riesco - GMail account at Jan 22, 2014 at 10:11 pm
    What is the explanation why using:
           var s [N]string
    instead of
           var s [2*N]int
    makes such a big difference in the executable size?



    PS: I've also tried to declare s as a global and the difference in size
    is still important:
    587520 Jan 22 21:55 issue-no-string
    587600 Jan 22 22:08 issue-no-string-global
    10590872 Jan 22 21:55 issue-string
    5589736 Jan 22 22:03 issue-string-global

    This time I'm using go version devel +f9e8a970798c Wed Jan 22 16:39:39
    2014 -0500 linux/amd64

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedJan 22, '14 at 7:36a
activeJan 23, '14 at 7:36a
posts18
users5
websitegolang.org

People

Translate

site design / logo © 2022 Grokbase