FAQ
This evening I was prompted by this post<http://www.reddit.com/r/rust/comments/1xaum0/standalone_rust_app/>to work on a Go implementation of the code
here<http://mechanical-sympathy.blogspot.com.au/2012/10/compact-off-heap-structurestuples-in.html>.
My implementation is in this gist<https://gist.github.com/logicchains/8882549>.
I also made a C implementation, here<https://gist.github.com/logicchains/8883534>.
While the C code works fine, the Go code fails with 'killed' when I try to
run it. If I change the static array in the Go version to a slice, then
upon attempting to allocate room for that slice the program crashes with
"fatal error: runtime: out of memory".

My question is: why does the C version work where the Go version doesn't?
The array/slice in question is around 2.5 gigabytes in size, but the system
I'm currently testing the program on has 4 gigabytes of memory, and in my
experience the Go version normally wouldn't use much more memory for the
array than the C version does. Note that the Java implementation also
throws an "Exception: java.lang.OutOfMemoryError thrown from the
UncaughtExceptionHandler in thread "main"" error, but I'd have expected the
Go implementation to use much less memory than Java.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Jonathan T Barnard at Feb 8, 2014 at 2:20 pm
    Okay, I discovered that if I increase the size of the array in the C
    version by more than 6% then it gets killed, so Go need only have been
    using 6% more memory to cause a crash. Nevermind.
    On Saturday, 8 February 2014 22:10:59 UTC+8, jonathan....@gmail.com wrote:

    This evening I was prompted by this post<http://www.reddit.com/r/rust/comments/1xaum0/standalone_rust_app/>to work on a Go implementation of the code
    here<http://mechanical-sympathy.blogspot.com.au/2012/10/compact-off-heap-structurestuples-in.html>.
    My implementation is in this gist<https://gist.github.com/logicchains/8882549>.
    I also made a C implementation, here<https://gist.github.com/logicchains/8883534>.
    While the C code works fine, the Go code fails with 'killed' when I try to
    run it. If I change the static array in the Go version to a slice, then
    upon attempting to allocate room for that slice the program crashes with
    "fatal error: runtime: out of memory".

    My question is: why does the C version work where the Go version doesn't?
    The array/slice in question is around 2.5 gigabytes in size, but the system
    I'm currently testing the program on has 4 gigabytes of memory, and in my
    experience the Go version normally wouldn't use much more memory for the
    array than the C version does. Note that the Java implementation also
    throws an "Exception: java.lang.OutOfMemoryError thrown from the
    UncaughtExceptionHandler in thread "main"" error, but I'd have expected the
    Go implementation to use much less memory than Java.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Egon at Feb 8, 2014 at 9:16 pm
    The "int" in C may differ from sizeof "int" in Go.

    https://gist.github.com/egonelbre/8890467

    Try and see whether they give different answers?

    For Go I got: "56, 48, 40"; for C I got: "48, 48, 40". Of course it will
    depend on which versions of Go/C you have installed. i.e. are both for the
    same architecture or not.

    + egon

    On Saturday, February 8, 2014 4:19:56 PM UTC+2, jonathan....@gmail.com
    wrote:
    Okay, I discovered that if I increase the size of the array in the C
    version by more than 6% then it gets killed, so Go need only have been
    using 6% more memory to cause a crash. Nevermind.
    On Saturday, 8 February 2014 22:10:59 UTC+8, jonathan....@gmail.com wrote:

    This evening I was prompted by this post<http://www.reddit.com/r/rust/comments/1xaum0/standalone_rust_app/>to work on a Go implementation of the code
    here<http://mechanical-sympathy.blogspot.com.au/2012/10/compact-off-heap-structurestuples-in.html>.
    My implementation is in this gist<https://gist.github.com/logicchains/8882549>.
    I also made a C implementation, here<https://gist.github.com/logicchains/8883534>.
    While the C code works fine, the Go code fails with 'killed' when I try to
    run it. If I change the static array in the Go version to a slice, then
    upon attempting to allocate room for that slice the program crashes with
    "fatal error: runtime: out of memory".

    My question is: why does the C version work where the Go version doesn't?
    The array/slice in question is around 2.5 gigabytes in size, but the system
    I'm currently testing the program on has 4 gigabytes of memory, and in my
    experience the Go version normally wouldn't use much more memory for the
    array than the C version does. Note that the Java implementation also
    throws an "Exception: java.lang.OutOfMemoryError thrown from the
    UncaughtExceptionHandler in thread "main"" error, but I'd have expected the
    Go implementation to use much less memory than Java.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Jonathan T Barnard at Feb 9, 2014 at 3:39 am
    Thanks for that! I got the same results as you; seems a Go int is 64bits on
    this Linux x86-64 architecture while a C int is 32bit. Definitely something
    I'll need to bear in mind in future.

    Amazingly, the Go version now seems to be slightly faster than the C one,
    which makes me suspect I'm doing something wrong. Anyway, In case anyone's
    interested, at 500k records in the array the naive Java one from the
    original post runs in 1.01 seconds, the optimised unsafe Java one from the
    original post in 0.14 seconds, the C version in 0.088 seconds and the Go in
    0.086 seconds. Java doesn't come out of the comparison to favourably,
    although I suspect the original Java code be improved.
    On Sunday, 9 February 2014 05:16:13 UTC+8, egon wrote:

    The "int" in C may differ from sizeof "int" in Go.

    https://gist.github.com/egonelbre/8890467

    Try and see whether they give different answers?

    For Go I got: "56, 48, 40"; for C I got: "48, 48, 40". Of course it will
    depend on which versions of Go/C you have installed. i.e. are both for the
    same architecture or not.

    + egon

    On Saturday, February 8, 2014 4:19:56 PM UTC+2, jonathan....@gmail.comwrote:
    Okay, I discovered that if I increase the size of the array in the C
    version by more than 6% then it gets killed, so Go need only have been
    using 6% more memory to cause a crash. Nevermind.

    On Saturday, 8 February 2014 22:10:59 UTC+8, jonathan....@gmail.comwrote:
    This evening I was prompted by this post<http://www.reddit.com/r/rust/comments/1xaum0/standalone_rust_app/>to work on a Go implementation of the code
    here<http://mechanical-sympathy.blogspot.com.au/2012/10/compact-off-heap-structurestuples-in.html>.
    My implementation is in this gist<https://gist.github.com/logicchains/8882549>.
    I also made a C implementation, here<https://gist.github.com/logicchains/8883534>.
    While the C code works fine, the Go code fails with 'killed' when I try to
    run it. If I change the static array in the Go version to a slice, then
    upon attempting to allocate room for that slice the program crashes with
    "fatal error: runtime: out of memory".

    My question is: why does the C version work where the Go version
    doesn't? The array/slice in question is around 2.5 gigabytes in size, but
    the system I'm currently testing the program on has 4 gigabytes of memory,
    and in my experience the Go version normally wouldn't use much more memory
    for the array than the C version does. Note that the Java implementation
    also throws an "Exception: java.lang.OutOfMemoryError thrown from the
    UncaughtExceptionHandler in thread "main"" error, but I'd have expected the
    Go implementation to use much less memory than Java.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Alex Skinner at Feb 11, 2014 at 4:13 am
    Hi,

    Are you going to add your results and write up to your blog like the last
    couple? I quite enjoyed the comparisons, strategies, and challenge of
    microoptimizing. I'd also be interested in seeing all benchmarks ran with
    with updated versions as they are released - specifically go and rust.
    They are both shaping up quite nicely, and have a lot of smart people
    onboard.

    Thanks,
    Alex


    On Saturday, February 8, 2014 10:39:43 PM UTC-5, jonathan....@gmail.com
    wrote:
    Thanks for that! I got the same results as you; seems a Go int is 64bits
    on this Linux x86-64 architecture while a C int is 32bit. Definitely
    something I'll need to bear in mind in future.

    Amazingly, the Go version now seems to be slightly faster than the C one,
    which makes me suspect I'm doing something wrong. Anyway, In case anyone's
    interested, at 500k records in the array the naive Java one from the
    original post runs in 1.01 seconds, the optimised unsafe Java one from the
    original post in 0.14 seconds, the C version in 0.088 seconds and the Go in
    0.086 seconds. Java doesn't come out of the comparison to favourably,
    although I suspect the original Java code be improved.
    On Sunday, 9 February 2014 05:16:13 UTC+8, egon wrote:

    The "int" in C may differ from sizeof "int" in Go.

    https://gist.github.com/egonelbre/8890467

    Try and see whether they give different answers?

    For Go I got: "56, 48, 40"; for C I got: "48, 48, 40". Of course it will
    depend on which versions of Go/C you have installed. i.e. are both for the
    same architecture or not.

    + egon

    On Saturday, February 8, 2014 4:19:56 PM UTC+2, jonathan....@gmail.comwrote:
    Okay, I discovered that if I increase the size of the array in the C
    version by more than 6% then it gets killed, so Go need only have been
    using 6% more memory to cause a crash. Nevermind.

    On Saturday, 8 February 2014 22:10:59 UTC+8, jonathan....@gmail.comwrote:
    This evening I was prompted by this post<http://www.reddit.com/r/rust/comments/1xaum0/standalone_rust_app/>to work on a Go implementation of the code
    here<http://mechanical-sympathy.blogspot.com.au/2012/10/compact-off-heap-structurestuples-in.html>.
    My implementation is in this gist<https://gist.github.com/logicchains/8882549>.
    I also made a C implementation, here<https://gist.github.com/logicchains/8883534>.
    While the C code works fine, the Go code fails with 'killed' when I try to
    run it. If I change the static array in the Go version to a slice, then
    upon attempting to allocate room for that slice the program crashes with
    "fatal error: runtime: out of memory".

    My question is: why does the C version work where the Go version
    doesn't? The array/slice in question is around 2.5 gigabytes in size, but
    the system I'm currently testing the program on has 4 gigabytes of memory,
    and in my experience the Go version normally wouldn't use much more memory
    for the array than the C version does. Note that the Java implementation
    also throws an "Exception: java.lang.OutOfMemoryError thrown from the
    UncaughtExceptionHandler in thread "main"" error, but I'd have expected the
    Go implementation to use much less memory than Java.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedFeb 8, '14 at 2:11p
activeFeb 11, '14 at 4:13a
posts5
users3
websitegolang.org

People

Translate

site design / logo © 2022 Grokbase