FAQ
Hi all,

Given the ongoing discussions about Go's performance in the benchmark
game, I thought I'd take a look to see if there was any low hanging
fruit to pick. Based on

http://benchmarksgame.alioth.debian.org/u32/benchmark.php?test=all&lang=go&lang2=java&data=u32

the binary-trees benchmark looked interesting. I profiled it and it
seems to spend most it's time allocating memory and GC:ing it again,
which is reasonable. So I changed it to use a slice based binary tree
instead of individually allocated nodes:

http://play.golang.org/p/Ys2HKeuhue

That executes in about one ninth of the time of the original, on my
laptop, putting it roughly on par with gcc and above Java.

So is this a more or less relevant benchmark? On the one hand, it's no
longer doing the same thing as for example the C or Java
implementations. On the other hand, it's solving the same "problem"
with the same(-ish) data structure and one of the points about Go is
being able to control the memory layout...

(Lets skip the obvious "it's just a synthetic benchmark, who cares"
discussion since that's the premise of the game here.)

//jb

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Dave Cheney at Sep 24, 2013 at 6:57 am
    Sounds reasonable to me.

    Why not propose it as a change and see what people think.
    On 24 Sep 2013, at 16:53, Jakob Borg wrote:

    Hi all,

    Given the ongoing discussions about Go's performance in the benchmark
    game, I thought I'd take a look to see if there was any low hanging
    fruit to pick. Based on

    http://benchmarksgame.alioth.debian.org/u32/benchmark.php?test=all&lang=go&lang2=java&data=u32

    the binary-trees benchmark looked interesting. I profiled it and it
    seems to spend most it's time allocating memory and GC:ing it again,
    which is reasonable. So I changed it to use a slice based binary tree
    instead of individually allocated nodes:

    http://play.golang.org/p/Ys2HKeuhue

    That executes in about one ninth of the time of the original, on my
    laptop, putting it roughly on par with gcc and above Java.

    So is this a more or less relevant benchmark? On the one hand, it's no
    longer doing the same thing as for example the C or Java
    implementations. On the other hand, it's solving the same "problem"
    with the same(-ish) data structure and one of the points about Go is
    being able to control the memory layout...

    (Lets skip the obvious "it's just a synthetic benchmark, who cares"
    discussion since that's the premise of the game here.)

    //jb

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Egon at Sep 24, 2013 at 8:47 am
    http://benchmarksgame.alioth.debian.org/u32/performance.php?test=binarytrees#about

    That test is for benchmarking memory use and GC by allocating a lot of
    small objects.... so, no custom memory pool or free list.

    +egon
    On Tuesday, September 24, 2013 9:53:16 AM UTC+3, Jakob Borg wrote:

    Hi all,

    Given the ongoing discussions about Go's performance in the benchmark
    game, I thought I'd take a look to see if there was any low hanging
    fruit to pick. Based on


    http://benchmarksgame.alioth.debian.org/u32/benchmark.php?test=all&lang=go&lang2=java&data=u32

    the binary-trees benchmark looked interesting. I profiled it and it
    seems to spend most it's time allocating memory and GC:ing it again,
    which is reasonable. So I changed it to use a slice based binary tree
    instead of individually allocated nodes:

    http://play.golang.org/p/Ys2HKeuhue

    That executes in about one ninth of the time of the original, on my
    laptop, putting it roughly on par with gcc and above Java.

    So is this a more or less relevant benchmark? On the one hand, it's no
    longer doing the same thing as for example the C or Java
    implementations. On the other hand, it's solving the same "problem"
    with the same(-ish) data structure and one of the points about Go is
    being able to control the memory layout...

    (Lets skip the obvious "it's just a synthetic benchmark, who cares"
    discussion since that's the premise of the game here.)

    //jb
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Jesse McNelis at Sep 24, 2013 at 9:35 am

    On 24/09/2013 6:48 PM, "egon" wrote:
    http://benchmarksgame.alioth.debian.org/u32/performance.php?test=binarytrees#about
    That test is for benchmarking memory use and GC by allocating a lot of
    small objects.... so, no custom memory pool or free list.
    >

    The top C result and C++ result is using a memory pool. I think a lot of
    cheating is going on.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Dave Cheney at Sep 24, 2013 at 9:41 am
    All is fair in love and benchmarking.
    On 24 Sep 2013, at 19:35, Jesse McNelis wrote:

    On 24/09/2013 6:48 PM, "egon" wrote:

    http://benchmarksgame.alioth.debian.org/u32/performance.php?test=binarytrees#about

    That test is for benchmarking memory use and GC by allocating a lot of small objects.... so, no custom memory pool or free list.
    The top C result and C++ result is using a memory pool. I think a lot of cheating is going on.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Egon at Sep 24, 2013 at 9:47 am
    I guess the exact wording was "Please don't implement your own custom
    memory pool or free list."... so if it's third-party it can be used? Which
    is a bit weird....

    + egon
    On Tuesday, September 24, 2013 12:35:35 PM UTC+3, Jesse McNelis wrote:

    On 24/09/2013 6:48 PM, "egon" <egon...@gmail.com <javascript:>> wrote:

    http://benchmarksgame.alioth.debian.org/u32/performance.php?test=binarytrees#about
    That test is for benchmarking memory use and GC by allocating a lot of
    small objects.... so, no custom memory pool or free list.
    The top C result and C++ result is using a memory pool. I think a lot of
    cheating is going on.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Юрий Соколов at Sep 24, 2013 at 10:52 am
    I used to use such preallocate technique a lot :
    http://play.golang.org/p/5jFBBxC3LV

    вторник, 24 сентября 2013 г., 13:35:35 UTC+4 пользователь Jesse McNelis
    написал:
    On 24/09/2013 6:48 PM, "egon" <egon...@gmail.com <javascript:>> wrote:

    http://benchmarksgame.alioth.debian.org/u32/performance.php?test=binarytrees#about
    That test is for benchmarking memory use and GC by allocating a lot of
    small objects.... so, no custom memory pool or free list.
    The top C result and C++ result is using a memory pool. I think a lot of
    cheating is going on.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Michael Jones at Sep 24, 2013 at 11:06 am
    There is a teachable moment here for the meta-issue --- performance follows
    contextual design --- and that can be an issue for little benchmarks and
    big systems. If you say, "I'll implement X using this precise design in Go,
    Lisp, Forth, and Haskell" to see which is fastest, part of your actual
    measurement is which language's inner nature best aligns with your rigid
    design. (The other part is a measure of the language's ability to say X and
    a third part is about the quality of the language toolchain.)

    Once you move from "programming in Java but expressed in Go" to
    "programming in Go" (or vice versa) you will have better performance,
    smaller memory use, and a much better time. This is a universal truth. Even
    in assembly language, the "idiomatic approach" is contextual in register
    pressure, memory hierarchy, and instruction cycle counts. (Is Increment
    slower than Add One Immediate? Often so on x86.) Not being flexible about
    implementation reality means being slow.

    Michael

    On Tue, Sep 24, 2013 at 4:52 AM, Юрий Соколов wrote:

    I used to use such preallocate technique a lot :
    http://play.golang.org/p/5jFBBxC3LV

    вторник, 24 сентября 2013 г., 13:35:35 UTC+4 пользователь Jesse McNelis
    написал:
    On 24/09/2013 6:48 PM, "egon" wrote:

    http://benchmarksgame.alioth.**debian.org/u32/performance.**
    php?test=binarytrees#about<http://benchmarksgame.alioth.debian.org/u32/performance.php?test=binarytrees#about>
    That test is for benchmarking memory use and GC by allocating a lot of
    small objects.... so, no custom memory pool or free list.
    The top C result and C++ result is using a memory pool. I think a lot of
    cheating is going on.
    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.


    --
    Michael T. Jones | Chief Technology Advocate | mtj@google.com | +1
    650-335-5765

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Sean Russell at Sep 24, 2013 at 11:12 am

    On Tuesday, September 24, 2013 5:35:35 AM UTC-4, Jesse McNelis wrote:

    On 24/09/2013 6:48 PM, "egon" <egon...@gmail.com <javascript:>> wrote:
    ...
    That test is for benchmarking memory use and GC by allocating a lot of
    small objects.... so, no custom memory pool or free list.
    The top C result and C++ result is using a memory pool. I think a lot of
    cheating is going on.
    You are absolutely right. The C version uses apr_pools, so pools are being
    used. Egon is referring to the comment in the benchmark description that
    says:

    *Please don't implement your own custom memory pool or free list*.
    However, earlier in the description the notes say:

    Note: this is an adaptation of a benchmark for testing GC so we are
    interested in the whole tree being allocated before any nodes are GC'd -
    which probably excludes lazy evaluation.
    It's pretty clear from this that the goal is to end up with the entire tree
    in memory, not to force a bunch of allocations and de-allocations.
    Consequently, I'd suggest that the C program isn't cheating and also that
    the Go version is doing more work than it needs to.

    Jakob, I didn't see a post from you on the benchmark discussion forum; I
    think it'd be worth mentioning the comment about C using pools, and for
    clarification about that comment about not implementing custom pools. If
    you don't want to spend the time, I'll post it.

    --- SER

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Jakob Borg at Sep 24, 2013 at 12:16 pm

    2013/9/24 Sean Russell <seanerussell@gmail.com>:
    Jakob, I didn't see a post from you on the benchmark discussion forum; I
    think it'd be worth mentioning the comment about C using pools, and for
    clarification about that comment about not implementing custom pools. If
    you don't want to spend the time, I'll post it.
    Please do! I haven't been at all involved with the benchmarksgame
    before so wasn't really aware of the forum or there being a custom to
    discuss the code there. I saw the instructions for "playing" and
    created a ticket for the new code earlier though.

    BR,
    Jakob

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Alex at Sep 24, 2013 at 4:28 pm
    I submitted this one -
    http://benchmarksgame.alioth.debian.org/u32/program.php?test=binarytrees&lang=go&id=6
    a while back based on work of some others. It was rejected for not
    following the rules regarding memory pools/free lists, which is silly as
    this is how the C version acts, which makes me question how well the
    maintainer -
    1 - knows c
    2 - knows go
    3 - is biased for or against some languages?

    Ah well, I wish you luck.

    Thanks,
    Alex

    On Tuesday, September 24, 2013 8:16:44 AM UTC-4, Jakob Borg wrote:

    2013/9/24 Sean Russell <seaner...@gmail.com <javascript:>>:
    Jakob, I didn't see a post from you on the benchmark discussion forum; I
    think it'd be worth mentioning the comment about C using pools, and for
    clarification about that comment about not implementing custom pools. If
    you don't want to spend the time, I'll post it.
    Please do! I haven't been at all involved with the benchmarksgame
    before so wasn't really aware of the forum or there being a custom to
    discuss the code there. I saw the instructions for "playing" and
    created a ticket for the new code earlier though.

    BR,
    Jakob
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Isaac Gouy at Sep 24, 2013 at 10:05 pm

    On Tuesday, September 24, 2013 9:28:11 AM UTC-7, al...@lx.lc wrote:
    It was rejected for not following the rules regarding memory pools/free
    lists, which is silly as this is how the C version acts...
    "Please don't implement *your own* custom memory pool or free list."Apache
    Portable Runtime was not written to solve the benchmarks game binary-trees
    task ;-)

    Note that C programs are also rejected for the same reason, for example C
    gcc #2 and C gcc #9

    Note that the Go program being discussed doesn't even print the correct
    results.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Jakob Borg at Sep 24, 2013 at 10:12 am
    Yeah, looking at the instructions it might indeed be cheating. However it speaks mostly about allocating many _trees_, which I haven't done anything to optimize away.

    Ah well, in any case it goes to show that Go's gc and allocation may not be the fastest but that there might be better approaches instead, even if it doesn't fit the benchmark. :)

    //jb

    (Sent from my phone - please excuse brevity and typos.)
    On 24 sep 2013, at 10:47, egon wrote:

    http://benchmarksgame.alioth.debian.org/u32/performance.php?test=binarytrees#about

    That test is for benchmarking memory use and GC by allocating a lot of small objects.... so, no custom memory pool or free list.

    +egon
    On Tuesday, September 24, 2013 9:53:16 AM UTC+3, Jakob Borg wrote:
    Hi all,

    Given the ongoing discussions about Go's performance in the benchmark
    game, I thought I'd take a look to see if there was any low hanging
    fruit to pick. Based on

    http://benchmarksgame.alioth.debian.org/u32/benchmark.php?test=all&lang=go&lang2=java&data=u32

    the binary-trees benchmark looked interesting. I profiled it and it
    seems to spend most it's time allocating memory and GC:ing it again,
    which is reasonable. So I changed it to use a slice based binary tree
    instead of individually allocated nodes:

    http://play.golang.org/p/Ys2HKeuhue

    That executes in about one ninth of the time of the original, on my
    laptop, putting it roughly on par with gcc and above Java.

    So is this a more or less relevant benchmark? On the one hand, it's no
    longer doing the same thing as for example the C or Java
    implementations. On the other hand, it's solving the same "problem"
    with the same(-ish) data structure and one of the points about Go is
    being able to control the memory layout...

    (Lets skip the obvious "it's just a synthetic benchmark, who cares"
    discussion since that's the premise of the game here.)

    //jb
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Benjamin Measures at Sep 24, 2013 at 11:12 pm

    On Tuesday, 24 September 2013 11:12:34 UTC+1, Jakob Borg wrote:

    Go's gc and allocation may not be the fastest but that there might be
    better approaches instead, even if it doesn't fit the benchmark. :)
    TLDR; "it's just a synthetic benchmark, who cares"

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Wes Freeman at Sep 24, 2013 at 11:27 pm
    This C++ benchmark uses SSE2 floating point stuff, which made it
    significantly faster than the C program (and all of the other programs) at
    the time. Eventually someone updated the C program, but still it feels like
    a bit much. :P

    http://benchmarksgame.alioth.debian.org/u32/program.php?test=nbody&lang=gpp&id=7

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Isaac Gouy at Sep 25, 2013 at 3:04 am

    On Tuesday, September 24, 2013 4:27:03 PM UTC-7, Wes Freeman wrote:
    This C++ benchmark uses SSE2 floating point stuff, which made it
    significantly faster than the C program (and all of the other programs) at
    the time. Eventually someone updated the C program, but still it feels like
    a bit much. :P


    http://benchmarksgame.alioth.debian.org/u32/program.php?test=nbody&lang=gpp&id=7

    Probably more appropriate to complain about that in the benchmarks game
    discussion forum; and explain in detail your "feels like a bit much"
    criteria :-)

    The Go nbody<http://benchmarksgame.alioth.debian.org/u64/program.php?test=nbody&lang=go&id=1>program is based on this C
    nbody<http://benchmarksgame.alioth.debian.org/u64/program.php?test=nbody&lang=gcc&id=1>program.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Wes Freeman at Sep 25, 2013 at 3:16 am
    This is the current C nbody program. ~twice as fast as the one you linked.

    http://benchmarksgame.alioth.debian.org/u64/program.php?test=nbody&lang=gcc

    I guess the Go program needs updating.

    Wes
    On Tue, Sep 24, 2013 at 11:04 PM, Isaac Gouy wrote:


    On Tuesday, September 24, 2013 4:27:03 PM UTC-7, Wes Freeman wrote:

    This C++ benchmark uses SSE2 floating point stuff, which made it
    significantly faster than the C program (and all of the other programs) at
    the time. Eventually someone updated the C program, but still it feels like
    a bit much. :P

    http://benchmarksgame.alioth.**debian.org/u32/program.php?**
    test=nbody&lang=gpp&id=7<http://benchmarksgame.alioth.debian.org/u32/program.php?test=nbody&lang=gpp&id=7>

    Probably more appropriate to complain about that in the benchmarks game
    discussion forum; and explain in detail your "feels like a bit much"
    criteria :-)

    The Go nbody<http://benchmarksgame.alioth.debian.org/u64/program.php?test=nbody&lang=go&id=1>program is based on this C
    nbody<http://benchmarksgame.alioth.debian.org/u64/program.php?test=nbody&lang=gcc&id=1>program.

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Isaac Gouy at Sep 25, 2013 at 3:18 pm
    On Tuesday, September 24, 2013 8:15:34 PM UTC-7, Wes Freeman wrote:

    This is the current C nbody program
    >

    The *current* C nbody program? *Currently* 6 C nbody programs are shown.


    I guess the Go program needs updating.
    I guess how best to accomplish that actually is an appropriate topic for a
    Go discussion group.


    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Wes Freeman at Sep 26, 2013 at 2:10 am
    Current fastest/best/most optimized/etc.

    Perhaps a similarly optimized go program can be written? (SSE2 double
    floating point stores, conversions and sqrts). Not really sure where to
    begin myself.

    Wes
    On Wed, Sep 25, 2013 at 11:18 AM, Isaac Gouy wrote:



    On Tuesday, September 24, 2013 8:15:34 PM UTC-7, Wes Freeman wrote:

    This is the current C nbody program
    The *current* C nbody program? *Currently* 6 C nbody programs are shown.


    I guess the Go program needs updating.
    I guess how best to accomplish that actually is an appropriate topic for a
    Go discussion group.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Isaac Gouy at Sep 25, 2013 at 2:53 am

    On Tuesday, September 24, 2013 4:12:41 PM UTC-7, Benjamin Measures wrote:
    On Tuesday, 24 September 2013 11:12:34 UTC+1, Jakob Borg wrote:

    Go's gc and allocation may not be the fastest but that there might be
    better approaches instead, even if it doesn't fit the benchmark. :)
    TLDR; "it's just a synthetic benchmark, who cares"

    *Your application is **the ultimate benchmark*<http://benchmarksgame.alioth.debian.org/dont-jump-to-conclusions.php#app>


    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedSep 24, '13 at 6:53a
activeSep 26, '13 at 2:10a
posts20
users11
websitegolang.org

People

Translate

site design / logo © 2021 Grokbase