FAQ
I wanted to see how changing main data type in the cayley graph database
from a pointer type to a concrete struct (the type contains 4 strings).

For the common cases the outcome looks pretty good. But for one of our
pathological benchmarks, the change causes a significant regression (in
commit message at [1], but also below).


Comparison of -short benchmarks in cayley.

$ benchcmp pointer.bench concrete.bench
benchmark old ns/op new ns/op delta
BenchmarkNamePredicate 1673276 1655093 -1.09%
BenchmarkLargeSetsNoIntersection 318985907 261499984 -18.02%
BenchmarkNetAndSpeed 104403743 41516981 -60.23%
BenchmarkKeanuAndNet 17309258 16857513 -2.61%
BenchmarkKeanuAndSpeed 20159161 19282833 -4.35%

Long benchmarks are run separately to avoid timeout by testing, and are
less good.

benchmark old ns/op new ns/op delta
BenchmarkVeryLargeSetsSmallIntersection 55269775527 246084606672 +345.24%
BenchmarkHelplessContainsChecker 23436501319 24308906949 +3.72%


I had a look at the cpu profile of the worst case and the cayley code
that's being run doesn't end up anywhere near the top of the list.

Can someone explain what is going on here?


Pointer:
Total: 6121 samples
     1973 32.2% 32.2% 1973 32.2% runtime.findfunc
      773 12.6% 44.9% 773 12.6% readvarint
      510 8.3% 53.2% 511 8.3% step
      409 6.7% 59.9% 410 6.7% runtime.gentraceback
      390 6.4% 66.2% 391 6.4% pcvalue
      215 3.5% 69.8% 215 3.5% runtime.funcdata
      181 3.0% 72.7% 181 3.0% checkframecopy
      118 1.9% 74.6% 119 1.9% runtime.funcspdelta
       96 1.6% 76.2% 96 1.6% runtime.topofstack
       76 1.2% 77.5% 76 1.2% scanblock

Concrete:
Total: 25027 samples
     9437 37.7% 37.7% 9437 37.7% runtime.findfunc
     3853 15.4% 53.1% 3853 15.4% readvarint
     2366 9.5% 62.6% 2366 9.5% step
     2186 8.7% 71.3% 2186 8.7% runtime.gentraceback
     1816 7.3% 78.5% 1816 7.3% pcvalue
     1016 4.1% 82.6% 1016 4.1% runtime.funcdata
      859 3.4% 86.0% 859 3.4% checkframecopy
      506 2.0% 88.1% 506 2.0% runtime.funcspdelta
      410 1.6% 89.7% 410 1.6% runtime.topofstack
      303 1.2% 90.9% 303 1.2% runtime.newstack


thanks
Dan

[1]https://github.com/kortschak/cayley/commit/6acfdcc5d6b9259e180cd576e0fa3e95a05bfd13

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Gyu-Ho Lee at Aug 5, 2014 at 3:31 pm
    Interesting. Can you provide the code for
    `BenchmarkVeryLargeSetsSmallIntersection` ?
    Do you mean that you iterate the large set of vertices with small
    intersection?

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dan Kortschak at Aug 5, 2014 at 7:50 pm
    The tests are at https://github.com/kortschak/cayley/blob/master/cayley_test.go

    The branch with the non-pointer type is 'concrete'.

    My suspicion is that has to do with stack growth.
    On 06/08/2014, at 1:01 AM, "Gyu-Ho Lee" wrote:

    Interesting. Can you provide the code for `BenchmarkVeryLargeSetsSmallIntersection` ?
    Do you mean that you iterate the large set of vertices with small intersection?
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dan Kortschak at Aug 6, 2014 at 12:49 am
    More complete top lists below.

    I can't see anything that explains why the concrete type takes so much
    longer. It just looks like there space-time continuum is stretched in
    that case (there are more samples taken, but there doesn't seem to be
    any particular function that is over represented in the concrete case).

    I'm profiling on linux am64, so the osX issue is not the problem. What
    things are not caught by the profiler that might be causing this slow
    down?


    Pointer-based:
    Total: 5110 samples
         1685 33.0% 33.0% 1685 33.0% runtime.findfunc
          678 13.3% 46.2% 678 13.3% readvarint
          429 8.4% 54.6% 429 8.4% runtime.gentraceback
          362 7.1% 61.7% 362 7.1% step
          324 6.3% 68.1% 324 6.3% pcvalue
          165 3.2% 71.3% 165 3.2% checkframecopy
          156 3.1% 74.3% 156 3.1% runtime.funcdata
          107 2.1% 76.4% 107 2.1% runtime.funcspdelta
           67 1.3% 77.7% 67 1.3% runtime.topofstack
           63 1.2% 79.0% 63 1.2% runtime.newstack
           54 1.1% 80.0% 54 1.1% scanblock
           42 0.8% 80.9% 42 0.8% copyabletopsegment
           40 0.8% 81.6% 69 1.4% github.com/google/cayley/quad/cquads.parse
           39 0.8% 82.4% 88 1.7% runtime.convT2I
           37 0.7% 83.1% 214 4.2% github.com/petar/GoLLRB/llrb.less
           36 0.7% 83.8% 89 1.7% github.com/google/cayley/graph/memstore.Int64.Less
           35 0.7% 84.5% 35 0.7% runtime.memclr
           31 0.6% 85.1% 42 0.8% copyin
           29 0.6% 85.7% 29 0.6% runtime.memhash
           28 0.5% 86.2% 134 2.6% github.com/petar/GoLLRB/llrb.(*LLRB).replaceOrInsert
           28 0.5% 86.8% 73 1.4% runtime.mallocgc
           25 0.5% 87.3% 25 0.5% runtime.oldstack
           24 0.5% 87.7% 31 0.6% copyout
           24 0.5% 88.2% 24 0.5% flushptrbuf
           23 0.5% 88.7% 52 1.0% assertI2Tret
           23 0.5% 89.1% 116 2.3% github.com/petar/GoLLRB/llrb.(*LLRB).Get
           20 0.4% 89.5% 34 0.7% runtime.mapaccess2_faststr
           18 0.4% 89.9% 18 0.4% runtime.stackalloc
           17 0.3% 90.2% 19 0.4% runtime.mapaccess2_fast64
           16 0.3% 90.5% 31 0.6% github.com/barakmich/glog.V
           16 0.3% 90.8% 16 0.3% runtime.atomicloadp
           15 0.3% 91.1% 67 1.3% runtime.assertI2T
           13 0.3% 91.4% 21 0.4% runtime.mapaccess1_faststr
           13 0.3% 91.6% 13 0.3% runtime.stackfree
           12 0.2% 91.9% 24 0.5% MCentral_Grow
           12 0.2% 92.1% 61 1.2% fmt.(*pp).doPrintf
           12 0.2% 92.3% 13 0.3% github.com/petar/GoLLRB/llrb.walkUpRot23
           12 0.2% 92.6% 12 0.2% runtime.MSpan_Sweep
           11 0.2% 92.8% 22 0.4% evacuate
           11 0.2% 93.0% 69 1.4% github.com/petar/GoLLRB/llrb.(*LLRB).ascendGreaterOrEqual
           11 0.2% 93.2% 11 0.2% runtime.charntorune
           11 0.2% 93.4% 11 0.2% runtime.lessstack
           11 0.2% 93.6% 11 0.2% runtime.memcopy64
           11 0.2% 93.9% 26 0.5% runtime.stringtoslicerune
           10 0.2% 94.1% 10 0.2% markonly
           10 0.2% 94.2% 27 0.5% runtime.slicerunetostring
            9 0.2% 94.4% 9 0.2% runtime.gogo
            9 0.2% 94.6% 9 0.2% runtime.rewindmorestack
            9 0.2% 94.8% 9 0.2% runtime.round2
            8 0.2% 94.9% 10 0.2% fmt.(*fmt).integer

    Non-pointer-based:
    Total: 21026 samples
         7972 37.9% 37.9% 7972 37.9% runtime.findfunc
         3280 15.6% 53.5% 3280 15.6% readvarint
         1917 9.1% 62.6% 1917 9.1% runtime.gentraceback
         1771 8.4% 71.1% 1771 8.4% step
         1505 7.2% 78.2% 1505 7.2% pcvalue
          874 4.2% 82.4% 874 4.2% runtime.funcdata
          844 4.0% 86.4% 844 4.0% checkframecopy
          428 2.0% 88.4% 428 2.0% runtime.funcspdelta
          373 1.8% 90.2% 373 1.8% runtime.topofstack
          223 1.1% 91.3% 223 1.1% runtime.newstack
          179 0.9% 92.1% 179 0.9% copyabletopsegment
           96 0.5% 92.6% 96 0.5% runtime.oldstack
           95 0.5% 93.0% 95 0.5% runtime.memclr
           94 0.4% 93.5% 347 1.7% github.com/petar/GoLLRB/llrb.less
           75 0.4% 93.8% 75 0.4% runtime.round2
           73 0.3% 94.2% 96 0.5% copyin
           64 0.3% 94.5% 64 0.3% runtime.stackalloc
           56 0.3% 94.7% 56 0.3% scanblock
           53 0.3% 95.0% 277 1.3% github.com/petar/GoLLRB/llrb.(*LLRB).Get
           51 0.2% 95.2% 51 0.2% runtime.gotraceback
           46 0.2% 95.4% 113 0.5% github.com/google/cayley/graph/memstore.Int64.Less
           44 0.2% 95.7% 139 0.7% runtime.convT2I
           42 0.2% 95.9% 42 0.2% runtime.stackfree
           40 0.2% 96.0% 40 0.2% runtime.rewindmorestack
           38 0.2% 96.2% 106 0.5% github.com/google/cayley/quad/cquads.Parse
           35 0.2% 96.4% 35 0.2% runtime.lessstack
           33 0.2% 96.6% 44 0.2% copyout
           26 0.1% 96.7% 26 0.1% runtime.memhash
           25 0.1% 96.8% 25 0.1% runtime.gogo
           24 0.1% 96.9% 134 0.6% github.com/petar/GoLLRB/llrb.(*LLRB).replaceOrInsert
           23 0.1% 97.0% 58 0.3% assertI2Tret
           21 0.1% 97.1% 64 0.3% runtime.mallocgc
           18 0.1% 97.2% 18 0.1% runtime.mapaccess2_fast64
           16 0.1% 97.3% 16 0.1% flushptrbuf
           16 0.1% 97.4% 20 0.1% runtime.mapaccess1_faststr
           16 0.1% 97.4% 16 0.1% runtime.memmove
           15 0.1% 97.5% 15 0.1% runtime.atomicloadp
           14 0.1% 97.6% 66 0.3% github.com/petar/GoLLRB/llrb.(*LLRB).ascendGreaterOrEqual
           14 0.1% 97.6% 14 0.1% github.com/petar/GoLLRB/llrb.walkUpRot23
           14 0.1% 97.7% 14 0.1% itab
           14 0.1% 97.8% 14 0.1% runtime.memcopy64
           14 0.1% 97.8% 34 0.2% runtime.stringtoslicerune
           13 0.1% 97.9% 38 0.2% github.com/barakmich/glog.V
           13 0.1% 98.0% 71 0.3% runtime.assertI2T
           13 0.1% 98.0% 13 0.1% runtime.atomicload
           13 0.1% 98.1% 13 0.1% runtime.charntorune
           13 0.1% 98.1% 13 0.1% runtime.gostartcall
           13 0.1% 98.2% 34 0.2% runtime.slicerunetostring
           12 0.1% 98.3% 85 0.4% fmt.(*pp).doPrintf
           12 0.1% 98.3% 15 0.1% runtime.MSpan_Sweep


    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dave Cheney at Aug 6, 2014 at 1:23 am
    Can we get those as a .svg ? I have a suspicion more pressure has been
    placed on the stack.
    On Wednesday, 6 August 2014 10:50:00 UTC+10, kortschak wrote:

    More complete top lists below.

    I can't see anything that explains why the concrete type takes so much
    longer. It just looks like there space-time continuum is stretched in
    that case (there are more samples taken, but there doesn't seem to be
    any particular function that is over represented in the concrete case).

    I'm profiling on linux am64, so the osX issue is not the problem. What
    things are not caught by the profiler that might be causing this slow
    down?


    Pointer-based:
    Total: 5110 samples
    1685 33.0% 33.0% 1685 33.0% runtime.findfunc
    678 13.3% 46.2% 678 13.3% readvarint
    429 8.4% 54.6% 429 8.4% runtime.gentraceback
    362 7.1% 61.7% 362 7.1% step
    324 6.3% 68.1% 324 6.3% pcvalue
    165 3.2% 71.3% 165 3.2% checkframecopy
    156 3.1% 74.3% 156 3.1% runtime.funcdata
    107 2.1% 76.4% 107 2.1% runtime.funcspdelta
    67 1.3% 77.7% 67 1.3% runtime.topofstack
    63 1.2% 79.0% 63 1.2% runtime.newstack
    54 1.1% 80.0% 54 1.1% scanblock
    42 0.8% 80.9% 42 0.8% copyabletopsegment
    40 0.8% 81.6% 69 1.4%
    github.com/google/cayley/quad/cquads.parse
    39 0.8% 82.4% 88 1.7% runtime.convT2I
    37 0.7% 83.1% 214 4.2% github.com/petar/GoLLRB/llrb.less
    36 0.7% 83.8% 89 1.7%
    github.com/google/cayley/graph/memstore.Int64.Less
    35 0.7% 84.5% 35 0.7% runtime.memclr
    31 0.6% 85.1% 42 0.8% copyin
    29 0.6% 85.7% 29 0.6% runtime.memhash
    28 0.5% 86.2% 134 2.6%
    github.com/petar/GoLLRB/llrb.(*LLRB).replaceOrInsert
    28 0.5% 86.8% 73 1.4% runtime.mallocgc
    25 0.5% 87.3% 25 0.5% runtime.oldstack
    24 0.5% 87.7% 31 0.6% copyout
    24 0.5% 88.2% 24 0.5% flushptrbuf
    23 0.5% 88.7% 52 1.0% assertI2Tret
    23 0.5% 89.1% 116 2.3%
    github.com/petar/GoLLRB/llrb.(*LLRB).Get
    20 0.4% 89.5% 34 0.7% runtime.mapaccess2_faststr
    18 0.4% 89.9% 18 0.4% runtime.stackalloc
    17 0.3% 90.2% 19 0.4% runtime.mapaccess2_fast64
    16 0.3% 90.5% 31 0.6% github.com/barakmich/glog.V
    16 0.3% 90.8% 16 0.3% runtime.atomicloadp
    15 0.3% 91.1% 67 1.3% runtime.assertI2T
    13 0.3% 91.4% 21 0.4% runtime.mapaccess1_faststr
    13 0.3% 91.6% 13 0.3% runtime.stackfree
    12 0.2% 91.9% 24 0.5% MCentral_Grow
    12 0.2% 92.1% 61 1.2% fmt.(*pp).doPrintf
    12 0.2% 92.3% 13 0.3%
    github.com/petar/GoLLRB/llrb.walkUpRot23
    12 0.2% 92.6% 12 0.2% runtime.MSpan_Sweep
    11 0.2% 92.8% 22 0.4% evacuate
    11 0.2% 93.0% 69 1.4%
    github.com/petar/GoLLRB/llrb.(*LLRB).ascendGreaterOrEqual
    11 0.2% 93.2% 11 0.2% runtime.charntorune
    11 0.2% 93.4% 11 0.2% runtime.lessstack
    11 0.2% 93.6% 11 0.2% runtime.memcopy64
    11 0.2% 93.9% 26 0.5% runtime.stringtoslicerune
    10 0.2% 94.1% 10 0.2% markonly
    10 0.2% 94.2% 27 0.5% runtime.slicerunetostring
    9 0.2% 94.4% 9 0.2% runtime.gogo
    9 0.2% 94.6% 9 0.2% runtime.rewindmorestack
    9 0.2% 94.8% 9 0.2% runtime.round2
    8 0.2% 94.9% 10 0.2% fmt.(*fmt).integer

    Non-pointer-based:
    Total: 21026 samples
    7972 37.9% 37.9% 7972 37.9% runtime.findfunc
    3280 15.6% 53.5% 3280 15.6% readvarint
    1917 9.1% 62.6% 1917 9.1% runtime.gentraceback
    1771 8.4% 71.1% 1771 8.4% step
    1505 7.2% 78.2% 1505 7.2% pcvalue
    874 4.2% 82.4% 874 4.2% runtime.funcdata
    844 4.0% 86.4% 844 4.0% checkframecopy
    428 2.0% 88.4% 428 2.0% runtime.funcspdelta
    373 1.8% 90.2% 373 1.8% runtime.topofstack
    223 1.1% 91.3% 223 1.1% runtime.newstack
    179 0.9% 92.1% 179 0.9% copyabletopsegment
    96 0.5% 92.6% 96 0.5% runtime.oldstack
    95 0.5% 93.0% 95 0.5% runtime.memclr
    94 0.4% 93.5% 347 1.7% github.com/petar/GoLLRB/llrb.less
    75 0.4% 93.8% 75 0.4% runtime.round2
    73 0.3% 94.2% 96 0.5% copyin
    64 0.3% 94.5% 64 0.3% runtime.stackalloc
    56 0.3% 94.7% 56 0.3% scanblock
    53 0.3% 95.0% 277 1.3%
    github.com/petar/GoLLRB/llrb.(*LLRB).Get
    51 0.2% 95.2% 51 0.2% runtime.gotraceback
    46 0.2% 95.4% 113 0.5%
    github.com/google/cayley/graph/memstore.Int64.Less
    44 0.2% 95.7% 139 0.7% runtime.convT2I
    42 0.2% 95.9% 42 0.2% runtime.stackfree
    40 0.2% 96.0% 40 0.2% runtime.rewindmorestack
    38 0.2% 96.2% 106 0.5%
    github.com/google/cayley/quad/cquads.Parse
    35 0.2% 96.4% 35 0.2% runtime.lessstack
    33 0.2% 96.6% 44 0.2% copyout
    26 0.1% 96.7% 26 0.1% runtime.memhash
    25 0.1% 96.8% 25 0.1% runtime.gogo
    24 0.1% 96.9% 134 0.6%
    github.com/petar/GoLLRB/llrb.(*LLRB).replaceOrInsert
    23 0.1% 97.0% 58 0.3% assertI2Tret
    21 0.1% 97.1% 64 0.3% runtime.mallocgc
    18 0.1% 97.2% 18 0.1% runtime.mapaccess2_fast64
    16 0.1% 97.3% 16 0.1% flushptrbuf
    16 0.1% 97.4% 20 0.1% runtime.mapaccess1_faststr
    16 0.1% 97.4% 16 0.1% runtime.memmove
    15 0.1% 97.5% 15 0.1% runtime.atomicloadp
    14 0.1% 97.6% 66 0.3%
    github.com/petar/GoLLRB/llrb.(*LLRB).ascendGreaterOrEqual
    14 0.1% 97.6% 14 0.1%
    github.com/petar/GoLLRB/llrb.walkUpRot23
    14 0.1% 97.7% 14 0.1% itab
    14 0.1% 97.8% 14 0.1% runtime.memcopy64
    14 0.1% 97.8% 34 0.2% runtime.stringtoslicerune
    13 0.1% 97.9% 38 0.2% github.com/barakmich/glog.V
    13 0.1% 98.0% 71 0.3% runtime.assertI2T
    13 0.1% 98.0% 13 0.1% runtime.atomicload
    13 0.1% 98.1% 13 0.1% runtime.charntorune
    13 0.1% 98.1% 13 0.1% runtime.gostartcall
    13 0.1% 98.2% 34 0.2% runtime.slicerunetostring
    12 0.1% 98.3% 85 0.4% fmt.(*pp).doPrintf
    12 0.1% 98.3% 15 0.1% runtime.MSpan_Sweep

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dan Kortschak at Aug 6, 2014 at 1:39 am

    On Tue, 2014-08-05 at 18:23 -0700, Dave Cheney wrote:
    Can we get those as a .svg ? I have a suspicion more pressure has been
    placed on the stack.
    Thanks, Dave. That's sort of what I thought might be going on. SVGs
    attached.

    Dan

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dave Cheney at Aug 6, 2014 at 2:14 am
    Very hard to say, both look like they are equally stressing the stack
    copying logic; all other actions are dwarfed by this operation so a
    small change in the way you use the stack probably accounts for the
    difference in runtime, even though your change may have otherwise been
    innocuous.

    On Wed, Aug 6, 2014 at 11:39 AM, Dan Kortschak
    wrote:
    On Tue, 2014-08-05 at 18:23 -0700, Dave Cheney wrote:
    Can we get those as a .svg ? I have a suspicion more pressure has been
    placed on the stack.
    Thanks, Dave. That's sort of what I thought might be going on. SVGs
    attached.

    Dan
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dan Kortschak at Aug 6, 2014 at 2:17 am
    Yeah, that was pretty much how I was seeing it so thanks for giving your
    view. It's unfortunate that a relatively small change that otherwise
    gives quite good benefits has such a significant downside in the
    pathological case.
    On Wed, 2014-08-06 at 12:14 +1000, Dave Cheney wrote:
    Very hard to say, both look like they are equally stressing the stack
    copying logic; all other actions are dwarfed by this operation so a
    small change in the way you use the stack probably accounts for the
    difference in runtime, even though your change may have otherwise been
    innocuous.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dmitry Vyukov at Aug 6, 2014 at 7:20 am
    Why does the program spend so much time in stack management? Do you
    have lots of very short running goroutines, each requiring lots of
    stack?


    On Wed, Aug 6, 2014 at 6:17 AM, Dan Kortschak
    wrote:
    Yeah, that was pretty much how I was seeing it so thanks for giving your
    view. It's unfortunate that a relatively small change that otherwise
    gives quite good benefits has such a significant downside in the
    pathological case.
    On Wed, 2014-08-06 at 12:14 +1000, Dave Cheney wrote:
    Very hard to say, both look like they are equally stressing the stack
    copying logic; all other actions are dwarfed by this operation so a
    small change in the way you use the stack probably accounts for the
    difference in runtime, even though your change may have otherwise been
    innocuous.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dave Cheney at Aug 6, 2014 at 8:01 am
    That was my suspicion. I'm guessing that each request or translation is a
    new goroutine which starts life by loading up a big buffer of those quad
    structures.
    On 6 Aug 2014 17:20, "Dmitry Vyukov" wrote:

    Why does the program spend so much time in stack management? Do you
    have lots of very short running goroutines, each requiring lots of
    stack?


    On Wed, Aug 6, 2014 at 6:17 AM, Dan Kortschak
    wrote:
    Yeah, that was pretty much how I was seeing it so thanks for giving your
    view. It's unfortunate that a relatively small change that otherwise
    gives quite good benefits has such a significant downside in the
    pathological case.
    On Wed, 2014-08-06 at 12:14 +1000, Dave Cheney wrote:
    Very hard to say, both look like they are equally stressing the stack
    copying logic; all other actions are dwarfed by this operation so a
    small change in the way you use the stack probably accounts for the
    difference in runtime, even though your change may have otherwise been
    innocuous.

    --
    You received this message because you are subscribed to the Google
    Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dan Kortschak at Aug 6, 2014 at 8:21 am
    I'm afraid that is something that I can't answer - I have not spent any
    time in the query handling code.

    Query handling in this benchmark is done via javascript through otto. A
    cursory glance shows no significant use of goroutines. I'll point the
    author of that code to your suggestion though.

    thanks
    On Wed, 2014-08-06 at 11:20 +0400, Dmitry Vyukov wrote:
    Why does the program spend so much time in stack management? Do you
    have lots of very short running goroutines, each requiring lots of
    stack?

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dave Cheney at Aug 6, 2014 at 8:26 am
    Its not really a suggestion, more accurately a shot in the dark.
    On 6 Aug 2014 18:21, "Dan Kortschak" wrote:

    I'm afraid that is something that I can't answer - I have not spent any
    time in the query handling code.

    Query handling in this benchmark is done via javascript through otto. A
    cursory glance shows no significant use of goroutines. I'll point the
    author of that code to your suggestion though.

    thanks
    On Wed, 2014-08-06 at 11:20 +0400, Dmitry Vyukov wrote:
    Why does the program spend so much time in stack management? Do you
    have lots of very short running goroutines, each requiring lots of
    stack?
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dan Kortschak at Aug 6, 2014 at 8:41 am

    On Wed, 2014-08-06 at 18:26 +1000, Dave Cheney wrote:
    Its not really a suggestion, more accurately a shot in the dark.
    Tomayto, tomahto. It's a start.

    thanks

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dmitry Vyukov at Aug 6, 2014 at 8:59 am

    On Wed, Aug 6, 2014 at 12:41 PM, Dan Kortschak wrote:
    On Wed, 2014-08-06 at 18:26 +1000, Dave Cheney wrote:
    Its not really a suggestion, more accurately a shot in the dark.
    Tomayto, tomahto. It's a start.


    I think I see what happens. THere is runtime.oldstack in the profile.
    It means that the code uses segmented stacks. Probably it splits stack
    on some non-copyable C function in runtime (most likely some interface
    conversion, also in the profile), analyzes current stack and figures
    out that it's non-copyable, so uses an additional segment. Then
    quickly returns from the C function, releases the segment. And the
    story repeats. So it's hot split problem worsened by growable stacks.
    This will be fixed in 1.5. The program will become 5 times faster.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dan Kortschak at Aug 6, 2014 at 9:04 am

    On Wed, 2014-08-06 at 12:59 +0400, Dmitry Vyukov wrote:
    I think I see what happens. THere is runtime.oldstack in the profile.
    It means that the code uses segmented stacks. Probably it splits stack
    on some non-copyable C function in runtime (most likely some interface
    conversion, also in the profile), analyzes current stack and figures
    out that it's non-copyable, so uses an additional segment. Then
    quickly returns from the C function, releases the segment. And the
    story repeats. So it's hot split problem worsened by growable stacks.
    This will be fixed in 1.5. The program will become 5 times faster.
    Thanks, Dmitri. That makes sense.

    Dan

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedAug 5, '14 at 2:06p
activeAug 6, '14 at 9:04a
posts15
users4
websitegolang.org

People

Translate

site design / logo © 2021 Grokbase