FAQ
I‘m building a toy key-value store and I’m trying to compute time stats as
I run unit tests. I get rubbish numbers; they are way too high for Put/
Delete operations, but they make sense for Get. I‘ve been trying to figure
this out all night and can’t see clear anymore, I‘d love a second pair of
eyes. I’ve investigated integer overflows, however time.Duration is a int64which I don't think should overflow.

To gather my metrics, I have a channel that received time.Duration objects:

https://github.com/aybabtme/dskvs/blob/proto/concurrency_test.go#L188

Depending on the operation I want to measure, I fill the channel this way
(see ctx.dur):

https://github.com/aybabtme/dskvs/blob/proto/concurrency_test.go#L196

In the case of a Get request (line 211), I get usable numbers:

2013/07/04 07:38:56 Get operations
2013/07/04 07:38:56 N=2048
      total = 0.004893s
      min = 1527ns
      avg = 2389ns
      med = 1919ns
      max = 137159ns
      p50 = 2968ns
      p9 = 6549ns
      p99 = 31446ns
      p999 = 116621ns
      p9999 = 0ns

However for Put and Delete, the number are rubbish (I know they are because
the test doesn't take 145s to run, neither is the smallest Put done in
22ms):

2013/07/04 07:38:56 Put operations
2013/07/04 07:38:56 N=2048
      total = 145.633327s
      min = 21378504ns
      avg = 71110022ns
      med = 70096754ns
      max = 115325380ns
      p50 = 92633901ns
      p9 = 110740221ns
      p99 = 114924456ns
      p999 = 115311230ns
      p9999 = 0ns

I've handrolled my own little stats struct, this is where the numbers come
from:

https://github.com/aybabtme/dskvs/blob/proto/stats.go

Does anybody have an idea as to what's going on?

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Antoine Grondin at Jul 4, 2013 at 7:59 am
    Sorry for the fancy colors, something got wrong with my Markdown chrome
    plugin.
    On Thursday, July 4, 2013 3:54:14 AM UTC-4, Antoine Grondin wrote:

    I‘m building a toy key-value store and I’m trying to compute time stats as
    I run unit tests. I get rubbish numbers; they are way too high for Put/
    Delete operations, but they make sense for Get. I‘ve been trying to
    figure this out all night and can’t see clear anymore, I‘d love a second
    pair of eyes. I’ve investigated integer overflows, however time.Durationis a
    int64 which I don't think should overflow.

    To gather my metrics, I have a channel that received time.Durationobjects:

    https://github.com/aybabtme/dskvs/blob/proto/concurrency_test.go#L188

    Depending on the operation I want to measure, I fill the channel this way
    (see ctx.dur):

    https://github.com/aybabtme/dskvs/blob/proto/concurrency_test.go#L196

    In the case of a Get request (line 211), I get usable numbers:

    2013/07/04 07:38:56 Get operations
    2013/07/04 07:38:56 N=2048
    total = 0.004893s
    min = 1527ns
    avg = 2389ns
    med = 1919ns
    max = 137159ns
    p50 = 2968ns
    p9 = 6549ns
    p99 = 31446ns
    p999 = 116621ns
    p9999 = 0ns

    However for Put and Delete, the number are rubbish (I know they are
    because the test doesn't take 145s to run, neither is the smallest Put done
    in 22ms):

    2013/07/04 07:38:56 Put operations
    2013/07/04 07:38:56 N=2048
    total = 145.633327s
    min = 21378504ns
    avg = 71110022ns
    med = 70096754ns
    max = 115325380ns
    p50 = 92633901ns
    p9 = 110740221ns
    p99 = 114924456ns
    p999 = 115311230ns
    p9999 = 0ns

    I've handrolled my own little stats struct, this is where the numbers come
    from:

    https://github.com/aybabtme/dskvs/blob/proto/stats.go

    Does anybody have an idea as to what's going on?
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Rémy Oudompheng at Jul 4, 2013 at 8:10 am
    Le 4 juil. 2013 08:54, "Antoine Grondin" <antoinegrondin@gmail.com> a
    écrit :
    I‘m building a toy key-value store and I’m trying to compute time stats
    as I run unit tests. I get rubbish numbers; they are way too high for
    Put/Delete operations, but they make sense for Get. I‘ve been trying to
    figure this out all night and can’t see clear anymore, I‘d love a second
    pair of eyes. I’ve investigated integer overflows, however time.Duration is
    a int64 which I don't think should overflow.
    >

    You are measuring durations that overlap because they come from concurrent
    goroutines.

    Rémy

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Antoine Grondin at Jul 4, 2013 at 8:14 am
    I'm not sure I follow, what would be the right way to do?
    On Thursday, July 4, 2013 4:10:39 AM UTC-4, Rémy Oudompheng wrote:


    Le 4 juil. 2013 08:54, "Antoine Grondin" <antoine...@gmail.com<javascript:>>
    a écrit :
    I‘m building a toy key-value store and I’m trying to compute time stats
    as I run unit tests. I get rubbish numbers; they are way too high for
    Put/Delete operations, but they make sense for Get. I‘ve been trying to
    figure this out all night and can’t see clear anymore, I‘d love a second
    pair of eyes. I’ve investigated integer overflows, however time.Duration is
    a int64 which I don't think should overflow.
    You are measuring durations that overlap because they come from concurrent
    goroutines.

    Rémy
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Antoine Grondin at Jul 5, 2013 at 1:32 am
    I think I understand, the `t0 := time.Now()` are all executed at once, then
    later all the `dT := time.Since(t0)` are also done at the same time. So
    they all measure the same duration instead of measuring their individual
    ones.

    To measure each calls individually (and get an accurate total seconds), I
    have to run the Put statements in the same goroutine.

    Please correct me if I'm wrong.
    On Thursday, July 4, 2013 4:10:39 AM UTC-4, Rémy Oudompheng wrote:


    Le 4 juil. 2013 08:54, "Antoine Grondin" <antoine...@gmail.com<javascript:>>
    a écrit :
    I‘m building a toy key-value store and I’m trying to compute time stats
    as I run unit tests. I get rubbish numbers; they are way too high for
    Put/Delete operations, but they make sense for Get. I‘ve been trying to
    figure this out all night and can’t see clear anymore, I‘d love a second
    pair of eyes. I’ve investigated integer overflows, however time.Duration is
    a int64 which I don't think should overflow.
    You are measuring durations that overlap because they come from concurrent
    goroutines.

    Rémy
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedJul 4, '13 at 7:54a
activeJul 5, '13 at 1:32a
posts5
users2
websitegolang.org

People

Translate

site design / logo © 2022 Grokbase