Grokbase › Groups › Go › golang-dev › September 2012
FAQ

On 2012/09/11 19:50:49, Sebastien Paolacci wrote:
On 2012/09/11 03:25:12, dvyukov wrote:
On 2012/08/29 06:53:20, Sebastien Paolacci wrote:
PTAL.

As already observed by Dmitry, numbers really get crazy (and
stable) for any
cpu
number greater than 8.
Does anybody have an explanation? It seems to correlate with number
of
connections. Is it a kernel issue?
Kernel is the easy culprit.. but I honestly first thought about Go's
scheduler;)
. I have the very same behavior on both 2.6.32 and 3.2 kernels, but
didn't found
any time to further investigate.
I observe it on 3.6.38 with both current and my new scheduler and poll
server.
I didn't investigate around the "Persistent*" variants being
slowed-down for
cpu=1, it just seems counter intuitive at first.
I think it has to do with the fact that now you have 8 additional
polling
goroutines. It must go away once you limit number of poll servers.
I think I still don't get it. I'm puzzled about how the connection
persistence
"alone" can affect that patch so much.
It obviously must have some explanation, but it is not natural to me. I'll run
some more tests (the patch was actually left alone since the initial
upload).

Each pollServer is a separate thread and goroutine. The hypothesis is
when you have 8 pollServers with GOMAXPROCS=1, there are additional
latencies and penalties caused by constant thread and goroutine
switching where each goroutine does very little work.


http://codereview.appspot.com/6496054/

Search Discussions

  • Dvyukov at Sep 16, 2012 at 5:07 am ⇧

    On 2012/09/11 19:30:26, Sebastien Paolacci wrote:
    On 2012/09/11 02:58:02, rsc wrote:
    It would probably be okay to create poll servers on demand up to the
    max?
    A pollServer, despite its name, might not be a so heavy beast. I'll try to
    measure it tomorrow (out of curiosity), and go for a lazy
    instantiation unless
    someone really advocates the opposite.
    Each pollServer is a dedicated persistent thread + I think it may
    benefit from some amount of batching.

    http://codereview.appspot.com/6496054/
  • Dvyukov at Sep 16, 2012 at 5:09 am ⇧

    On 2012/09/11 20:05:38, Sebastien Paolacci wrote:
    On 2012/09/11 03:29:55, dvyukov wrote:
    It should be limited by min(GOMAXPROCS, NumCPU), because it makes no
    sense to
    have 8 poll servers with GOMAXPROCS=1 (I believe that is what causes
    slowdown
    on
    BenchmarkTCPPersistent-1).
    I think we should avoid GOMAXPROCS and use something more static.
    Every process
    that is playing with GOMAXPROC, e.g `go test', would otherwise miss the point
    (and I would avoid dynamically reassigning fds).
    I would consider that anything beyond NumCPU is useless, that anything smaller
    is not really damaging, and that anything greater than 8 currently
    doesn't bring
    any improvements.
    Min(NumCPU, 8) + lazy instantiation as suggested by Dave/Russ ?
    25% performance penalty does not look particularly cool. I would
    experiment with Min(GOMAXPROCS, NumCPU, 8). I think you do not need to
    re-distribute fds, just grow up to Min(GOMAXPROCS, NumCPU, 8) using
    current GOMAXPROCS value. I do not care a lot about programs that "play
    with GOMAXPROCS" (especially about go test).


    http://codereview.appspot.com/6496054/

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-dev @
categoriesgo
postedSep 16, '12 at 5:05a
activeSep 16, '12 at 5:09a
posts3
users1
websitegolang.org

1 user in discussion

Dvyukov: 3 posts

Content

People

Support

Translate

site design / logo © 2023 Grokbase