Grokbase › Groups › Go › golang-nuts › September 2012
FAQ
Hello,

It seems one of the requirements of creating a go server that handles a
large number of requests per second is creating little garbage and making
sure that large data structures follow certain rules. This seems to boil
down to eliminating the use of pointers passed between functions or through
channels, using offsets into arrays instead of pointers to elements within
large data structures, and in general not using pointers in large data
structures.

I am assuming that the use of an interface results in heap allocated data
which is then pointed to by the interface value, therefore interfaces can
be lumped into this.

This means you have to remove all pointers/interfaces from function
signatures and channels that are involved in the critical path. This would
be no trivial task particularly considering the standard library makes
heavy use of interfaces, not to mention interfaces being one of the key
abstractions in go.

It seems to me that go tries to be a clean and simple language, but when
you're writing high performance software, you end up having to write go in
a rather obtuse way... and I'm mainly talking about avoiding the passing of
pointers and interfaces between functions or across channels. The data
structure restrictions are pretty inconvenient but, ultimately I think you
don't have to sacrifice too much to follow best practices there.

I'd like to see comments related to the above relative to go1. Also, how
might go change in the future regarding improved GC, escape analysis, and
maybe even addition to the language syntax itself to help with any of these
issues.

Thanks,
-Jason

Search Discussions

  • Kyle Lemons at Sep 4, 2012 at 11:39 pm ⇧

    On Tue, Sep 4, 2012 at 1:52 PM, wrote:

    Hello,

    It seems one of the requirements of creating a go server that handles a
    large number of requests per second is creating little garbage and making
    sure that large data structures follow certain rules. This seems to boil
    down to eliminating the use of pointers passed between functions or through
    channels, using offsets into arrays instead of pointers to elements within
    large data structures, and in general not using pointers in large data
    structures.

    I am assuming that the use of an interface results in heap allocated data
    which is then pointed to by the interface value, therefore interfaces can
    be lumped into this.

    This means you have to remove all pointers/interfaces from function
    signatures and channels that are involved in the critical path. This would
    be no trivial task particularly considering the standard library makes
    heavy use of interfaces, not to mention interfaces being one of the key
    abstractions in go.

    It seems to me that go tries to be a clean and simple language, but when
    you're writing high performance software, you end up having to write go in
    a rather obtuse way... and I'm mainly talking about avoiding the passing of
    pointers and interfaces between functions or across channels. The data
    structure restrictions are pretty inconvenient but, ultimately I think you
    don't have to sacrifice too much to follow best practices there.

    I'd like to see comments related to the above relative to go1. Also, how
    might go change in the future regarding improved GC, escape analysis, and
    maybe even addition to the language syntax itself to help with any of these
    issues.
    There have already been improvements since go1, and there will continue to
    be more.

    Opinion:
    If (and I stress, *if*) you have written an idiomatic piece of Go software
    that is bumping against its performance limits after common-sense
    optimization and you cannot scale out in a different direction for whatever
    reason, then you can start to do more thorough benchmarks and streamline
    the things you mention above. In terms of the amount of money you pay in
    programmer time versus the cost of an extra server, my possibly naive
    suspicion is that this trade-off won't be appropriate very often. In any
    other case, just write it the "right" way and let the compilers and runtime
    continue to improve. The goal is not to require you to think about such
    things in general, especially since a sufficiently advanced compiler or
    runtime can optimize many of them for you.
  • Jason Eggleston at Sep 5, 2012 at 4:35 am ⇧
    Kyle,

    If the server currently has GC times of 500+ milliseconds, the
    solution of using 100 times more hardware to get GC times to 5
    milliseconds doesn't sound like a reasonable answer to me. And that's
    assuming that 100% of the memory the GC has to traverse is related to
    garbage generated per request.

    However there is one technique I did leave off of the table which is
    to use channels / free lists to re-use memory allocations. I suppose
    that is a way to continue to use interfaces and pointers without
    generating more garbage.

    Thanks,
    -Jason
    On Tue, Sep 4, 2012 at 4:02 PM, Kyle Lemons wrote:
    On Tue, Sep 4, 2012 at 1:52 PM, wrote:

    Hello,

    It seems one of the requirements of creating a go server that handles a
    large number of requests per second is creating little garbage and making
    sure that large data structures follow certain rules. This seems to boil
    down to eliminating the use of pointers passed between functions or through
    channels, using offsets into arrays instead of pointers to elements within
    large data structures, and in general not using pointers in large data
    structures.

    I am assuming that the use of an interface results in heap allocated data
    which is then pointed to by the interface value, therefore interfaces can be
    lumped into this.

    This means you have to remove all pointers/interfaces from function
    signatures and channels that are involved in the critical path. This would
    be no trivial task particularly considering the standard library makes heavy
    use of interfaces, not to mention interfaces being one of the key
    abstractions in go.

    It seems to me that go tries to be a clean and simple language, but when
    you're writing high performance software, you end up having to write go in a
    rather obtuse way... and I'm mainly talking about avoiding the passing of
    pointers and interfaces between functions or across channels. The data
    structure restrictions are pretty inconvenient but, ultimately I think you
    don't have to sacrifice too much to follow best practices there.

    I'd like to see comments related to the above relative to go1. Also, how
    might go change in the future regarding improved GC, escape analysis, and
    maybe even addition to the language syntax itself to help with any of these
    issues.

    There have already been improvements since go1, and there will continue to
    be more.

    Opinion:
    If (and I stress, *if*) you have written an idiomatic piece of Go software
    that is bumping against its performance limits after common-sense
    optimization and you cannot scale out in a different direction for whatever
    reason, then you can start to do more thorough benchmarks and streamline the
    things you mention above. In terms of the amount of money you pay in
    programmer time versus the cost of an extra server, my possibly naive
    suspicion is that this trade-off won't be appropriate very often. In any
    other case, just write it the "right" way and let the compilers and runtime
    continue to improve. The goal is not to require you to think about such
    things in general, especially since a sufficiently advanced compiler or
    runtime can optimize many of them for you.
  • Rémy Oudompheng at Sep 5, 2012 at 6:07 am ⇧

    On 2012/9/5 Jason Eggleston wrote:
    Kyle,

    If the server currently has GC times of 500+ milliseconds, the
    solution of using 100 times more hardware to get GC times to 5
    milliseconds doesn't sound like a reasonable answer to me. And that's
    assuming that 100% of the memory the GC has to traverse is related to
    garbage generated per request.
    What is the situation you are talking about? What kind of server are
    you writing ?

    Rémy.
  • Jason Eggleston at Sep 5, 2012 at 1:16 pm ⇧
    All I can say is it is a server performing hundreds of requests per
    second. The raw speed of go in between garbage collection runs is
    very impressive. I suspect free lists are the idiomatic way to reduce
    the per request garbage generated at this point.

    -Jason

    On Tue, Sep 4, 2012 at 11:00 PM, Rémy Oudompheng
    wrote:
    What is the situation you are talking about? What kind of server are
    you writing ?

    Rémy.
  • Roger peppe at Sep 5, 2012 at 6:03 am ⇧

    On 4 September 2012 21:52, wrote:
    I am assuming that the use of an interface results in heap allocated data
    which is then pointed to by the interface value, therefore interfaces can be
    lumped into this.
    Not necessarily. If the data held in the interface fits within the size
    of a pointer, then no allocation is necessary.
  • Paul at Sep 5, 2012 at 1:44 pm ⇧
    Hallo Jason,

    thanks for the question. I assume you are talking about a server that
    services requests coming out of the Internet.

    There is a new runtime/scheduler in development right now. It is already
    showing very substancial performance improvements in its relatively early
    stages, also in terms of scalability. Have a look at the last few posts in
    this thread:
    https://groups.google.com/forum/?fromgroups=#!topic/golang-nuts/hkd5fjWIGmY

    I think the new scheduler is still quite a ways from being finished
    however.

    So the overall increase in throughput would likely leave more time for GC
    in a given time window. It seems to me the biggest issue with the GC now is
    that it hast to stop the world.


    On Tuesday, September 4, 2012 10:52:45 PM UTC+2, [email protected] wrote:

    Hello,

    It seems one of the requirements of creating a go server that handles a
    large number of requests per second is creating little garbage and making
    sure that large data structures follow certain rules. This seems to boil
    down to eliminating the use of pointers passed between functions or through
    channels, using offsets into arrays instead of pointers to elements within
    large data structures, and in general not using pointers in large data
    structures.

    I am assuming that the use of an interface results in heap allocated data
    which is then pointed to by the interface value, therefore interfaces can
    be lumped into this.

    This means you have to remove all pointers/interfaces from function
    signatures and channels that are involved in the critical path. This would
    be no trivial task particularly considering the standard library makes
    heavy use of interfaces, not to mention interfaces being one of the key
    abstractions in go.

    It seems to me that go tries to be a clean and simple language, but when
    you're writing high performance software, you end up having to write go in
    a rather obtuse way... and I'm mainly talking about avoiding the passing of
    pointers and interfaces between functions or across channels. The data
    structure restrictions are pretty inconvenient but, ultimately I think you
    don't have to sacrifice too much to follow best practices there.

    I'd like to see comments related to the above relative to go1. Also, how
    might go change in the future regarding improved GC, escape analysis, and
    maybe even addition to the language syntax itself to help with any of these
    issues.

    Thanks,
    -Jason

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedSep 4, '12 at 10:35p
activeSep 5, '12 at 1:44p
posts7
users5
websitegolang.org

5 users in discussion

Jason Eggleston: 3 posts Rémy Oudompheng: 1 post Paul: 1 post Roger peppe: 1 post Kyle Lemons: 1 post

Content

People

Support

Translate

site design / logo © 2023 Grokbase