FAQ
Hi all,

I just had an interesting conversation with a friend. I think the
problem we are talking about is quite open ended. So I would like to
share it and see what others say about it.

Let's first take a look at the problem: My friend is working on a
service which needs to handle lots of concurrent TCP connections. As you
can imagine, I suggest him to use Go.

He then told me that his company would like to handle ~100K connections
per server. His colleague made a quick benchmark in Go the other day and
the Go program handles 5K connections by utilizing 200% of the CPU.

5K connections is apparently not the maximum that a Go program can get.
(As far as I know, vitess claims it can handle 10K connections.)
Moreover, such benchmark relies on the underlying hardware, OS
configuration, etc. (I'm unable to get details about the benchmark or
the configuration information.)

Although it's not a new problem to this field, his problem made me
google some existing solutions/suggestions about handling large amount
of concurrent TCP connections. For something like node.js, we need some
tweaks[1]. Eventually, "The kernel isn’t the solution. The kernel is the
problem."

Specific to Go, I didn't find anything related to handling large amount
of concurrent TCP connections. But I believe there must be someone who
has the experience of working on such scale. It would be great if you
can share here. Any suggestion/discussion is welcome. As I mentioned,
this is an open ended problem. I won't mind if you throw me some crazy idea.

[1]
http://blog.caustik.com/2012/04/08/scaling-node-js-to-100k-concurrent-connections/
[2]
http://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concurrent-connections-the-kernel-i.html

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Dave Cheney at Jul 22, 2013 at 4:31 am
    We've heard on this mailing list form people who (I believe) run large
    chat servers that host upwards of half a million concurrent
    connections. The main problem the OP had was the amount of per
    connection memory his application used. From memory it was 28k per
    connection.
    On Mon, Jul 22, 2013 at 2:28 PM, Monnand wrote:
    Hi all,

    I just had an interesting conversation with a friend. I think the problem we
    are talking about is quite open ended. So I would like to share it and see
    what others say about it.

    Let's first take a look at the problem: My friend is working on a service
    which needs to handle lots of concurrent TCP connections. As you can
    imagine, I suggest him to use Go.

    He then told me that his company would like to handle ~100K connections per
    server. His colleague made a quick benchmark in Go the other day and the Go
    program handles 5K connections by utilizing 200% of the CPU.

    5K connections is apparently not the maximum that a Go program can get. (As
    far as I know, vitess claims it can handle 10K connections.) Moreover, such
    benchmark relies on the underlying hardware, OS configuration, etc. (I'm
    unable to get details about the benchmark or the configuration information.)

    Although it's not a new problem to this field, his problem made me google
    some existing solutions/suggestions about handling large amount of
    concurrent TCP connections. For something like node.js, we need some
    tweaks[1]. Eventually, "The kernel isn’t the solution. The kernel is the
    problem."

    Specific to Go, I didn't find anything related to handling large amount of
    concurrent TCP connections. But I believe there must be someone who has the
    experience of working on such scale. It would be great if you can share
    here. Any suggestion/discussion is welcome. As I mentioned, this is an open
    ended problem. I won't mind if you throw me some crazy idea.

    [1]
    http://blog.caustik.com/2012/04/08/scaling-node-js-to-100k-concurrent-connections/
    [2]
    http://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concurrent-connections-the-kernel-i.html

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Dmitry Vyukov at Jul 22, 2013 at 9:57 am

    On Mon, Jul 22, 2013 at 8:28 AM, Monnand wrote:
    Hi all,

    I just had an interesting conversation with a friend. I think the problem we
    are talking about is quite open ended. So I would like to share it and see
    what others say about it.

    Let's first take a look at the problem: My friend is working on a service
    which needs to handle lots of concurrent TCP connections. As you can
    imagine, I suggest him to use Go.

    He then told me that his company would like to handle ~100K connections per
    server. His colleague made a quick benchmark in Go the other day and the Go
    program handles 5K connections by utilizing 200% of the CPU.

    5K connections is apparently not the maximum that a Go program can get. (As
    far as I know, vitess claims it can handle 10K connections.) Moreover, such
    benchmark relies on the underlying hardware, OS configuration, etc. (I'm
    unable to get details about the benchmark or the configuration information.)

    Although it's not a new problem to this field, his problem made me google
    some existing solutions/suggestions about handling large amount of
    concurrent TCP connections. For something like node.js, we need some
    tweaks[1]. Eventually, "The kernel isn’t the solution. The kernel is the
    problem."

    Specific to Go, I didn't find anything related to handling large amount of
    concurrent TCP connections. But I believe there must be someone who has the
    experience of working on such scale. It would be great if you can share
    here. Any suggestion/discussion is welcome. As I mentioned, this is an open
    ended problem. I won't mind if you throw me some crazy idea.

    Go is not as efficient as C yet. But 100K should be possible on a
    beefy machine (both RAM and CPU).

    I've tested vitess with 10K connections and it runs fine. However I
    was more interested in total throughput (I was able to achieve ~120K
    fat (including external request to memcached) requests per second).

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Rajiv Kurian at Jul 22, 2013 at 10:15 am
    Is this an implementation detail of the net package or more of a state of the go compiler?

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Dave Cheney at Jul 22, 2013 at 10:21 am
    Neither. 100k connections consumes lots of memory in both the kernel
    and the application. If you are using one goroutine per connection,
    you need to account for at least 4k per connection, plus overhead for
    actually reading and processing data. On the kernel side you need to
    account for memory for SKBs and other data structures.
    On Mon, Jul 22, 2013 at 8:15 PM, Rajiv Kurian wrote:
    Is this an implementation detail of the net package or more of a state of the go compiler?

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Rajiv Kurian at Jul 22, 2013 at 10:28 am
    100k is a very possible number using epoll straight up or even using java NIO. These numbers have been achieved on very bland EC2 machines - http://urbanairship.com/blog/2010/08/24/c500k-in-action-at-urban-airship/

    Given that this is not an OS limitation, I'll assume that the overhead of a go routine per connection is a limitation.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Dave Cheney at Jul 22, 2013 at 10:31 am
    This is a problem of memory. Goroutines consume memory, so do threads,
    so do buffers, and so on.
    On Mon, Jul 22, 2013 at 8:28 PM, Rajiv Kurian wrote:
    100k is a very possible number using epoll straight up or even using java NIO. These numbers have been achieved on very bland EC2 machines - http://urbanairship.com/blog/2010/08/24/c500k-in-action-at-urban-airship/

    Given that this is not an OS limitation, I'll assume that the overhead of a go routine per connection is a limitation.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Rajiv Kurian at Jul 22, 2013 at 10:34 am
    Yeah, so the go routine memory overhead then.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Dmitry Vyukov at Jul 22, 2013 at 10:32 am

    On Mon, Jul 22, 2013 at 2:28 PM, Rajiv Kurian wrote:
    100k is a very possible number using epoll straight up or even using java NIO. These numbers have been achieved on very bland EC2 machines - http://urbanairship.com/blog/2010/08/24/c500k-in-action-at-urban-airship/

    Given that this is not an OS limitation, I'll assume that the overhead of a go routine per connection is a limitation.

    A goroutine adds ~6.5K. We need to reduce it in long term...

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Robert Melton at Jul 22, 2013 at 12:08 pm

    On Mon, Jul 22, 2013 at 12:28 AM, Monnand wrote:

    He then told me that his company would like to handle ~100K connections
    per server. His colleague made a quick benchmark in Go the other day and
    the Go program handles 5K connections by utilizing 200% of the CPU.
    Any chance he could post that test? So far, my experience with
    semi-useless toys tests has been easy (effortless actually) scaling into
    the six digits concurrent assuming the memory is available. Maybe he hit a
    hidden gotcha and wasn't aware of it?

    --
    Robert Melton

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Ngaut1 at Jul 22, 2013 at 12:59 pm
    1000k connections is ok if you got enough memory.
    bug gc is the problem, you program may pause ~3 seconds

    在 2013年7月21日星期日UTC-8下午8时28分34秒,Monnand写道:
    Hi all,

    I just had an interesting conversation with a friend. I think the
    problem we are talking about is quite open ended. So I would like to
    share it and see what others say about it.

    Let's first take a look at the problem: My friend is working on a
    service which needs to handle lots of concurrent TCP connections. As you
    can imagine, I suggest him to use Go.

    He then told me that his company would like to handle ~100K connections
    per server. His colleague made a quick benchmark in Go the other day and
    the Go program handles 5K connections by utilizing 200% of the CPU.

    5K connections is apparently not the maximum that a Go program can get.
    (As far as I know, vitess claims it can handle 10K connections.)
    Moreover, such benchmark relies on the underlying hardware, OS
    configuration, etc. (I'm unable to get details about the benchmark or
    the configuration information.)

    Although it's not a new problem to this field, his problem made me
    google some existing solutions/suggestions about handling large amount
    of concurrent TCP connections. For something like node.js, we need some
    tweaks[1]. Eventually, "The kernel isn’t the solution. The kernel is the
    problem."

    Specific to Go, I didn't find anything related to handling large amount
    of concurrent TCP connections. But I believe there must be someone who
    has the experience of working on such scale. It would be great if you
    can share here. Any suggestion/discussion is welcome. As I mentioned,
    this is an open ended problem. I won't mind if you throw me some crazy
    idea.

    [1]

    http://blog.caustik.com/2012/04/08/scaling-node-js-to-100k-concurrent-connections/
    [2]

    http://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concurrent-connections-the-kernel-i.html
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Ngaut1 at Jul 22, 2013 at 12:59 pm
    1000k connections is ok if you got enough memory.
    bug gc is the problem, you program may pause ~3 seconds

    在 2013年7月21日星期日UTC-8下午8时28分34秒,Monnand写道:
    Hi all,

    I just had an interesting conversation with a friend. I think the
    problem we are talking about is quite open ended. So I would like to
    share it and see what others say about it.

    Let's first take a look at the problem: My friend is working on a
    service which needs to handle lots of concurrent TCP connections. As you
    can imagine, I suggest him to use Go.

    He then told me that his company would like to handle ~100K connections
    per server. His colleague made a quick benchmark in Go the other day and
    the Go program handles 5K connections by utilizing 200% of the CPU.

    5K connections is apparently not the maximum that a Go program can get.
    (As far as I know, vitess claims it can handle 10K connections.)
    Moreover, such benchmark relies on the underlying hardware, OS
    configuration, etc. (I'm unable to get details about the benchmark or
    the configuration information.)

    Although it's not a new problem to this field, his problem made me
    google some existing solutions/suggestions about handling large amount
    of concurrent TCP connections. For something like node.js, we need some
    tweaks[1]. Eventually, "The kernel isn’t the solution. The kernel is the
    problem."

    Specific to Go, I didn't find anything related to handling large amount
    of concurrent TCP connections. But I believe there must be someone who
    has the experience of working on such scale. It would be great if you
    can share here. Any suggestion/discussion is welcome. As I mentioned,
    this is an open ended problem. I won't mind if you throw me some crazy
    idea.

    [1]

    http://blog.caustik.com/2012/04/08/scaling-node-js-to-100k-concurrent-connections/
    [2]

    http://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concurrent-connections-the-kernel-i.html
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Atomly at Jul 22, 2013 at 7:46 pm
    honestly if he needs to handle 100k concurrent connections easily, then i
    would say he would have a very easy time with anything based around the
    standard event multiplexing models (e.g. epoll on linux) such as
    java.nio/netty...

    :: atomly ::

    [ atomly@atomly.com : www.atomly.com : http://blog.atomly.com/ ...
    [ atomiq records : new york city : +1.347.692.8661 ...
    [ e-mail atomly-news-subscribe@atomly.com for atomly info and updates ...

    On Mon, Jul 22, 2013 at 12:28 AM, Monnand wrote:

    Hi all,

    I just had an interesting conversation with a friend. I think the problem
    we are talking about is quite open ended. So I would like to share it and
    see what others say about it.

    Let's first take a look at the problem: My friend is working on a service
    which needs to handle lots of concurrent TCP connections. As you can
    imagine, I suggest him to use Go.

    He then told me that his company would like to handle ~100K connections
    per server. His colleague made a quick benchmark in Go the other day and
    the Go program handles 5K connections by utilizing 200% of the CPU.

    5K connections is apparently not the maximum that a Go program can get.
    (As far as I know, vitess claims it can handle 10K connections.) Moreover,
    such benchmark relies on the underlying hardware, OS configuration, etc.
    (I'm unable to get details about the benchmark or the configuration
    information.)

    Although it's not a new problem to this field, his problem made me google
    some existing solutions/suggestions about handling large amount of
    concurrent TCP connections. For something like node.js, we need some
    tweaks[1]. Eventually, "The kernel isn’t the solution. The kernel is the
    problem."

    Specific to Go, I didn't find anything related to handling large amount of
    concurrent TCP connections. But I believe there must be someone who has the
    experience of working on such scale. It would be great if you can share
    here. Any suggestion/discussion is welcome. As I mentioned, this is an open
    ended problem. I won't mind if you throw me some crazy idea.

    [1] http://blog.caustik.com/2012/**04/08/scaling-node-js-to-100k-**
    concurrent-connections/<http://blog.caustik.com/2012/04/08/scaling-node-js-to-100k-concurrent-connections/>
    [2] http://highscalability.com/**blog/2013/5/13/the-secret-to-**
    10-million-concurrent-**connections-the-kernel-i.html<http://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concurrent-connections-the-kernel-i.html>

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@**googlegroups.com<golang-nuts%2Bunsubscribe@googlegroups.com>
    .
    For more options, visit https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out>
    .

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Monnand at Jul 22, 2013 at 10:05 pm
    Thank you all your replies! All of them are very useful.

    On 07/22/2013 12:28 AM, Monnand wrote:
    <snip>
    He then told me that his company would like to handle ~100K connections
    per server. His colleague made a quick benchmark in Go the other day and
    the Go program handles 5K connections by utilizing 200% of the CPU.
    I'm pretty sure that this guy made a wrong benchmark. According to my
    friend's description, he just wanted to make a simple program and see
    how many connections it can handle.

    I wrote a quite simple echo server and tested on my own laptop. It can
    easily handle 20K connections on my laptop running both server and
    clients (this means 40K fds in total. Considering I was using firefox
    watching youtube, thunderbird receiving email, etc, it should work much
    faster on a dedicated server. No need to say more advanced hardware on
    those servers -- I'm pretty sure they are not using EC2 or any publicly
    available cloud instances) I will contact my friend to know more details
    about the benchmark. I think it is totally safe to simply ignore the
    results of his colleague's benchmark.

    The source code of my test programs (both server and client) is
    available here:

    https://github.com/monnand/c10k-go

    With only 46 lines of code, the server scales linearly with the number
    of connections. Goroutines have very low footprint.

    Regards,
    -Monnand

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Sugu Sougoumarane at Jul 23, 2013 at 12:43 am
    https://groups.google.com/forum/#!searchin/golang-nuts/Garbage$20collecting/golang-nuts/S9goEGuoMRM/FZyd2M6uiVMJ

    This person was using 600k connections in a production environment, and he
    was using a very old version of go. Things are even better now.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Monnand at Jul 23, 2013 at 8:19 am

    On 07/22/2013 08:43 PM, Sugu Sougoumarane wrote:
    https://groups.google.com/forum/#!searchin/golang-nuts/Garbage$20collecting/golang-nuts/S9goEGuoMRM/FZyd2M6uiVMJ

    This person was using 600k connections in a production environment, and
    he was using a very old version of go. Things are even better now.
    This is very useful! Thank you very much!

    -Monnand

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedJul 22, '13 at 4:28a
activeJul 23, '13 at 8:19a
posts16
users8
websitegolang.org

People

Translate

site design / logo © 2021 Grokbase