FAQ
I am working on a high volume (7-10k reqs/sec) http application that is at
times getting overwhelmed with occasional traffic spikes during peak
traffic times. When this happens, the number of CLOSE_WAIT connections
grows into the 10s of thousands, Reducing tcp kernel timeouts helps
recovery, but right now I am totally reactive and using linux command line
tools to monitor. I would like to detect this situation in the application
where I can remediate. Is there a way to hook into the tcp state
transitions in the app? RIght now I start a http ServerMux using
ListenAndServe(). Do I need to roll my own? Any tips or links to docs would
be appreciated.

Thanks in advance!
Karim

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Dave Cheney at Nov 2, 2013 at 7:37 am
    Can you give a few more details about your environment ? Ie, which os
    (i'm guessing linux inside ec2), which version of go, etc.
    On Sat, Nov 2, 2013 at 6:24 PM, wrote:
    I am working on a high volume (7-10k reqs/sec) http application that is at
    times getting overwhelmed with occasional traffic spikes during peak traffic
    times. When this happens, the number of CLOSE_WAIT connections grows into
    the 10s of thousands, Reducing tcp kernel timeouts helps recovery, but
    right now I am totally reactive and using linux command line tools to
    monitor. I would like to detect this situation in the application where I
    can remediate. Is there a way to hook into the tcp state transitions in the
    app? RIght now I start a http ServerMux using ListenAndServe(). Do I need to
    roll my own? Any tips or links to docs would be appreciated.

    Thanks in advance!
    Karim

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Karim Nassar at Nov 2, 2013 at 7:41 am
    Sure. go version go1.1.2 linux/amd64, running debian wheezy 7.2 on google
    compute "high cpu" instances (8 cpu, 7.2GB memory)

    --
    Karim Nassar
    Software Engineer, eclipse.io
    (310) 598-0003

    On Sat, Nov 2, 2013 at 12:37 AM, Dave Cheney wrote:

    Can you give a few more details about your environment ? Ie, which os
    (i'm guessing linux inside ec2), which version of go, etc.
    On Sat, Nov 2, 2013 at 6:24 PM, wrote:
    I am working on a high volume (7-10k reqs/sec) http application that is at
    times getting overwhelmed with occasional traffic spikes during peak traffic
    times. When this happens, the number of CLOSE_WAIT connections grows into
    the 10s of thousands, Reducing tcp kernel timeouts helps recovery, but
    right now I am totally reactive and using linux command line tools to
    monitor. I would like to detect this situation in the application where I
    can remediate. Is there a way to hook into the tcp state transitions in the
    app? RIght now I start a http ServerMux using ListenAndServe(). Do I need to
    roll my own? Any tips or links to docs would be appreciated.

    Thanks in advance!
    Karim

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Dave Cheney at Nov 2, 2013 at 7:45 am
    I'd start by doing a net.Listen then wrapping that listener in
    something that counted the number of outstanding connections by
    returning a wrapped net.Conn where you hook the Close() method.
    Elevated CLOSE_WAITs _may_ indicate that the server is not responding
    to the clients' close requests in time, just a guess. You could also
    be bashing your head against some ec2 load balancer problem, also a
    guess.
    On Sat, Nov 2, 2013 at 6:41 PM, Karim Nassar wrote:
    Sure. go version go1.1.2 linux/amd64, running debian wheezy 7.2 on google
    compute "high cpu" instances (8 cpu, 7.2GB memory)

    --
    Karim Nassar
    Software Engineer, eclipse.io
    (310) 598-0003

    On Sat, Nov 2, 2013 at 12:37 AM, Dave Cheney wrote:

    Can you give a few more details about your environment ? Ie, which os
    (i'm guessing linux inside ec2), which version of go, etc.
    On Sat, Nov 2, 2013 at 6:24 PM, wrote:
    I am working on a high volume (7-10k reqs/sec) http application that is
    at
    times getting overwhelmed with occasional traffic spikes during peak
    traffic
    times. When this happens, the number of CLOSE_WAIT connections grows
    into
    the 10s of thousands, Reducing tcp kernel timeouts helps recovery, but
    right now I am totally reactive and using linux command line tools to
    monitor. I would like to detect this situation in the application where
    I
    can remediate. Is there a way to hook into the tcp state transitions in
    the
    app? RIght now I start a http ServerMux using ListenAndServe(). Do I
    need to
    roll my own? Any tips or links to docs would be appreciated.

    Thanks in advance!
    Karim

    --
    You received this message because you are subscribed to the Google
    Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedNov 2, '13 at 7:24a
activeNov 2, '13 at 7:45a
posts4
users2
websitegolang.org

2 users in discussion

Dave Cheney: 2 posts Karim Nassar: 2 posts

People

Translate

site design / logo © 2022 Grokbase