FAQ
Apologies for not explaining the problem in detail. My application is a
Dynamic Reverse Proxy(lets call it DRP). The clients(lets call them
CLIENTS) to DRP are servers(not browsers), which send http request with a
header X-purl which contains the backend URL(e.g:- X-purl: http://<url>) to
which DRP should forward the request to. Lets call backend servers as
BSERVERS. DRP provides BSERVER a time of 200 milliseconds to respond to a
request that DRP forwards it.

When I mentioned that netstat showed 20K connections, most of these
connections are with the CLIENTS, not with BSERVERS.

The application memory keeps increasing(over a couple of weeks). A lot of
goroutines ~20K are in IO wait state in the same call trace. Call trace is
mentioned above. Even when I decrease the traffic on a server, the
goroutines are not coming down.

Is there a real leak of connections or goroutines which are not being
garbage collected? Is the IO wait time cumulative?
Are there 20K goroutines because DRP has 20K keep-alive connections with
CLIENTS? Please note the CLIENTS also have timeout at their end(say 150ms)
and if DRP doesn't respond within that time, CLIENT closes the connection.

I don't understand why you have multiple ReverseProxies, each with
their own transport. Each one has to maintain it's own idle connection
pool, and and will multiply the number of connections that you keep
open.

A lot of your code seems to be around synchronizing multiple proxy
structs. It would be a lot simpler if you just use a single instance,
and it would make the use if idle connections a lot more efficient.
Thanks for your patience of going through my code. The reason I am using
multipe reverse-proxies(per URL) is because I observed heavy lock
contention on the transport, i.e the lock which is taken for "putIdleConn"
and "getIdleConn"(transport.go)

All of your idle connections need to be in IO wait in order to receive
data. I'd expect it to be very high maintaining lost of open
connections.
So are you saying that IO wait is cumulative?

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 10 of 14 | next ›
Discussion Overview
groupgolang-nuts @
categoriesgo
postedSep 2, '15 at 12:29p
activeSep 4, '15 at 1:48p
posts14
users5
websitegolang.org

People

Translate

site design / logo © 2021 Grokbase