[go-nuts] Max Connections to Host (http.DefaultTransport.(*http.Transport).MaxIdleConnsPerHost)

I think this is really the way to go. If the number you need for
MaxIdleConnections to accommodate your peak concurrency is greater than the
number of ports available, make more ports!

You can set the local address in a net.Dialer, which can provide the Dial
function for your client's http.Transport.

http://golang.org/pkg/net/#Dialer
http://golang.org/pkg/net/http/#Transport

Thanks for the links - will check that.

The problem with creating virtual network interfaces is - our communication
is outside private network and hence need public ips.

I fee that, the usage of MaxIdleConnection should be bit different - at
peak concurrency, when the connections exceed this limit, that generally
means that connections are not idle and thus new connections shall not be
immediately closed (as it degrades the performance, because we need to
again establish connection immediately with same host, and thus too many
connections goes into TIME_WAIT state, rapidly decreasing the available
ports) and instead be cached. I think the optimization should be at the
global connection pool. For the partners whose connections are idle, they
can be closed, but one who is in demand now, shall scale upto the limits
allowed - by closing the idle connections with other hosts.

Thanks for the links - will check that.

The problem with creating virtual network interfaces is - communication can
be with outside world and then need public IP.

I fee that, the usage of MaxIdleConnection should be bit different - at
peak concurrency, when the connections exceed this limit, that generally
means that connections are not idle and thus new connections shall not be
immediately closed (as it degrades the performance, because we need to
again establish connection immediately with same host, and thus too many
connections goes into TIME_WAIT state, rapidly decreasing the available
ports) and instead be cached. I think the optimization should be at the
global connection pool. For the partners whose connections are idle, they
can be closed, but one who is in demand now, shall scale upto the limits
allowed - by closing the idle connections with other hosts.

Why not create such pooling yourself?

Are you certain that this is adversely affecting you? It seems like only a
slight optimization of the current behavior, with a much more complication
implementation (not to mention, it makes MaxIdleConnsPerHost not a hard
limit as it would imply)

That limit is specifically for "Idle" connections. If MaxIdleConnsPerHost
is 10, and you have 1000 active connections, and 10 are released, you
should have exactly 10 idle and waiting and 990 active. No [keepalive]
connection is immediately closed if there are no idle connections already.

James, Thats what my expectation is and if it works that way, then thats
great. But observation is different - during stress testing, till the total
connections are below MaxIdleConnsPerHost, number of connections are
constant, but as I increase the concurrency for stress testing, when the
total connections from server to outside clients exceed
MaxIdleConnsPerHost, the connections count explodes as most of them (above
the limit) go in TIME_WAIT state and it does not become stable. Code
- http://play.golang.org/p/yK6Knh8RkE.
The backend services, are simple, accept the request, sleep for 80ms and
send the response.

I am using wrk to do stress testing. After initial set of testing
connections were constant and 10000 -

concurrency - a:
ESTABLISHED=> 10000
TIME_WAIT => 0

concurrency - 2a:(By starting another wrk client):
ESTABLISHED => 10000 -> 11000 (stable around here)
TIME_WAIT: Increasing -> 1939 - > 3714 -> 4156 -> ..->15066 -> ...42000
->...

My understanding is after some time - most of the connections should be
established, and TIME_WAIT should be very low, since connections are not
idle, as the stress testing is on.
Please let me know if there is issue in the code, or this is expected the
behavior.

Came across - https://code.google.com/p/go/issues/detail?id=6785. Similar
issue is already reported.

Ah, that makes sense.
Luckily that's not likely to come up in real world use for most people.

Indeed. I'm the original reporter; fixing it looked slightly
nontrivial so I decided to let it slide for the time being, and as far
as I know the problem has never popped up in production, just when
attempting local benchmarks.

//jb

Thanks Jakob!

Though currently I am just prototyping the system, I think, this issue can
easily come into production - if
  - The backend services are slow.
- You have to communicate with many backend services and hosts.

Observation is - As soon as the MaxIdleConnsPerHost exceeds, the subsequent
connections are not persistent and results into lot of connections in
TIME_WAIT.

Even a basic test case with MaxIdleConnsPerHost = 1 and stress testing with
concurrency 2 results in > 5000 connections in TIME_WAIT.
Setup:
- Only one backend service is called. It responds after 5 ms of sleep.
- Main HTTP server responds if timeout happens (10ms) or when response is
received from backend service.
- Concurrency test - weighttp -n 10000 -c 2 -t 1 -k
"http://127.0.0.1:8084/".

This basic test also results in > 5000 connections in TIME_WAIT. In
practice, this test case should be handled in 2 persistent connections.
When I changed MaxIdleConnsPerHost to 4, keeping everything else same, the
net connections established to the backend service are 4.

Reported issue at - https://code.google.com/p/go/issues/detail?id=8536.
Also mentioned the link
for https://code.google.com/p/go/issues/detail?id=6785.
I think there is a difference in two - earlier one asks for max limit on
number of connections. This one asks for better utilization of the
established connections.

This may not related to the issue in golang but You may want to look at
good explanation about TCP tuple from CloudFlare.
http://blog.cloudflare.com/cloudflare-now-supports-websockets
This allow you to reuse outgoing port to n http servers you have just like
an incoming port.