[go-nuts] How to decrease contention in re-using connections in http.Client

As it is already raised as issue here
- https://code.google.com/p/go/issues/detail?id=6785, I am trying to get
opinions from folks on how to reduce the contention till issue is fixed -

When there is an attempt to reuse connections at higher rate, there is
contention in GetIdleCon and PutIdleCon -

316 @ 0x415989 0x415a0b 0x429eee 0x42a130 0x49ad56 0x45056e 0x450ab8 0x44ef26 0x43390d 0x4330db 0x43456f 0x433bbf 0x400e4f 0x415c20
# 0x42a130 sync.runtime_Semacquire+0x30 /usr/local/go/src/pkg/runtime/sema.goc:199
# 0x49ad56 sync.(*Mutex).Lock+0xd6 /usr/local/go/src/pkg/sync/mutex.go:66
# 0x45056e net/http.(*Transport).getIdleConn+0x9e /usr/local/go/src/pkg/net/http/transport.go:402
# 0x450ab8 net/http.(*Transport).getConn+0xa8 /usr/local/go/src/pkg/net/http/transport.go:452
# 0x44ef26 net/http.(*Transport).RoundTrip+0x416 /usr/local/go/src/pkg/net/http/transport.go:201
# 0x43390d net/http.send+0x43d /usr/local/go/src/pkg/net/http/client.go:195
# 0x4330db net/http.(*Client).send+0x15b /usr/local/go/src/pkg/net/http/client.go:118
# 0x43456f net/http.(*Client).doFollowingRedirects+0x97f /usr/local/go/src/pkg/net/http/client.go:343
# 0x433bbf net/http.(*Client).Get+0xaf /usr/local/go/src/pkg/net/http/client.go:275
# 0x400e4f main.startClient+0x16f

347 @ 0x415989 0x415a0b 0x429eee 0x42a130 0x49ad56 0x44fd5e 0x455b87 0x454c1b 0x454931 0x483291 0x480f6d 0x400f0b 0x415c20
# 0x42a130 sync.runtime_Semacquire+0x30 /usr/local/go/src/pkg/runtime/sema.goc:199
# 0x49ad56 sync.(*Mutex).Lock+0xd6 /usr/local/go/src/pkg/sync/mutex.go:66
# 0x44fd5e net/http.(*Transport).putIdleConn+0xce /usr/local/go/src/pkg/net/http/transport.go:342
# 0x455b87 net/http.func·023+0x97 /usr/local/go/src/pkg/net/http/transport.go:853
# 0x454c1b net/http.(*bodyEOFSignal).condfn+0xab /usr/local/go/src/pkg/net/http/transport.go:1161
# 0x454931 net/http.(*bodyEOFSignal).Read+0x321 /usr/local/go/src/pkg/net/http/transport.go:1133
# 0x483291 io/ioutil.devNull.ReadFrom+0xb1 /usr/local/go/src/pkg/io/ioutil/ioutil.go:151
# 0x480f6d io.Copy+0x13d /usr/local/go/src/pkg/io/io.go:349
# 0x400f0b main.startClient+0x22b


This results in -

1. New connections being made and teared down very fast and thus exceeding the ports available.

2. System resources are not used optimal. For a simple hello world server, for loop clients using http.Client gives 112k rps with 64% CPU. Network is not bottleneck here. Because even with 1k response and 100ms response time from server I still get 82k rps with this client at high concurrency.

Code : http://play.golang.org/p/MA-a2ZAVNe . N go-routines are created for startClient in main, where n is the concurrency desired by user. I used client (*http.Client) object per go-routine instead of global, as I thought, contention will be across one client object. But is it not the case?


Can someone please suggest alternatives/options to avoid contention in putting and getting connection?