On Fri, Jun 14, 2013 at 4:15 PM, James A. Robinson wrote:
On Fri, Jun 14, 2013 at 1:05 PM, Robert Melton wrote:
In my experience, the reason for rate limiting tends to have major
implications on implementation. Limiting for license reasons
(either on a back-end license, or your own license limiting a
customer), limiting for DB load, limiting for memory, limiting for
cpu... multifaceted limiting. Do you have a specific use case?
Rate limiting to ensure quality of service. I want to set things up
so that a health check can be used to determine if more servers need
to be spun up and load balanced to.
Basically I'm planning on the model of something like an F5 load
balancer or nginx reverse proxy sitting in front of two or more
instances of the go-based server. Once the go-based servers reach
some point of saturation I want to prevent them from getting
overloaded with processing new requests, and I want to detect the fact
so that we can automate the spinning up of new go-based servers to
Hopefully that makes sense. :)
Yep, and if you are checking multiple things (network usage, cpu usage,
memory usage, disk usage) having a reasonable polling health service is a
great way to handle it. I think using actual machine stats rather than
some random "X connections" is a far better and more flexible way to handle
this, as it will handle different sized hosts, different sized workloads,
and is nice and disconnected form the specifics.
In the past, I have had my machines poll themselves and report back to my
load balance machine(s) their raw stats, the load balancer would weight the
stats and come up with scores for all the machines. Then I would load them
according to their weighted scores. This would happen every few seconds,
once a machine was overloaded it would simply stop getting requests until
its score came down... regardless of why the score was high (could have
been network load, or disk swapping or whatever). It wasn't elegant, but
it was highly effective for a small cluster of about 50 machines. You
could simply make combined score thresholds for starting new go-based
servers on the fly, and when the scores dropped low enough for X time, you
could drain and shutdown go-based servers.
The code on the servers should be simple. The load balancer would be more
complex due to scoring, starting up new machines and doing the routing...