FAQ
I am designing a simple ad selection server in Golang.

My system works like following:
Server will receive an adrequest comprising one or more impressions/ads,
for each impression/ad I have to execute some CPU processing stufff like
targeting etc consuming approximately 5-15ms.

Then for each impression/ad I will be making an upstream call to another
webserver to fetch some related details.

After this IO is done, for each impression/ad I will be selecting winning
creative whichh may again consume CPU of 5-10ms per impression/ad.
& finally I will be writing the response.

MY machine configuration would be typically: 16 cores, 32 GB

How the processing pipelines for such typical system will look like?

I want to exploit full parallelism but at the same time the scheduling of
Goroutines should be minimum.

For handling CPU processing what is best strategy? Continue to do
processing for all impressions/ads in single Goroutine or fork a separate
one for each impression/ad?

For handling IO I think I can fork one Goroutine for each call to be made.

I have been checking the Concurrent Design Patterns in Golang & found
something here: http://blog.golang.org/pipelines

So shall I go for 3 Goroutine worker pools one for preprocessing of
request, one for making external IO calls & one for postprocessing &
synchronize execution between them through channels?

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Dave Cheney at Jan 6, 2015 at 9:24 am
    Start with one goroutine per incoming request and learn how to use the block mode of the profiler which will show you if and where your bottleneck may be (but I think you'll probably be blocked on network io)

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Ajay Bodhe at Jan 6, 2015 at 10:20 am
    Ok.
    Will having separate goroutines to serve or perform CPU intensive tasks in
    a server getting 5k-6k requests per second enahance performance by any
    means.
    I am going to start small & then try differenr combinations but till then
    just for some more thoughts & knowledge shooting these questions.
    On Tuesday, 6 January 2015 14:54:12 UTC+5:30, Dave Cheney wrote:

    Start with one goroutine per incoming request and learn how to use the
    block mode of the profiler which will show you if and where your bottleneck
    may be (but I think you'll probably be blocked on network io)
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedJan 6, '15 at 7:07a
activeJan 6, '15 at 10:20a
posts3
users2
websitegolang.org

2 users in discussion

Ajay Bodhe: 2 posts Dave Cheney: 1 post

People

Translate

site design / logo © 2021 Grokbase