FAQ
I am trying to receive UDP packets and process them, while also setting up
periodic tickers to do some other work. For simplicity (it isn't much work)
I want to do it all in one goroutine so I don't have concurrent access to
the data structures. I find that if no channel is ready to read, the
select{} blocks, NOT until "some" channel is ready to read, but until the
"last" one is. I don't expect this according to the specification, which
says when some channel is ready, it will be read.

Example: http://play.golang.org/p/UDxpur2wrd

If I start the program just shown, it prints nothing. I expect it to print
nothing for 10 seconds, then start printing timestamps from the ticker.
Instead, nothing happens until I send some data to port 8032 on UDP. Then,
it starts printing timestamps AND prints the size of packets received,
which is what I expect -- and then, when I stop sending anything to the UDP
port, the timestamps continue to be received and printed, also what I
expect.

What I don't expect is the initial blocking of the select{} statement while
no data arrives at UDP.

Am I wrong, or is there a bug, or should I do this completely differently?

--

Search Discussions

  • Kyle Lemons at Nov 12, 2012 at 6:32 pm
    I would personally bite the bullet and do it in multiple goroutines. I
    can't comment on your specific problem unless you expand a bit, but usually
    you can avoid having to lock some data structure if you think about how
    they need to communicate, not about what data they need to share (hence the
    now-familiar adage "Don't communicate by sharing memory, share memory by
    communicating"). It makes it easier to write, read, test, understand, and
    maintain.

    On Mon, Nov 12, 2012 at 10:20 AM, Boris wrote:

    I am trying to receive UDP packets and process them, while also setting up
    periodic tickers to do some other work. For simplicity (it isn't much work)
    I want to do it all in one goroutine so I don't have concurrent access to
    the data structures. I find that if no channel is ready to read, the
    select{} blocks, NOT until "some" channel is ready to read, but until the
    "last" one is. I don't expect this according to the specification, which
    says when some channel is ready, it will be read.

    Example: http://play.golang.org/p/UDxpur2wrd

    If I start the program just shown, it prints nothing. I expect it to print
    nothing for 10 seconds, then start printing timestamps from the ticker.
    Instead, nothing happens until I send some data to port 8032 on UDP. Then,
    it starts printing timestamps AND prints the size of packets received,
    which is what I expect -- and then, when I stop sending anything to the UDP
    port, the timestamps continue to be received and printed, also what I
    expect.

    What I don't expect is the initial blocking of the select{} statement
    while no data arrives at UDP.

    Am I wrong, or is there a bug, or should I do this completely differently?

    --

    --
  • Kyle Lemons at Nov 12, 2012 at 6:36 pm
    oops again... forgot to paste my comments on your code: (they're inline)
    http://play.golang.org/p/pZkuzd-tdJ

    Basically you're still sharing memory without locking.

    On Mon, Nov 12, 2012 at 10:31 AM, Kyle Lemons wrote:

    I would personally bite the bullet and do it in multiple goroutines. I
    can't comment on your specific problem unless you expand a bit, but usually
    you can avoid having to lock some data structure if you think about how
    they need to communicate, not about what data they need to share (hence the
    now-familiar adage "Don't communicate by sharing memory, share memory by
    communicating"). It makes it easier to write, read, test, understand, and
    maintain.

    On Mon, Nov 12, 2012 at 10:20 AM, Boris wrote:

    I am trying to receive UDP packets and process them, while also setting
    up periodic tickers to do some other work. For simplicity (it isn't much
    work) I want to do it all in one goroutine so I don't have concurrent
    access to the data structures. I find that if no channel is ready to read,
    the select{} blocks, NOT until "some" channel is ready to read, but until
    the "last" one is. I don't expect this according to the specification,
    which says when some channel is ready, it will be read.

    Example: http://play.golang.org/p/UDxpur2wrd

    If I start the program just shown, it prints nothing. I expect it to
    print nothing for 10 seconds, then start printing timestamps from the
    ticker. Instead, nothing happens until I send some data to port 8032 on
    UDP. Then, it starts printing timestamps AND prints the size of packets
    received, which is what I expect -- and then, when I stop sending anything
    to the UDP port, the timestamps continue to be received and printed, also
    what I expect.

    What I don't expect is the initial blocking of the select{} statement
    while no data arrives at UDP.

    Am I wrong, or is there a bug, or should I do this completely differently?

    --

    --
  • Boris Solovyov at Nov 12, 2012 at 8:35 pm
    The specific problem is that I am receiving UDP messages, aggregating
    them, and once per second taking snapshots of what has been
    aggregated. Once per minute I need to send all the once-per-second
    results to an API and mark everything as "sent" (but not delete/erase
    it; I need it as a reference point for the beginning of the next
    minute). I don't want to concurrently receive and do this aggregation
    and snapshotting, because I don't want inconsistencies. I don't expect
    the snapshotting to take very long, so I think it is OK to set up the
    once-per-interval ticker events to run in the same goroutine and not
    run concurrently. I think it will be more complex to run them in
    separate goroutines and ensure access to the aggregated data is atomic
    and consistent, but I could be wrong; I do not know Go idioms yet and
    there may be something easy or elegant that I'm not thinking about.
    On Mon, Nov 12, 2012 at 1:31 PM, Kyle Lemons wrote:
    I would personally bite the bullet and do it in multiple goroutines. I
    can't comment on your specific problem unless you expand a bit, but usually
    you can avoid having to lock some data structure if you think about how they
    need to communicate, not about what data they need to share (hence the
    now-familiar adage "Don't communicate by sharing memory, share memory by
    communicating"). It makes it easier to write, read, test, understand, and
    maintain.

    On Mon, Nov 12, 2012 at 10:20 AM, Boris wrote:

    I am trying to receive UDP packets and process them, while also setting up
    periodic tickers to do some other work. For simplicity (it isn't much work)
    I want to do it all in one goroutine so I don't have concurrent access to
    the data structures. I find that if no channel is ready to read, the
    select{} blocks, NOT until "some" channel is ready to read, but until the
    "last" one is. I don't expect this according to the specification, which
    says when some channel is ready, it will be read.

    Example: http://play.golang.org/p/UDxpur2wrd

    If I start the program just shown, it prints nothing. I expect it to print
    nothing for 10 seconds, then start printing timestamps from the ticker.
    Instead, nothing happens until I send some data to port 8032 on UDP. Then,
    it starts printing timestamps AND prints the size of packets received, which
    is what I expect -- and then, when I stop sending anything to the UDP port,
    the timestamps continue to be received and printed, also what I expect.

    What I don't expect is the initial blocking of the select{} statement
    while no data arrives at UDP.

    Am I wrong, or is there a bug, or should I do this completely differently?
    --
  • Dustin Sallings at Nov 12, 2012 at 8:58 pm
    Boris Solovyov <boris.solovyov@gmail.com>
    writes:
    The specific problem is that I am receiving UDP messages, aggregating
    them, and once per second taking snapshots of what has been
    aggregated. Once per minute I need to send all the once-per-second
    results to an API and mark everything as "sent" (but not delete/erase
    it; I need it as a reference point for the beginning of the next
    minute). I don't want to concurrently receive and do this aggregation
    and snapshotting, because I don't want inconsistencies. I don't expect
    the snapshotting to take very long, so I think it is OK to set up the
    once-per-interval ticker events to run in the same goroutine and not
    run concurrently. I think it will be more complex to run them in
    separate goroutines and ensure access to the aggregated data is atomic
    and consistent, but I could be wrong; I do not know Go idioms yet and
    there may be something easy or elegant that I'm not thinking about.
    Just send all the concurrent UDP messages into something like this:

    func aggregator(in <-chan reading, out chan<- result) {
    t := timer.Tick(time.Second)

    for {
    select {
    case r := <-in:
    addReading(r)
    case <-t:
    sendResultsAndClearState()
    }
    }
    }

    --
    dustin

    --
  • Boris Solovyov at Nov 12, 2012 at 9:56 pm
    Hi,
    On Mon, Nov 12, 2012 at 3:57 PM, Dustin Sallings wrote:
    Boris Solovyov <boris.solovyov@gmail.com>
    writes:
    The specific problem is that I am receiving UDP messages, aggregating
    them, and once per second taking snapshots of what has been
    aggregated. Once per minute I need to send all the once-per-second
    results to an API and mark everything as "sent" (but not delete/erase
    it; I need it as a reference point for the beginning of the next
    minute). I don't want to concurrently receive and do this aggregation
    and snapshotting, because I don't want inconsistencies. I don't expect
    the snapshotting to take very long, so I think it is OK to set up the
    once-per-interval ticker events to run in the same goroutine and not
    run concurrently. I think it will be more complex to run them in
    separate goroutines and ensure access to the aggregated data is atomic
    and consistent, but I could be wrong; I do not know Go idioms yet and
    there may be something easy or elegant that I'm not thinking about.
    Just send all the concurrent UDP messages into something like this:

    func aggregator(in <-chan reading, out chan<- result) {
    t := timer.Tick(time.Second)

    for {
    select {
    case r := <-in:
    addReading(r)
    case <-t:
    sendResultsAndClearState()
    }
    }
    }
    I neglected to say WHY I was starting the timer after a sleep period.
    It is because I want the ticker to start on a particular time
    boundary. If I can do that in some other way, that would also solve my
    needs. (I know I should have read
    http://www.catb.org/esr/faqs/smart-questions.html before I asked this
    question.) While the ticker is waiting to start, though, I do want to
    process incoming UDP messages.

    --
  • Dustin Sallings at Nov 12, 2012 at 10:31 pm
    Boris Solovyov <boris.solovyov@gmail.com>
    writes:
    I neglected to say WHY I was starting the timer after a sleep period.
    It is because I want the ticker to start on a particular time
    boundary. If I can do that in some other way, that would also solve my
    needs. (I know I should have read
    http://www.catb.org/esr/faqs/smart-questions.html before I asked this
    question.) While the ticker is waiting to start, though, I do want to
    process incoming UDP messages.
    Then stick the sleep above the tick creation? That's a fairly small
    detail. You could even have the uninitialized variable in the select
    loop as you asynchronously assign it to the timer.

    One safe and easy way to do that would look something like this:

    func stuffdoer(in <-chan stuff) {

    var tick <-chan time.Time
    var tickinit := time.After(syncupTime)

    for {
    select {
    case thing <- in:
    store(thing)
    case <-tick:
    aggregateAndClear()
    case <-tickinit:
    tick = time.Tick(duration)
    tickinit = nil
    }
    }
    }

    --
    dustin

    --
  • Boris Solovyov at Nov 12, 2012 at 10:54 pm

    var tick <-chan time.Time
    var tickinit := time.After(syncupTime)
    I think that time.After is what I was looking for all this time :-) Thank you!

    --
  • Skip Tavakkolian at Nov 12, 2012 at 10:46 pm
    you could forward contents of the ticker channel to sTick in a function literal:

    http://play.golang.org/p/CC7zz8CkjP

    -Skip

    On Mon, Nov 12, 2012 at 1:56 PM, Boris Solovyov
    wrote:
    Hi,
    On Mon, Nov 12, 2012 at 3:57 PM, Dustin Sallings wrote:
    Boris Solovyov <boris.solovyov@gmail.com>
    writes:
    The specific problem is that I am receiving UDP messages, aggregating
    them, and once per second taking snapshots of what has been
    aggregated. Once per minute I need to send all the once-per-second
    results to an API and mark everything as "sent" (but not delete/erase
    it; I need it as a reference point for the beginning of the next
    minute). I don't want to concurrently receive and do this aggregation
    and snapshotting, because I don't want inconsistencies. I don't expect
    the snapshotting to take very long, so I think it is OK to set up the
    once-per-interval ticker events to run in the same goroutine and not
    run concurrently. I think it will be more complex to run them in
    separate goroutines and ensure access to the aggregated data is atomic
    and consistent, but I could be wrong; I do not know Go idioms yet and
    there may be something easy or elegant that I'm not thinking about.
    Just send all the concurrent UDP messages into something like this:

    func aggregator(in <-chan reading, out chan<- result) {
    t := timer.Tick(time.Second)

    for {
    select {
    case r := <-in:
    addReading(r)
    case <-t:
    sendResultsAndClearState()
    }
    }
    }
    I neglected to say WHY I was starting the timer after a sleep period.
    It is because I want the ticker to start on a particular time
    boundary. If I can do that in some other way, that would also solve my
    needs. (I know I should have read
    http://www.catb.org/esr/faqs/smart-questions.html before I asked this
    question.) While the ticker is waiting to start, though, I do want to
    process incoming UDP messages.

    --
    --
  • Boris Solovyov at Nov 12, 2012 at 10:55 pm
    This also looks like it would do what I need. I just don't know enough
    Go yet to know which tools to reach for. Thanks!

    On Mon, Nov 12, 2012 at 5:46 PM, Skip Tavakkolian
    wrote:
    you could forward contents of the ticker channel to sTick in a function literal:

    http://play.golang.org/p/CC7zz8CkjP
    --
  • Dustin Sallings at Nov 13, 2012 at 1:37 am
    Skip Tavakkolian
    <skip.tavakkolian@gmail.com> writes:

    you could forward contents of the ticker channel to sTick in a function literal:

    http://play.golang.org/p/CC7zz8CkjP
    It probably doesn't make a huge difference, but it seems a little
    heavy to have a goroutine and a channel just to forward the results of
    another channel of the same type.

    --
    dustin

    --
  • Skip Tavakkolian at Nov 13, 2012 at 7:40 pm
    the original question was about how to start a ticker after waiting
    for a few seconds. at any rate, the overhead of an extra channel and
    goroutine will be tiny compared to network io; and the flow is more
    obvious.

    -Skip
    On Mon, Nov 12, 2012 at 5:37 PM, Dustin Sallings wrote:
    Skip Tavakkolian
    <skip.tavakkolian@gmail.com> writes:

    you could forward contents of the ticker channel to sTick in a function literal:

    http://play.golang.org/p/CC7zz8CkjP
    It probably doesn't make a huge difference, but it seems a little
    heavy to have a goroutine and a channel just to forward the results of
    another channel of the same type.

    --
    dustin

    --
    --
  • Dustin Sallings at Nov 13, 2012 at 9:01 pm
    Skip Tavakkolian
    <skip.tavakkolian@gmail.com> writes:

    the original question was about how to start a ticker after waiting
    for a few seconds. at any rate, the overhead of an extra channel and
    goroutine will be tiny compared to network io; and the flow is more
    obvious.
    I think obvious is in the eye of the reader.

    When the normal case requires all of your timer messages to be
    duplicated, I find that to be a bit of a distraction I'd stumble over.

    In my case, there's a similar select loop with one outlier case that
    only fires once. I'm sure someone may find the single-use case
    distracting in what is otherwise the same loop as well, but I find the
    one that doesn't add anything onto the scheduler to be easier to think
    about.


    --
    dustin

    --
  • Zizon Qiu at Nov 13, 2012 at 1:44 am
    select will block until:
    1.one of the channel is ready or
    2.a default case present.

    see selectgo in
    http://golang.org/src/pkg/runtime/chan.c?h=selectgo
    On Tue, Nov 13, 2012 at 2:20 AM, Boris wrote:

    I am trying to receive UDP packets and process them, while also setting up
    periodic tickers to do some other work. For simplicity (it isn't much work)
    I want to do it all in one goroutine so I don't have concurrent access to
    the data structures. I find that if no channel is ready to read, the
    select{} blocks, NOT until "some" channel is ready to read, but until the
    "last" one is. I don't expect this according to the specification, which
    says when some channel is ready, it will be read.

    Example: http://play.golang.org/p/UDxpur2wrd

    If I start the program just shown, it prints nothing. I expect it to print
    nothing for 10 seconds, then start printing timestamps from the ticker.
    Instead, nothing happens until I send some data to port 8032 on UDP. Then,
    it starts printing timestamps AND prints the size of packets received,
    which is what I expect -- and then, when I stop sending anything to the UDP
    port, the timestamps continue to be received and printed, also what I
    expect.

    What I don't expect is the initial blocking of the select{} statement
    while no data arrives at UDP.

    Am I wrong, or is there a bug, or should I do this completely differently?

    --

    --

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedNov 12, '12 at 6:20p
activeNov 13, '12 at 9:01p
posts14
users5
websitegolang.org

People

Translate

site design / logo © 2021 Grokbase