FAQ
Are there any good resources on buffered channels?

I have some questions to which I have not found answers:


    - Can you examine the size of the buffer in the channel, e.g. to see if
    it is empty or full, without modifying the channel?
    - What happens to the buffer if the channel is closed? use case -> I
    have an emitter that emits to a buffered channel. If it finishes and tries
    to close the channel but data remains in the buffer, is it lost?
    - Are buffered channels strictly FIFO?

Thanks!

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Jan Mercl at Nov 17, 2014 at 9:14 pm
    The language specs.

    len(ch). It's inherently racy, correct usage is rarely seen but such cases
    exist.

    No. Closing a channel is just sending a special value to it (in effect,
    implementation details aside).

    Yes, but concurrent writes to a channel have no order.

    -j (phone)

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Jesper Louis Andersen at Nov 18, 2014 at 3:17 pm

    On Mon, Nov 17, 2014 at 10:04 PM, Philip Thomas wrote:

    Can you examine the size of the buffer in the channel, e.g. to see if it
    is empty or full, without modifying the channel?

    Don't do this by asking for the length of the channel. Just try to read a
    value out of the channel without blocking and handle the events from there.
    It is pretty important to make sure your channel operations are safe from
    data races. Otherwise you introduce small bugs in the code base which are
    hard to find later on. It tend to be better to drive it by events on the
    channels rather than looking at "side information".


    --
    J.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Philip Thomas at Nov 18, 2014 at 4:39 pm
    I guess the main problem I am having is coordinating the shutdown of
    multiple what I would call "queue workers" - basically one input channel,
    one output channel, and about 1000 goroutines processing input, dropping
    some data, and sending on the output channel. What's the standard pattern
    for recognizing that the input channel has closed, then closing the output
    channel when processing has finished without losing data (e.g. "ERROR -
    output channel already closed"). I thought that the input shutdown message
    could only be received by one queue worker, not all of them.

    Philip
    On Tue, Nov 18, 2014 at 7:17 AM, Jesper Louis Andersen wrote:

    On Mon, Nov 17, 2014 at 10:04 PM, Philip Thomas wrote:

    Can you examine the size of the buffer in the channel, e.g. to see if it
    is empty or full, without modifying the channel?

    Don't do this by asking for the length of the channel. Just try to read a
    value out of the channel without blocking and handle the events from there.
    It is pretty important to make sure your channel operations are safe from
    data races. Otherwise you introduce small bugs in the code base which are
    hard to find later on. It tend to be better to drive it by events on the
    channels rather than looking at "side information".


    --
    J.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Francesc Campoy Flores at Nov 18, 2014 at 4:56 pm
    You should definitely check the talk by John Graham-Cumming:

    https://www.youtube.com/watch?v=woCg2zaIVzQ

    On Tue, Nov 18, 2014 at 8:38 AM, Philip Thomas wrote:

    I guess the main problem I am having is coordinating the shutdown of
    multiple what I would call "queue workers" - basically one input channel,
    one output channel, and about 1000 goroutines processing input, dropping
    some data, and sending on the output channel. What's the standard pattern
    for recognizing that the input channel has closed, then closing the output
    channel when processing has finished without losing data (e.g. "ERROR -
    output channel already closed"). I thought that the input shutdown message
    could only be received by one queue worker, not all of them.

    Philip

    On Tue, Nov 18, 2014 at 7:17 AM, Jesper Louis Andersen <
    jesper.louis.andersen@gmail.com> wrote:
    On Mon, Nov 17, 2014 at 10:04 PM, Philip Thomas <philipithomas@gmail.com>
    wrote:
    Can you examine the size of the buffer in the channel, e.g. to see if it
    is empty or full, without modifying the channel?

    Don't do this by asking for the length of the channel. Just try to read a
    value out of the channel without blocking and handle the events from there.
    It is pretty important to make sure your channel operations are safe from
    data races. Otherwise you introduce small bugs in the code base which are
    hard to find later on. It tend to be better to drive it by events on the
    channels rather than looking at "side information".


    --
    J.
    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.


    --
    --
    Francesc Campoy
    http://twitter.com/francesc

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Chris Hines at Nov 18, 2014 at 5:04 pm
    I believe this blog post also has the answers you are looking for:

    http://blog.golang.org/pipelines

    On Tuesday, November 18, 2014 11:56:26 AM UTC-5, Francesc Campoy Flores
    wrote:
    You should definitely check the talk by John Graham-Cumming:

    https://www.youtube.com/watch?v=woCg2zaIVzQ


    On Tue, Nov 18, 2014 at 8:38 AM, Philip Thomas <philip...@gmail.com
    <javascript:>> wrote:
    I guess the main problem I am having is coordinating the shutdown of
    multiple what I would call "queue workers" - basically one input channel,
    one output channel, and about 1000 goroutines processing input, dropping
    some data, and sending on the output channel. What's the standard pattern
    for recognizing that the input channel has closed, then closing the output
    channel when processing has finished without losing data (e.g. "ERROR -
    output channel already closed"). I thought that the input shutdown message
    could only be received by one queue worker, not all of them.

    Philip

    On Tue, Nov 18, 2014 at 7:17 AM, Jesper Louis Andersen <
    jesper.lou...@gmail.com <javascript:>> wrote:
    On Mon, Nov 17, 2014 at 10:04 PM, Philip Thomas <philip...@gmail.com
    <javascript:>> wrote:
    Can you examine the size of the buffer in the channel, e.g. to see if
    it is empty or full, without modifying the channel?

    Don't do this by asking for the length of the channel. Just try to read
    a value out of the channel without blocking and handle the events from
    there. It is pretty important to make sure your channel operations are safe
    from data races. Otherwise you introduce small bugs in the code base which
    are hard to find later on. It tend to be better to drive it by events on
    the channels rather than looking at "side information".


    --
    J.
    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts...@googlegroups.com <javascript:>.
    For more options, visit https://groups.google.com/d/optout.


    --
    --
    Francesc Campoy
    http://twitter.com/francesc
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Shawn Milochik at Nov 18, 2014 at 5:04 pm

    On Tue, Nov 18, 2014 at 11:56 AM, Francesc Campoy Flores wrote:

    You should definitely check the talk by John Graham-Cumming:

    https://www.youtube.com/watch?v=woCg2zaIVzQ

    Not that Francesc needs any seconding, but yes, this is a very exciting
    talk. I recommend showing it to anyone you're trying to get interested in
    Go in gerenal.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Nick Patavalis at Nov 18, 2014 at 10:56 pm

    On Tuesday, November 18, 2014 6:39:27 PM UTC+2, Philip Thomas wrote:
    I guess the main problem I am having is coordinating the shutdown of
    multiple what I would call "queue workers" - basically one input channel,
    one output channel, and about 1000 goroutines processing input, dropping
    some data, and sending on the output channel. What's the standard pattern
    for recognizing that the input channel has closed, then closing the output
    channel when processing has finished without losing data (e.g. "ERROR -
    output channel already closed"). I thought that the input shutdown message
    could only be received by one queue worker, not all of them.
    1. *Do* watch the video, and do read the article suggested above. They are
    great.

    2. For your specific question: Reading from a closed channel never blocks.
    Instead, you get the zero-value for the channel's element type. If the
    type's zero value is valid data in your application, you can check if a
    channel is closed by receiving from it like this:

         v, ok := <- cin
         if ! ok {
             // channel closed
         }

    You can receive as many times as you want from a closed channel (and by as
    many goroutines as you want). All receives will return v == zero and ok ==
    false.

    Once a worker goroutine detects that its input channel is closed it should
    do whatever cleanup necessary and then exit. It should *not* close the
    output channel.

    The output channel must be closed when *ALL* worker goroutines have
    finished (because if any of them are still running they may have results to
    send).

    Another goroutine (*not* one of the workers, and most-likely the one that
    started them) should monitor when all worker-goroutines have finished (e.g.
    by waiting on a WaitGroup, or by some other means). When it detects that
    they have, *it* closes the output channel.

    /npat

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Nick Patavalis at Nov 18, 2014 at 11:04 pm

    On Tuesday, November 18, 2014 6:39:27 PM UTC+2, Philip Thomas wrote:
    I guess the main problem I am having is coordinating the shutdown of
    multiple what I would call "queue workers" - basically one input channel,
    one output channel, and about 1000 goroutines processing input, dropping
    some data, and sending on the output channel. What's the standard pattern
    for recognizing that the input channel has closed, then closing the output
    channel when processing has finished without losing data (e.g. "ERROR -
    output channel already closed"). I thought that the input shutdown message
    could only be received by one queue worker, not all of them.
    Also: When you receive from a *closed buffered channel*, you first receive
    all the data in the buffer normally (as if the channel had not been closed) *and
    then* you receive the channel-closed indication (the zero-value or the ok
    == false).

    /npat


    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedNov 17, '14 at 9:04p
activeNov 18, '14 at 11:04p
posts9
users7
websitegolang.org

People

Translate

site design / logo © 2021 Grokbase