FAQ
Ill outline my problem and a solution, I'd appreciate well reasoned
feedback as to why this would be a good idea, or a bad idea.

The problem:

I have a computation pipeline where data flow is simply flowing one way
down a go-routine pipeline through channels.

go1 -> go2 -> go3 -> go4

There is an issue when any of the go-routines experiences an error, leaving
previous goroutines blocked is a memory leak.

To solve this I can either complicate logic by adding a select to each
goroutine send which either sends to the next channel or recieves an error
and panics + recover to terminate the nested call stack, or I can add this
code to each stage:

defer readChannelUntilEmpty(prevStage)

This change means that if any goroutine fails, the previous ones wont be
deadlocked and leaking memory as it is now guaranteed to run to completion.
This logic is far simpler than adding selects to every send, however it's
obvious flaw is whenever there is an error, you are wasting work, moreover,
if the input is an infinite sequence this will fail.

Both of these solutions feel akin to manual memory management to ensure
goroutines dont leak, and if enough goroutines leak in a faulty program,
the whole program will crash eventually. (we invented a GC to avoid this
crap)


Solution:

change all channels into two separate parts:

  readonly,writeonly := make(chan int)

then a write is attempted to the write end of a channel whos read only end
has been garbage collected, the runtime knows there is a deadlock and can
run a panic in the sending goroutine killing it, knowing that it is
deadlocked permantly.

when a read occurs on a channel who has its write end garbage collected,
then either it could be treated as an implicitly closed channel, or a panic
could be raised.

This two sided channel system would make using goroutines far simpler in my
opinion, in the same way that garbage collection makes memory management
easier. I think this idea has been suggested before, but I don't think the
panic on the sender idea has been presented.

Basically deadlocked goroutines are a bug which will crash a process
eventually, so why no make this panic explicit? A program can then add a
recover to the goroutine start if they want to explicitly allow the
goroutines to be garbage collected.



Also, what are the arguments against this? I cant really see a downside,
and can see HUGE upsides. It could even work with existing go channels with
some runtime logic regarding double ended channels and casting them to
single sided channels. Was this overlooked in the original design of Go, or
is there a good reason against it?









--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Andrewchamberss at Jul 2, 2014 at 11:35 pm
    In fact, this doesn't require breaking backwards compatibility, and it only
    would cause panics in already broken programs.

    Is the only reason this isn't done due to requiring big changes to the
    channel and garbage collector runtime?
    On Thursday, July 3, 2014 11:30:35 AM UTC+12, andrewc...@gmail.com wrote:

    Ill outline my problem and a solution, I'd appreciate well reasoned
    feedback as to why this would be a good idea, or a bad idea.

    The problem:

    I have a computation pipeline where data flow is simply flowing one way
    down a go-routine pipeline through channels.

    go1 -> go2 -> go3 -> go4

    There is an issue when any of the go-routines experiences an error,
    leaving previous goroutines blocked is a memory leak.

    To solve this I can either complicate logic by adding a select to each
    goroutine send which either sends to the next channel or recieves an error
    and panics + recover to terminate the nested call stack, or I can add this
    code to each stage:

    defer readChannelUntilEmpty(prevStage)

    This change means that if any goroutine fails, the previous ones wont be
    deadlocked and leaking memory as it is now guaranteed to run to completion.
    This logic is far simpler than adding selects to every send, however it's
    obvious flaw is whenever there is an error, you are wasting work, moreover,
    if the input is an infinite sequence this will fail.

    Both of these solutions feel akin to manual memory management to ensure
    goroutines dont leak, and if enough goroutines leak in a faulty program,
    the whole program will crash eventually. (we invented a GC to avoid this
    crap)


    Solution:

    change all channels into two separate parts:

    readonly,writeonly := make(chan int)

    then a write is attempted to the write end of a channel whos read only end
    has been garbage collected, the runtime knows there is a deadlock and can
    run a panic in the sending goroutine killing it, knowing that it is
    deadlocked permantly.

    when a read occurs on a channel who has its write end garbage collected,
    then either it could be treated as an implicitly closed channel, or a panic
    could be raised.

    This two sided channel system would make using goroutines far simpler in
    my opinion, in the same way that garbage collection makes memory management
    easier. I think this idea has been suggested before, but I don't think the
    panic on the sender idea has been presented.

    Basically deadlocked goroutines are a bug which will crash a process
    eventually, so why no make this panic explicit? A program can then add a
    recover to the goroutine start if they want to explicitly allow the
    goroutines to be garbage collected.



    Also, what are the arguments against this? I cant really see a downside,
    and can see HUGE upsides. It could even work with existing go channels with
    some runtime logic regarding double ended channels and casting them to
    single sided channels. Was this overlooked in the original design of Go, or
    is there a good reason against it?








    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Jesse McNelis at Jul 2, 2014 at 11:55 pm

    On Thu, Jul 3, 2014 at 9:35 AM, wrote:
    In fact, this doesn't require breaking backwards compatibility, and it only
    would cause panics in already broken programs.

    Is the only reason this isn't done due to requiring big changes to the
    channel and garbage collector runtime?
    It's hard to do right, impossible to do under certain conditions, and
    is a problem that results from having an unrelated bug in your
    program.
    The panic from the deadlock is going to come from a goroutine that
    isn't the one with the bug, and likely some amount of time after the
    bug happened meaning the context of the bug is already gone.

    It's better to catch the original bug than to catch the resulting deadlock.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Andrewchamberss at Jul 3, 2014 at 12:07 am
    When is it ever impossible to do? It's always possible to detect this
    specific deadlock.

    Your point about finding the original cause of the error is valid, but the
    current behaviour is worse than proposed behaviour. Current behaviour
    relies on a user seeing that there are goroutines blocked forever and
    working out when it blocked. Proposed behaviour triggers it as soon as the
    runtime detects an issue making it easier to pinpoint the cause in logs.
    On Thursday, July 3, 2014 11:55:45 AM UTC+12, Jesse McNelis wrote:

    On Thu, Jul 3, 2014 at 9:35 AM, <andrewc...@gmail.com <javascript:>>
    wrote:
    In fact, this doesn't require breaking backwards compatibility, and it only
    would cause panics in already broken programs.

    Is the only reason this isn't done due to requiring big changes to the
    channel and garbage collector runtime?
    It's hard to do right, impossible to do under certain conditions, and
    is a problem that results from having an unrelated bug in your
    program.
    The panic from the deadlock is going to come from a goroutine that
    isn't the one with the bug, and likely some amount of time after the
    bug happened meaning the context of the bug is already gone.

    It's better to catch the original bug than to catch the resulting
    deadlock.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Matt Harden at Jul 3, 2014 at 12:34 am
    If one of the goroutines is panicking, why not let it terminate the whole
    program, as it does by default?

    On Wed, Jul 2, 2014 at 7:07 PM, wrote:

    When is it ever impossible to do? It's always possible to detect this
    specific deadlock.

    Your point about finding the original cause of the error is valid, but the
    current behaviour is worse than proposed behaviour. Current behaviour
    relies on a user seeing that there are goroutines blocked forever and
    working out when it blocked. Proposed behaviour triggers it as soon as the
    runtime detects an issue making it easier to pinpoint the cause in logs.
    On Thursday, July 3, 2014 11:55:45 AM UTC+12, Jesse McNelis wrote:
    On Thu, Jul 3, 2014 at 9:35 AM, wrote:
    In fact, this doesn't require breaking backwards compatibility, and it only
    would cause panics in already broken programs.

    Is the only reason this isn't done due to requiring big changes to the
    channel and garbage collector runtime?
    It's hard to do right, impossible to do under certain conditions, and
    is a problem that results from having an unrelated bug in your
    program.
    The panic from the deadlock is going to come from a goroutine that
    isn't the one with the bug, and likely some amount of time after the
    bug happened meaning the context of the bug is already gone.

    It's better to catch the original bug than to catch the resulting
    deadlock.
    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Andrewchamberss at Jul 3, 2014 at 12:39 am
    That's what I want by default., the program will crash, as it should if you
    have a deadlocked goroutine.
    Then if you actually intended for the goroutine to be garbage collected
    then you would just add a recover at the start and the goroutine now safely
    unwinds it's stack and terminates when collected.
    On Thursday, July 3, 2014 12:34:53 PM UTC+12, Matt Harden wrote:

    If one of the goroutines is panicking, why not let it terminate the whole
    program, as it does by default?


    On Wed, Jul 2, 2014 at 7:07 PM, <andrewc...@gmail.com <javascript:>>
    wrote:
    When is it ever impossible to do? It's always possible to detect this
    specific deadlock.

    Your point about finding the original cause of the error is valid, but
    the current behaviour is worse than proposed behaviour. Current behaviour
    relies on a user seeing that there are goroutines blocked forever and
    working out when it blocked. Proposed behaviour triggers it as soon as the
    runtime detects an issue making it easier to pinpoint the cause in logs.
    On Thursday, July 3, 2014 11:55:45 AM UTC+12, Jesse McNelis wrote:
    On Thu, Jul 3, 2014 at 9:35 AM, wrote:
    In fact, this doesn't require breaking backwards compatibility, and it only
    would cause panics in already broken programs.

    Is the only reason this isn't done due to requiring big changes to the
    channel and garbage collector runtime?
    It's hard to do right, impossible to do under certain conditions, and
    is a problem that results from having an unrelated bug in your
    program.
    The panic from the deadlock is going to come from a goroutine that
    isn't the one with the bug, and likely some amount of time after the
    bug happened meaning the context of the bug is already gone.

    It's better to catch the original bug than to catch the resulting
    deadlock.
    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts...@googlegroups.com <javascript:>.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Jesse McNelis at Jul 3, 2014 at 1:05 am

    On Thu, Jul 3, 2014 at 10:39 AM, wrote:
    That's what I want by default., the program will crash, as it should if you
    have a deadlocked goroutine.
    But by this stage you've already had an issue that your program hasn't
    handled correctly.
    It's either recover()ed from a panic that it shouldn't have or the
    goroutine has exited cleanly without communicating this correctly to
    those that depend on it.
    This is more complicated when you have multiple senders and receivers,
    which one should panic and how should the others deal with this
    unexpected issue?
    Then if you actually intended for the goroutine to be garbage collected then
    you would just add a recover at the start and the goroutine now safely
    unwinds it's stack and terminates when collected.
    If this is your intention then your program has a design problem.
    Discouraging this kind of design is the other reason for the current
    behaviour.
    There are quite a few threads on this mailing list covering this. Down
    this road is madness.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Andrewchamberss at Jul 3, 2014 at 1:38 am

    On Thursday, July 3, 2014 1:05:13 PM UTC+12, Jesse McNelis wrote:
    On Thu, Jul 3, 2014 at 10:39 AM, <andrewc...@gmail.com <javascript:>>
    wrote:
    That's what I want by default., the program will crash, as it should if you
    have a deadlocked goroutine.
    But by this stage you've already had an issue that your program hasn't
    handled correctly.
    In the current behaviour this is a bug issue that requires a full restart
    of the go process, there is no way to recover at all. In my proposed
    behaviour, the program has the option to recover if the programmer knows
    how to handle it, but the default is still a process restart. Moreover, it
    allows more elegant code for aborting streams of computation if the
    programmer explicitly recovers....

    You are suggesting "write a program with no bugs". Or you are suggesting
    "It is bad for the runtime to detect allocation errors". Would you also
    argue against normal garbage collection? You are essentially arguing
    against catching bugs automatically, and removing a class of bugs entirely.

    This is more complicated when you have multiple senders and receivers,
    which one should panic and how should the others deal with this
    unexpected issue?
    How is it more complicated? any write to a channel which cannot possibly
    ever read MUST be a bug, the reciever has been garbage collected...
    therefore any channel writing to it should fail rather than deadlock. The
    runtime already detects full program deadlocks and panics and no one
    complains about that.

    There is a blog post dedicated to resolving this type of situation
    http://blog.golang.org/pipeline
    the cancellation channels are an unneeded hack if the runtime detects a
    deadlock and allows a recovery.



    If this is your intention then your program has a design problem.
    Discouraging this kind of design is the other reason for the current
    behaviour.
    There is no design problem in my program... This change only flags real
    bugs, or allows more elegant solutions when explicitly specified.

    There are quite a few threads on this mailing list covering this. Down
    this road is madness.
    Could you provide me with actual code which demonstrates why this is a bad
    idea?
    I'll write some examples of why this is a good idea and post them here
    tonight.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Matt Harden at Jul 3, 2014 at 1:06 am
    I'm trying to understand the situation you're describing. What condition
    would cause one of the goroutines to stop reading from its input channel
    without panicking?

    On Wed, Jul 2, 2014 at 7:39 PM, wrote:

    That's what I want by default., the program will crash, as it should if
    you have a deadlocked goroutine.
    Then if you actually intended for the goroutine to be garbage collected
    then you would just add a recover at the start and the goroutine now safely
    unwinds it's stack and terminates when collected.

    On Thursday, July 3, 2014 12:34:53 PM UTC+12, Matt Harden wrote:

    If one of the goroutines is panicking, why not let it terminate the whole
    program, as it does by default?

    On Wed, Jul 2, 2014 at 7:07 PM, wrote:

    When is it ever impossible to do? It's always possible to detect this
    specific deadlock.

    Your point about finding the original cause of the error is valid, but
    the current behaviour is worse than proposed behaviour. Current behaviour
    relies on a user seeing that there are goroutines blocked forever and
    working out when it blocked. Proposed behaviour triggers it as soon as the
    runtime detects an issue making it easier to pinpoint the cause in logs.
    On Thursday, July 3, 2014 11:55:45 AM UTC+12, Jesse McNelis wrote:
    On Thu, Jul 3, 2014 at 9:35 AM, wrote:
    In fact, this doesn't require breaking backwards compatibility, and it only
    would cause panics in already broken programs.

    Is the only reason this isn't done due to requiring big changes to the
    channel and garbage collector runtime?
    It's hard to do right, impossible to do under certain conditions, and
    is a problem that results from having an unrelated bug in your
    program.
    The panic from the deadlock is going to come from a goroutine that
    isn't the one with the bug, and likely some amount of time after the
    bug happened meaning the context of the bug is already gone.

    It's better to catch the original bug than to catch the resulting
    deadlock.
    --
    You received this message because you are subscribed to the Google
    Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to golang-nuts...@googlegroups.com.

    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Andrewchamberss at Jul 3, 2014 at 1:30 am
    I'll post some examples tonight. Basically if there is a bug in the program
    where the reading end has terminated but left the channel open. The current
    behaviour of Go is a permanent memory leak, In the proposed behaviour the
    runtime detects this and causes a panic since it must be a bug.

    The proposed behaviour also allows programmer to explicitly allow
    goroutines to be freed by the GC rather than needing to add a close back
    channel.
    On Thursday, July 3, 2014 1:07:10 PM UTC+12, Matt Harden wrote:

    I'm trying to understand the situation you're describing. What condition
    would cause one of the goroutines to stop reading from its input channel
    without panicking?


    On Wed, Jul 2, 2014 at 7:39 PM, <andrewc...@gmail.com <javascript:>>
    wrote:
    That's what I want by default., the program will crash, as it should if
    you have a deadlocked goroutine.
    Then if you actually intended for the goroutine to be garbage collected
    then you would just add a recover at the start and the goroutine now safely
    unwinds it's stack and terminates when collected.

    On Thursday, July 3, 2014 12:34:53 PM UTC+12, Matt Harden wrote:

    If one of the goroutines is panicking, why not let it terminate the
    whole program, as it does by default?

    On Wed, Jul 2, 2014 at 7:07 PM, wrote:

    When is it ever impossible to do? It's always possible to detect this
    specific deadlock.

    Your point about finding the original cause of the error is valid, but
    the current behaviour is worse than proposed behaviour. Current behaviour
    relies on a user seeing that there are goroutines blocked forever and
    working out when it blocked. Proposed behaviour triggers it as soon as the
    runtime detects an issue making it easier to pinpoint the cause in logs.
    On Thursday, July 3, 2014 11:55:45 AM UTC+12, Jesse McNelis wrote:
    On Thu, Jul 3, 2014 at 9:35 AM, wrote:
    In fact, this doesn't require breaking backwards compatibility, and it only
    would cause panics in already broken programs.

    Is the only reason this isn't done due to requiring big changes to the
    channel and garbage collector runtime?
    It's hard to do right, impossible to do under certain conditions, and
    is a problem that results from having an unrelated bug in your
    program.
    The panic from the deadlock is going to come from a goroutine that
    isn't the one with the bug, and likely some amount of time after the
    bug happened meaning the context of the bug is already gone.

    It's better to catch the original bug than to catch the resulting
    deadlock.
    --
    You received this message because you are subscribed to the Google
    Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to golang-nuts...@googlegroups.com.

    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts...@googlegroups.com <javascript:>.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Matt Harden at Jul 3, 2014 at 3:09 am

    On Wed, Jul 2, 2014 at 8:30 PM, wrote:
    Basically if there is a bug in the program where the reading end has
    terminated but left the channel open.
    But how did it terminate? I see three cases.

    1. The goroutine function exited normally (no panic). You should ensure
    that the goroutine only returns normally if it detected the channel had
    been closed. To me it would be very easy to ensure this in the code,
    especially in the case you described, where each goroutine reads from just
    one channel and writes to another. For instance, a simple `for x :=
    range(chan) { // process x }` would do that.

    2. The goroutine did not exit normally; that is, it panicked, and you did
    not recover() from it. Then the whole program ended and gave you a stack
    trace showing where the panic occurred.

    3. An unexpected panic occurred and your code recover()'d from it. That is
    a bad design. Don't do that. Let panic() do its job.

      The current behaviour of Go is a permanent memory leak, In the proposed
    behaviour the runtime detects this and causes a panic since it must be a
    bug.

    The proposed behaviour also allows programmer to explicitly allow
    goroutines to be freed by the GC rather than needing to add a close back
    channel.

    On Thursday, July 3, 2014 1:07:10 PM UTC+12, Matt Harden wrote:

    I'm trying to understand the situation you're describing. What condition
    would cause one of the goroutines to stop reading from its input channel
    without panicking?

    On Wed, Jul 2, 2014 at 7:39 PM, wrote:

    That's what I want by default., the program will crash, as it should if
    you have a deadlocked goroutine.
    Then if you actually intended for the goroutine to be garbage collected
    then you would just add a recover at the start and the goroutine now safely
    unwinds it's stack and terminates when collected.

    On Thursday, July 3, 2014 12:34:53 PM UTC+12, Matt Harden wrote:

    If one of the goroutines is panicking, why not let it terminate the
    whole program, as it does by default?

    On Wed, Jul 2, 2014 at 7:07 PM, wrote:

    When is it ever impossible to do? It's always possible to detect this
    specific deadlock.

    Your point about finding the original cause of the error is valid, but
    the current behaviour is worse than proposed behaviour. Current behaviour
    relies on a user seeing that there are goroutines blocked forever and
    working out when it blocked. Proposed behaviour triggers it as soon as the
    runtime detects an issue making it easier to pinpoint the cause in logs.
    On Thursday, July 3, 2014 11:55:45 AM UTC+12, Jesse McNelis wrote:
    On Thu, Jul 3, 2014 at 9:35 AM, wrote:
    In fact, this doesn't require breaking backwards compatibility, and it only
    would cause panics in already broken programs.

    Is the only reason this isn't done due to requiring big changes to the
    channel and garbage collector runtime?
    It's hard to do right, impossible to do under certain conditions, and
    is a problem that results from having an unrelated bug in your
    program.
    The panic from the deadlock is going to come from a goroutine that
    isn't the one with the bug, and likely some amount of time after the
    bug happened meaning the context of the bug is already gone.

    It's better to catch the original bug than to catch the resulting
    deadlock.
    --
    You received this message because you are subscribed to the Google
    Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to golang-nuts...@googlegroups.com.

    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google
    Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to golang-nuts...@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Andrewchamberss at Jul 3, 2014 at 3:36 am
    Here is a program with a bug. It uses infinite channels to compute some
    simple values. It has a clear goroutine leak.
    Currently the runtime will just run out of memory and never complain. In my
    proposal there will be a panic to at least notify the programmer.
    http://play.golang.org/p/CWte4ZTqJz

    Moreover, with my proposal the following will be perfectly valid. The
    solution is very elegant allowing infinite channels to be used no problem.

    http://play.golang.org/p/HOdMyiJ0MQ

    Can you guys rewrite the first example with no leak but using three
    goroutines to do the pipeline of numbers, evens and printing?
    I want to see how you do cancellation in this case without a leak using
    whatever method you see fit that works today.
    Specifically I want to be shown how to code the cancellation in a way that
    is as simple or simpler than my second example.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Andrewchamberss at Jul 3, 2014 at 3:43 am
    I also want you guys to imagine that the bug is intermittent and in much
    more complicated code. It's a big help if the runtime can at least flag
    this bug.
    On Thursday, July 3, 2014 3:36:00 PM UTC+12, andrewc...@gmail.com wrote:

    Here is a program with a bug. It uses infinite channels to compute some
    simple values. It has a clear goroutine leak.
    Currently the runtime will just run out of memory and never complain. In
    my proposal there will be a panic to at least notify the programmer.
    http://play.golang.org/p/CWte4ZTqJz

    Moreover, with my proposal the following will be perfectly valid. The
    solution is very elegant allowing infinite channels to be used no problem.

    http://play.golang.org/p/HOdMyiJ0MQ

    Can you guys rewrite the first example with no leak but using three
    goroutines to do the pipeline of numbers, evens and printing?
    I want to see how you do cancellation in this case without a leak using
    whatever method you see fit that works today.
    Specifically I want to be shown how to code the cancellation in a way that
    is as simple or simpler than my second example.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Andrewchamberss at Jul 3, 2014 at 3:48 am
    And as a further addition, imagine that the any of the stages can fail for
    its own reasons, requiring that whatever stage failed propagate the failure
    back to each previous stage, aswell as propagate the error to a handler who
    can log why the pipeline failed. I encourage anyone who thinks my proposal
    is bad to write this code as an excercise.
    On Thursday, July 3, 2014 3:43:36 PM UTC+12, andrewc...@gmail.com wrote:

    I also want you guys to imagine that the bug is intermittent and in much
    more complicated code. It's a big help if the runtime can at least flag
    this bug.
    On Thursday, July 3, 2014 3:36:00 PM UTC+12, andrewc...@gmail.com wrote:

    Here is a program with a bug. It uses infinite channels to compute some
    simple values. It has a clear goroutine leak.
    Currently the runtime will just run out of memory and never complain. In
    my proposal there will be a panic to at least notify the programmer.
    http://play.golang.org/p/CWte4ZTqJz

    Moreover, with my proposal the following will be perfectly valid. The
    solution is very elegant allowing infinite channels to be used no problem.

    http://play.golang.org/p/HOdMyiJ0MQ

    Can you guys rewrite the first example with no leak but using three
    goroutines to do the pipeline of numbers, evens and printing?
    I want to see how you do cancellation in this case without a leak using
    whatever method you see fit that works today.
    Specifically I want to be shown how to code the cancellation in a way
    that is as simple or simpler than my second example.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Egon at Jul 3, 2014 at 4:23 am

    On Thursday, 3 July 2014 06:36:00 UTC+3, andrewc...@gmail.com wrote:
    Here is a program with a bug. It uses infinite channels to compute some
    simple values. It has a clear goroutine leak.
    Currently the runtime will just run out of memory and never complain. In
    my proposal there will be a panic to at least notify the programmer.
    http://play.golang.org/p/CWte4ZTqJz

    Moreover, with my proposal the following will be perfectly valid. The
    solution is very elegant allowing infinite channels to be used no problem.

    http://play.golang.org/p/HOdMyiJ0MQ

    Can you guys rewrite the first example with no leak but using three
    goroutines to do the pipeline of numbers, evens and printing?
    I want to see how you do cancellation in this case without a leak using
    whatever method you see fit that works today.
    Specifically I want to be shown how to code the cancellation in a way that
    is as simple or simpler than my second example.
    See the tomb package (
    http://blog.labix.org/2011/10/09/death-of-goroutines-under-control
    http://godoc.org/gopkg.in/tomb.v2 ).

    + egon

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Andrewchamberss at Jul 3, 2014 at 4:48 am
    Thanks Egon,

    Could you recreate my first example with the tomb package as a github gist
    or something like that? I'm glad this package exists, but I think the fact
    the package exists in the first place means there is at least some room for
    improvement in runtime support.

    On Thursday, July 3, 2014 4:23:40 PM UTC+12, egon wrote:


    On Thursday, 3 July 2014 06:36:00 UTC+3, andrewc...@gmail.com wrote:

    Here is a program with a bug. It uses infinite channels to compute some
    simple values. It has a clear goroutine leak.
    Currently the runtime will just run out of memory and never complain. In
    my proposal there will be a panic to at least notify the programmer.
    http://play.golang.org/p/CWte4ZTqJz

    Moreover, with my proposal the following will be perfectly valid. The
    solution is very elegant allowing infinite channels to be used no problem.

    http://play.golang.org/p/HOdMyiJ0MQ

    Can you guys rewrite the first example with no leak but using three
    goroutines to do the pipeline of numbers, evens and printing?
    I want to see how you do cancellation in this case without a leak using
    whatever method you see fit that works today.
    Specifically I want to be shown how to code the cancellation in a way
    that is as simple or simpler than my second example.
    See the tomb package (
    http://blog.labix.org/2011/10/09/death-of-goroutines-under-control
    http://godoc.org/gopkg.in/tomb.v2 ).

    + egon
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Egon at Jul 3, 2014 at 5:18 am

    On Thursday, 3 July 2014 07:48:24 UTC+3, andrewc...@gmail.com wrote:
    Thanks Egon,

    Could you recreate my first example with the tomb package as a github gist
    or something like that? I'm glad this package exists, but I think the fact
    the package exists in the first place means there is at least some room for
    improvement in runtime support.
    https://gist.github.com/egonelbre/44bf352952f96df7ce57

    It's one possible solution, but there could be simpler solutions.

    On Thursday, July 3, 2014 4:23:40 PM UTC+12, egon wrote:


    On Thursday, 3 July 2014 06:36:00 UTC+3, andrewc...@gmail.com wrote:

    Here is a program with a bug. It uses infinite channels to compute some
    simple values. It has a clear goroutine leak.
    Currently the runtime will just run out of memory and never complain. In
    my proposal there will be a panic to at least notify the programmer.
    http://play.golang.org/p/CWte4ZTqJz

    Moreover, with my proposal the following will be perfectly valid. The
    solution is very elegant allowing infinite channels to be used no problem.

    http://play.golang.org/p/HOdMyiJ0MQ

    Can you guys rewrite the first example with no leak but using three
    goroutines to do the pipeline of numbers, evens and printing?
    I want to see how you do cancellation in this case without a leak using
    whatever method you see fit that works today.
    Specifically I want to be shown how to code the cancellation in a way
    that is as simple or simpler than my second example.
    See the tomb package (
    http://blog.labix.org/2011/10/09/death-of-goroutines-under-control
    http://godoc.org/gopkg.in/tomb.v2 ).

    + egon
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Andrewchamberss at Jul 3, 2014 at 5:33 am
    It looks like Tomb is doing an ok job, but in my opinion its not as simple
    as my second example requiring alot of boiler plate. Infinite lazy lists
    seems like such an idiomatic use for channels, and sadly they cant be
    automatically collected when the read end is gone.
    On Thursday, July 3, 2014 5:18:05 PM UTC+12, egon wrote:


    On Thursday, 3 July 2014 07:48:24 UTC+3, andrewc...@gmail.com wrote:

    Thanks Egon,

    Could you recreate my first example with the tomb package as a github
    gist or something like that? I'm glad this package exists, but I think the
    fact the package exists in the first place means there is at least some
    room for improvement in runtime support.
    https://gist.github.com/egonelbre/44bf352952f96df7ce57

    It's one possible solution, but there could be simpler solutions.

    On Thursday, July 3, 2014 4:23:40 PM UTC+12, egon wrote:


    On Thursday, 3 July 2014 06:36:00 UTC+3, andrewc...@gmail.com wrote:

    Here is a program with a bug. It uses infinite channels to compute some
    simple values. It has a clear goroutine leak.
    Currently the runtime will just run out of memory and never complain.
    In my proposal there will be a panic to at least notify the programmer.
    http://play.golang.org/p/CWte4ZTqJz

    Moreover, with my proposal the following will be perfectly valid. The
    solution is very elegant allowing infinite channels to be used no problem.

    http://play.golang.org/p/HOdMyiJ0MQ

    Can you guys rewrite the first example with no leak but using three
    goroutines to do the pipeline of numbers, evens and printing?
    I want to see how you do cancellation in this case without a leak using
    whatever method you see fit that works today.
    Specifically I want to be shown how to code the cancellation in a way
    that is as simple or simpler than my second example.
    See the tomb package (
    http://blog.labix.org/2011/10/09/death-of-goroutines-under-control
    http://godoc.org/gopkg.in/tomb.v2 ).

    + egon
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Egon at Jul 3, 2014 at 5:41 am

    On Thursday, 3 July 2014 08:33:27 UTC+3, andrewc...@gmail.com wrote:
    It looks like Tomb is doing an ok job, but in my opinion its not as simple
    as my second example requiring alot of boiler plate. Infinite lazy lists
    seems like such an idiomatic use for channels, and sadly they cant be
    automatically collected when the read end is gone.
    https://gist.github.com/egonelbre/1322cf22cc0043d6b276

    an alternative solution.

    On Thursday, July 3, 2014 5:18:05 PM UTC+12, egon wrote:


    On Thursday, 3 July 2014 07:48:24 UTC+3, andrewc...@gmail.com wrote:

    Thanks Egon,

    Could you recreate my first example with the tomb package as a github
    gist or something like that? I'm glad this package exists, but I think the
    fact the package exists in the first place means there is at least some
    room for improvement in runtime support.
    https://gist.github.com/egonelbre/44bf352952f96df7ce57

    It's one possible solution, but there could be simpler solutions.

    On Thursday, July 3, 2014 4:23:40 PM UTC+12, egon wrote:


    On Thursday, 3 July 2014 06:36:00 UTC+3, andrewc...@gmail.com wrote:

    Here is a program with a bug. It uses infinite channels to compute
    some simple values. It has a clear goroutine leak.
    Currently the runtime will just run out of memory and never complain.
    In my proposal there will be a panic to at least notify the programmer.
    http://play.golang.org/p/CWte4ZTqJz

    Moreover, with my proposal the following will be perfectly valid. The
    solution is very elegant allowing infinite channels to be used no problem.

    http://play.golang.org/p/HOdMyiJ0MQ

    Can you guys rewrite the first example with no leak but using three
    goroutines to do the pipeline of numbers, evens and printing?
    I want to see how you do cancellation in this case without a leak
    using whatever method you see fit that works today.
    Specifically I want to be shown how to code the cancellation in a way
    that is as simple or simpler than my second example.
    See the tomb package (
    http://blog.labix.org/2011/10/09/death-of-goroutines-under-control
    http://godoc.org/gopkg.in/tomb.v2 ).

    + egon
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Andrewchamberss at Jul 3, 2014 at 5:51 am
    Not only are these solutions more code, they seem more error prone, and
    don't automatically detect errors. In this one you are essentially
    implementing better channel semantics with defers and locks.

    I'm convinced its possible to work around this issue, but I'm still not
    convinced the solution is as useful overall as the proposed solution in the
    thread.
    On Thursday, July 3, 2014 5:41:42 PM UTC+12, egon wrote:


    On Thursday, 3 July 2014 08:33:27 UTC+3, andrewc...@gmail.com wrote:

    It looks like Tomb is doing an ok job, but in my opinion its not as
    simple as my second example requiring alot of boiler plate. Infinite lazy
    lists seems like such an idiomatic use for channels, and sadly they cant be
    automatically collected when the read end is gone.
    https://gist.github.com/egonelbre/1322cf22cc0043d6b276

    an alternative solution.

    On Thursday, July 3, 2014 5:18:05 PM UTC+12, egon wrote:


    On Thursday, 3 July 2014 07:48:24 UTC+3, andrewc...@gmail.com wrote:

    Thanks Egon,

    Could you recreate my first example with the tomb package as a github
    gist or something like that? I'm glad this package exists, but I think the
    fact the package exists in the first place means there is at least some
    room for improvement in runtime support.
    https://gist.github.com/egonelbre/44bf352952f96df7ce57

    It's one possible solution, but there could be simpler solutions.

    On Thursday, July 3, 2014 4:23:40 PM UTC+12, egon wrote:


    On Thursday, 3 July 2014 06:36:00 UTC+3, andrewc...@gmail.com wrote:

    Here is a program with a bug. It uses infinite channels to compute
    some simple values. It has a clear goroutine leak.
    Currently the runtime will just run out of memory and never complain.
    In my proposal there will be a panic to at least notify the programmer.
    http://play.golang.org/p/CWte4ZTqJz

    Moreover, with my proposal the following will be perfectly valid. The
    solution is very elegant allowing infinite channels to be used no problem.

    http://play.golang.org/p/HOdMyiJ0MQ

    Can you guys rewrite the first example with no leak but using three
    goroutines to do the pipeline of numbers, evens and printing?
    I want to see how you do cancellation in this case without a leak
    using whatever method you see fit that works today.
    Specifically I want to be shown how to code the cancellation in a way
    that is as simple or simpler than my second example.
    See the tomb package (
    http://blog.labix.org/2011/10/09/death-of-goroutines-under-control
    http://godoc.org/gopkg.in/tomb.v2 ).

    + egon
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Andrewchamberss at Jul 3, 2014 at 5:45 am
    The real core of this idea is the runtime should detect channels with no
    reachable recievers and automatically close them. The worst case scenario
    is that it exposes resource leaks in existing programs as the writers
    panic.

    The main advantage is it allows code to include idiomatic lazy sequences
    which are garbage collected. One example would be
    http://golang.org/pkg/time/#Ticker not leaking if you left it. Other
    examples are inifinte channels of prime numbers etc. It really has alot of
    uses.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Egon at Jul 3, 2014 at 5:56 am

    On Thursday, 3 July 2014 08:45:16 UTC+3, andrewc...@gmail.com wrote:
    The real core of this idea is the runtime should detect channels with no
    reachable recievers and automatically close them.
    I can not imagine a good solution that would detect this. Even a simple

    ch := make(chan int)
    ch <- 1
    fmt.Println(<-ch)

    would be difficult to detect, although not impossible. It becomes more
    difficult with multiple goroutines:

    ch := make(chan int)
    go func(){ for { time.Sleep(1*time.Second) }; <-ch }
    ch <- 1

    I'm guessing detection of such deadlocks is as complex as the halting
    problem.

    + egon


    >
    The worst case scenario is that it exposes resource leaks in existing
    programs as the writers panic.

    The main advantage is it allows code to include idiomatic lazy sequences
    which are garbage collected. One example would be
    http://golang.org/pkg/time/#Ticker not leaking if you left it. Other
    examples are inifinte channels of prime numbers etc. It really has alot of
    uses.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Andrewchamberss at Jul 3, 2014 at 5:58 am
    I don't think you are understanding me.Garbage collection is 100 percent
    precise. I'm suggesting only closing the channel when every single recieve
    end is collected. This is 100 percent possible and reasonable.
    On Thursday, July 3, 2014 5:56:54 PM UTC+12, egon wrote:

    On Thursday, 3 July 2014 08:45:16 UTC+3, andrewc...@gmail.com wrote:

    The real core of this idea is the runtime should detect channels with no
    reachable recievers and automatically close them.
    I can not imagine a good solution that would detect this. Even a simple

    ch := make(chan int)
    ch <- 1
    fmt.Println(<-ch)

    would be difficult to detect, although not impossible. It becomes more
    difficult with multiple goroutines:

    ch := make(chan int)
    go func(){ for { time.Sleep(1*time.Second) }; <-ch }
    ch <- 1

    I'm guessing detection of such deadlocks is as complex as the halting
    problem.

    + egon


    The worst case scenario is that it exposes resource leaks in existing
    programs as the writers panic.

    The main advantage is it allows code to include idiomatic lazy sequences
    which are garbage collected. One example would be
    http://golang.org/pkg/time/#Ticker not leaking if you left it. Other
    examples are inifinte channels of prime numbers etc. It really has alot of
    uses.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Andrewchamberss at Jul 3, 2014 at 6:00 am
    It only requires that both channel ends are two different objects in the
    eyes of the garbage collector.
    On Thursday, July 3, 2014 5:58:56 PM UTC+12, andrewc...@gmail.com wrote:

    I don't think you are understanding me.Garbage collection is 100 percent
    precise. I'm suggesting only closing the channel when every single recieve
    end is collected. This is 100 percent possible and reasonable.
    On Thursday, July 3, 2014 5:56:54 PM UTC+12, egon wrote:

    On Thursday, 3 July 2014 08:45:16 UTC+3, andrewc...@gmail.com wrote:

    The real core of this idea is the runtime should detect channels with no
    reachable recievers and automatically close them.
    I can not imagine a good solution that would detect this. Even a simple

    ch := make(chan int)
    ch <- 1
    fmt.Println(<-ch)

    would be difficult to detect, although not impossible. It becomes more
    difficult with multiple goroutines:

    ch := make(chan int)
    go func(){ for { time.Sleep(1*time.Second) }; <-ch }
    ch <- 1

    I'm guessing detection of such deadlocks is as complex as the halting
    problem.

    + egon


    The worst case scenario is that it exposes resource leaks in existing
    programs as the writers panic.

    The main advantage is it allows code to include idiomatic lazy sequences
    which are garbage collected. One example would be
    http://golang.org/pkg/time/#Ticker not leaking if you left it. Other
    examples are inifinte channels of prime numbers etc. It really has alot of
    uses.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Egon at Jul 3, 2014 at 6:07 am

    On Thursday, 3 July 2014 08:58:56 UTC+3, andrewc...@gmail.com wrote:
    I don't think you are understanding me.Garbage collection is 100 percent
    precise. I'm suggesting only closing the channel when every single recieve
    end is collected. This is 100 percent possible and reasonable.
    I don't think it's sufficient for complicated systems, although it would
    catch some deadlocks. It would make more sense to do a proper handling.

    With actor systems you would create timeouts, e.g. if send hasn't happened
    in 10sec assume something has went wrong, so shutdown with an timeout error
    and give control to supervisor.

    + egon

    On Thursday, July 3, 2014 5:56:54 PM UTC+12, egon wrote:

    On Thursday, 3 July 2014 08:45:16 UTC+3, andrewc...@gmail.com wrote:

    The real core of this idea is the runtime should detect channels with no
    reachable recievers and automatically close them.
    I can not imagine a good solution that would detect this. Even a simple

    ch := make(chan int)
    ch <- 1
    fmt.Println(<-ch)

    would be difficult to detect, although not impossible. It becomes more
    difficult with multiple goroutines:

    ch := make(chan int)
    go func(){ for { time.Sleep(1*time.Second) }; <-ch }
    ch <- 1

    I'm guessing detection of such deadlocks is as complex as the halting
    problem.

    + egon


    The worst case scenario is that it exposes resource leaks in existing
    programs as the writers panic.

    The main advantage is it allows code to include idiomatic lazy sequences
    which are garbage collected. One example would be
    http://golang.org/pkg/time/#Ticker not leaking if you left it. Other
    examples are inifinte channels of prime numbers etc. It really has alot of
    uses.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Andrewchamberss at Jul 3, 2014 at 6:10 am
    I agree it wouldn't catch every deadlock, but it catches a trivial case,
    and it also allows infinite generators to clean themselves up which is
    incredibly useful.
    On Thursday, July 3, 2014 6:07:56 PM UTC+12, egon wrote:
    On Thursday, 3 July 2014 08:58:56 UTC+3, andrewc...@gmail.com wrote:

    I don't think you are understanding me.Garbage collection is 100 percent
    precise. I'm suggesting only closing the channel when every single recieve
    end is collected. This is 100 percent possible and reasonable.
    I don't think it's sufficient for complicated systems, although it would
    catch some deadlocks. It would make more sense to do a proper handling.

    With actor systems you would create timeouts, e.g. if send hasn't happened
    in 10sec assume something has went wrong, so shutdown with an timeout error
    and give control to supervisor.

    + egon

    On Thursday, July 3, 2014 5:56:54 PM UTC+12, egon wrote:

    On Thursday, 3 July 2014 08:45:16 UTC+3, andrewc...@gmail.com wrote:

    The real core of this idea is the runtime should detect channels with
    no reachable recievers and automatically close them.
    I can not imagine a good solution that would detect this. Even a simple

    ch := make(chan int)
    ch <- 1
    fmt.Println(<-ch)

    would be difficult to detect, although not impossible. It becomes more
    difficult with multiple goroutines:

    ch := make(chan int)
    go func(){ for { time.Sleep(1*time.Second) }; <-ch }
    ch <- 1

    I'm guessing detection of such deadlocks is as complex as the halting
    problem.

    + egon


    The worst case scenario is that it exposes resource leaks in existing
    programs as the writers panic.

    The main advantage is it allows code to include idiomatic lazy
    sequences which are garbage collected. One example would be
    http://golang.org/pkg/time/#Ticker not leaking if you left it. Other
    examples are inifinte channels of prime numbers etc. It really has alot of
    uses.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Andrewchamberss at Jul 3, 2014 at 10:56 am
    A final note after discussing with other people just so that it is on
    record and searchable:

    you can detect that a channel is unreachable from goroutines other than
    those goroutines that are blocked on it -- all the blocked goroutines are
    deadlocked
    forever. This technique applies to sync.Mutex too. This is doable with
    changes to the garbage collector collector.

    Performing a panic in the blocked goroutine is a language change so its
    impossible to do for Go 1.

    Performing a silent cancel of a detected blocked goroutine changes no
    behaviour and fixes memory leaks, but allows no deferred calls for cleanup,
    goroutines will disappear making the bug harder to fix, and if libraries
    depend on this to stop leaks it means they are non portable.

    Conclusion:

    This is a good idea provided Go 2 semantics state that provably deadlocked
    goroutines must panic.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Matt Harden at Jul 4, 2014 at 1:56 am
    I don't think it's a good idea. It encourages people to rely on
    (implementation details of) the GC for correct program behavior. Go is not
    Haskell, and channels are not lazy lists. Don't get me wrong; I'm a huge
    fan of lazy functional programming, but it's not the Go way.

    On Thu, Jul 3, 2014 at 5:56 AM, wrote:

    A final note after discussing with other people just so that it is on
    record and searchable:

    you can detect that a channel is unreachable from goroutines other than
    those goroutines that are blocked on it -- all the blocked goroutines are
    deadlocked
    forever. This technique applies to sync.Mutex too. This is doable with
    changes to the garbage collector collector.

    Performing a panic in the blocked goroutine is a language change so its
    impossible to do for Go 1.

    Performing a silent cancel of a detected blocked goroutine changes no
    behaviour and fixes memory leaks, but allows no deferred calls for cleanup,
    goroutines will disappear making the bug harder to fix, and if libraries
    depend on this to stop leaks it means they are non portable.

    Conclusion:

    This is a good idea provided Go 2 semantics state that provably deadlocked
    goroutines must panic.

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Andrewchamberss at Jul 4, 2014 at 2:06 am
    This could be true, the detection might be useful for debugging if it cant
    affect runtime behaviour.

    On Friday, July 4, 2014 1:56:56 PM UTC+12, Matt Harden wrote:

    I don't think it's a good idea. It encourages people to rely on
    (implementation details of) the GC for correct program behavior. Go is not
    Haskell, and channels are not lazy lists. Don't get me wrong; I'm a huge
    fan of lazy functional programming, but it's not the Go way.


    On Thu, Jul 3, 2014 at 5:56 AM, <andrewc...@gmail.com <javascript:>>
    wrote:
    A final note after discussing with other people just so that it is on
    record and searchable:

    you can detect that a channel is unreachable from goroutines other than
    those goroutines that are blocked on it -- all the blocked goroutines
    are deadlocked
    forever. This technique applies to sync.Mutex too. This is doable with
    changes to the garbage collector collector.

    Performing a panic in the blocked goroutine is a language change so its
    impossible to do for Go 1.

    Performing a silent cancel of a detected blocked goroutine changes no
    behaviour and fixes memory leaks, but allows no deferred calls for cleanup,
    goroutines will disappear making the bug harder to fix, and if libraries
    depend on this to stop leaks it means they are non portable.

    Conclusion:

    This is a good idea provided Go 2 semantics state that provably
    deadlocked goroutines must panic.

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts...@googlegroups.com <javascript:>.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Matt Harden at Jul 4, 2014 at 2:11 am
    This works with pretty minimal changes from your example.

    http://play.golang.org/p/za9qlqCaxn

    On Thu, Jul 3, 2014 at 9:06 PM, wrote:


    This could be true, the detection might be useful for debugging if it cant
    affect runtime behaviour.


    On Friday, July 4, 2014 1:56:56 PM UTC+12, Matt Harden wrote:

    I don't think it's a good idea. It encourages people to rely on
    (implementation details of) the GC for correct program behavior. Go is not
    Haskell, and channels are not lazy lists. Don't get me wrong; I'm a huge
    fan of lazy functional programming, but it's not the Go way.

    On Thu, Jul 3, 2014 at 5:56 AM, wrote:

    A final note after discussing with other people just so that it is on
    record and searchable:

    you can detect that a channel is unreachable from goroutines other than
    those goroutines that are blocked on it -- all the blocked goroutines
    are deadlocked
    forever. This technique applies to sync.Mutex too. This is doable with
    changes to the garbage collector collector.

    Performing a panic in the blocked goroutine is a language change so its
    impossible to do for Go 1.

    Performing a silent cancel of a detected blocked goroutine changes no
    behaviour and fixes memory leaks, but allows no deferred calls for cleanup,
    goroutines will disappear making the bug harder to fix, and if libraries
    depend on this to stop leaks it means they are non portable.

    Conclusion:

    This is a good idea provided Go 2 semantics state that provably
    deadlocked goroutines must panic.

    --
    You received this message because you are subscribed to the Google
    Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to golang-nuts...@googlegroups.com.

    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Andrewchamberss at Jul 4, 2014 at 2:23 am
    Thats not a bad design for this case, my argument is the api now requires a
    defer close() which might as well be the same as mem := malloc(); defer
    free(); It works in some cases, but not all cases. Even gcc based C has
    __attribute__(cleanup(free)) which is the same thing. It's not really a
    substitute for garbage collection.

    I suppose it has to do for now though.
    On Friday, July 4, 2014 2:12:06 PM UTC+12, Matt Harden wrote:

    This works with pretty minimal changes from your example.

    http://play.golang.org/p/za9qlqCaxn


    On Thu, Jul 3, 2014 at 9:06 PM, <andrewc...@gmail.com <javascript:>>
    wrote:
    This could be true, the detection might be useful for debugging if it
    cant affect runtime behaviour.


    On Friday, July 4, 2014 1:56:56 PM UTC+12, Matt Harden wrote:

    I don't think it's a good idea. It encourages people to rely on
    (implementation details of) the GC for correct program behavior. Go is not
    Haskell, and channels are not lazy lists. Don't get me wrong; I'm a huge
    fan of lazy functional programming, but it's not the Go way.

    On Thu, Jul 3, 2014 at 5:56 AM, wrote:

    A final note after discussing with other people just so that it is on
    record and searchable:

    you can detect that a channel is unreachable from goroutines other
    than those goroutines that are blocked on it -- all the blocked
    goroutines are deadlocked
    forever. This technique applies to sync.Mutex too. This is doable with
    changes to the garbage collector collector.

    Performing a panic in the blocked goroutine is a language change so its
    impossible to do for Go 1.

    Performing a silent cancel of a detected blocked goroutine changes no
    behaviour and fixes memory leaks, but allows no deferred calls for cleanup,
    goroutines will disappear making the bug harder to fix, and if libraries
    depend on this to stop leaks it means they are non portable.

    Conclusion:

    This is a good idea provided Go 2 semantics state that provably
    deadlocked goroutines must panic.

    --
    You received this message because you are subscribed to the Google
    Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to golang-nuts...@googlegroups.com.

    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts...@googlegroups.com <javascript:>.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Jesse McNelis at Jul 4, 2014 at 4:23 am

    On Thu, Jul 3, 2014 at 8:56 PM, wrote:
    A final note after discussing with other people just so that it is on record
    and searchable:

    you can detect that a channel is unreachable from goroutines other than
    those goroutines that are blocked on it -- all the blocked goroutines are
    deadlocked
    forever. This technique applies to sync.Mutex too. This is doable with
    changes to the garbage collector collector.
    Aside from the 'at a distance' and implicit nature of having the
    garbage collector do communications between goroutines and the fact
    that the majority of deadlock/livelock bugs in goroutine communication
    aren't going to be resolved by this change, when you discuss changes
    to the semantics of channel operations you have to consider 'select'.
    The current channel semantics are consistent whether or not they
    appear in a 'select' block.

    This consistency has some, perhaps, unintuitive side effects.
    Most commonly the way receiving on a nil channel works.

    Receiving on a nil channel blocks forever, a nil channel is never
    ready to receive.
    This is useless and a bug if done outside of a select block because it
    will deadlock the goroutine.
    Inside a select block this behaviour is useful, the fact that a nil
    channel is never ready to receive
    allows other channel operations to proceed, allowing you to easily
    swap out which channels you want to receive on.

    There was discussion of panic'ing when receiving from a nil channel
    outside of a select block, but this creates an inconsistency.

    If receiving on a channel without a sender was to panic, should it
    also panic inside a select block?
    It's not a deadlock, but it is a receive on a channel that can never
    be ready to receive.

    There are many ways to deadlock a goroutine, should all of these cases panic?
    http://play.golang.org/p/GEC__u5NCW

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Jason Woods at Jul 3, 2014 at 5:55 am
    Hi
    On 3 Jul 2014, at 00:30, andrewchamberss@gmail.com wrote:

    To solve this I can either complicate logic by adding a select to each goroutine send which either sends to the next channel or recieves an error and panics + recover to terminate the nested call stack, or I can add this code to each stage:

    defer readChannelUntilEmpty(prevStage)

    This change means that if any goroutine fails, the previous ones wont be deadlocked and leaking memory as it is now guaranteed to run to completion.
    I had a similar problem. I just deferred a close() of the input channel.

    If the go routine returns due to logical bug and stops reading from channel it closes it in the defer. All sends then subsequently panic due to sending on a closed channel.

    Is this what you are after?

    It won't help debug the problem though but at least an auto restart can workaround it while you find time to fix it.

    Jason

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedJul 2, '14 at 11:30p
activeJul 4, '14 at 4:23a
posts33
users5
websitegolang.org

People

Translate

site design / logo © 2022 Grokbase