FAQ
I have a program in which a goroutine executes a long-running computation
that may be canceled. I'm not sure of the best way to do this. I tried
passing the goroutine a "quit" channel that it performs nonblocking
receives on in order to get the message to stop, and I was surprised to
learn that when GOMAXPROCS is 1, the runtime never switches away from the
goroutine to allow another part of the program to actually put something on
that channel. It behaves essentially the same as putting "for { }" in a
goroutine, but in my case it's not an empty or non-communicating loop --
it's doing non-blocking reads.

See http://play.golang.org/p/iZsCXv7vdw for a toy example. The real code
doesn't actually do an infinite loop, just a long computation, but the
result is essentially the same -- I can't stop it.

I know one solution is to call runtime.Gosched() before trying to read from
the channel, but is that the only way? I've read in multiple places people
discouraging the use of this function. Is there another way I should
structure my code? It's possible to just let the goroutine run but ignore
its result, but that would be too inefficient. I actually only want to stop
the goroutine in order to start another one with different inputs. Running
even two at the same time will really slow things down, and if I didn't
stop them before starting new ones I could get several.

Thanks for your help,
-Ryan

--

Search Discussions

  • Matt Kane's Brain at Nov 1, 2012 at 2:40 pm
    What's the problem with running with GOMAXPROCS=2?

    Any operation that could block within your long-running loop would
    give another goroutine a chance to run. This could be as simple as
    fmt.Println("still here!")
    On Thu, Nov 1, 2012 at 10:22 AM, Ryan Tarpine wrote:
    I have a program in which a goroutine executes a long-running computation
    that may be canceled. I'm not sure of the best way to do this. I tried
    passing the goroutine a "quit" channel that it performs nonblocking receives
    on in order to get the message to stop, and I was surprised to learn that
    when GOMAXPROCS is 1, the runtime never switches away from the goroutine to
    allow another part of the program to actually put something on that channel.
    It behaves essentially the same as putting "for { }" in a goroutine, but in
    my case it's not an empty or non-communicating loop -- it's doing
    non-blocking reads.

    See http://play.golang.org/p/iZsCXv7vdw for a toy example. The real code
    doesn't actually do an infinite loop, just a long computation, but the
    result is essentially the same -- I can't stop it.

    I know one solution is to call runtime.Gosched() before trying to read from
    the channel, but is that the only way? I've read in multiple places people
    discouraging the use of this function. Is there another way I should
    structure my code? It's possible to just let the goroutine run but ignore
    its result, but that would be too inefficient. I actually only want to stop
    the goroutine in order to start another one with different inputs. Running
    even two at the same time will really slow things down, and if I didn't stop
    them before starting new ones I could get several.

    Thanks for your help,
    -Ryan

    --


    --
    matt kane's brain
    http://hydrogenproject.com

    --
  • Lars Pensjö at Nov 1, 2012 at 2:44 pm
    For controlling life and death of goroutines, I recommend the Tomb package.
    See https://launchpad.net/tomb/ and http://go.pkgdoc.org/launchpad.net/tomb.

    On Thursday, 1 November 2012 15:40:37 UTC+1, mkb wrote:

    What's the problem with running with GOMAXPROCS=2?

    Any operation that could block within your long-running loop would
    give another goroutine a chance to run. This could be as simple as
    fmt.Println("still here!")
    On Thu, Nov 1, 2012 at 10:22 AM, Ryan Tarpine wrote:
    I have a program in which a goroutine executes a long-running
    computation
    that may be canceled. I'm not sure of the best way to do this. I tried
    passing the goroutine a "quit" channel that it performs nonblocking receives
    on in order to get the message to stop, and I was surprised to learn that
    when GOMAXPROCS is 1, the runtime never switches away from the goroutine to
    allow another part of the program to actually put something on that channel.
    It behaves essentially the same as putting "for { }" in a goroutine, but in
    my case it's not an empty or non-communicating loop -- it's doing
    non-blocking reads.

    See http://play.golang.org/p/iZsCXv7vdw for a toy example. The real code
    doesn't actually do an infinite loop, just a long computation, but the
    result is essentially the same -- I can't stop it.

    I know one solution is to call runtime.Gosched() before trying to read from
    the channel, but is that the only way? I've read in multiple places people
    discouraging the use of this function. Is there another way I should
    structure my code? It's possible to just let the goroutine run but ignore
    its result, but that would be too inefficient. I actually only want to stop
    the goroutine in order to start another one with different inputs. Running
    even two at the same time will really slow things down, and if I didn't stop
    them before starting new ones I could get several.

    Thanks for your help,
    -Ryan

    --


    --
    matt kane's brain
    http://hydrogenproject.com
    --
  • Ryan Tarpine at Nov 1, 2012 at 3:13 pm
    Is it possible in general to know whether a program could get stuck or not?
    In this program, with just a single goroutine, it's simple. Two is enough.
    But I'd worry that in a larger program with many long-running (possibly
    infinitely-looping) goroutines, even setting it to a large number like 16
    may still fail if it needed e.g. 17. I want to know that my program will
    always run, but that tweaking GOMAXPROCS might get me better performance.

    -Ryan
    On Thursday, November 1, 2012 10:40:37 AM UTC-4, mkb wrote:

    What's the problem with running with GOMAXPROCS=2?

    Any operation that could block within your long-running loop would
    give another goroutine a chance to run. This could be as simple as
    fmt.Println("still here!")
    On Thu, Nov 1, 2012 at 10:22 AM, Ryan Tarpine wrote:
    I have a program in which a goroutine executes a long-running
    computation
    that may be canceled. I'm not sure of the best way to do this. I tried
    passing the goroutine a "quit" channel that it performs nonblocking receives
    on in order to get the message to stop, and I was surprised to learn that
    when GOMAXPROCS is 1, the runtime never switches away from the goroutine to
    allow another part of the program to actually put something on that channel.
    It behaves essentially the same as putting "for { }" in a goroutine, but in
    my case it's not an empty or non-communicating loop -- it's doing
    non-blocking reads.

    See http://play.golang.org/p/iZsCXv7vdw for a toy example. The real code
    doesn't actually do an infinite loop, just a long computation, but the
    result is essentially the same -- I can't stop it.

    I know one solution is to call runtime.Gosched() before trying to read from
    the channel, but is that the only way? I've read in multiple places people
    discouraging the use of this function. Is there another way I should
    structure my code? It's possible to just let the goroutine run but ignore
    its result, but that would be too inefficient. I actually only want to stop
    the goroutine in order to start another one with different inputs. Running
    even two at the same time will really slow things down, and if I didn't stop
    them before starting new ones I could get several.

    Thanks for your help,
    -Ryan

    --


    --
    matt kane's brain
    http://hydrogenproject.com
    --
  • Patrick Mylund Nielsen at Nov 1, 2012 at 3:18 pm
    Is it possible in general to know whether a program could get stuck or
    not?

    No! :) Not in general.

    http://en.wikipedia.org/wiki/Halting_problem

    On Thu, Nov 1, 2012 at 4:13 PM, Ryan Tarpine wrote:

    Is it possible in general to know whether a program could get stuck or
    not? In this program, with just a single goroutine, it's simple. Two is
    enough. But I'd worry that in a larger program with many long-running
    (possibly infinitely-looping) goroutines, even setting it to a large number
    like 16 may still fail if it needed e.g. 17. I want to know that my program
    will always run, but that tweaking GOMAXPROCS might get me better
    performance.

    -Ryan
    On Thursday, November 1, 2012 10:40:37 AM UTC-4, mkb wrote:

    What's the problem with running with GOMAXPROCS=2?

    Any operation that could block within your long-running loop would
    give another goroutine a chance to run. This could be as simple as
    fmt.Println("still here!")
    On Thu, Nov 1, 2012 at 10:22 AM, Ryan Tarpine wrote:
    I have a program in which a goroutine executes a long-running
    computation
    that may be canceled. I'm not sure of the best way to do this. I tried
    passing the goroutine a "quit" channel that it performs nonblocking receives
    on in order to get the message to stop, and I was surprised to learn that
    when GOMAXPROCS is 1, the runtime never switches away from the
    goroutine to
    allow another part of the program to actually put something on that channel.
    It behaves essentially the same as putting "for { }" in a goroutine, but in
    my case it's not an empty or non-communicating loop -- it's doing
    non-blocking reads.

    See http://play.golang.org/p/**iZsCXv7vdw<http://play.golang.org/p/iZsCXv7vdw>for a toy example. The real code
    doesn't actually do an infinite loop, just a long computation, but the
    result is essentially the same -- I can't stop it.

    I know one solution is to call runtime.Gosched() before trying to read from
    the channel, but is that the only way? I've read in multiple places people
    discouraging the use of this function. Is there another way I should
    structure my code? It's possible to just let the goroutine run but ignore
    its result, but that would be too inefficient. I actually only want to stop
    the goroutine in order to start another one with different inputs. Running
    even two at the same time will really slow things down, and if I didn't stop
    them before starting new ones I could get several.

    Thanks for your help,
    -Ryan

    --


    --
    matt kane's brain
    http://hydrogenproject.com
    --

    --
  • Ryan Tarpine at Nov 1, 2012 at 3:25 pm
    You know that's not what I meant ;-) To clarify, how can I be sure my
    program runs correctly on GOMAXPROCS=n if I'm already sure it would run
    correctly on GOMAXPROCS=∞?

    On Thursday, November 1, 2012 11:18:32 AM UTC-4, Patrick Mylund Nielsen
    wrote:
    Is it possible in general to know whether a program could get stuck or
    not?

    No! :) Not in general.

    http://en.wikipedia.org/wiki/Halting_problem


    On Thu, Nov 1, 2012 at 4:13 PM, Ryan Tarpine <[email protected]<javascript:>
    wrote:
    Is it possible in general to know whether a program could get stuck or
    not? In this program, with just a single goroutine, it's simple. Two is
    enough. But I'd worry that in a larger program with many long-running
    (possibly infinitely-looping) goroutines, even setting it to a large number
    like 16 may still fail if it needed e.g. 17. I want to know that my program
    will always run, but that tweaking GOMAXPROCS might get me better
    performance.

    -Ryan
    On Thursday, November 1, 2012 10:40:37 AM UTC-4, mkb wrote:

    What's the problem with running with GOMAXPROCS=2?

    Any operation that could block within your long-running loop would
    give another goroutine a chance to run. This could be as simple as
    fmt.Println("still here!")

    On Thu, Nov 1, 2012 at 10:22 AM, Ryan Tarpine <[email protected]>
    wrote:
    I have a program in which a goroutine executes a long-running
    computation
    that may be canceled. I'm not sure of the best way to do this. I tried
    passing the goroutine a "quit" channel that it performs nonblocking receives
    on in order to get the message to stop, and I was surprised to learn that
    when GOMAXPROCS is 1, the runtime never switches away from the
    goroutine to
    allow another part of the program to actually put something on that channel.
    It behaves essentially the same as putting "for { }" in a goroutine, but in
    my case it's not an empty or non-communicating loop -- it's doing
    non-blocking reads.

    See http://play.golang.org/p/**iZsCXv7vdw<http://play.golang.org/p/iZsCXv7vdw>for a toy example. The real code
    doesn't actually do an infinite loop, just a long computation, but the
    result is essentially the same -- I can't stop it.

    I know one solution is to call runtime.Gosched() before trying to read from
    the channel, but is that the only way? I've read in multiple places people
    discouraging the use of this function. Is there another way I should
    structure my code? It's possible to just let the goroutine run but ignore
    its result, but that would be too inefficient. I actually only want to stop
    the goroutine in order to start another one with different inputs. Running
    even two at the same time will really slow things down, and if I
    didn't stop
    them before starting new ones I could get several.

    Thanks for your help,
    -Ryan

    --


    --
    matt kane's brain
    http://hydrogenproject.com
    --

    --
  • Patrick Mylund Nielsen at Nov 1, 2012 at 3:27 pm
    Sorry. Couldn't help myself.

    On Thu, Nov 1, 2012 at 4:25 PM, Ryan Tarpine wrote:

    You know that's not what I meant ;-) To clarify, how can I be sure my
    program runs correctly on GOMAXPROCS=n if I'm already sure it would run
    correctly on GOMAXPROCS=∞?


    On Thursday, November 1, 2012 11:18:32 AM UTC-4, Patrick Mylund Nielsen
    wrote:
    Is it possible in general to know whether a program could get stuck or
    not?

    No! :) Not in general.

    http://en.wikipedia.org/wiki/**Halting_problem<http://en.wikipedia.org/wiki/Halting_problem>

    On Thu, Nov 1, 2012 at 4:13 PM, Ryan Tarpine wrote:

    Is it possible in general to know whether a program could get stuck or
    not? In this program, with just a single goroutine, it's simple. Two is
    enough. But I'd worry that in a larger program with many long-running
    (possibly infinitely-looping) goroutines, even setting it to a large number
    like 16 may still fail if it needed e.g. 17. I want to know that my program
    will always run, but that tweaking GOMAXPROCS might get me better
    performance.

    -Ryan
    On Thursday, November 1, 2012 10:40:37 AM UTC-4, mkb wrote:

    What's the problem with running with GOMAXPROCS=2?

    Any operation that could block within your long-running loop would
    give another goroutine a chance to run. This could be as simple as
    fmt.Println("still here!")

    On Thu, Nov 1, 2012 at 10:22 AM, Ryan Tarpine <[email protected]>
    wrote:
    I have a program in which a goroutine executes a long-running
    computation
    that may be canceled. I'm not sure of the best way to do this. I tried
    passing the goroutine a "quit" channel that it performs nonblocking receives
    on in order to get the message to stop, and I was surprised to learn that
    when GOMAXPROCS is 1, the runtime never switches away from the
    goroutine to
    allow another part of the program to actually put something on that channel.
    It behaves essentially the same as putting "for { }" in a goroutine, but in
    my case it's not an empty or non-communicating loop -- it's doing
    non-blocking reads.

    See http://play.golang.org/p/**iZsCX**v7vdw<http://play.golang.org/p/iZsCXv7vdw>for a toy example. The real code
    doesn't actually do an infinite loop, just a long computation, but the
    result is essentially the same -- I can't stop it.

    I know one solution is to call runtime.Gosched() before trying to read from
    the channel, but is that the only way? I've read in multiple places people
    discouraging the use of this function. Is there another way I should
    structure my code? It's possible to just let the goroutine run but ignore
    its result, but that would be too inefficient. I actually only want to stop
    the goroutine in order to start another one with different inputs. Running
    even two at the same time will really slow things down, and if I
    didn't stop
    them before starting new ones I could get several.

    Thanks for your help,
    -Ryan

    --


    --
    matt kane's brain
    http://hydrogenproject.com
    --

    --
    --
  • Si guy at Nov 1, 2012 at 3:34 pm
    Maybe this is a good candidate for a compiler warning, it is implementation specific after all and many people seem to run into this issue.

    Would it be sufficient to detect non unrolled loops that make no blocking syscalls?

    --
  • Stevewang at Nov 1, 2012 at 3:23 pm
    Such sort of programs always run on a basis that you can always guarantee
    by setting proper GOMAXPROCS in the beginning of main function.
    On Thursday, November 1, 2012 11:13:29 PM UTC+8, Ryan Tarpine wrote:

    Is it possible in general to know whether a program could get stuck or
    not? In this program, with just a single goroutine, it's simple. Two is
    enough. But I'd worry that in a larger program with many long-running
    (possibly infinitely-looping) goroutines, even setting it to a large number
    like 16 may still fail if it needed e.g. 17. I want to know that my program
    will always run, but that tweaking GOMAXPROCS might get me better
    performance.

    -Ryan
    On Thursday, November 1, 2012 10:40:37 AM UTC-4, mkb wrote:

    What's the problem with running with GOMAXPROCS=2?

    Any operation that could block within your long-running loop would
    give another goroutine a chance to run. This could be as simple as
    fmt.Println("still here!")
    On Thu, Nov 1, 2012 at 10:22 AM, Ryan Tarpine wrote:
    I have a program in which a goroutine executes a long-running
    computation
    that may be canceled. I'm not sure of the best way to do this. I tried
    passing the goroutine a "quit" channel that it performs nonblocking receives
    on in order to get the message to stop, and I was surprised to learn that
    when GOMAXPROCS is 1, the runtime never switches away from the
    goroutine to
    allow another part of the program to actually put something on that channel.
    It behaves essentially the same as putting "for { }" in a goroutine, but in
    my case it's not an empty or non-communicating loop -- it's doing
    non-blocking reads.

    See http://play.golang.org/p/iZsCXv7vdw for a toy example. The real code
    doesn't actually do an infinite loop, just a long computation, but the
    result is essentially the same -- I can't stop it.

    I know one solution is to call runtime.Gosched() before trying to read from
    the channel, but is that the only way? I've read in multiple places people
    discouraging the use of this function. Is there another way I should
    structure my code? It's possible to just let the goroutine run but ignore
    its result, but that would be too inefficient. I actually only want to stop
    the goroutine in order to start another one with different inputs. Running
    even two at the same time will really slow things down, and if I didn't stop
    them before starting new ones I could get several.

    Thanks for your help,
    -Ryan

    --


    --
    matt kane's brain
    http://hydrogenproject.com
    --
  • Zeebo at Nov 1, 2012 at 2:46 pm
    If your work loop genuinely doesn't yield to the scheduler (which includes
    many things like i/o, channel operations, etc) then you have to use
    runtime.Gosched. That's just how cooperative scheduling works. In my
    experience, this is very rare.

    I don't know enough about your problem to suggest a structure where these
    yields would be a side effect, but perhaps instead of having one long
    running operation that you want to cancel, you could break up the problem
    into many smaller concurrent tasks and send smaller work units to them?
    On Thursday, November 1, 2012 10:22:46 AM UTC-4, Ryan Tarpine wrote:

    I have a program in which a goroutine executes a long-running computation
    that may be canceled. I'm not sure of the best way to do this. I tried
    passing the goroutine a "quit" channel that it performs nonblocking
    receives on in order to get the message to stop, and I was surprised to
    learn that when GOMAXPROCS is 1, the runtime never switches away from the
    goroutine to allow another part of the program to actually put something on
    that channel. It behaves essentially the same as putting "for { }" in a
    goroutine, but in my case it's not an empty or non-communicating loop --
    it's doing non-blocking reads.

    See http://play.golang.org/p/iZsCXv7vdw for a toy example. The real code
    doesn't actually do an infinite loop, just a long computation, but the
    result is essentially the same -- I can't stop it.

    I know one solution is to call runtime.Gosched() before trying to read
    from the channel, but is that the only way? I've read in multiple places
    people discouraging the use of this function. Is there another way I should
    structure my code? It's possible to just let the goroutine run but ignore
    its result, but that would be too inefficient. I actually only want to stop
    the goroutine in order to start another one with different inputs. Running
    even two at the same time will really slow things down, and if I didn't
    stop them before starting new ones I could get several.

    Thanks for your help,
    -Ryan
    --
  • Timon ter Braak at Nov 1, 2012 at 4:52 pm

    On 11/01/2012 03:46 PM, zeebo wrote:
    If your work loop genuinely doesn't yield to the scheduler (which
    includes many things like i/o, channel operations, etc) then you have to
    use runtime.Gosched. That's just how cooperative scheduling works. In my
    experience, this is very rare.
    I have a similar problem; a numerical optimization problem that does
    non-blocking reads on a 'stop' channel, and yields increasingly better
    solutions on a result channel. However, it might takes quite long before
    a solution is found, such that the goroutine does not seem to be
    responding. So, 'very rare' depends on your field of work...

    --
  • Stevewang at Nov 1, 2012 at 3:05 pm

    On Thursday, November 1, 2012 10:22:46 PM UTC+8, Ryan Tarpine wrote:
    I have a program in which a goroutine executes a long-running computation
    that may be canceled. I'm not sure of the best way to do this. I tried
    passing the goroutine a "quit" channel that it performs nonblocking
    receives on in order to get the message to stop, and I was surprised to
    learn that when GOMAXPROCS is 1, the runtime never switches away from the
    goroutine to allow another part of the program to actually put something on
    that channel. It behaves essentially the same as putting "for { }" in a
    goroutine, but in my case it's not an empty or non-communicating loop --
    it's doing non-blocking reads.

    See http://play.golang.org/p/iZsCXv7vdw for a toy example. The real code
    doesn't actually do an infinite loop, just a long computation, but the
    result is essentially the same -- I can't stop it.

    I know one solution is to call runtime.Gosched() before trying to read
    from the channel, but is that the only way? I've read in multiple places
    people discouraging the use of this function.
    Here encourage you to have a break(runtime.Gosched()) periodically in you
    long-running loop no matter how many cpus you have and use:)

    Is there another way I should structure my code? It's possible to just let
    the goroutine run but ignore its result, but that would be too inefficient.
    I actually only want to stop the goroutine in order to start another one
    with different inputs. Running even two at the same time will really slow
    things down, and if I didn't stop them before starting new ones I could get
    several.

    Thanks for your help,
    -Ryan
    --
  • Stevewang at Nov 1, 2012 at 3:06 pm
    Here I encourage you to have a break(runtime.Gosched()) periodically in you
    long-running loop no matter how many cpus you have and use:)
    On Thursday, November 1, 2012 11:05:32 PM UTC+8, stevewang wrote:


    On Thursday, November 1, 2012 10:22:46 PM UTC+8, Ryan Tarpine wrote:

    I have a program in which a goroutine executes a long-running computation
    that may be canceled. I'm not sure of the best way to do this. I tried
    passing the goroutine a "quit" channel that it performs nonblocking
    receives on in order to get the message to stop, and I was surprised to
    learn that when GOMAXPROCS is 1, the runtime never switches away from the
    goroutine to allow another part of the program to actually put something on
    that channel. It behaves essentially the same as putting "for { }" in a
    goroutine, but in my case it's not an empty or non-communicating loop --
    it's doing non-blocking reads.

    See http://play.golang.org/p/iZsCXv7vdw for a toy example. The real code
    doesn't actually do an infinite loop, just a long computation, but the
    result is essentially the same -- I can't stop it.

    I know one solution is to call runtime.Gosched() before trying to read
    from the channel, but is that the only way? I've read in multiple places
    people discouraging the use of this function.
    Here encourage you to have a break(runtime.Gosched()) periodically in you
    long-running loop no matter how many cpus you have and use:)

    Is there another way I should structure my code? It's possible to just let
    the goroutine run but ignore its result, but that would be too inefficient.
    I actually only want to stop the goroutine in order to start another one
    with different inputs. Running even two at the same time will really slow
    things down, and if I didn't stop them before starting new ones I could get
    several.

    Thanks for your help,
    -Ryan
    --
  • Dustin at Nov 1, 2012 at 4:56 pm

    On Thursday, November 1, 2012 7:22:46 AM UTC-7, Ryan Tarpine wrote:
    I have a program in which a goroutine executes a long-running computation
    that may be canceled. I'm not sure of the best way to do this. I tried
    passing the goroutine a "quit" channel that it performs nonblocking
    receives on in order to get the message to stop, and I was surprised to
    learn that when GOMAXPROCS is 1, the runtime never switches away from the
    goroutine to allow another part of the program to actually put something on
    that channel. It behaves essentially the same as putting "for { }" in a
    goroutine, but in my case it's not an empty or non-communicating loop --
    it's doing non-blocking reads.

    See http://play.golang.org/p/iZsCXv7vdw for a toy example. The real code
    doesn't actually do an infinite loop, just a long computation, but the
    result is essentially the same -- I can't stop it.

    I know one solution is to call runtime.Gosched() before trying to read
    from the channel, but is that the only way? I've read in multiple places
    people discouraging the use of this function. Is there another way I should
    structure my code? It's possible to just let the goroutine run but ignore
    its result, but that would be too inefficient. I actually only want to stop
    the goroutine in order to start another one with different inputs. Running
    even two at the same time will really slow things down, and if I didn't
    stop them before starting new ones I could get several.
    Gosched is a workaround, but not for anything *you're* doing. At some
    point, the scheduler will understand it can pick up another runnable
    goroutine and do it at the right time (perhaps one of those times you're
    doing the non-blocking channel receive would be a great hint).

    --

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedNov 1, '12 at 2:22p
activeNov 1, '12 at 4:56p
posts14
users9
websitegolang.org

People

Translate

site design / logo © 2023 Grokbase