Grokbase › Groups › Go › golang-nuts › December 2013
FAQ
Hello, I would like to know the best way to close a channel in a recursive
function ?

Here is a typical example is a simple function to generate all possible *1-3
letter words* :

package main
import "fmt"
func main() {
     result := make(chan string)
     go words(3, "", result)
     for v := range result {
         fmt.Printf("%s\n", v)
     }

     fmt.Println("got here")}
func words(length int , prefix string, result chan string) {
     if length < 1 {
         return
     }
     letter := ""
     for i := 97 ; i < 123 ; i++ {
         letter = fmt.Sprintf("%s%c", prefix, i)
         result <- letter
         words(length - 1, letter , result)
     }}


- How do you effectively *close result *or is there a better way to achieve
the same result ?
- What the best way to increase performance when working with eg go words(
10, "", result) because go words(length - 1, letter , result) can lead to
too many goroutines

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Egon at Dec 21, 2013 at 1:16 pm ⇧

    On Saturday, December 21, 2013 2:30:02 PM UTC+2, Oleku Konko wrote:
    Hello, I would like to know the best way to close a channel in a recursive
    function ?

    Here is a typical example is a simple function to generate all possible *1-3
    letter words* :

    package main
    import "fmt"
    func main() {
    result := make(chan string)
    go words(3, "", result)
    go func(){
       words(3, "", result)
       close(result)
    }()

    for v := range result {
    fmt.Printf("%s\n", v)
    }

    fmt.Println("got here")}
    func words(length int , prefix string, result chan string) {
    if length < 1 {
    return
    }
    letter := ""
    for i := 97 ; i < 123 ; i++ {
    letter = fmt.Sprintf("%s%c", prefix, i)
    result <- letter
    words(length - 1, letter , result)
    }}


    - How do you effectively *close result *or is there a better way to
    achieve the same result ?
    - What the best way to increase performance when working with eg go
    words(10, "", result) because go words(length - 1, letter , result) can
    lead to too many goroutines
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Egon at Dec 21, 2013 at 2:13 pm ⇧
    The parallel version, that avoids too many goroutines is a bit more
    complicated... http://play.golang.org/p/b-9x0UpuwW

    + egon
    On Saturday, December 21, 2013 2:30:02 PM UTC+2, Oleku Konko wrote:

    Hello, I would like to know the best way to close a channel in a recursive
    function ?

    Here is a typical example is a simple function to generate all possible *1-3
    letter words* :

    package main
    import "fmt"
    func main() {
    result := make(chan string)
    go words(3, "", result)
    for v := range result {
    fmt.Printf("%s\n", v)
    }

    fmt.Println("got here")}
    func words(length int , prefix string, result chan string) {
    if length < 1 {
    return
    }
    letter := ""
    for i := 97 ; i < 123 ; i++ {
    letter = fmt.Sprintf("%s%c", prefix, i)
    result <- letter
    words(length - 1, letter , result)
    }}


    - How do you effectively *close result *or is there a better way to
    achieve the same result ?
    - What the best way to increase performance when working with eg go
    words(10, "", result) because go words(length - 1, letter , result) can
    lead to too many goroutines
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Oleku Konko at Dec 21, 2013 at 5:08 pm ⇧
    - Thanks, did not thinking about wrapping it in a closure
    - The parallel version is more complicated as expected, i would take my
    time to study it and see if i can simplify it.

    On Saturday, December 21, 2013 3:13:29 PM UTC+1, egon wrote:

    The parallel version, that avoids too many goroutines is a bit more
    complicated... http://play.golang.org/p/b-9x0UpuwW

    + egon
    On Saturday, December 21, 2013 2:30:02 PM UTC+2, Oleku Konko wrote:

    Hello, I would like to know the best way to close a channel in a
    recursive function ?

    Here is a typical example is a simple function to generate all possible *1-3
    letter words* :

    package main
    import "fmt"
    func main() {
    result := make(chan string)
    go words(3, "", result)
    for v := range result {
    fmt.Printf("%s\n", v)
    }

    fmt.Println("got here")}
    func words(length int , prefix string, result chan string) {
    if length < 1 {
    return
    }
    letter := ""
    for i := 97 ; i < 123 ; i++ {
    letter = fmt.Sprintf("%s%c", prefix, i)
    result <- letter
    words(length - 1, letter , result)
    }}


    - How do you effectively *close result *or is there a better way to
    achieve the same result ?
    - What the best way to increase performance when working with eg go
    words(10, "", result) because go words(length - 1, letter , result) can
    lead to too many goroutines
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Egon at Dec 21, 2013 at 5:35 pm ⇧

    On Saturday, December 21, 2013 7:08:44 PM UTC+2, Oleku Konko wrote:


    - Thanks, did not thinking about wrapping it in a closure
    - The parallel version is more complicated as expected, i would take my
    time to study it and see if i can simplify it.
    Mostly, I would say no... I once already spent several days on it actually
    to get it fully correct and safe. I.e. everything there is by necessity.

    Although, I now was able to remove the alldone, by using close signaling...
    http://play.golang.org/p/iOJySmEUP0

    * the todo chan, is a easy way of implementing a semaphore... it's there to
    notify that there are new jobs and block items.
    * mu is there to protect the internal queue and other variables
    * active counts the number of active routines... this is to prevent worker
    death from starvation and to see when we are actually done
    * queue simply manages the work queue

    Maybe there is a simpler design, I could not find one... (i.e. one that is
    thread-safe and finishes all go-routines properly when done)

    + egon

    On Saturday, December 21, 2013 3:13:29 PM UTC+1, egon wrote:

    The parallel version, that avoids too many goroutines is a bit more
    complicated... http://play.golang.org/p/b-9x0UpuwW

    + egon
    On Saturday, December 21, 2013 2:30:02 PM UTC+2, Oleku Konko wrote:

    Hello, I would like to know the best way to close a channel in a
    recursive function ?

    Here is a typical example is a simple function to generate all possible *1-3
    letter words* :

    package main
    import "fmt"
    func main() {
    result := make(chan string)
    go words(3, "", result)
    for v := range result {
    fmt.Printf("%s\n", v)
    }

    fmt.Println("got here")}
    func words(length int , prefix string, result chan string) {
    if length < 1 {
    return
    }
    letter := ""
    for i := 97 ; i < 123 ; i++ {
    letter = fmt.Sprintf("%s%c", prefix, i)
    result <- letter
    words(length - 1, letter , result)
    }}


    - How do you effectively *close result *or is there a better way to
    achieve the same result ?
    - What the best way to increase performance when working with eg go
    words(10, "", result) because go words(length - 1, letter , result) can
    lead to too many goroutines
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Egon at Dec 21, 2013 at 5:44 pm ⇧
    ... and by modifying, immediately made a mistake... :/
    Here should be a fixed one http://play.golang.org/p/ZZQ5HCYfrn
    On Saturday, December 21, 2013 7:35:10 PM UTC+2, egon wrote:


    On Saturday, December 21, 2013 7:08:44 PM UTC+2, Oleku Konko wrote:



    - Thanks, did not thinking about wrapping it in a closure
    - The parallel version is more complicated as expected, i would take my
    time to study it and see if i can simplify it.
    Mostly, I would say no... I once already spent several days on it actually
    to get it fully correct and safe. I.e. everything there is by necessity.

    Although, I now was able to remove the alldone, by using close
    signaling... http://play.golang.org/p/iOJySmEUP0

    * the todo chan, is a easy way of implementing a semaphore... it's there
    to notify that there are new jobs and block items.
    * mu is there to protect the internal queue and other variables
    * active counts the number of active routines... this is to prevent worker
    death from starvation and to see when we are actually done
    * queue simply manages the work queue

    Maybe there is a simpler design, I could not find one... (i.e. one that is
    thread-safe and finishes all go-routines properly when done)

    + egon

    On Saturday, December 21, 2013 3:13:29 PM UTC+1, egon wrote:

    The parallel version, that avoids too many goroutines is a bit more
    complicated... http://play.golang.org/p/b-9x0UpuwW

    + egon
    On Saturday, December 21, 2013 2:30:02 PM UTC+2, Oleku Konko wrote:

    Hello, I would like to know the best way to close a channel in a
    recursive function ?

    Here is a typical example is a simple function to generate all possible *1-3
    letter words* :

    package main
    import "fmt"
    func main() {
    result := make(chan string)
    go words(3, "", result)
    for v := range result {
    fmt.Printf("%s\n", v)
    }

    fmt.Println("got here")}
    func words(length int , prefix string, result chan string) {
    if length < 1 {
    return
    }
    letter := ""
    for i := 97 ; i < 123 ; i++ {
    letter = fmt.Sprintf("%s%c", prefix, i)
    result <- letter
    words(length - 1, letter , result)
    }}


    - How do you effectively *close result *or is there a better way to
    achieve the same result ?
    - What the best way to increase performance when working with eg go
    words(10, "", result) because go words(length - 1, letter , result) can
    lead to too many goroutines
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Egon at Dec 21, 2013 at 6:12 pm ⇧
    But, there is an alternate design using a condition
    variable http://play.golang.org/p/06v_0JiGNy

    + egon
    On Saturday, December 21, 2013 7:44:39 PM UTC+2, egon wrote:

    ... and by modifying, immediately made a mistake... :/
    Here should be a fixed one http://play.golang.org/p/ZZQ5HCYfrn
    On Saturday, December 21, 2013 7:35:10 PM UTC+2, egon wrote:


    On Saturday, December 21, 2013 7:08:44 PM UTC+2, Oleku Konko wrote:



    - Thanks, did not thinking about wrapping it in a closure
    - The parallel version is more complicated as expected, i would take my
    time to study it and see if i can simplify it.
    Mostly, I would say no... I once already spent several days on it
    actually to get it fully correct and safe. I.e. everything there is by
    necessity.

    Although, I now was able to remove the alldone, by using close
    signaling... http://play.golang.org/p/iOJySmEUP0

    * the todo chan, is a easy way of implementing a semaphore... it's there
    to notify that there are new jobs and block items.
    * mu is there to protect the internal queue and other variables
    * active counts the number of active routines... this is to prevent
    worker death from starvation and to see when we are actually done
    * queue simply manages the work queue

    Maybe there is a simpler design, I could not find one... (i.e. one that
    is thread-safe and finishes all go-routines properly when done)

    + egon

    On Saturday, December 21, 2013 3:13:29 PM UTC+1, egon wrote:

    The parallel version, that avoids too many goroutines is a bit more
    complicated... http://play.golang.org/p/b-9x0UpuwW

    + egon
    On Saturday, December 21, 2013 2:30:02 PM UTC+2, Oleku Konko wrote:

    Hello, I would like to know the best way to close a channel in a
    recursive function ?

    Here is a typical example is a simple function to generate all
    possible *1-3 letter words* :

    package main
    import "fmt"
    func main() {
    result := make(chan string)
    go words(3, "", result)
    for v := range result {
    fmt.Printf("%s\n", v)
    }

    fmt.Println("got here")}
    func words(length int , prefix string, result chan string) {
    if length < 1 {
    return
    }
    letter := ""
    for i := 97 ; i < 123 ; i++ {
    letter = fmt.Sprintf("%s%c", prefix, i)
    result <- letter
    words(length - 1, letter , result)
    }}


    - How do you effectively *close result *or is there a better way to
    achieve the same result ?
    - What the best way to increase performance when working with eg go
    words(10, "", result) because go words(length - 1, letter , result) can
    lead to too many goroutines
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Oleku Konko at Dec 22, 2013 at 2:19 pm ⇧
    Still very much intense than i expected .. thanks looking at the code at
    the moment
    On Saturday, December 21, 2013 7:12:48 PM UTC+1, egon wrote:

    But, there is an alternate design using a condition variable
    http://play.golang.org/p/06v_0JiGNy

    + egon
    On Saturday, December 21, 2013 7:44:39 PM UTC+2, egon wrote:

    ... and by modifying, immediately made a mistake... :/
    Here should be a fixed one http://play.golang.org/p/ZZQ5HCYfrn
    On Saturday, December 21, 2013 7:35:10 PM UTC+2, egon wrote:


    On Saturday, December 21, 2013 7:08:44 PM UTC+2, Oleku Konko wrote:



    - Thanks, did not thinking about wrapping it in a closure
    - The parallel version is more complicated as expected, i would take my
    time to study it and see if i can simplify it.
    Mostly, I would say no... I once already spent several days on it
    actually to get it fully correct and safe. I.e. everything there is by
    necessity.

    Although, I now was able to remove the alldone, by using close
    signaling... http://play.golang.org/p/iOJySmEUP0

    * the todo chan, is a easy way of implementing a semaphore... it's there
    to notify that there are new jobs and block items.
    * mu is there to protect the internal queue and other variables
    * active counts the number of active routines... this is to prevent
    worker death from starvation and to see when we are actually done
    * queue simply manages the work queue

    Maybe there is a simpler design, I could not find one... (i.e. one that
    is thread-safe and finishes all go-routines properly when done)

    + egon

    On Saturday, December 21, 2013 3:13:29 PM UTC+1, egon wrote:

    The parallel version, that avoids too many goroutines is a bit more
    complicated... http://play.golang.org/p/b-9x0UpuwW

    + egon
    On Saturday, December 21, 2013 2:30:02 PM UTC+2, Oleku Konko wrote:

    Hello, I would like to know the best way to close a channel in a
    recursive function ?

    Here is a typical example is a simple function to generate all
    possible *1-3 letter words* :

    package main
    import "fmt"
    func main() {
    result := make(chan string)
    go words(3, "", result)
    for v := range result {
    fmt.Printf("%s\n", v)
    }

    fmt.Println("got here")}
    func words(length int , prefix string, result chan string) {
    if length < 1 {
    return
    }
    letter := ""
    for i := 97 ; i < 123 ; i++ {
    letter = fmt.Sprintf("%s%c", prefix, i)
    result <- letter
    words(length - 1, letter , result)
    }}


    - How do you effectively *close result *or is there a better way to
    achieve the same result ?
    - What the best way to increase performance when working with eg go
    words(10, "", result) because go words(length - 1, letter ,
    result) can lead to too many goroutines
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Dmitry Vyukov at Dec 22, 2013 at 5:15 pm ⇧
    In such cases, you usually do parallel recursive decomposition until
    you reach the required grain size:
    http://play.golang.org/p/epvbA0vjZX

    On Sun, Dec 22, 2013 at 6:19 PM, Oleku Konko wrote:
    Still very much intense than i expected .. thanks looking at the code at the
    moment

    On Saturday, December 21, 2013 7:12:48 PM UTC+1, egon wrote:

    But, there is an alternate design using a condition variable
    http://play.golang.org/p/06v_0JiGNy

    + egon
    On Saturday, December 21, 2013 7:44:39 PM UTC+2, egon wrote:

    ... and by modifying, immediately made a mistake... :/
    Here should be a fixed one http://play.golang.org/p/ZZQ5HCYfrn
    On Saturday, December 21, 2013 7:35:10 PM UTC+2, egon wrote:


    On Saturday, December 21, 2013 7:08:44 PM UTC+2, Oleku Konko wrote:



    - Thanks, did not thinking about wrapping it in a closure
    - The parallel version is more complicated as expected, i would take my
    time to study it and see if i can simplify it.

    Mostly, I would say no... I once already spent several days on it
    actually to get it fully correct and safe. I.e. everything there is by
    necessity.

    Although, I now was able to remove the alldone, by using close
    signaling... http://play.golang.org/p/iOJySmEUP0

    * the todo chan, is a easy way of implementing a semaphore... it's there
    to notify that there are new jobs and block items.
    * mu is there to protect the internal queue and other variables
    * active counts the number of active routines... this is to prevent
    worker death from starvation and to see when we are actually done
    * queue simply manages the work queue

    Maybe there is a simpler design, I could not find one... (i.e. one that
    is thread-safe and finishes all go-routines properly when done)

    + egon

    On Saturday, December 21, 2013 3:13:29 PM UTC+1, egon wrote:

    The parallel version, that avoids too many goroutines is a bit more
    complicated... http://play.golang.org/p/b-9x0UpuwW

    + egon
    On Saturday, December 21, 2013 2:30:02 PM UTC+2, Oleku Konko wrote:

    Hello, I would like to know the best way to close a channel in a
    recursive function ?

    Here is a typical example is a simple function to generate all
    possible 1-3 letter words :

    package main

    import "fmt"

    func main() {
    result := make(chan string)
    go words(3, "", result)
    for v := range result {
    fmt.Printf("%s\n", v)
    }

    fmt.Println("got here")
    }

    func words(length int , prefix string, result chan string) {
    if length < 1 {
    return
    }
    letter := ""
    for i := 97 ; i < 123 ; i++ {
    letter = fmt.Sprintf("%s%c", prefix, i)
    result <- letter
    words(length - 1, letter , result)
    }
    }


    - How do you effectively close result or is there a better way to
    achieve the same result ?
    - What the best way to increase performance when working with eg go
    words(10, "", result) because go words(length - 1, letter , result) can
    lead to too many goroutines
    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Egon at Dec 22, 2013 at 7:46 pm ⇧
    Indeed, that would be a simpler solution for this problem.

    Now to think of it, my problem was somewhat different... i.e. I didn't know
    in advance how deep some recursion branch went and potentially had to deal
    with the state going to several hunderd GBs; so I had a potential need for
    persisting the state... although I never actually got to the persisting
    part. The recursive parallel decomposition for me was hard to control...
    since I also needed to have a particular order, how to recurse... otherwise
    the 100GB+ went easily into 1TB+...

    But yes, if your problems have predictable recursion, then that solution is
    better.

    + egon
    On Sunday, December 22, 2013 7:15:24 PM UTC+2, Dmitry Vyukov wrote:

    In such cases, you usually do parallel recursive decomposition until
    you reach the required grain size:
    http://play.golang.org/p/epvbA0vjZX

    On Sun, Dec 22, 2013 at 6:19 PM, Oleku Konko wrote:
    Still very much intense than i expected .. thanks looking at the code at the
    moment

    On Saturday, December 21, 2013 7:12:48 PM UTC+1, egon wrote:

    But, there is an alternate design using a condition variable
    http://play.golang.org/p/06v_0JiGNy

    + egon
    On Saturday, December 21, 2013 7:44:39 PM UTC+2, egon wrote:

    ... and by modifying, immediately made a mistake... :/
    Here should be a fixed one http://play.golang.org/p/ZZQ5HCYfrn
    On Saturday, December 21, 2013 7:35:10 PM UTC+2, egon wrote:


    On Saturday, December 21, 2013 7:08:44 PM UTC+2, Oleku Konko wrote:



    - Thanks, did not thinking about wrapping it in a closure
    - The parallel version is more complicated as expected, i would take
    my
    time to study it and see if i can simplify it.

    Mostly, I would say no... I once already spent several days on it
    actually to get it fully correct and safe. I.e. everything there is
    by
    necessity.

    Although, I now was able to remove the alldone, by using close
    signaling... http://play.golang.org/p/iOJySmEUP0

    * the todo chan, is a easy way of implementing a semaphore... it's
    there
    to notify that there are new jobs and block items.
    * mu is there to protect the internal queue and other variables
    * active counts the number of active routines... this is to prevent
    worker death from starvation and to see when we are actually done
    * queue simply manages the work queue

    Maybe there is a simpler design, I could not find one... (i.e. one
    that
    is thread-safe and finishes all go-routines properly when done)

    + egon

    On Saturday, December 21, 2013 3:13:29 PM UTC+1, egon wrote:

    The parallel version, that avoids too many goroutines is a bit more
    complicated... http://play.golang.org/p/b-9x0UpuwW

    + egon
    On Saturday, December 21, 2013 2:30:02 PM UTC+2, Oleku Konko wrote:

    Hello, I would like to know the best way to close a channel in a
    recursive function ?

    Here is a typical example is a simple function to generate all
    possible 1-3 letter words :

    package main

    import "fmt"

    func main() {
    result := make(chan string)
    go words(3, "", result)
    for v := range result {
    fmt.Printf("%s\n", v)
    }

    fmt.Println("got here")
    }

    func words(length int , prefix string, result chan string) {
    if length < 1 {
    return
    }
    letter := ""
    for i := 97 ; i < 123 ; i++ {
    letter = fmt.Sprintf("%s%c", prefix, i)
    result <- letter
    words(length - 1, letter , result)
    }
    }


    - How do you effectively close result or is there a better way to
    achieve the same result ?
    - What the best way to increase performance when working with eg
    go
    words(10, "", result) because go words(length - 1, letter ,
    result) can
    lead to too many goroutines
    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts...@googlegroups.com <javascript:>.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedDec 21, '13 at 12:30p
activeDec 22, '13 at 7:46p
posts10
users3
websitegolang.org

3 users in discussion

Egon: 6 posts Oleku Konko: 3 posts Dmitry Vyukov: 1 post

Content

People

Support

Translate

site design / logo © 2021 Grokbase