FAQ
I'm using filepath.Walk on a directory and processing each file I find
there. I'd like the filepath.Walkfunc that I define to run as a goroutine
but I'm fairly new to go so I'm having some trouble mapping the examples
I've seen to an anonymous function like the one I defined for my walkfunc.

Are there examples of using a walkfunc as a goroutine somewhere and is it
advised to do it in this way with that function in a filepath.Walk?

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Jan Mercl at Jun 19, 2013 at 11:39 am

    On Wed, Jun 19, 2013 at 1:29 PM, Scott Turnbull wrote:
    I'm using filepath.Walk on a directory and processing each file I find
    there. I'd like the filepath.Walkfunc that I define to run as a goroutine
    but I'm fairly new to go so I'm having some trouble mapping the examples
    I've seen to an anonymous function like the one I defined for my walkfunc.

    Are there examples of using a walkfunc as a goroutine somewhere and is it
    advised to do it in this way with that function in a filepath.Walk?
    Instead of (run in this goroutine)

             err := filepath.Walk(myRoot, myWalkFunc)

    write

             c := make(chan error)
             go func() { c <- filepath.Walk(myRoot, myWalkFunc) }()
             ...
             err := <-c // Walk done, check the error

    which will run _both_ the 'Walk' function and the 'myWalkFunc' in a
    new goroutine.

    -j

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Scott Turnbull at Jun 19, 2013 at 2:24 pm
    I may not be understanding what's going on here conceptually well enough.
      What I would expect from that code is the entire Walk to run as a
    goroutine but I'm not sure how each individual instance of the WalkFunc
    would be run a go routine.

    I modified the code to run with the example you provided but my benchmark
    tests don't really show any performance improvement. Since I'm struggling
    with this a bit I must be doing something wrong. I'm including the
    specific method I'm creating as an example. When you get to the bottom of
    the method you'll see the Walk running as a go routine. Above the visit
    function defines the callback and in that I'm taking the file I find and
    doing a fixity check on it and putting the filepath and fixity result into
    a map so I can use it elsewhere. I'm trying to get the callback for fixity
    to run as a go routine to improve performance.

    // Performs an add on every file under the directory supplied to the
    // method. Returns a map of the filename and its fixity falue and a
    // list of errors.
    func (p *Payload) AddAll(src string, hsh hash.Hash) (fxs map[string]string, errs []error) {

      fxs = make(map[string]string) // Mapping filepaths to fixity results.

             // Implementation of my Walkfunc
      visit := func(pth string, info os.FileInfo, err error) error {
       hsh.Reset() // Reset the hash value everytime a new fixity starts.
       if err != nil {
        errs = append([]error{err})
       }
       if !info.IsDir() {
        dstPath := strings.TrimPrefix(pth, src)
        fx, err := p.Add(pth, dstPath, hsh) // run add on the file.
        if err != nil {
         return err
        }
        fxs[dstPath] = fx
       }
       return nil
      }

      c := make(chan error)

      go func() {
       c <- filepath.Walk(src, visit)
      }()
      if err := <-c; err != nil {
       errs = append([]error{err})
      }

      return
    }

    On Wednesday, June 19, 2013 7:38:46 AM UTC-4, Jan Mercl wrote:
    On Wed, Jun 19, 2013 at 1:29 PM, Scott Turnbull wrote:
    I'm using filepath.Walk on a directory and processing each file I find
    there. I'd like the filepath.Walkfunc that I define to run as a goroutine
    but I'm fairly new to go so I'm having some trouble mapping the examples
    I've seen to an anonymous function like the one I defined for my walkfunc.
    Are there examples of using a walkfunc as a goroutine somewhere and is it
    advised to do it in this way with that function in a filepath.Walk?
    Instead of (run in this goroutine)

    err := filepath.Walk(myRoot, myWalkFunc)

    write

    c := make(chan error)
    go func() { c <- filepath.Walk(myRoot, myWalkFunc) }()
    ...
    err := <-c // Walk done, check the error

    which will run _both_ the 'Walk' function and the 'myWalkFunc' in a
    new goroutine.

    -j
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Jan Mercl at Jun 19, 2013 at 3:12 pm

    On Wed, Jun 19, 2013 at 4:24 PM, Scott Turnbull wrote:
    I may not be understanding what's going on here conceptually well enough.
    What I would expect from that code is the entire Walk to run as a goroutine
    but I'm not sure how each individual instance of the WalkFunc would be run a
    go routine.
    Ah, I think I've misunderstood you question, sorry.

    Well, no. Running myWalkFunc in a separate goroutine makes little
    sense, because filepath.Walk is blocked until myWalkFunc returns - as
    it needs the returned value to decide what to do next. IOW,
    filepath.Walk implements a synchronous algorithm, which myWalkFunc is
    a part of.

    OTOH, collecting the walked paths in/by myWalkFunc and _afterwards_
    running a goroutine for each path _does make_ sense.

    Sorry for any confusion created by my previous post.

    -j

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Sethamclean at Mar 11, 2014 at 1:19 am

    On Wednesday, June 19, 2013 7:29:55 AM UTC-4, Scott Turnbull wrote:
    I'm using filepath.Walk on a directory and processing each file I find
    there. I'd like the filepath.Walkfunc that I define to run as a goroutine
    but I'm fairly new to go so I'm having some trouble mapping the examples
    I've seen to an anonymous function like the one I defined for my walkfunc.

    Are there examples of using a walkfunc as a goroutine somewhere and is it
    advised to do it in this way with that function in a filepath.Walk?
    Not sure if this is exactly what you ware looking for or not. I know this
    is an old thread but I thought this might be helpful:

       <script src="https://gist.github.com/sethamclean/9475737.js"></script>

    https://gist.github.com/sethamclean/9475737

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Sethamclean at Mar 11, 2014 at 1:19 am

    On Wednesday, June 19, 2013 7:29:55 AM UTC-4, Scott Turnbull wrote:
    I'm using filepath.Walk on a directory and processing each file I find
    there. I'd like the filepath.Walkfunc that I define to run as a goroutine
    but I'm fairly new to go so I'm having some trouble mapping the examples
    I've seen to an anonymous function like the one I defined for my walkfunc.

    Are there examples of using a walkfunc as a goroutine somewhere and is it
    advised to do it in this way with that function in a filepath.Walk?
    package main

    import (
         "path/filepath"
         "fmt"
         "os"
    )

    func main(){
         location := "../gocode/"
         chann := GoWalk(location)
         for msg := range chann {
             fmt.Println(msg)
         }
         return
    }

    func GoWalk(location string) (chan string) {
         chann := make(chan string)
         go func(){
             filepath.Walk(location, func(path string, _ os.FileInfo, _ error)(err error){
                 chann <- path
                 return
             })
             defer close(chann)
         }()
             return chann

    }

    https://gist.github.com/sethamclean/9475737


    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedJun 19, '13 at 11:30a
activeMar 11, '14 at 1:19a
posts6
users3
websitegolang.org

People

Translate

site design / logo © 2022 Grokbase