FAQ
Hello,

I am trying to learn how/whether recursive fsnotify watchers can be
implemented on top of the "gopkg.in/fsnotify.v1" interface. While playing
with this I run into a race condition which I have trouble to work around
(see code below):

* if a new subdirectory is added, I install a new watcher for this directory
* if further sub-subdirectories are added before the watcher is in place,
these are missed.

The problem is not only theoretical. If I do "mkdir -p
a/b/c/d/e/f/g/h/i/j/k" on MacOS X, my code below never notices the
subdirectories "a/b/...". I assume I could try to fix this as follows:

* install the new watcher
* use filepath.Walk() on the newly watched directory, and add further
watchers for every subdirectory found therein

My problem with this fix is that there may now be races between
filepath.Walk reporting a file and the newly installed watcher receiving a
REMOVE event. Is there a known way to avoid these races? is there
publicly available, racefree code (using the "gopkg.in/fsnotify.v1"
interface) I could look at?

Many thanks,
Jochen

===============

Here is the relevant excerpt of my code containing the race. Full code is
at http://play.golang.org/p/MSgnIEAQCC (but doesn't run in the playground).

func StartWatcher(root string) error {
watcher, err := fsnotify.NewWatcher()
if err != nil {
return err
}

watchedDirs := map[string]bool{}
go func() {
for {
select {
case ev := <-watcher.Events:
switch {
case ev.Op&fsnotify.Create != 0:
                                         ...

// race condition here: we may miss
// directories/files added before the new watcher
// is in place.

if !watchedDirs[ev.Name] {
fi, err := os.Stat(ev.Name)
if err == nil && fi.IsDir() {
log.Print("watch ", ev.Name)
watchedDirs[ev.Name] = true
watcher.Add(ev.Name)
}
}
case ev.Op&(fsnotify.Remove|fsnotify.Rename) != 0:
if watchedDirs[ev.Name] {
log.Print("unwatch ", ev.Name)
watcher.Remove(ev.Name)
delete(watchedDirs, ev.Name)
}
                                         ...
}
case err := <-watcher.Errors:
log.Printf("fsnotify watcher error %q", err.Error())
}
}
}()

         ...
}
--
http://seehuhn.de/

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Daniel Eloff at Feb 17, 2015 at 8:11 pm
    Your solution looks correct, but you need to synchronize between fsnotify
    events and your fielpath.Walk routine. If the walk routine takes a mutex,
    and every event handler takes the same mutex, then when it's adding
    subdirectories you won't get any REMOVE or otherwise events until after the
    walk routine has finished. Depending on your application this might be the
    easiest solution or it might be impractically slow. I would advise to try
    it and see, because failing that you really are just going to have to pile
    hacks on hacks to try and live with those race conditions, and that's going
    to be painful and bug prone. How will you know you've handled every case?

    If there were a recursive watcher, it would solve your problem, but I don't
    see anything like that in the interface. The underlying operating systems
    may provide it (e.g. inotify.)

    Cheers,
    Dan

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedFeb 14, '15 at 11:45p
activeFeb 17, '15 at 8:11p
posts2
users2
websitegolang.org

2 users in discussion

Jochen Voss: 1 post Daniel Eloff: 1 post

People

Translate

site design / logo © 2022 Grokbase