FAQ
So I've got a little program that is watching a file for updates via a
goroutine.

When that goroutine detects a change, it reads in the data and stores it in
a an attribute of a struct, then notifies all the listeners that the file
has change.

Kinda like:

func (b *blah) update() {
   // Watch file and update our data when it changes.
   f := Watcher("some file")
   t := f.Content()
   b.content := &t

   // Update all listeners who want to know the file changed. Only wait for
waitTime to prevent blocking.
   for _, c := range b.listeners {
     select {
     case c <- true:
     case time.After(waitTime):
       break
   }
}

My listeners before accessing the content make a pointer copy:

copyPtr := b.content
<do stuff with copyPtr>

I do this to make sure the pointer I'm using doesn't change out while
working with the set of data.

However I'm not using sync.Mutex or other sync primitives to protect the
pointer. I could be sending the data to all listeners through the channel
instead of just an update(well, not everywhere, because not all readers
listen on a channel.)... But I'm wondering with Go's memory model what the
outcome here is going to be? Could I get away with what I'm doing without
sync.Mutex since I have guaranteed synchronous writes with async reads and
copy the pointer before reading (the memory model seems to indicate no)?
  If I need a lock, could I get away with just atomic.CompareAndSwapPointer
around the write and not put sync.Mutex on all my reads.

Testing it a few thousand times seems to work the way I expected it to, but
I'm still very doubtful. I have the feeling that an update will occur at
some point and the listener receiving from the update channel will miss the
updated data.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Dmitry Vyukov at Jun 24, 2013 at 4:24 pm

    On Mon, Jun 24, 2013 at 8:10 PM, John wrote:
    So I've got a little program that is watching a file for updates via a
    goroutine.

    When that goroutine detects a change, it reads in the data and stores it in
    a an attribute of a struct, then notifies all the listeners that the file
    has change.

    Kinda like:

    func (b *blah) update() {
    // Watch file and update our data when it changes.
    f := Watcher("some file")
    t := f.Content()
    b.content := &t

    // Update all listeners who want to know the file changed. Only wait for
    waitTime to prevent blocking.
    for _, c := range b.listeners {
    select {
    case c <- true:
    case time.After(waitTime):
    break
    }
    }

    My listeners before accessing the content make a pointer copy:

    copyPtr := b.content
    <do stuff with copyPtr>

    I do this to make sure the pointer I'm using doesn't change out while
    working with the set of data.

    However I'm not using sync.Mutex or other sync primitives to protect the
    pointer. I could be sending the data to all listeners through the channel
    instead of just an update(well, not everywhere, because not all readers
    listen on a channel.)... But I'm wondering with Go's memory model what the
    outcome here is going to be? Could I get away with what I'm doing without
    sync.Mutex since I have guaranteed synchronous writes with async reads and
    copy the pointer before reading (the memory model seems to indicate no)? If
    I need a lock, could I get away with just atomic.CompareAndSwapPointer
    around the write and not put sync.Mutex on all my reads.

    Testing it a few thousand times seems to work the way I expected it to, but
    I'm still very doubtful. I have the feeling that an update will occur at
    some point and the listener receiving from the update channel will miss the
    updated data.

    This can break.
    Update the pointer with atomic.StorePointer (do not use CompareAndSwap
    to do a store, use Store to do a store, that's what it is for).
    Read the pointer with atomic.LoadPointer.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • John Doak at Jun 24, 2013 at 11:19 pm
    Thanks for the answer. It leaves me with a few follow-ups:

    Can you explain why it breaks in the memory model?

    Also, why do StorePoitner and LoadPointer instead of CompareAndSwap?




    On Mon, Jun 24, 2013 at 9:24 AM, Dmitry Vyukov wrote:
    On Mon, Jun 24, 2013 at 8:10 PM, John wrote:
    So I've got a little program that is watching a file for updates via a
    goroutine.

    When that goroutine detects a change, it reads in the data and stores it in
    a an attribute of a struct, then notifies all the listeners that the file
    has change.

    Kinda like:

    func (b *blah) update() {
    // Watch file and update our data when it changes.
    f := Watcher("some file")
    t := f.Content()
    b.content := &t

    // Update all listeners who want to know the file changed. Only wait for
    waitTime to prevent blocking.
    for _, c := range b.listeners {
    select {
    case c <- true:
    case time.After(waitTime):
    break
    }
    }

    My listeners before accessing the content make a pointer copy:

    copyPtr := b.content
    <do stuff with copyPtr>

    I do this to make sure the pointer I'm using doesn't change out while
    working with the set of data.

    However I'm not using sync.Mutex or other sync primitives to protect the
    pointer. I could be sending the data to all listeners through the channel
    instead of just an update(well, not everywhere, because not all readers
    listen on a channel.)... But I'm wondering with Go's memory model what the
    outcome here is going to be? Could I get away with what I'm doing without
    sync.Mutex since I have guaranteed synchronous writes with async reads and
    copy the pointer before reading (the memory model seems to indicate no)? If
    I need a lock, could I get away with just atomic.CompareAndSwapPointer
    around the write and not put sync.Mutex on all my reads.

    Testing it a few thousand times seems to work the way I expected it to, but
    I'm still very doubtful. I have the feeling that an update will occur at
    some point and the listener receiving from the update channel will miss the
    updated data.

    This can break.
    Update the pointer with atomic.StorePointer (do not use CompareAndSwap
    to do a store, use Store to do a store, that's what it is for).
    Read the pointer with atomic.LoadPointer.


    --
    John Doak
    http://flavors.me/johnsiilver

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Henrik Johansson at Jun 25, 2013 at 5:00 am
    If I understand correctly the two complementary Load/Store methods behave
    conceptually as a volatile variable in say Java. A memory barrier is
    inserted in the correct location respectively thus ensuring memory
    coherence across goroutines (really the underlying threads and cpus).

    Reading shared variables like you do in your code is afaik not guaranteed
    to yield consistent results under the go memory model as per specification.

    As for not using swap, well I guess there is no need to swap anything with
    only one producer?
    On Jun 25, 2013 1:19 AM, "John Doak" wrote:

    Thanks for the answer. It leaves me with a few follow-ups:

    Can you explain why it breaks in the memory model?

    Also, why do StorePoitner and LoadPointer instead of CompareAndSwap?




    On Mon, Jun 24, 2013 at 9:24 AM, Dmitry Vyukov wrote:
    On Mon, Jun 24, 2013 at 8:10 PM, John wrote:
    So I've got a little program that is watching a file for updates via a
    goroutine.

    When that goroutine detects a change, it reads in the data and stores it in
    a an attribute of a struct, then notifies all the listeners that the file
    has change.

    Kinda like:

    func (b *blah) update() {
    // Watch file and update our data when it changes.
    f := Watcher("some file")
    t := f.Content()
    b.content := &t

    // Update all listeners who want to know the file changed. Only wait for
    waitTime to prevent blocking.
    for _, c := range b.listeners {
    select {
    case c <- true:
    case time.After(waitTime):
    break
    }
    }

    My listeners before accessing the content make a pointer copy:

    copyPtr := b.content
    <do stuff with copyPtr>

    I do this to make sure the pointer I'm using doesn't change out while
    working with the set of data.

    However I'm not using sync.Mutex or other sync primitives to protect the
    pointer. I could be sending the data to all listeners through the channel
    instead of just an update(well, not everywhere, because not all readers
    listen on a channel.)... But I'm wondering with Go's memory model what the
    outcome here is going to be? Could I get away with what I'm doing without
    sync.Mutex since I have guaranteed synchronous writes with async reads and
    copy the pointer before reading (the memory model seems to indicate no)? If
    I need a lock, could I get away with just atomic.CompareAndSwapPointer
    around the write and not put sync.Mutex on all my reads.

    Testing it a few thousand times seems to work the way I expected it to, but
    I'm still very doubtful. I have the feeling that an update will occur at
    some point and the listener receiving from the update channel will miss the
    updated data.

    This can break.
    Update the pointer with atomic.StorePointer (do not use CompareAndSwap
    to do a store, use Store to do a store, that's what it is for).
    Read the pointer with atomic.LoadPointer.


    --
    John Doak
    http://flavors.me/johnsiilver

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Dmitry Vyukov at Jun 25, 2013 at 10:09 am

    On Tue, Jun 25, 2013 at 9:00 AM, Henrik Johansson wrote:
    If I understand correctly the two complementary Load/Store methods behave
    conceptually as a volatile variable in say Java.

    Yes, Go atomic operations are pretty much the same as Java volatiles.
    However, there is no intention (at least for now) to fully define
    semantics of data races (as in Java).

    A memory barrier is
    inserted in the correct location respectively thus ensuring memory coherence
    across goroutines (really the underlying threads and cpus).

    Reading shared variables like you do in your code is afaik not guaranteed to
    yield consistent results under the go memory model as per specification.

    As for not using swap, well I guess there is no need to swap anything with
    only one producer?
    On Jun 25, 2013 1:19 AM, "John Doak" wrote:

    Thanks for the answer. It leaves me with a few follow-ups:

    Can you explain why it breaks in the memory model?

    Also, why do StorePoitner and LoadPointer instead of CompareAndSwap?




    On Mon, Jun 24, 2013 at 9:24 AM, Dmitry Vyukov wrote:
    On Mon, Jun 24, 2013 at 8:10 PM, John wrote:
    So I've got a little program that is watching a file for updates via a
    goroutine.

    When that goroutine detects a change, it reads in the data and stores
    it in
    a an attribute of a struct, then notifies all the listeners that the
    file
    has change.

    Kinda like:

    func (b *blah) update() {
    // Watch file and update our data when it changes.
    f := Watcher("some file")
    t := f.Content()
    b.content := &t

    // Update all listeners who want to know the file changed. Only wait
    for
    waitTime to prevent blocking.
    for _, c := range b.listeners {
    select {
    case c <- true:
    case time.After(waitTime):
    break
    }
    }

    My listeners before accessing the content make a pointer copy:

    copyPtr := b.content
    <do stuff with copyPtr>

    I do this to make sure the pointer I'm using doesn't change out while
    working with the set of data.

    However I'm not using sync.Mutex or other sync primitives to protect
    the
    pointer. I could be sending the data to all listeners through the
    channel
    instead of just an update(well, not everywhere, because not all readers
    listen on a channel.)... But I'm wondering with Go's memory model what
    the
    outcome here is going to be? Could I get away with what I'm doing
    without
    sync.Mutex since I have guaranteed synchronous writes with async reads
    and
    copy the pointer before reading (the memory model seems to indicate
    no)? If
    I need a lock, could I get away with just atomic.CompareAndSwapPointer
    around the write and not put sync.Mutex on all my reads.

    Testing it a few thousand times seems to work the way I expected it to,
    but
    I'm still very doubtful. I have the feeling that an update will occur
    at
    some point and the listener receiving from the update channel will miss
    the
    updated data.

    This can break.
    Update the pointer with atomic.StorePointer (do not use CompareAndSwap
    to do a store, use Store to do a store, that's what it is for).
    Read the pointer with atomic.LoadPointer.



    --
    John Doak
    http://flavors.me/johnsiilver

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Jesse McNelis at Jun 25, 2013 at 5:15 am

    On Tue, Jun 25, 2013 at 9:19 AM, John Doak wrote:

    Thanks for the answer. It leaves me with a few follow-ups:

    Can you explain why it breaks in the memory model?

    I'm guessing you're testing on some kind of x86/amd64 where pointer sized
    assignments are going to be atomic.
    But the Go memory model covers additional architectures(ARM etc.) where
    this isn't guaranteed.
    The memory model doesn't guranteee any assignments are atomic.
    Technically you could end up with a half assigned pointer that didn't point
    to anything.

    Also, why do StorePoitner and LoadPointer instead of CompareAndSwap?
    Do you actually care if the pointer has changed since you last saw it?
    What are you planning to do if the CompareAndSwap returns false?



    --
    =====================
    http://jessta.id.au

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Dmitry Vyukov at Jun 25, 2013 at 10:10 am

    On Tue, Jun 25, 2013 at 3:19 AM, John Doak wrote:
    Thanks for the answer. It leaves me with a few follow-ups:

    Can you explain why it breaks in the memory model?
    The behavior of programs with data races is undefined.
    Also, why do StorePoitner and LoadPointer instead of CompareAndSwap?
    Well, because you need to store a pointer and load a pointer. I would
    put the question the other way around -- why CompareAndSwap to store a
    value?




    On Mon, Jun 24, 2013 at 9:24 AM, Dmitry Vyukov wrote:
    On Mon, Jun 24, 2013 at 8:10 PM, John wrote:
    So I've got a little program that is watching a file for updates via a
    goroutine.

    When that goroutine detects a change, it reads in the data and stores it
    in
    a an attribute of a struct, then notifies all the listeners that the
    file
    has change.

    Kinda like:

    func (b *blah) update() {
    // Watch file and update our data when it changes.
    f := Watcher("some file")
    t := f.Content()
    b.content := &t

    // Update all listeners who want to know the file changed. Only wait
    for
    waitTime to prevent blocking.
    for _, c := range b.listeners {
    select {
    case c <- true:
    case time.After(waitTime):
    break
    }
    }

    My listeners before accessing the content make a pointer copy:

    copyPtr := b.content
    <do stuff with copyPtr>

    I do this to make sure the pointer I'm using doesn't change out while
    working with the set of data.

    However I'm not using sync.Mutex or other sync primitives to protect the
    pointer. I could be sending the data to all listeners through the
    channel
    instead of just an update(well, not everywhere, because not all readers
    listen on a channel.)... But I'm wondering with Go's memory model what
    the
    outcome here is going to be? Could I get away with what I'm doing
    without
    sync.Mutex since I have guaranteed synchronous writes with async reads
    and
    copy the pointer before reading (the memory model seems to indicate no)?
    If
    I need a lock, could I get away with just atomic.CompareAndSwapPointer
    around the write and not put sync.Mutex on all my reads.

    Testing it a few thousand times seems to work the way I expected it to,
    but
    I'm still very doubtful. I have the feeling that an update will occur
    at
    some point and the listener receiving from the update channel will miss
    the
    updated data.

    This can break.
    Update the pointer with atomic.StorePointer (do not use CompareAndSwap
    to do a store, use Store to do a store, that's what it is for).
    Read the pointer with atomic.LoadPointer.



    --
    John Doak
    http://flavors.me/johnsiilver
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • John Nagle at Jun 25, 2013 at 6:23 pm

    On 6/25/2013 3:10 AM, Dmitry Vyukov wrote:
    On Tue, Jun 25, 2013 at 3:19 AM, John Doak wrote:
    Thanks for the answer. It leaves me with a few follow-ups:

    Can you explain why it breaks in the memory model?
    The behavior of programs with data races is undefined.
    Also, why do StorePoitner and LoadPointer instead of CompareAndSwap?
    Well, because you need to store a pointer and load a pointer. I would
    put the question the other way around -- why CompareAndSwap to store a
    value?
         First, if you're waiting for an event at the file system time
    scale, trying to use lockless programming techniques is overkill.
    Knocking a few hundred nanoseconds off the locking operation is
    pointless for such slow events. Those approaches are usually used
    only in very low level programming such as inside the scheduler of a
    multiprocessor operating system.

        Atomic compare and swap is quite useful, but it's generally used
    for explicit synchronization between different CPUs. Read:

        http://en.wikipedia.org/wiki/Compare-and-swap

    Here's a more detailed discussion of compare-and-swap usage on
    x86 Linux kernel implementations:

        http://heather.cs.ucdavis.edu/~matloff/50/PLN/lock.pdf

    This describes in some detail what's happening at the assembler
    and machine level.

        If you use a classic compare-and-swap spinlock at the Go
    source level, and retry without waiting on a fail, your
    program will go into an infinite loop if it ever spinlocks
    with GOMAXPROCS=1.

        Just use mutexes, until you get to the point that
    profiling tells you that your 64-processor supercomputer
    is bottlenecking on some simple list update operation.
    Or "share memory by communicating, not by sharing memory".

         John Nagle

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Nathan Youngman at Jun 26, 2013 at 5:46 am
    Why not forward the file event onto another channel? (share by
    communicating)

    Here's how I approached file watching:
    https://github.com/gophertown/looper/blob/master/watch.go

    Correct me if I'm wrong, but I kind've feel like the low-level Mutexes and
    stuff are appropriate for custom data structures and the like, not
    necessarily flow control like this.

    Nathan.


    On Monday, 24 June 2013 10:10:15 UTC-6, John wrote:

    So I've got a little program that is watching a file for updates via a
    goroutine.

    When that goroutine detects a change, it reads in the data and stores it
    in a an attribute of a struct, then notifies all the listeners that the
    file has change.

    Kinda like:

    func (b *blah) update() {
    // Watch file and update our data when it changes.
    f := Watcher("some file")
    t := f.Content()
    b.content := &t

    // Update all listeners who want to know the file changed. Only wait
    for waitTime to prevent blocking.
    for _, c := range b.listeners {
    select {
    case c <- true:
    case time.After(waitTime):
    break
    }
    }

    My listeners before accessing the content make a pointer copy:

    copyPtr := b.content
    <do stuff with copyPtr>

    I do this to make sure the pointer I'm using doesn't change out while
    working with the set of data.

    However I'm not using sync.Mutex or other sync primitives to protect the
    pointer. I could be sending the data to all listeners through the channel
    instead of just an update(well, not everywhere, because not all readers
    listen on a channel.)... But I'm wondering with Go's memory model what the
    outcome here is going to be? Could I get away with what I'm doing without
    sync.Mutex since I have guaranteed synchronous writes with async reads and
    copy the pointer before reading (the memory model seems to indicate no)?
    If I need a lock, could I get away with just atomic.CompareAndSwapPointer
    around the write and not put sync.Mutex on all my reads.

    Testing it a few thousand times seems to work the way I expected it to,
    but I'm still very doubtful. I have the feeling that an update will occur
    at some point and the listener receiving from the update channel will miss
    the updated data.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • John at Jul 2, 2013 at 6:42 am
    Thank for the explanation and links guys. This sorted out what I needed to
    know.
    Sorry I had forgotten the courtesy of a response.
    On Tuesday, June 25, 2013 10:46:47 PM UTC-7, Nathan Youngman wrote:


    Why not forward the file event onto another channel? (share by
    communicating)

    Here's how I approached file watching:
    https://github.com/gophertown/looper/blob/master/watch.go

    Correct me if I'm wrong, but I kind've feel like the low-level Mutexes and
    stuff are appropriate for custom data structures and the like, not
    necessarily flow control like this.

    Nathan.


    On Monday, 24 June 2013 10:10:15 UTC-6, John wrote:

    So I've got a little program that is watching a file for updates via a
    goroutine.

    When that goroutine detects a change, it reads in the data and stores it
    in a an attribute of a struct, then notifies all the listeners that the
    file has change.

    Kinda like:

    func (b *blah) update() {
    // Watch file and update our data when it changes.
    f := Watcher("some file")
    t := f.Content()
    b.content := &t

    // Update all listeners who want to know the file changed. Only wait
    for waitTime to prevent blocking.
    for _, c := range b.listeners {
    select {
    case c <- true:
    case time.After(waitTime):
    break
    }
    }

    My listeners before accessing the content make a pointer copy:

    copyPtr := b.content
    <do stuff with copyPtr>

    I do this to make sure the pointer I'm using doesn't change out while
    working with the set of data.

    However I'm not using sync.Mutex or other sync primitives to protect the
    pointer. I could be sending the data to all listeners through the channel
    instead of just an update(well, not everywhere, because not all readers
    listen on a channel.)... But I'm wondering with Go's memory model what the
    outcome here is going to be? Could I get away with what I'm doing without
    sync.Mutex since I have guaranteed synchronous writes with async reads and
    copy the pointer before reading (the memory model seems to indicate no)?
    If I need a lock, could I get away with just atomic.CompareAndSwapPointer
    around the write and not put sync.Mutex on all my reads.

    Testing it a few thousand times seems to work the way I expected it to,
    but I'm still very doubtful. I have the feeling that an update will occur
    at some point and the listener receiving from the update channel will miss
    the updated data.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedJun 24, '13 at 4:10p
activeJul 2, '13 at 6:42a
posts10
users6
websitegolang.org

People

Translate

site design / logo © 2021 Grokbase