FAQ
I want to create a map and fill it with values, in init().

After init(), this map will only be accessed for reading, never for writing.

I imagine that multiple goroutines can access the map to read its content
   - at the same time
   - even if they run on different OS threads
   - without needing to protect access to this map with e.g. sync.Mutex


But I don't find anything in Go specification or other document saying that
a map is safe to use like this.

One can imagine that for performance reason, an implementation of the map
may make some internal reorganization of its data during a read operation,
or add a little cache to store the last value retrieved, or things like
that.
In this case, the map won't be safe any more for concurrent read operations.

In fact, such behaviour existed and has been fixed, in Issue 5179 "runtime:
concurrent map reads grow map and corrupt map internals"
http://code.google.com/p/go/issues/detail?id=5179.

Does such guarantee exists somewhere, that a map will never ever change
anything in its internal state during read operations ?




--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Jesse McNelis at Nov 15, 2013 at 3:06 am

    On Fri, Nov 15, 2013 at 4:32 AM, wrote:

    I want to create a map and fill it with values, in init().

    After init(), this map will only be accessed for reading, never for
    writing.

    I imagine that multiple goroutines can access the map to read its content
    - at the same time
    - even if they run on different OS threads
    - without needing to protect access to this map with e.g. sync.Mutex
    But I don't find anything in Go specification or other document saying
    that a map is safe to use like this.
    It's not specified.

    One can imagine that for performance reason, an implementation of the map
    may make some internal reorganization of its data during a read operation,
    or add a little cache to store the last value retrieved, or things like
    that.
    In this case, the map won't be safe any more for concurrent read
    operations.
    Yep, an implementation of the Go language could make reads from maps unsafe
    for concurrent access.

    In fact, such behaviour existed and has been fixed, in Issue 5179
    "runtime: concurrent map reads grow map and corrupt map internals"
    http://code.google.com/p/go/issues/detail?id=5179.

    Does such guarantee exists somewhere, that a map will never ever change
    anything in its internal state during read operations ?
    I believe the Go developers have decided to keep map reads thread-safe in
    the standard implementation(they have a test for it), but since the spec
    doesn't guarantee it other implementations could make map reads not
    thread-safe.
    Since they aren't specified to be safe, I assume they are unsafe and use a
    sync.Mutex to protect my concurrent map accesses.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Ian Lance Taylor at Nov 15, 2013 at 5:19 am

    On Thu, Nov 14, 2013 at 7:06 PM, Jesse McNelis wrote:
    On Fri, Nov 15, 2013 at 4:32 AM, wrote:

    I want to create a map and fill it with values, in init().

    After init(), this map will only be accessed for reading, never for
    writing.

    I imagine that multiple goroutines can access the map to read its content
    - at the same time
    - even if they run on different OS threads
    - without needing to protect access to this map with e.g. sync.Mutex


    But I don't find anything in Go specification or other document saying
    that a map is safe to use like this.

    It's not specified.
    A read from a map is like any other read from a variable. Thus it's
    fine if multiple goroutines read from a map simultaneously. But if
    one goroutine reads from a map while another writes to a map, or if
    two goroutines write to a map, then the program must synchronize those
    goroutines in some way.

    Ian

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Dominik Honnef at Nov 15, 2013 at 5:58 am

    Ian Lance Taylor writes:

    It's not specified.
    A read from a map is like any other read from a variable. Thus it's
    fine if multiple goroutines read from a map simultaneously. But if
    one goroutine reads from a map while another writes to a map, or if
    two goroutines write to a map, then the program must synchronize those
    goroutines in some way.

    Ian
    Does the existing specification actually guarantee this? Because there
    are certainly ways to implement it that don't have that guarantee.
    https://code.google.com/p/go/source/detail?r=af469280a34b comes to mind
    in particular.

    I know that this implementation of Go seems to guarantee it, but could
    there be an alternate implementation that implements maps that aren't
    safe for concurrent reads and still implement the specification?

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Kevin Gillette at Nov 15, 2013 at 9:53 am
    Implementation details aside, from a usability perspective it's a rather
    unfortunate thing when concurrent no-write access results in race
    conditions.

    Other such questions about read-only map access have been answered with
    "yes, it's safe," and it's a widely enough held assumption that concurrent
    no-write access to anything in Go is safe; it may as well be specified,
    since any implementation breaking that assumption will break a lot of
    programs.
    On Thursday, November 14, 2013 10:58:13 PM UTC-7, Dominik Honnef wrote:

    Ian Lance Taylor <ia...@golang.org <javascript:>> writes:
    It's not specified.
    A read from a map is like any other read from a variable. Thus it's
    fine if multiple goroutines read from a map simultaneously. But if
    one goroutine reads from a map while another writes to a map, or if
    two goroutines write to a map, then the program must synchronize those
    goroutines in some way.

    Ian
    Does the existing specification actually guarantee this? Because there
    are certainly ways to implement it that don't have that guarantee.
    https://code.google.com/p/go/source/detail?r=af469280a34b comes to mind
    in particular.

    I know that this implementation of Go seems to guarantee it, but could
    there be an alternate implementation that implements maps that aren't
    safe for concurrent reads and still implement the specification?
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Vincent Callanan at Nov 15, 2013 at 10:01 am
    Assurances aside, this really needs to be firmed up in the spec as it is so
    fundamental.
    On Friday, 15 November 2013 09:53:12 UTC, Kevin Gillette wrote:

    Implementation details aside, from a usability perspective it's a rather
    unfortunate thing when concurrent no-write access results in race
    conditions.

    Other such questions about read-only map access have been answered with
    "yes, it's safe," and it's a widely enough held assumption that concurrent
    no-write access to anything in Go is safe; it may as well be specified,
    since any implementation breaking that assumption will break a lot of
    programs.
    On Thursday, November 14, 2013 10:58:13 PM UTC-7, Dominik Honnef wrote:

    Ian Lance Taylor <ia...@golang.org> writes:
    It's not specified.
    A read from a map is like any other read from a variable. Thus it's
    fine if multiple goroutines read from a map simultaneously. But if
    one goroutine reads from a map while another writes to a map, or if
    two goroutines write to a map, then the program must synchronize those
    goroutines in some way.

    Ian
    Does the existing specification actually guarantee this? Because there
    are certainly ways to implement it that don't have that guarantee.
    https://code.google.com/p/go/source/detail?r=af469280a34b comes to mind
    in particular.

    I know that this implementation of Go seems to guarantee it, but could
    there be an alternate implementation that implements maps that aren't
    safe for concurrent reads and still implement the specification?
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Ian Lance Taylor at Nov 15, 2013 at 6:12 pm

    On Thu, Nov 14, 2013 at 9:58 PM, Dominik Honnef wrote:
    Ian Lance Taylor <iant@golang.org> writes:
    It's not specified.
    A read from a map is like any other read from a variable. Thus it's
    fine if multiple goroutines read from a map simultaneously. But if
    one goroutine reads from a map while another writes to a map, or if
    two goroutines write to a map, then the program must synchronize those
    goroutines in some way.
    Does the existing specification actually guarantee this? Because there
    are certainly ways to implement it that don't have that guarantee.
    https://code.google.com/p/go/source/detail?r=af469280a34b comes to mind
    in particular.

    I know that this implementation of Go seems to guarantee it, but could
    there be an alternate implementation that implements maps that aren't
    safe for concurrent reads and still implement the specification?
    The relevant document is here is the memory model
    (http://golang.org/ref/mem). The memory model is written in terms of
    reads and writes to a variable. You are essentially saying "a map is
    not a variable, so we need a separate specification for the behaviour
    of a map." I am simply saying: a map is a variable.

    I'm not opposed to adding a sentence about maps to the memory model.
    I'm just not sure it's required.

    You are of course correct that maps can be implemented such that they
    do not meet the requirements of the Go memory model. It's possible to
    implement non-map variables that way too, e.g., by caching reads and
    writes across synchronization boundaries. Either way it would be a
    case where the implementation fails to meet the spec, or, in other
    words, it would be a bug.

    Ian

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Vincent Callanan at Nov 16, 2013 at 10:08 am

    You are of course correct that maps can be implemented such that they
    do not meet the requirements of the Go memory model. It's possible to
    implement non-map variables that way too, e.g., by caching reads and
    writes across synchronization boundaries. Either way it would be a
    case where the implementation fails to meet the spec, or, in other
    words, it would be a bug.
    ...not to mention the CPU cache itself which is why we never want to have
    to use read mutexes for "fixed" map variables.

    The "variable" get-out clause is surely a little too "facile" in this
    instance.

    Arrays/slices are special cases (witness race conditions)
    So too maps (there is already a statement somewhere about thread-safety of
    maps)

    Remember, a non-novice poster to this thread has already stated:
    "Since they aren't specified to be safe, I assume they are unsafe and use a
    sync.Mutex to protect my concurrent map accesses".

    We wouldn't want that to become the norm.
    Go maps very speedy for "const string lookups" even over relatively small
    data sets.
    Having to use mutexes in these situations would be unthinkable.

    By firming up the spec, the intent is absolutely clear.

    Vincent

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Dmitry Vyukov at Nov 16, 2013 at 11:04 am

    On Fri, Nov 15, 2013 at 10:12 PM, Ian Lance Taylor wrote:
    On Thu, Nov 14, 2013 at 9:58 PM, Dominik Honnef wrote:
    Ian Lance Taylor <iant@golang.org> writes:
    It's not specified.
    A read from a map is like any other read from a variable. Thus it's
    fine if multiple goroutines read from a map simultaneously. But if
    one goroutine reads from a map while another writes to a map, or if
    two goroutines write to a map, then the program must synchronize those
    goroutines in some way.
    Does the existing specification actually guarantee this? Because there
    are certainly ways to implement it that don't have that guarantee.
    https://code.google.com/p/go/source/detail?r=af469280a34b comes to mind
    in particular.

    I know that this implementation of Go seems to guarantee it, but could
    there be an alternate implementation that implements maps that aren't
    safe for concurrent reads and still implement the specification?
    The relevant document is here is the memory model
    (http://golang.org/ref/mem). The memory model is written in terms of
    reads and writes to a variable. You are essentially saying "a map is
    not a variable, so we need a separate specification for the behaviour
    of a map." I am simply saying: a map is a variable.

    I'm not opposed to adding a sentence about maps to the memory model.
    I'm just not sure it's required.

    Maps are not "as safe as int". "as safe as int" implies that different
    goroutines can freely operate on different variables of the type. Maps
    are as safe as *int. I am not sure it's clearly and explicitly stated
    in the docs.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Nicolas Riesch at Nov 16, 2013 at 3:08 pm

    Maps are not "as safe as int". "as safe as int" implies that different
    goroutines can freely operate on different variables of the type. Maps
    are as safe as *int. I am not sure it's clearly and explicitly stated
    in the docs.
    This time, I think this is correct : a map[int]Type_A is as safe as a
    pointer to an array of Type_A.




    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Vincent Callanan at Nov 16, 2013 at 7:20 pm
    de facto or de jure?
    :-)
    On Saturday, 16 November 2013 15:08:36 UTC, nicolas...@gmail.com wrote:

    Maps are not "as safe as int". "as safe as int" implies that different
    goroutines can freely operate on different variables of the type. Maps
    are as safe as *int. I am not sure it's clearly and explicitly stated
    in the docs.
    This time, I think this is correct : a map[int]Type_A is as safe as a
    pointer to an array of Type_A.


    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Nicolas Riesch at Nov 15, 2013 at 4:56 pm
    In C, using a pointer to type A to write data, and reading the same data
    with a pointer to type B was also considered "safe" for decades.
    But one day, compilers made more agressive optimizations and such
    programmatic idiom did not work any more (strict-aliasing problem).
    Indeed, this broke a lot of programs, even Linux kernel. The answer was :
    such thing has never been explicitely allowed by the C specs.

    The question is : something works today, but will it work tomorrow ?
    The problem that occurred ten years ago in C with this strict-aliasing
    problem shows that "break a lot of programs" or "widely enough held
    assumption" are not enough to ensure that something is "safe" to do.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Ian Lance Taylor at Nov 15, 2013 at 6:19 pm

    On Fri, Nov 15, 2013 at 8:11 AM, wrote:
    In C, using a pointer to type A to write data, and reading the same data
    with a pointer to type B was also considered "safe" for decades.
    But one day, compilers made more agressive optimizations and such
    programmatic idiom did not work any more (strict-aliasing problem).
    Indeed, this broke a lot of programs, even Linux kernel. The answer was :
    such thing has never been explicitely allowed by the C specs.
    That may well be the perspective of a C programmer, but I don't think
    it's an accurate representation of how C developed. During the
    initial C standardization process, this issue was discussed. At that
    time, the standard writers decided that they would not require that
    data written by a pointer to type A be readable using a pointer to
    type B. This is explicitly spelled out in the ANSI C89 standard aka
    the ISO C90 standard, and similar wording has carried forward into
    later C and C++ standards. I believe this decision was made based on
    experience with Fortran compilers.

    Over time C compiler implementors modified their compilers to follow
    the new standard more closely. They also strengthened their
    optimizations. This increasingly caused programs to fail due to type
    aliasing. At that time programmers complained, but the answer was not
    "such thing has never been explicitely allowed by the C specs." The
    answer was "the C language standard specifically permit compilers to
    make this optimization." And, of course, most production compilers
    provided options to disable these optimizations.

    None of this has anything to do with Go, which is a different
    language with a different philosophy.

    Ian

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Nicolas Riesch at Nov 15, 2013 at 9:07 pm
    Excuse me, Ian, I did not understand your first answer when I read it.

    But is is clear, now : "A MAP IS A VARIABLE", exactly like an int or a
    string, just a more complex one.

    For a Go programmer, a map must be considered "macroscopically" like a very
    basic type, like int or string, which is provided by Go. The inner
    mechanism of a map is not relevant for the programmer.

    It is obvious that if an "int" variable is assigned a value in init(), and
    no other assigment ever occurs, many goroutines can read it safely without
    synchronization.

    The fact that in Go, a "map" can be considered like an "int" make it
    possible to replace the word "int" by "map" in the above sentence, and all
    becomes clear !

    The difficult "philosophical" thing to understand was just : "a map is a
    VARIABLE" ;-)))

    Thank you very much for your explanation, because I think it is something
    fundamental to understand how Go works.

    (and thank you for the explanation about C strict-aliasing, which has
    bitten me at that time ;-)


    nicolas riesch

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • John Nagle at Nov 17, 2013 at 12:12 am

    On 11/14/2013 9:32 AM, nicolas.riesch@gmail.com wrote:
    I want to create a map and fill it with values, in init().

    After init(), this map will only be accessed for reading, never for writing.
        You could use a read/write lock around it, and when done writing,
    lock it for reading and never unlock it again.

         John Nagle

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Nicolas Riesch at Nov 17, 2013 at 2:25 pm
    Le dimanche 17 novembre 2013 01:12:25 UTC+1, John Nagle a écrit :
    You could use a read/write lock around it, and when done writing,
    lock it for reading and never unlock it again.
    John Nagle

    The answer of Ian seems to be clear. There is no need for a lock in this
    case.

      As an exemple, the sqlite driver code
    http://code.google.com/p/gosqlite/source/browse/sqlite/sqlite.go has a map
    var errText = map[Errno]string{
             1: "SQL error or missing database",
             2: "Internal logic error in SQLite",
             3: "Access permission denied",
             ,,,
    }
    which access is not protected by any lock. The map is not initialized in
    init(), but it makes no difference.


    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedNov 15, '13 at 2:43a
activeNov 17, '13 at 2:25p
posts16
users8
websitegolang.org

People

Translate

site design / logo © 2022 Grokbase