FAQ
Hi,
when in next releases of go compacting GC move pointers,
does map on poiner types will work ?
http://play.golang.org/p/kyw1poFKiU

Djadala

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Kevin Malachowski at Oct 20, 2014 at 6:30 am
    If and when Go uses a compacting/moving GC all valid Go code will still be valid, as per the Go1 guarantee.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Carlos Castillo at Oct 20, 2014 at 9:50 pm
    Case in point: http://play.golang.org/p/jWG5cOn8p7 - Current Go (1.3) moves
    stacks when they need to grow/shrink, and valid pointers to that memory get
    updated (addresses stored in uintptrs using unsafe, are not updated).

    The issues that will arise with moving heap and stack objects will mostly
    affect C code called via cgo, as Go can't find/update any C pointers. As
    shown above, Go code that uses "unsafe" can also be affected (eg: when it
    turns ptrs into non-ptrs).
    On Sunday, October 19, 2014 11:30:10 PM UTC-7, Kevin Malachowski wrote:

    If and when Go uses a compacting/moving GC all valid Go code will still be
    valid, as per the Go1 guarantee.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Khr at Oct 20, 2014 at 11:50 pm
    Djadala, yes, maps of pointers are one of the major stumbling blocks to
    moving objects. We're thinking about it. Rest assured we will not break
    that functionality.
    On Monday, October 20, 2014 2:50:05 PM UTC-7, Carlos Castillo wrote:

    Case in point: http://play.golang.org/p/jWG5cOn8p7 - Current Go (1.3)
    moves stacks when they need to grow/shrink, and valid pointers to that
    memory get updated (addresses stored in uintptrs using unsafe, are not
    updated).

    The issues that will arise with moving heap and stack objects will mostly
    affect C code called via cgo, as Go can't find/update any C pointers. As
    shown above, Go code that uses "unsafe" can also be affected (eg: when it
    turns ptrs into non-ptrs).
    On Sunday, October 19, 2014 11:30:10 PM UTC-7, Kevin Malachowski wrote:

    If and when Go uses a compacting/moving GC all valid Go code will still
    be valid, as per the Go1 guarantee.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Djadala at Oct 21, 2014 at 5:43 am

    On Tuesday, October 21, 2014 2:49:57 AM UTC+3, k...@golang.org wrote:
    Djadala, yes, maps of pointers are one of the major stumbling blocks to
    moving objects. We're thinking about it. Rest assured we will not break
    that functionality.
    On Monday, October 20, 2014 2:50:05 PM UTC-7, Carlos Castillo wrote:

    Case in point: http://play.golang.org/p/jWG5cOn8p7 - Current Go (1.3)
    moves stacks when they need to grow/shrink, and valid pointers to that
    memory get updated (addresses stored in uintptrs using unsafe, are not
    updated).

    The issues that will arise with moving heap and stack objects will mostly
    affect C code called via cgo, as Go can't find/update any C pointers. As
    shown above, Go code that uses "unsafe" can also be affected (eg: when it
    turns ptrs into non-ptrs).
    On Sunday, October 19, 2014 11:30:10 PM UTC-7, Kevin Malachowski wrote:

    If and when Go uses a compacting/moving GC all valid Go code will still
    be valid, as per the Go1 guarantee.
    thanks for explanation.
    One way to solve this problem is to add internal map
    map[pointer]pointerid, accesible only from runtime.
    when GC move pointer, it delete old pointer and add new one in that map.
    then all maps of type map[pointer]... are actually of type map[pointerid]...


    Djadala

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dmitry Vyukov at Oct 21, 2014 at 6:42 am

    On Tue, Oct 21, 2014 at 9:43 AM, wrote:
    On Tuesday, October 21, 2014 2:49:57 AM UTC+3, k...@golang.org wrote:

    Djadala, yes, maps of pointers are one of the major stumbling blocks to
    moving objects. We're thinking about it. Rest assured we will not break
    that functionality.
    On Monday, October 20, 2014 2:50:05 PM UTC-7, Carlos Castillo wrote:

    Case in point: http://play.golang.org/p/jWG5cOn8p7 - Current Go (1.3)
    moves stacks when they need to grow/shrink, and valid pointers to that
    memory get updated (addresses stored in uintptrs using unsafe, are not
    updated).

    The issues that will arise with moving heap and stack objects will mostly
    affect C code called via cgo, as Go can't find/update any C pointers. As
    shown above, Go code that uses "unsafe" can also be affected (eg: when it
    turns ptrs into non-ptrs).
    On Sunday, October 19, 2014 11:30:10 PM UTC-7, Kevin Malachowski wrote:

    If and when Go uses a compacting/moving GC all valid Go code will still
    be valid, as per the Go1 guarantee.

    thanks for explanation.
    One way to solve this problem is to add internal map map[pointer]pointerid,
    accesible only from runtime.
    when GC move pointer, it delete old pointer and add new one in that map.
    then all maps of type map[pointer]... are actually of type map[pointerid]...

    A few complications to this scheme.
    A map key can be of type struct { p1 *int; val int; p2 *string; }.
    Both pointers can change separately due to compaction. Hash of the
    whole struct must remain persistent.
    When we get to compaction Go GC will be concurrent. So this scheme
    needs to work reasonably well when GC moves objects and rehashes
    pointers and the application concurrently reads and writes maps.
    The memory overhead and the performance overhead of the additional map
    is probably OK during GC. But ideally we don't want the overheads when
    GC is not running. Potentially doubled memory consumption can be
    prohibitive.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Djadala at Oct 21, 2014 at 7:49 am

    On Tuesday, October 21, 2014 9:42:36 AM UTC+3, Dmitry Vyukov wrote:
    On Tue, Oct 21, 2014 at 9:43 AM, <dja...@gmail.com <javascript:>> wrote:

    On Tuesday, October 21, 2014 2:49:57 AM UTC+3, k...@golang.org wrote:

    Djadala, yes, maps of pointers are one of the major stumbling blocks to
    moving objects. We're thinking about it. Rest assured we will not
    break
    that functionality.
    On Monday, October 20, 2014 2:50:05 PM UTC-7, Carlos Castillo wrote:

    Case in point: http://play.golang.org/p/jWG5cOn8p7 - Current Go (1.3)
    moves stacks when they need to grow/shrink, and valid pointers to that
    memory get updated (addresses stored in uintptrs using unsafe, are not
    updated).

    The issues that will arise with moving heap and stack objects will
    mostly
    affect C code called via cgo, as Go can't find/update any C pointers.
    As
    shown above, Go code that uses "unsafe" can also be affected (eg: when
    it
    turns ptrs into non-ptrs).

    On Sunday, October 19, 2014 11:30:10 PM UTC-7, Kevin Malachowski
    wrote:
    If and when Go uses a compacting/moving GC all valid Go code will
    still
    be valid, as per the Go1 guarantee.

    thanks for explanation.
    One way to solve this problem is to add internal map
    map[pointer]pointerid,
    accesible only from runtime.
    when GC move pointer, it delete old pointer and add new one in that map.
    then all maps of type map[pointer]... are actually of type
    map[pointerid]...


    A few complications to this scheme.
    A map key can be of type struct { p1 *int; val int; p2 *string; }.
    for hashing, p1 && p2 are replaced with their pointerid.

    Both pointers can change separately due to compaction. Hash of the
    whole struct must remain persistent.
    When we get to compaction Go GC will be concurrent. So this scheme
    needs to work reasonably well when GC moves objects and rehashes
    pointers and the application concurrently reads and writes maps.
    The memory overhead and the performance overhead of the additional map
    is probably OK during GC. But ideally we don't want the overheads when
    GC is not running. Potentially doubled memory consumption can be
    prohibitive.
    yes, this sheme ruin concurrency, and memory overhead also is big.

    remaining solutions ?
    1 pointer points to pointerid+data ( surprise, sizeof( struct{}) > 0 )
    2 pointer include pointerid ( sizeof(pointer) > C.sizeof( pointer))
    ?

    Djadala




    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dmitry Vyukov at Oct 21, 2014 at 8:05 am

    On Tue, Oct 21, 2014 at 11:49 AM, wrote:
    On Tuesday, October 21, 2014 9:42:36 AM UTC+3, Dmitry Vyukov wrote:
    On Tue, Oct 21, 2014 at 9:43 AM, wrote:

    On Tuesday, October 21, 2014 2:49:57 AM UTC+3, k...@golang.org wrote:

    Djadala, yes, maps of pointers are one of the major stumbling blocks to
    moving objects. We're thinking about it. Rest assured we will not
    break
    that functionality.
    On Monday, October 20, 2014 2:50:05 PM UTC-7, Carlos Castillo wrote:

    Case in point: http://play.golang.org/p/jWG5cOn8p7 - Current Go (1.3)
    moves stacks when they need to grow/shrink, and valid pointers to that
    memory get updated (addresses stored in uintptrs using unsafe, are not
    updated).

    The issues that will arise with moving heap and stack objects will
    mostly
    affect C code called via cgo, as Go can't find/update any C pointers.
    As
    shown above, Go code that uses "unsafe" can also be affected (eg: when
    it
    turns ptrs into non-ptrs).

    On Sunday, October 19, 2014 11:30:10 PM UTC-7, Kevin Malachowski
    wrote:
    If and when Go uses a compacting/moving GC all valid Go code will
    still
    be valid, as per the Go1 guarantee.

    thanks for explanation.
    One way to solve this problem is to add internal map
    map[pointer]pointerid,
    accesible only from runtime.
    when GC move pointer, it delete old pointer and add new one in that map.
    then all maps of type map[pointer]... are actually of type
    map[pointerid]...

    A few complications to this scheme.
    A map key can be of type struct { p1 *int; val int; p2 *string; }.

    for hashing, p1 && p2 are replaced with their pointerid.
    Both pointers can change separately due to compaction. Hash of the
    whole struct must remain persistent.
    When we get to compaction Go GC will be concurrent. So this scheme
    needs to work reasonably well when GC moves objects and rehashes
    pointers and the application concurrently reads and writes maps.
    The memory overhead and the performance overhead of the additional map
    is probably OK during GC. But ideally we don't want the overheads when
    GC is not running. Potentially doubled memory consumption can be
    prohibitive.

    yes, this sheme ruin concurrency, and memory overhead also is big.

    remaining solutions ?
    1 pointer points to pointerid+data ( surprise, sizeof( struct{}) > 0 )
    A user can take pointer of any member of a struct. This means that
    every other member must be a pointer id. This can make struct up to
    16x larger...
    2 pointer include pointerid ( sizeof(pointer) > C.sizeof( pointer))
    ?
    Memory overhead and counter-intuitiveness aside; this works well when
    you only copy pointer; but when you obtain a new pointer you need to
    get its pointer id from somewhere (a global hashmap?).
    Djadala




    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedOct 20, '14 at 5:52a
activeOct 21, '14 at 8:05a
posts8
users5
websitegolang.org

People

Translate

site design / logo © 2021 Grokbase