FAQ
I want to insure that all memory I am using inside my specific function
"func memhungry()" is released at the end of that function.

Therefore, inside memhungry(), I would like to hook all memory allocations,
and add them to a list. At the end of the function, I would like to use
defer or other means would free those allocations.

In effect, this is manually implementing a region/arena memory area for
this particular function.

Is this possible in golang? If so, how would I proceed to hook memory
allocations? I assume profiling tools already do this, no?

Thank you.

- J

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Dave Cheney at Nov 3, 2013 at 11:13 pm

    On Mon, Nov 4, 2013 at 10:09 AM, J wrote:
    I want to insure that all memory I am using inside my specific function
    "func memhungry()" is released at the end of that function.
    Why do you want to do this ? What would you do if you were able to do so ?
    Therefore, inside memhungry(), I would like to hook all memory allocations,
    and add them to a list. At the end of the function, I would like to use
    defer or other means would free those allocations.

    In effect, this is manually implementing a region/arena memory area for this
    particular function.

    Is this possible in golang?
    yes, use mmap.
    If so, how would I proceed to hook memory
    allocations? I assume profiling tools already do this, no?
    They do, but only allocations on the heap, and that part of the
    runtime is not visible to Go code.
    Thank you.

    - J

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • J at Nov 3, 2013 at 11:36 pm

    On Sunday, November 3, 2013 3:13:03 PM UTC-8, Dave Cheney wrote:
    On Mon, Nov 4, 2013 at 10:09 AM, J wrote:
    I want to insure that all memory I am using inside my specific function
    "func memhungry()" is released at the end of that function.
    Why do you want to do this ?
    I want to insure that all the temporarily allocated memory is cleaned up at
    the end of the function. This is to avoid extra garbage collection, and the
    associated pause times. This is a common strategy for games to meet
    frame-rate rendering requirements and create smooth animation without
    lags/pauses.
    If so, how would I proceed to hook memory
    allocations? I assume profiling tools already do this, no?
    They do, but only allocations on the heap, and that part of the
    runtime is not visible to Go code.
    Very good. It would nice to have an API to hook heap allocations, but I
    could hack the runtime source if need be. Could someone kindly point me at
    the source code for tools that might serve as an example of how to do
    allocation hooking. I would like to experiment with adding a
    reg.Start()/reg.Release() mechanism. reg.Start() would start a new region
    that would take the place of all subsequent user-code heap allocations.

    Thank you.

    - J

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Dave Cheney at Nov 3, 2013 at 11:38 pm

    On Mon, Nov 4, 2013 at 10:36 AM, J wrote:
    On Sunday, November 3, 2013 3:13:03 PM UTC-8, Dave Cheney wrote:
    On Mon, Nov 4, 2013 at 10:09 AM, J wrote:
    I want to insure that all memory I am using inside my specific function
    "func memhungry()" is released at the end of that function.
    Why do you want to do this ?

    I want to insure that all the temporarily allocated memory is cleaned up at
    the end of the function. This is to avoid extra garbage collection, and the
    associated pause times. This is a common strategy for games to meet
    frame-rate rendering requirements and create smooth animation without
    lags/pauses.
    Go provides no facility to do this outside of

    * stack allocation via escape analysis
    * using preallocated pools of values
    Very good. It would nice to have an API to hook heap allocations, but I
    could hack the runtime source if need be. Could someone kindly point me at
    the source code for tools that might serve as an example of how to do
    allocation hooking. I would like to experiment with adding a
    reg.Start()/reg.Release() mechanism. reg.Start() would start a new region
    that would take the place of all subsequent user-code heap allocations.

    Thank you.

    - J

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • J at Nov 3, 2013 at 11:54 pm

    On Sunday, November 3, 2013 3:38:51 PM UTC-8, Dave Cheney wrote:

    Go provides no facility to do this outside of
    * stack allocation via escape analysis
    * using preallocated pools of values


    Ah. Thanks for this insight, Dave. I find the escape analysis particularly
    interesting. It might naturally create the arena on the local stack if the
    analysis successfully identifies all the garbage. I wonder how I could
    tell if that was indeed happening.

    Is there any visibility available as to how effective the escape analysis
    has been? That is, I would like to find a means to double check that a
    given struct has been put on the local stack (lifetime corresponding to
    current function call) rather than on the infinite-lifetime heap.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Dave Cheney at Nov 4, 2013 at 12:02 am

    On Mon, Nov 4, 2013 at 10:54 AM, J wrote:
    On Sunday, November 3, 2013 3:38:51 PM UTC-8, Dave Cheney wrote:

    Go provides no facility to do this outside of
    * stack allocation via escape analysis
    * using preallocated pools of values


    Ah. Thanks for this insight, Dave. I find the escape analysis particularly
    interesting. It might naturally create the arena on the local stack if the
    analysis successfully identifies all the garbage. I wonder how I could tell
    if that was indeed happening.
    pass -gcflags -m to go build/install/run/test
    Is there any visibility available as to how effective the escape analysis
    has been? That is, I would like to find a means to double check that a given
    struct has been put on the local stack (lifetime corresponding to current
    function call) rather than on the infinite-lifetime heap.
    Use the flag above, over time you'll get a feeling for what is capable
    of being moved to the stack.
    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Tad Glines at Nov 4, 2013 at 7:16 pm

    On Sun, Nov 3, 2013 at 4:02 PM, Dave Cheney wrote:
    pass -gcflags -m to go build/install/run/test

    This is so cool. Is this documented somewhere? Some of the output is self
    explanatory (e.g. "X escapes to heap" or "moved to heap: X") put some
    output is a little cryptic. Foe example, does "leaking param: X" just mean
    that argument X is passed as an argument to another function, or is there
    some additional meaning?

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Dave Cheney at Nov 4, 2013 at 9:11 pm

    On 5 Nov 2013, at 6:15, Tad Glines wrote:
    On Sun, Nov 3, 2013 at 4:02 PM, Dave Cheney wrote:

    pass -gcflags -m to go build/install/run/test
    This is so cool. Is this documented somewhere?
    As always, use the source Luke, especially cmd/gc/esc.c
    Some of the output is self explanatory (e.g. "X escapes to heap" or "moved to heap: X") put some output is a little cryptic. Foe example, does "leaking param: X" just mean that argument X is passed as an argument to another function, or is there some additional meaning?
    It means the param leaks to the caller, outside the scope of escape analysis, or a param is passed to a function that escape analysis thinks leaks (by taking its address and holding it, or something)

    Fun fact, syscalls used to force any []byte passed to them to be heap allocated because escape analysis cannot inspect ASM functions. This was fixed in 1.2 with a small hack to annotate calls to syscall.Syscall as not leaking its arguments (because they don't)
    H a
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Jason E. Aten at Nov 4, 2013 at 10:50 pm
    Nice. A tidbit from reading the source: //no:escape can be used in front of
    declarations of extern functions to advise (with the hope of improving even
    further) the escape analysis. Seems like an important optimization I don't
    see readily documented elsewhere.

    On Mon, Nov 4, 2013 at 1:10 PM, Dave Cheney wrote:



    On 5 Nov 2013, at 6:15, Tad Glines wrote:
    On Sun, Nov 3, 2013 at 4:02 PM, Dave Cheney wrote:


    pass -gcflags -m to go build/install/run/test

    This is so cool. Is this documented somewhere?


    As always, use the source Luke, especially cmd/gc/esc.c

    Some of the output is self explanatory (e.g. "X escapes to heap" or "moved
    to heap: X") put some output is a little cryptic. Foe example, does
    "leaking param: X" just mean that argument X is passed as an argument to
    another function, or is there some additional meaning?


    It means the param leaks to the caller, outside the scope of escape
    analysis, or a param is passed to a function that escape analysis thinks
    leaks (by taking its address and holding it, or something)

    Fun fact, syscalls used to force any []byte passed to them to be heap
    allocated because escape analysis cannot inspect ASM functions. This was
    fixed in 1.2 with a small hack to annotate calls to syscall.Syscall as not
    leaking its arguments (because they don't)

    H a
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Rob Pike at Nov 4, 2013 at 11:04 pm
    //go:noescape you mean.

    -rob

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Jason E. Aten at Nov 4, 2013 at 11:19 pm

    On Nov 4, 2013, at 3:04 PM, Rob Pike wrote:

    //go:noescape you mean.
    Yeah, sorry for the mis-spelling. Much more info available if I don't goof that.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Dave Cheney at Nov 4, 2013 at 11:33 pm
    https://codereview.appspot.com/7289048
    On Tue, Nov 5, 2013 at 10:19 AM, Jason E. Aten wrote:
    On Nov 4, 2013, at 3:04 PM, Rob Pike wrote:

    //go:noescape you mean.
    Yeah, sorry for the mis-spelling. Much more info available if I don't goof that.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedNov 3, '13 at 11:09p
activeNov 4, '13 at 11:33p
posts12
users4
websitegolang.org

People

Translate

site design / logo © 2022 Grokbase