FAQ
Hello everyone,

I've been playing a bit with Go to check the garbage collector, and I found
an unexpected behaviour (unexpected for me). I don't know if it is a bug or
something that I don't really understand.

Everything has been tested on tip version.

Program is here: http://play.golang.org/p/QbfGh5xnhF

package main

import (
     "time"
     // "runtime/debug"
)

func main() {
     f()
     // debug.FreeOSMemory()
     // debug.FreeOSMemory()
     time.Sleep(10 * time.Hour)
}

func f() {
     a := make([]int, 100000000)
     for i := range a {
         a[i] = i
     }
}

The program allocates 763 MB of RAM (on 64-bits architecture) and then
sleeps for 10 hours. I've been monitoring the memory used and the GC work
by setting GODEBUG=gotrace=1

With the FreeOSMemory() lines commented, the memory is never GC'd nor
released back to the system. I've waited about 30 minutes.

With one single line of FreeOSMemory(), the memory is not freed immediately
(as I expected) but it seems that in the 2-3 next automatic GC runs it is
finally found and freed, and eventually, released back to the OS.

With two FreeOSMemory() lines, the memory is released immediately (I
suppose that in the second call).

I understand that this scenario is not very real. If I do some other work
instead of just waiting, eventually the GC is executed, there is more
garbage than my slice to be purged and everything is released.

Is this expected? In particular, that the 763 MB are not GC'd automatically
without FreeOSMemery() lines.

Thanks

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Stephen Gutekanst at May 17, 2014 at 10:43 pm
    I remember reading the GC does not return memory to the host OS unless you
    explicitly invoke FreeOSMemory() (but I think it's not important because of
    swap etc?).

    Stephen
    On Saturday, May 17, 2014 3:36:19 PM UTC-7, Ignacio Grande wrote:

    Hello everyone,

    I've been playing a bit with Go to check the garbage collector, and I
    found an unexpected behaviour (unexpected for me). I don't know if it is a
    bug or something that I don't really understand.

    Everything has been tested on tip version.

    Program is here: http://play.golang.org/p/QbfGh5xnhF

    package main

    import (
    "time"
    // "runtime/debug"
    )

    func main() {
    f()
    // debug.FreeOSMemory()
    // debug.FreeOSMemory()
    time.Sleep(10 * time.Hour)
    }

    func f() {
    a := make([]int, 100000000)
    for i := range a {
    a[i] = i
    }
    }

    The program allocates 763 MB of RAM (on 64-bits architecture) and then
    sleeps for 10 hours. I've been monitoring the memory used and the GC work
    by setting GODEBUG=gotrace=1

    With the FreeOSMemory() lines commented, the memory is never GC'd nor
    released back to the system. I've waited about 30 minutes.

    With one single line of FreeOSMemory(), the memory is not freed
    immediately (as I expected) but it seems that in the 2-3 next automatic GC
    runs it is finally found and freed, and eventually, released back to the OS.

    With two FreeOSMemory() lines, the memory is released immediately (I
    suppose that in the second call).

    I understand that this scenario is not very real. If I do some other work
    instead of just waiting, eventually the GC is executed, there is more
    garbage than my slice to be purged and everything is released.

    Is this expected? In particular, that the 763 MB are not GC'd
    automatically without FreeOSMemery() lines.

    Thanks
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Nacho at May 17, 2014 at 10:47 pm
    As I understand, memory is returned to the OS about 5 minutes after is has
    been marked as free by the GC. And the GC runs every two minutes top, if
    not triggered by an increase in memory use. So worst-case would be 7
    minutes to be freed.

    In this case, I think that the slice is not marked as freed, but in use, so
    it would never be returned to the OS.



    On Sun, May 18, 2014 at 12:43 AM, Stephen Gutekanst wrote:

    I remember reading the GC does not return memory to the host OS unless you
    explicitly invoke FreeOSMemory() (but I think it's not important because of
    swap etc?).

    Stephen
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Khr at May 18, 2014 at 4:42 am
    The behavior with one or two FreeOSMemory calls is expected. The array
    gets freed after 1 GC, although for technical reasons it takes another GC
    for that fact to make it to the gctrace info.

    With no FreeOSMemory calls, there's definitely a problem. The array should
    get freed eventually as it does when there are FreeOSMemory calls. I've
    opened bug 8018.

    On Saturday, May 17, 2014 3:47:35 PM UTC-7, Ignacio Grande wrote:

    As I understand, memory is returned to the OS about 5 minutes after is has
    been marked as free by the GC. And the GC runs every two minutes top, if
    not triggered by an increase in memory use. So worst-case would be 7
    minutes to be freed.

    In this case, I think that the slice is not marked as freed, but in use,
    so it would never be returned to the OS.




    On Sun, May 18, 2014 at 12:43 AM, Stephen Gutekanst <stephen....@gmail.com<javascript:>
    wrote:
    I remember reading the GC does not return memory to the host OS unless
    you explicitly invoke FreeOSMemory() (but I think it's not important
    because of swap etc?).

    Stephen
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dmitry Vyukov at May 18, 2014 at 7:26 am

    On Sun, May 18, 2014 at 8:42 AM, wrote:
    The behavior with one or two FreeOSMemory calls is expected. The array gets
    freed after 1 GC, although for technical reasons it takes another GC for
    that fact to make it to the gctrace info.
    I think it's actually a bug. Gctrace does not output amount of
    released memory, but if you look at top or Program Manager, the memory
    must be released instantly. Filed:
    https://code.google.com/p/go/issues/detail?id=8019

    With no FreeOSMemory calls, there's definitely a problem. The array should
    get freed eventually as it does when there are FreeOSMemory calls. I've
    opened bug 8018.

    On Saturday, May 17, 2014 3:47:35 PM UTC-7, Ignacio Grande wrote:

    As I understand, memory is returned to the OS about 5 minutes after is has
    been marked as free by the GC. And the GC runs every two minutes top, if not
    triggered by an increase in memory use. So worst-case would be 7 minutes to
    be freed.

    In this case, I think that the slice is not marked as freed, but in use,
    so it would never be returned to the OS.




    On Sun, May 18, 2014 at 12:43 AM, Stephen Gutekanst
    wrote:
    I remember reading the GC does not return memory to the host OS unless
    you explicitly invoke FreeOSMemory() (but I think it's not important because
    of swap etc?).

    Stephen
    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dmitry Vyukov at May 18, 2014 at 7:32 am
    OK, gctrace also prints this info.
    With one FreeOSMemory call:

    gc1(1): 9+0+173+0 us, 0 -> 0 MB, 20 (21-1) objects, 0/0/0 sweeps, 0(0)
    handoff, 0(0) steal, 0/0/0 yields
    gc2(1): 1+0+88+0 us, 0 -> 763 MB, 101 (103-2) objects, 15/0/0 sweeps,
    0(0) handoff, 0(0) steal, 0/0/0 yields
    gc3(1): 2+0+101+0 us, 763 -> 763 MB, 85 (108-23) objects, 24/0/0
    sweeps, 0(0) handoff, 0(0) steal, 0/0/0 yields
    scvg-1: 0 MB released
    scvg-1: inuse: 763, idle: 0, sys: 764, released: 0, consumed: 763 (MB)

    with two:

    gc1(1): 12+0+232+0 us, 0 -> 0 MB, 20 (21-1) objects, 0/0/0 sweeps,
    0(0) handoff, 0(0) steal, 0/0/0 yields
    gc2(1): 1+0+97+0 us, 0 -> 763 MB, 100 (102-2) objects, 15/0/0 sweeps,
    0(0) handoff, 0(0) steal, 0/0/0 yields
    gc3(1): 1+0+120+0 us, 763 -> 763 MB, 84 (107-23) objects, 24/0/0
    sweeps, 0(0) handoff, 0(0) steal, 0/0/0 yields
    scvg-1: 0 MB released
    scvg-1: inuse: 763, idle: 0, sys: 764, released: 0, consumed: 763 (MB)
    gc4(1): 0+35+68+0 us, 0 -> 0 MB, 83 (107-24) objects, 26/0/25 sweeps,
    0(0) handoff, 0(0) steal, 0/0/0 yields
    scvg-1: 762 MB released
    scvg-1: inuse: 0, idle: 763, sys: 764, released: 763, consumed: 0 (MB)


    On Sun, May 18, 2014 at 11:25 AM, Dmitry Vyukov wrote:
    On Sun, May 18, 2014 at 8:42 AM, wrote:
    The behavior with one or two FreeOSMemory calls is expected. The array gets
    freed after 1 GC, although for technical reasons it takes another GC for
    that fact to make it to the gctrace info.
    I think it's actually a bug. Gctrace does not output amount of
    released memory, but if you look at top or Program Manager, the memory
    must be released instantly. Filed:
    https://code.google.com/p/go/issues/detail?id=8019

    With no FreeOSMemory calls, there's definitely a problem. The array should
    get freed eventually as it does when there are FreeOSMemory calls. I've
    opened bug 8018.

    On Saturday, May 17, 2014 3:47:35 PM UTC-7, Ignacio Grande wrote:

    As I understand, memory is returned to the OS about 5 minutes after is has
    been marked as free by the GC. And the GC runs every two minutes top, if not
    triggered by an increase in memory use. So worst-case would be 7 minutes to
    be freed.

    In this case, I think that the slice is not marked as freed, but in use,
    so it would never be returned to the OS.




    On Sun, May 18, 2014 at 12:43 AM, Stephen Gutekanst
    wrote:
    I remember reading the GC does not return memory to the host OS unless
    you explicitly invoke FreeOSMemory() (but I think it's not important because
    of swap etc?).

    Stephen
    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedMay 17, '14 at 10:36p
activeMay 18, '14 at 7:32a
posts6
users4
websitegolang.org

People

Translate

site design / logo © 2022 Grokbase