FAQ
Hello,

I have a Go program which uses multi-threaded C library via cgo. After a load testing I noticed that program allocates (MemStats.Alloc) 200 MB of memory. pprof showed only about 2. I dig further and found that there are 4k of OS threads, most of them are sitting at sys_linux_amd64.s:238[1] (Go 1.0.3).

Does Go runtime created those threads to keep up with load? Does Go (1.0, 1.1) runtime ever shrinks thread pool?

[1] http://golang.org/src/pkg/runtime/sys_linux_amd64.s

–-–
Alexey "AlekSi" Palazhchenko

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

  • Minux at Apr 15, 2013 at 2:51 pm

    On Mon, Apr 15, 2013 at 9:44 PM, Alexey Palazhchenko wrote:
    I have a Go program which uses multi-threaded C library via cgo. After a
    load testing I noticed that program allocates (MemStats.Alloc) 200 MB of
    memory. pprof showed only about 2. I dig further and found that there are
    4k of OS threads, most of them are sitting at sys_linux_amd64.s:238[1] (Go
    1.0.3).
    Does Go runtime created those threads to keep up with load? Does Go (1.0,
    1.1) runtime ever shrinks thread pool?
    When one goroutine blocks in syscall on one of the OS thread and there is
    no available OS thread to
    host other ready goroutines, the runtime has no choice but to create
    another OS thread.

    The Go runtime currently doesn't ever shrink its thread pool.

    A related issue: https://code.google.com/p/go/issues/detail?id=4056
    for now, you will have to limit the number concurrent OS thread blocking
    yourself
    (mostly cgo calls, because calling into C world is treated the same as
    blocking in
    syscall).

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Alexei Sholik at Apr 18, 2013 at 2:32 pm
    So, it appears that for certain syscalls, like reading from a socket, the
    runtime uses some kind of polling or pushing provided by the OS allowing it
    to free the thread occupied by the goroutine blocked in such a call.

    What about the other calls? Is it the present issue only related to cgo? If
    not, is it documented which calls cause an OS thread to be blocked when the
    goroutine running on it blocks?

    I understand, that Go doesn't want to expose much info about the
    multiplexing mechanism to the user, but if user code may cause thousands of
    OS threads to be created, it should be common knowledge which syscalls do
    that.

    In Erlang, processes are scheduled preemptively and one OS thread can run
    thousands of concurrent processes. It also has an issue with calling to C
    code that is linked into the VM -- the scheduler can't preempt the process
    before it has returned from the C land. This caveat is known and understood
    by Erlang users.

    Go has cooperative goroutines that might be not so cooperative depending on
    the code they are running. So I think it should be well documented which
    use cases cause a goroutine to misbehave. Is it already?

    On Mon, Apr 15, 2013 at 5:51 PM, minux wrote:


    On Mon, Apr 15, 2013 at 9:44 PM, Alexey Palazhchenko <
    alexey.palazhchenko@gmail.com> wrote:
    I have a Go program which uses multi-threaded C library via cgo. After a
    load testing I noticed that program allocates (MemStats.Alloc) 200 MB of
    memory. pprof showed only about 2. I dig further and found that there are
    4k of OS threads, most of them are sitting at sys_linux_amd64.s:238[1] (Go
    1.0.3).
    Does Go runtime created those threads to keep up with load? Does Go
    (1.0, 1.1) runtime ever shrinks thread pool?
    When one goroutine blocks in syscall on one of the OS thread and there is
    no available OS thread to
    host other ready goroutines, the runtime has no choice but to create
    another OS thread.

    The Go runtime currently doesn't ever shrink its thread pool.

    A related issue: https://code.google.com/p/go/issues/detail?id=4056
    for now, you will have to limit the number concurrent OS thread blocking
    yourself
    (mostly cgo calls, because calling into C world is treated the same as
    blocking in
    syscall).

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.



    --
    Best regards
    Alexei Sholik

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Ian Lance Taylor at Apr 18, 2013 at 4:24 pm

    On Thu, Apr 18, 2013 at 7:32 AM, Alexei Sholik wrote:
    So, it appears that for certain syscalls, like reading from a socket, the
    runtime uses some kind of polling or pushing provided by the OS allowing it
    to free the thread occupied by the goroutine blocked in such a call. Yes.
    What about the other calls? Is it the present issue only related to cgo? If
    not, is it documented which calls cause an OS thread to be blocked when the
    goroutine running on it blocks?
    No. Any such documentation would be different for different Go
    implementations, and would change over time depending on changes to
    the scheduler.
    I understand, that Go doesn't want to expose much info about the
    multiplexing mechanism to the user, but if user code may cause thousands of
    OS threads to be created, it should be common knowledge which syscalls do
    that.
    https://code.google.com/p/go/issues/detail?id=4056
    Go has cooperative goroutines that might be not so cooperative depending on
    the code they are running. So I think it should be well documented which use
    cases cause a goroutine to misbehave. Is it already?
    No, because that too is implementation dependent. The implementations
    will improve over time and make it harder or, I hope, impossible, for
    a goroutine to misbehave in that way.

    Ian

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Ian Lance Taylor at Apr 15, 2013 at 4:51 pm

    On Mon, Apr 15, 2013 at 6:44 AM, Alexey Palazhchenko wrote:

    I have a Go program which uses multi-threaded C library via cgo. After a load testing I noticed that program allocates (MemStats.Alloc) 200 MB of memory. pprof showed only about 2. I dig further and found that there are 4k of OS threads, most of them are sitting at sys_linux_amd64.s:238[1] (Go 1.0.3).

    Does Go runtime created those threads to keep up with load? Does Go (1.0, 1.1) runtime ever shrinks thread pool?
    It does not.

    This is related to https://code.google.com/p/go/issues/detail?id=4056 .

    Ian

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Dmitry Vyukov at Apr 15, 2013 at 6:11 pm

    On Mon, Apr 15, 2013 at 6:44 AM, Alexey Palazhchenko wrote:

    Hello,

    I have a Go program which uses multi-threaded C library via cgo. After a
    load testing I noticed that program allocates (MemStats.Alloc) 200 MB of
    memory. pprof showed only about 2. I dig further and found that there are
    4k of OS threads, most of them are sitting at sys_linux_amd64.s:238[1] (Go
    1.0.3).

    Does Go runtime created those threads to keep up with load? Does Go (1.0,
    1.1) runtime ever shrinks thread pool?
    Can you please retest with Go1.1 beta/tip and report whether it's
    better/worser/same?

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Alexey Palazhchenko at Apr 18, 2013 at 9:20 am
    Hi,
    Actually, I would prefer recommended thread pool to size to hard limit. So idle threads would be slowly terminated one by one.

    Also, 200 MB is not a big problem. 50KB per thread is amazing.

    Can you please retest with Go1.1 beta/tip and report whether it's better/worser/same?
    Will do in few more days.

    –-–
    Alexey "AlekSi" Palazhchenko

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedApr 15, '13 at 1:45p
activeApr 18, '13 at 4:24p
posts7
users5
websitegolang.org

People

Translate

site design / logo © 2022 Grokbase