FAQ
I'm getting errors due to "too many open files":

Couldn't get hostname: open /proc/sys/kernel/hostname: too many open files


What tools/techniques could I use to track this down? I've
got deferred closes on the two locations in my program where I open files,
so I'm thinking it's in one of the third party libraries I'm using.

Doing a count on the number of file descriptors shows a definite problem:

% while [ 1 ] ; do
ls /proc/9787/fd | wc -l ; sleep 1
done
...
138
157
472
1023
1023
276
180
...


Sonia Hamilton.

--

Search Discussions

  • David Symonds at Sep 14, 2012 at 1:48 am
    The files under /proc/PID/fd/ should all look like symlinks to the
    real file that's opened. That might provide some clues.

    --
  • Yves Junqueira at Sep 14, 2012 at 2:26 am
    Try also:

    $ lsof -p <PID>

    (your problem is not related to Go so you may wanna try one of the Stack
    Exchange sites instead)
    On Sep 14, 2012 3:40 AM, "David Symonds" wrote:

    The files under /proc/PID/fd/ should all look like symlinks to the
    real file that's opened. That might provide some clues.

    --

    --
  • Greg Ward at Sep 14, 2012 at 4:37 am

    On 13 September 2012, Sonia Hamilton said:
    I'm getting errors due to "too many open files":

    Couldn't get hostname: open /proc/sys/kernel/hostname: too many open files


    What tools/techniques could I use to track this down? I've
    got deferred closes on the two locations in my program where I open files,
    so I'm thinking it's in one of the third party libraries I'm using.
    Code review?

    Years of experience with other garbage-collected languages (mainly
    Python and Java) has taught me to always, always, always explicitly
    release non-memory resources. IOW, if you open a file, you close it.
    If you are 100% sure that your own code religiously closes every file
    that it opens without relying on GC, then you should probably audit
    those third-party libraries.

    Alternately, write tiny test programs that just focus on one
    third-party library at a time until you have found the culprit... or
    convinced yourself that those libraries are all fine. ;-)

    Greg

    --
  • Dave Cheney at Sep 14, 2012 at 5:04 am
    It might be easier to work from the usual suspects. Sonia, are you
    using net/http ? If so, are you always calling request.Body.Close(),
    that is a frequent culprit.
    On Fri, Sep 14, 2012 at 11:07 AM, Greg Ward wrote:
    On 13 September 2012, Sonia Hamilton said:
    I'm getting errors due to "too many open files":

    Couldn't get hostname: open /proc/sys/kernel/hostname: too many open files


    What tools/techniques could I use to track this down? I've
    got deferred closes on the two locations in my program where I open files,
    so I'm thinking it's in one of the third party libraries I'm using.
    Code review?

    Years of experience with other garbage-collected languages (mainly
    Python and Java) has taught me to always, always, always explicitly
    release non-memory resources. IOW, if you open a file, you close it.
    If you are 100% sure that your own code religiously closes every file
    that it opens without relying on GC, then you should probably audit
    those third-party libraries.

    Alternately, write tiny test programs that just focus on one
    third-party library at a time until you have found the culprit... or
    convinced yourself that those libraries are all fine. ;-)

    Greg

    --
    --
  • Steve Phillips at Sep 14, 2012 at 6:33 am

    It might be easier to work from the usual suspects. Sonia, are you
    using net/http ? If so, are you always calling request.Body.Close(),
    that is a frequent culprit.
    If we don't use request.Body specifically, but _do_ use some other
    part (say, request.FormFile), then request.Body doesn't need to be
    Close()'d, does it? (I've had "too many open files" issues and don't
    Close Body unless I read from it...)

    --
  • Dave Cheney at Sep 14, 2012 at 6:42 am
    You should always close the body.

    http://code.google.com/p/go/source/browse/src/pkg/net/http/client.go?name=release#98
    On Fri, Sep 14, 2012 at 4:32 PM, Steve Phillips wrote:

    It might be easier to work from the usual suspects. Sonia, are you
    using net/http ? If so, are you always calling request.Body.Close(),
    that is a frequent culprit.
    If we don't use request.Body specifically, but _do_ use some other
    part (say, request.FormFile), then request.Body doesn't need to be
    Close()'d, does it? (I've had "too many open files" issues and don't
    Close Body unless I read from it...)

    --

    --
  • Zippoxer at Oct 21, 2012 at 10:22 am
    That says to close response body, not request body.
    On Friday, September 14, 2012 9:34:56 AM UTC+3, Dave Cheney wrote:

    You should always close the body.


    http://code.google.com/p/go/source/browse/src/pkg/net/http/client.go?name=release#98

    On Fri, Sep 14, 2012 at 4:32 PM, Steve Phillips <elimi...@gmail.com<javascript:>
    wrote:
    It might be easier to work from the usual suspects. Sonia, are you
    using net/http ? If so, are you always calling request.Body.Close(),
    that is a frequent culprit.
    If we don't use request.Body specifically, but _do_ use some other
    part (say, request.FormFile), then request.Body doesn't need to be
    Close()'d, does it? (I've had "too many open files" issues and don't
    Close Body unless I read from it...)

    --

    --
  • Sonia Hamilton at Sep 14, 2012 at 8:49 am
    Thanks guys for all your answers. @DaveCheney you were close - it was a
    call to net :)

    I (believe) I tracked the problem down to the snmp library I'm using, it
    was doing a net.DialTimeout without a deferred close; I've submitted a
    patch.

    conn, err := net.DialTimeout("udp", ...
    if err != nil {
    ...
    }
    defer conn.Close() // added

    Sonia.
    On Friday, 14 September 2012 10:57:41 UTC+10, Sonia Hamilton wrote:

    I'm getting errors due to "too many open files":

    Couldn't get hostname: open /proc/sys/kernel/hostname: too many open files


    What tools/techniques could I use to track this down? I've
    got deferred closes on the two locations in my program where I open files,
    so I'm thinking it's in one of the third party libraries I'm using.

    Doing a count on the number of file descriptors shows a definite problem:

    % while [ 1 ] ; do
    ls /proc/9787/fd | wc -l ; sleep 1
    done
    ...
    138
    157
    472
    1023
    1023
    276
    180
    ...


    Sonia Hamilton.
    --
  • Rémy Oudompheng at Sep 14, 2012 at 8:57 am

    On 2012/9/14 Sonia Hamilton wrote:
    Thanks guys for all your answers. @DaveCheney you were close - it was a call
    to net :)
    Open files are memory-allocated structures (os.File, for example).
    When they are leaking you should be able to see them in a memory
    profile and see what function allocated them;

    Rémy.

    --
  • Sonia Hamilton at Sep 14, 2012 at 11:33 pm
    Thanks Remy,

    Yes I've been using the profiler for another problem - and I've noticed
    encode/json seems to have a memory leak (similar to issue 2539 [1] for
    encoding/gob).

    It's the weekend here in Australia, I'm going to write a demo prog later
    and post it to the list, see if the problem is caused by my usage of
    encode/json or there is indeed a memory leak.

    [1]
    http://code.google.com/p/go/issues/detail?id=2539&q=json&colspec=ID%20Status%20Stars%20Priority%20Owner%20Reporter%20Summary

    Sonia.

    On Friday, 14 September 2012 10:57:41 UTC+10, Sonia Hamilton wrote:

    I'm getting errors due to "too many open files":

    Couldn't get hostname: open /proc/sys/kernel/hostname: too many open files


    What tools/techniques could I use to track this down? I've
    got deferred closes on the two locations in my program where I open files,
    so I'm thinking it's in one of the third party libraries I'm using.

    Doing a count on the number of file descriptors shows a definite problem:

    % while [ 1 ] ; do
    ls /proc/9787/fd | wc -l ; sleep 1
    done
    ...
    138
    157
    472
    1023
    1023
    276
    180
    ...


    Sonia Hamilton.
    --
  • Rémy Oudompheng at Sep 15, 2012 at 7:09 am
    In my experience I have not seem a memory leak in either encoding/json
    nor encoding/gob. It is possible that they use up more memory than
    sould be needed, but repeating the operation millions of times does
    not introduce a memory usage drift.

    The memory profile shows where the things were created, not where they 'live'.

    So if you have your own leaks, but the things you are leaking come
    from a JSON decoding, the memory profile will show where the leaking
    objects were decoded, not the place where the leak happens.

    Rémy.

    2012/9/15, Sonia Hamilton <sonia.snowfrog@gmail.com>:
    Thanks Remy,

    Yes I've been using the profiler for another problem - and I've noticed
    encode/json seems to have a memory leak (similar to issue 2539 [1] for
    encoding/gob).

    It's the weekend here in Australia, I'm going to write a demo prog later
    and post it to the list, see if the problem is caused by my usage of
    encode/json or there is indeed a memory leak.

    [1]
    http://code.google.com/p/go/issues/detail?id=2539&q=json&colspec=ID%20Status%20Stars%20Priority%20Owner%20Reporter%20Summary

    Sonia.

    On Friday, 14 September 2012 10:57:41 UTC+10, Sonia Hamilton wrote:

    I'm getting errors due to "too many open files":

    Couldn't get hostname: open /proc/sys/kernel/hostname: too many open
    files


    What tools/techniques could I use to track this down? I've
    got deferred closes on the two locations in my program where I open files,

    so I'm thinking it's in one of the third party libraries I'm using.

    Doing a count on the number of file descriptors shows a definite problem:

    % while [ 1 ] ; do
    ls /proc/9787/fd | wc -l ; sleep 1
    done
    ...
    138
    157
    472
    1023
    1023
    276
    180
    ...


    Sonia Hamilton.
    --

    --

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedSep 14, '12 at 12:57a
activeOct 21, '12 at 10:22a
posts12
users8
websitegolang.org

People

Translate

site design / logo © 2022 Grokbase