FAQ
Hey guys,

Long story short:

- How do you clean up main loops in libraries?

- How do you test code that uses SetFinalizer()?

Background:

Sometimes I use the "main loop" paradigm to make an object safe for
concurrent use. Basically, you have a New() routine that creates a new
object and starts a main loop in a goroutine. This main loop listens for
operations on a channel and you expose that channel directly (or indirectly
using accessor methods) to the user.

Great.

How do you clean this up when the object is no longer used? A .Close()
method works. I'd rather avoid that if I can, because .Close is extra work
(at the risk of forgetting it) for the user.

Ideally I'd like to stop the main loop once the garbage collector reclaims
the object.

Now I have something which *I think* works: how do I test it? It's a tricky
system and the tests I wrote succeed on my machine, but on Travis-CI not
even the simplest use of .SetFinalizer ever does anything.

Do I just leave it in and assume that because it works here, it works
everywhere?

Or does using a main loop like this require users to .Close()? Then I'd
rather use a lock.

Here is Travis not running any finalizers (not even a very simple one on an
integer):

https://travis-ci.org/hraban/testtravis/builds/22570080

This test passes every single time on my box. I tried crazy stuff to get
the GC juices flowing, to no avail.

It's a stripped down version of this project:

https://github.com/hraban/lrucache

Greetings,

Hraban

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Hraban Luyat at Apr 9, 2014 at 12:01 am
    P.S.: the project is go get'able:

    go get github.com/hraban/testtravis
    go test github.com/hraban/testtravis


    2014-04-09 1:43 GMT+02:00 Hraban Luyat <[email protected]>:
    Hey guys,

    Long story short:

    - How do you clean up main loops in libraries?

    - How do you test code that uses SetFinalizer()?

    Background:

    Sometimes I use the "main loop" paradigm to make an object safe for
    concurrent use. Basically, you have a New() routine that creates a new
    object and starts a main loop in a goroutine. This main loop listens for
    operations on a channel and you expose that channel directly (or indirectly
    using accessor methods) to the user.

    Great.

    How do you clean this up when the object is no longer used? A .Close()
    method works. I'd rather avoid that if I can, because .Close is extra work
    (at the risk of forgetting it) for the user.

    Ideally I'd like to stop the main loop once the garbage collector reclaims
    the object.

    Now I have something which *I think* works: how do I test it? It's a tricky
    system and the tests I wrote succeed on my machine, but on Travis-CI not
    even the simplest use of .SetFinalizer ever does anything.

    Do I just leave it in and assume that because it works here, it works
    everywhere?

    Or does using a main loop like this require users to .Close()? Then I'd
    rather use a lock.

    Here is Travis not running any finalizers (not even a very simple one on an
    integer):

    https://travis-ci.org/hraban/testtravis/builds/22570080

    This test passes every single time on my box. I tried crazy stuff to get the
    GC juices flowing, to no avail.

    It's a stripped down version of this project:

    https://github.com/hraban/lrucache

    Greetings,

    Hraban

    --
    You received this message because you are subscribed to a topic in the
    Google Groups "golang-nuts" group.
    To unsubscribe from this topic, visit
    https://groups.google.com/d/topic/golang-nuts/64SCXa0Ls0o/unsubscribe.
    To unsubscribe from this group and all its topics, send an email to
    [email protected].
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
  • Tamás Gulácsi at Apr 9, 2014 at 4:42 am
    Search for "atexit" on this list.
    The conclusion was: use .Close, document it, and the user can use defer to not forget it.

    Finalizers are not reliable.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
  • Dmitry Vyukov at Apr 9, 2014 at 6:54 am

    On Wed, Apr 9, 2014 at 3:43 AM, Hraban Luyat wrote:
    Hey guys,

    Long story short:

    - How do you clean up main loops in libraries?

    - How do you test code that uses SetFinalizer()?

    Background:

    Sometimes I use the "main loop" paradigm to make an object safe for
    concurrent use. Basically, you have a New() routine that creates a new
    object and starts a main loop in a goroutine. This main loop listens for
    operations on a channel and you expose that channel directly (or indirectly
    using accessor methods) to the user.

    Great.

    How do you clean this up when the object is no longer used? A .Close()
    method works. I'd rather avoid that if I can, because .Close is extra work
    (at the risk of forgetting it) for the user.

    Ideally I'd like to stop the main loop once the garbage collector reclaims
    the object.

    Now I have something which *I think* works: how do I test it? It's a tricky
    system and the tests I wrote succeed on my machine, but on Travis-CI not
    even the simplest use of .SetFinalizer ever does anything.

    Your "main loop" goroutine references the object, so it will never be
    garbage collected (finalizer will never run).
    You need to at least explicitly stop the main loop goroutine. And if
    you do it eplicitly with Close, it's simpler to cleanup everything
    else in that Close call.


    Do I just leave it in and assume that because it works here, it works
    everywhere?

    Or does using a main loop like this require users to .Close()? Then I'd
    rather use a lock.

    Here is Travis not running any finalizers (not even a very simple one on an
    integer):

    https://travis-ci.org/hraban/testtravis/builds/22570080

    This test passes every single time on my box. I tried crazy stuff to get the
    GC juices flowing, to no avail.

    It's a stripped down version of this project:

    https://github.com/hraban/lrucache

    Greetings,

    Hraban

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
  • Hraban Luyat at Apr 9, 2014 at 11:06 am
    Hi,

    Thanks for the replies. I'm sad you need .Close for something that
    uses goroutines this way, that makes using locks a better alternative
    from the user of your library's perspective.

    Just for the record: the main loop is not referencing the object
    (Cache struct), it's referencing a member (the operation channel). The
    struct goes out of scope at the end of leakingGoroutinesHelper().

    Greetings,

    Hraban

    2014-04-09 8:53 GMT+02:00 Dmitry Vyukov <[email protected]>:
    On Wed, Apr 9, 2014 at 3:43 AM, Hraban Luyat wrote:
    Hey guys,

    Long story short:

    - How do you clean up main loops in libraries?

    - How do you test code that uses SetFinalizer()?

    Background:

    Sometimes I use the "main loop" paradigm to make an object safe for
    concurrent use. Basically, you have a New() routine that creates a new
    object and starts a main loop in a goroutine. This main loop listens for
    operations on a channel and you expose that channel directly (or indirectly
    using accessor methods) to the user.

    Great.

    How do you clean this up when the object is no longer used? A .Close()
    method works. I'd rather avoid that if I can, because .Close is extra work
    (at the risk of forgetting it) for the user.

    Ideally I'd like to stop the main loop once the garbage collector reclaims
    the object.

    Now I have something which *I think* works: how do I test it? It's a tricky
    system and the tests I wrote succeed on my machine, but on Travis-CI not
    even the simplest use of .SetFinalizer ever does anything.

    Your "main loop" goroutine references the object, so it will never be
    garbage collected (finalizer will never run).
    You need to at least explicitly stop the main loop goroutine. And if
    you do it eplicitly with Close, it's simpler to cleanup everything
    else in that Close call.


    Do I just leave it in and assume that because it works here, it works
    everywhere?

    Or does using a main loop like this require users to .Close()? Then I'd
    rather use a lock.

    Here is Travis not running any finalizers (not even a very simple one on an
    integer):

    https://travis-ci.org/hraban/testtravis/builds/22570080

    This test passes every single time on my box. I tried crazy stuff to get the
    GC juices flowing, to no avail.

    It's a stripped down version of this project:

    https://github.com/hraban/lrucache

    Greetings,

    Hraban

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
  • Dmitry Vyukov at Apr 9, 2014 at 11:41 am

    On Wed, Apr 9, 2014 at 4:05 AM, Hraban Luyat wrote:
    Hi,

    Thanks for the replies. I'm sad you need .Close for something that
    uses goroutines this way, that makes using locks a better alternative
    from the user of your library's perspective.

    Just for the record: the main loop is not referencing the object
    (Cache struct), it's referencing a member (the operation channel). The
    struct goes out of scope at the end of leakingGoroutinesHelper().

    How the goroutine can act on the cache, if it's not referencing it?..
    Greetings,

    Hraban

    2014-04-09 8:53 GMT+02:00 Dmitry Vyukov <[email protected]>:
    On Wed, Apr 9, 2014 at 3:43 AM, Hraban Luyat wrote:
    Hey guys,

    Long story short:

    - How do you clean up main loops in libraries?

    - How do you test code that uses SetFinalizer()?

    Background:

    Sometimes I use the "main loop" paradigm to make an object safe for
    concurrent use. Basically, you have a New() routine that creates a new
    object and starts a main loop in a goroutine. This main loop listens for
    operations on a channel and you expose that channel directly (or indirectly
    using accessor methods) to the user.

    Great.

    How do you clean this up when the object is no longer used? A .Close()
    method works. I'd rather avoid that if I can, because .Close is extra work
    (at the risk of forgetting it) for the user.

    Ideally I'd like to stop the main loop once the garbage collector reclaims
    the object.

    Now I have something which *I think* works: how do I test it? It's a tricky
    system and the tests I wrote succeed on my machine, but on Travis-CI not
    even the simplest use of .SetFinalizer ever does anything.

    Your "main loop" goroutine references the object, so it will never be
    garbage collected (finalizer will never run).
    You need to at least explicitly stop the main loop goroutine. And if
    you do it eplicitly with Close, it's simpler to cleanup everything
    else in that Close call.


    Do I just leave it in and assume that because it works here, it works
    everywhere?

    Or does using a main loop like this require users to .Close()? Then I'd
    rather use a lock.

    Here is Travis not running any finalizers (not even a very simple one on an
    integer):

    https://travis-ci.org/hraban/testtravis/builds/22570080

    This test passes every single time on my box. I tried crazy stuff to get the
    GC juices flowing, to no avail.

    It's a stripped down version of this project:

    https://github.com/hraban/lrucache

    Greetings,

    Hraban

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
  • Hraban Luyat at Apr 9, 2014 at 11:48 am
    It's getting a reference to the cache wrapped in every operation (see
    func (c *Cache) Msg()). This reference is kept by the user of the
    library (leakingGoroutinesHelper() in this case).

    Once the user deletes his reference to the object, no references are
    left and the object should clean itself up. As you would expect from
    something that, e.g. uses locks.

    This untestability of SetFinalizer makes me wonder: what /is/ a good
    use-case for SetFinalizer? If it cannot be tested, why does it exist?

    Greetings,

    Hraban

    2014-04-09 13:40 GMT+02:00 Dmitry Vyukov <[email protected]>:
    On Wed, Apr 9, 2014 at 4:05 AM, Hraban Luyat wrote:
    Hi,

    Thanks for the replies. I'm sad you need .Close for something that
    uses goroutines this way, that makes using locks a better alternative
    from the user of your library's perspective.

    Just for the record: the main loop is not referencing the object
    (Cache struct), it's referencing a member (the operation channel). The
    struct goes out of scope at the end of leakingGoroutinesHelper().

    How the goroutine can act on the cache, if it's not referencing it?..
    Greetings,

    Hraban

    2014-04-09 8:53 GMT+02:00 Dmitry Vyukov <[email protected]>:
    On Wed, Apr 9, 2014 at 3:43 AM, Hraban Luyat wrote:
    Hey guys,

    Long story short:

    - How do you clean up main loops in libraries?

    - How do you test code that uses SetFinalizer()?

    Background:

    Sometimes I use the "main loop" paradigm to make an object safe for
    concurrent use. Basically, you have a New() routine that creates a new
    object and starts a main loop in a goroutine. This main loop listens for
    operations on a channel and you expose that channel directly (or indirectly
    using accessor methods) to the user.

    Great.

    How do you clean this up when the object is no longer used? A .Close()
    method works. I'd rather avoid that if I can, because .Close is extra work
    (at the risk of forgetting it) for the user.

    Ideally I'd like to stop the main loop once the garbage collector reclaims
    the object.

    Now I have something which *I think* works: how do I test it? It's a tricky
    system and the tests I wrote succeed on my machine, but on Travis-CI not
    even the simplest use of .SetFinalizer ever does anything.

    Your "main loop" goroutine references the object, so it will never be
    garbage collected (finalizer will never run).
    You need to at least explicitly stop the main loop goroutine. And if
    you do it eplicitly with Close, it's simpler to cleanup everything
    else in that Close call.


    Do I just leave it in and assume that because it works here, it works
    everywhere?

    Or does using a main loop like this require users to .Close()? Then I'd
    rather use a lock.

    Here is Travis not running any finalizers (not even a very simple one on an
    integer):

    https://travis-ci.org/hraban/testtravis/builds/22570080

    This test passes every single time on my box. I tried crazy stuff to get the
    GC juices flowing, to no avail.

    It's a stripped down version of this project:

    https://github.com/hraban/lrucache

    Greetings,

    Hraban

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
  • Dmitry Vyukov at Apr 9, 2014 at 11:51 am

    On Wed, Apr 9, 2014 at 4:47 AM, Hraban Luyat wrote:
    It's getting a reference to the cache wrapped in every operation (see
    func (c *Cache) Msg()). This reference is kept by the user of the
    library (leakingGoroutinesHelper() in this case).
    I've failed to find the sources.

    Once the user deletes his reference to the object, no references are
    left and the object should clean itself up. As you would expect from
    something that, e.g. uses locks.

    This untestability of SetFinalizer makes me wonder: what /is/ a good
    use-case for SetFinalizer? If it cannot be tested, why does it exist?
    We have several tests for finalizers in std lib:
    http://golang.org/search?q=SetFinalizer



    Greetings,

    Hraban

    2014-04-09 13:40 GMT+02:00 Dmitry Vyukov <[email protected]>:
    On Wed, Apr 9, 2014 at 4:05 AM, Hraban Luyat wrote:
    Hi,

    Thanks for the replies. I'm sad you need .Close for something that
    uses goroutines this way, that makes using locks a better alternative
    from the user of your library's perspective.

    Just for the record: the main loop is not referencing the object
    (Cache struct), it's referencing a member (the operation channel). The
    struct goes out of scope at the end of leakingGoroutinesHelper().

    How the goroutine can act on the cache, if it's not referencing it?..
    Greetings,

    Hraban

    2014-04-09 8:53 GMT+02:00 Dmitry Vyukov <[email protected]>:
    On Wed, Apr 9, 2014 at 3:43 AM, Hraban Luyat wrote:
    Hey guys,

    Long story short:

    - How do you clean up main loops in libraries?

    - How do you test code that uses SetFinalizer()?

    Background:

    Sometimes I use the "main loop" paradigm to make an object safe for
    concurrent use. Basically, you have a New() routine that creates a new
    object and starts a main loop in a goroutine. This main loop listens for
    operations on a channel and you expose that channel directly (or indirectly
    using accessor methods) to the user.

    Great.

    How do you clean this up when the object is no longer used? A .Close()
    method works. I'd rather avoid that if I can, because .Close is extra work
    (at the risk of forgetting it) for the user.

    Ideally I'd like to stop the main loop once the garbage collector reclaims
    the object.

    Now I have something which *I think* works: how do I test it? It's a tricky
    system and the tests I wrote succeed on my machine, but on Travis-CI not
    even the simplest use of .SetFinalizer ever does anything.

    Your "main loop" goroutine references the object, so it will never be
    garbage collected (finalizer will never run).
    You need to at least explicitly stop the main loop goroutine. And if
    you do it eplicitly with Close, it's simpler to cleanup everything
    else in that Close call.


    Do I just leave it in and assume that because it works here, it works
    everywhere?

    Or does using a main loop like this require users to .Close()? Then I'd
    rather use a lock.

    Here is Travis not running any finalizers (not even a very simple one on an
    integer):

    https://travis-ci.org/hraban/testtravis/builds/22570080

    This test passes every single time on my box. I tried crazy stuff to get the
    GC juices flowing, to no avail.

    It's a stripped down version of this project:

    https://github.com/hraban/lrucache

    Greetings,

    Hraban

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
    For more options, visit https://groups.google.com/d/optout.
  • Hraban Luyat at Apr 9, 2014 at 11:56 am

    2014-04-09 13:51 GMT+02:00 Dmitry Vyukov <[email protected]>:
    On Wed, Apr 9, 2014 at 4:47 AM, Hraban Luyat wrote:
    It's getting a reference to the cache wrapped in every operation (see
    func (c *Cache) Msg()). This reference is kept by the user of the
    library (leakingGoroutinesHelper() in this case).
    I've failed to find the sources.
    Sorry the version from the failing Travis runs we're talking about is here:

    https://github.com/hraban/testtravis/blob/cfa8e479f62081f28bf75abeb1cdef9e6754a124/lrucache.go

    https://github.com/hraban/testtravis/blob/cfa8e479f62081f28bf75abeb1cdef9e6754a124/lrucache_test.go

    Sincerely,

    Hraban

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedApr 8, '14 at 11:43p
activeApr 9, '14 at 11:56a
posts9
users3
websitegolang.org

People

Translate

site design / logo © 2023 Grokbase