FAQ
Hi.

There's a ThreadLocal field in SegmentReader (it's called termVectorsLocal).
Some value is put to it, but it's never cleared.
Is it ok? It looks like sometimes this behavior may lead to leaks.

This is the same in lucene-2.2.0 and lucene-2.3.2.

Thanks in advance.
--
View this message in context: http://www.nabble.com/ThreadLocal-in-SegmentReader-tp18306230p18306230.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Search Discussions

  • Michael McCandless at Jul 7, 2008 at 10:50 am
    Well ... if the thread dies, the value in its ThreadLocal should be
    GC'd.

    If the thread does not die (eg thread pool in an app server) then the
    ThreadLocal value remains, but that value is a shallow clone of the
    original TermVectorsReader and should not be consuming that much RAM
    per thread.

    So I think it's OK?

    Mike

    Roman Puchkovskiy wrote:
    Hi.

    There's a ThreadLocal field in SegmentReader (it's called
    termVectorsLocal).
    Some value is put to it, but it's never cleared.
    Is it ok? It looks like sometimes this behavior may lead to leaks.

    This is the same in lucene-2.2.0 and lucene-2.3.2.

    Thanks in advance.
    --
    View this message in context: http://www.nabble.com/ThreadLocal-in-SegmentReader-tp18306230p18306230.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Roman Puchkovskiy at Jul 7, 2008 at 11:07 am
    Unfortunately, it's not ok sometimes. For instance, when Lucene is loaded by
    a web-application from its WEB-INF/lib and SegmentReader is initialized
    during the application start-up (i.e. it's initialized in the thread which
    will never die), this causes problems with unloading of a classloader of the
    web-application. When web-application is undeployed, there's still a
    ThreadLocal in a thread which is external to webapp classloader, and this
    ThreadLocal contains an object which references its class, and this class
    was loaded through web-app classloader and hence references it... so we have
    a chain of hard links from an alive thread to our classloader. In the
    result, the classloader cannot be unloaded together will all its classes, so
    memory waste is considerable.

    I've found a way to work this around by creating a new thread during webapp
    start-up and executing code which eventually initializes Lucene indices from
    this thread, so all spawned ThreadLocals go to this short-lived thread and
    get garbage-collected shortly after the webapp start-up is finished. But
    this does not seem to be a pretty solution. Besides, one has to guess that
    ThreadLocals are the problem to invent such a work-around :)


    Michael McCandless-2 wrote:

    Well ... if the thread dies, the value in its ThreadLocal should be
    GC'd.

    If the thread does not die (eg thread pool in an app server) then the
    ThreadLocal value remains, but that value is a shallow clone of the
    original TermVectorsReader and should not be consuming that much RAM
    per thread.

    So I think it's OK?

    Mike

    Roman Puchkovskiy wrote:
    Hi.

    There's a ThreadLocal field in SegmentReader (it's called
    termVectorsLocal).
    Some value is put to it, but it's never cleared.
    Is it ok? It looks like sometimes this behavior may lead to leaks.

    This is the same in lucene-2.2.0 and lucene-2.3.2.

    Thanks in advance.
    --
    View this message in context:
    http://www.nabble.com/ThreadLocal-in-SegmentReader-tp18306230p18306230.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    --
    View this message in context: http://www.nabble.com/ThreadLocal-in-SegmentReader-tp18306230p18314310.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Michael McCandless at Jul 7, 2008 at 11:34 am
    Hmmm, I see... you're right. ThreadLocal is dangerous.

    So how would you recommend fixing it?

    One thing we can do, in SegmentReader.close, is to call
    termVectorsLocal.set(null). We do this eg in FieldsReader.close,
    which uses a ThreadLocal to hold thread-private clones of the
    fieldsStream

    However, that only works if the very same thread that had opened the
    reader also calls close... which likely often is the case but in
    general is not guaranteed and we should not expect/require.

    How about if we set termVectorsLocal itself to null? Will GC then "do
    the right thing", ie, recognize (eventually) that this ThreadLocal
    instance is no longer referenced, and clear all Objects for all
    threads that were held in it?

    Mike

    Roman Puchkovskiy wrote:
    Unfortunately, it's not ok sometimes. For instance, when Lucene is
    loaded by
    a web-application from its WEB-INF/lib and SegmentReader is
    initialized
    during the application start-up (i.e. it's initialized in the thread
    which
    will never die), this causes problems with unloading of a
    classloader of the
    web-application. When web-application is undeployed, there's still a
    ThreadLocal in a thread which is external to webapp classloader, and
    this
    ThreadLocal contains an object which references its class, and this
    class
    was loaded through web-app classloader and hence references it... so
    we have
    a chain of hard links from an alive thread to our classloader. In the
    result, the classloader cannot be unloaded together will all its
    classes, so
    memory waste is considerable.

    I've found a way to work this around by creating a new thread during
    webapp
    start-up and executing code which eventually initializes Lucene
    indices from
    this thread, so all spawned ThreadLocals go to this short-lived
    thread and
    get garbage-collected shortly after the webapp start-up is finished.
    But
    this does not seem to be a pretty solution. Besides, one has to
    guess that
    ThreadLocals are the problem to invent such a work-around :)


    Michael McCandless-2 wrote:

    Well ... if the thread dies, the value in its ThreadLocal should be
    GC'd.

    If the thread does not die (eg thread pool in an app server) then the
    ThreadLocal value remains, but that value is a shallow clone of the
    original TermVectorsReader and should not be consuming that much RAM
    per thread.

    So I think it's OK?

    Mike

    Roman Puchkovskiy wrote:
    Hi.

    There's a ThreadLocal field in SegmentReader (it's called
    termVectorsLocal).
    Some value is put to it, but it's never cleared.
    Is it ok? It looks like sometimes this behavior may lead to leaks.

    This is the same in lucene-2.2.0 and lucene-2.3.2.

    Thanks in advance.
    --
    View this message in context:
    http://www.nabble.com/ThreadLocal-in-SegmentReader-tp18306230p18306230.html
    Sent from the Lucene - Java Users mailing list archive at
    Nabble.com.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    --
    View this message in context: http://www.nabble.com/ThreadLocal-in-SegmentReader-tp18306230p18314310.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Roman Puchkovskiy at Jul 7, 2008 at 11:50 am
    Yes, calling set(null) does not seem a good fix. As for setting a reference
    to termVectorsLocal to null, not sure could this help or not, as this
    ThreadLocal will still be referenced by the thread (or threads).
    Anyway, I will try to test this approach and post the results here.


    Michael McCandless-2 wrote:

    Hmmm, I see... you're right. ThreadLocal is dangerous.

    So how would you recommend fixing it?

    One thing we can do, in SegmentReader.close, is to call
    termVectorsLocal.set(null). We do this eg in FieldsReader.close,
    which uses a ThreadLocal to hold thread-private clones of the
    fieldsStream

    However, that only works if the very same thread that had opened the
    reader also calls close... which likely often is the case but in
    general is not guaranteed and we should not expect/require.

    How about if we set termVectorsLocal itself to null? Will GC then "do
    the right thing", ie, recognize (eventually) that this ThreadLocal
    instance is no longer referenced, and clear all Objects for all
    threads that were held in it?

    Mike

    Roman Puchkovskiy wrote:
    Unfortunately, it's not ok sometimes. For instance, when Lucene is
    loaded by
    a web-application from its WEB-INF/lib and SegmentReader is
    initialized
    during the application start-up (i.e. it's initialized in the thread
    which
    will never die), this causes problems with unloading of a
    classloader of the
    web-application. When web-application is undeployed, there's still a
    ThreadLocal in a thread which is external to webapp classloader, and
    this
    ThreadLocal contains an object which references its class, and this
    class
    was loaded through web-app classloader and hence references it... so
    we have
    a chain of hard links from an alive thread to our classloader. In the
    result, the classloader cannot be unloaded together will all its
    classes, so
    memory waste is considerable.

    I've found a way to work this around by creating a new thread during
    webapp
    start-up and executing code which eventually initializes Lucene
    indices from
    this thread, so all spawned ThreadLocals go to this short-lived
    thread and
    get garbage-collected shortly after the webapp start-up is finished.
    But
    this does not seem to be a pretty solution. Besides, one has to
    guess that
    ThreadLocals are the problem to invent such a work-around :)


    Michael McCandless-2 wrote:

    Well ... if the thread dies, the value in its ThreadLocal should be
    GC'd.

    If the thread does not die (eg thread pool in an app server) then the
    ThreadLocal value remains, but that value is a shallow clone of the
    original TermVectorsReader and should not be consuming that much RAM
    per thread.

    So I think it's OK?

    Mike

    Roman Puchkovskiy wrote:
    Hi.

    There's a ThreadLocal field in SegmentReader (it's called
    termVectorsLocal).
    Some value is put to it, but it's never cleared.
    Is it ok? It looks like sometimes this behavior may lead to leaks.

    This is the same in lucene-2.2.0 and lucene-2.3.2.

    Thanks in advance.
    --
    View this message in context:
    http://www.nabble.com/ThreadLocal-in-SegmentReader-tp18306230p18306230.html
    Sent from the Lucene - Java Users mailing list archive at
    Nabble.com.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    --
    View this message in context:
    http://www.nabble.com/ThreadLocal-in-SegmentReader-tp18306230p18314310.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    --
    View this message in context: http://www.nabble.com/ThreadLocal-in-SegmentReader-tp18306230p18314959.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Roman Puchkovskiy at Jul 7, 2008 at 2:21 pm
    I've tested a little, and it seems that assigning a null is not sufficient.
    As expected...
    I don't see other ways how to fix this, but I'm not the Lucene developer :)
    Fortunately, there's the work-around with temporary thread.


    Michael McCandless-2 wrote:

    Hmmm, I see... you're right. ThreadLocal is dangerous.

    So how would you recommend fixing it?

    One thing we can do, in SegmentReader.close, is to call
    termVectorsLocal.set(null). We do this eg in FieldsReader.close,
    which uses a ThreadLocal to hold thread-private clones of the
    fieldsStream

    However, that only works if the very same thread that had opened the
    reader also calls close... which likely often is the case but in
    general is not guaranteed and we should not expect/require.

    How about if we set termVectorsLocal itself to null? Will GC then "do
    the right thing", ie, recognize (eventually) that this ThreadLocal
    instance is no longer referenced, and clear all Objects for all
    threads that were held in it?

    Mike

    Roman Puchkovskiy wrote:
    Unfortunately, it's not ok sometimes. For instance, when Lucene is
    loaded by
    a web-application from its WEB-INF/lib and SegmentReader is
    initialized
    during the application start-up (i.e. it's initialized in the thread
    which
    will never die), this causes problems with unloading of a
    classloader of the
    web-application. When web-application is undeployed, there's still a
    ThreadLocal in a thread which is external to webapp classloader, and
    this
    ThreadLocal contains an object which references its class, and this
    class
    was loaded through web-app classloader and hence references it... so
    we have
    a chain of hard links from an alive thread to our classloader. In the
    result, the classloader cannot be unloaded together will all its
    classes, so
    memory waste is considerable.

    I've found a way to work this around by creating a new thread during
    webapp
    start-up and executing code which eventually initializes Lucene
    indices from
    this thread, so all spawned ThreadLocals go to this short-lived
    thread and
    get garbage-collected shortly after the webapp start-up is finished.
    But
    this does not seem to be a pretty solution. Besides, one has to
    guess that
    ThreadLocals are the problem to invent such a work-around :)


    Michael McCandless-2 wrote:

    Well ... if the thread dies, the value in its ThreadLocal should be
    GC'd.

    If the thread does not die (eg thread pool in an app server) then the
    ThreadLocal value remains, but that value is a shallow clone of the
    original TermVectorsReader and should not be consuming that much RAM
    per thread.

    So I think it's OK?

    Mike

    Roman Puchkovskiy wrote:
    Hi.

    There's a ThreadLocal field in SegmentReader (it's called
    termVectorsLocal).
    Some value is put to it, but it's never cleared.
    Is it ok? It looks like sometimes this behavior may lead to leaks.

    This is the same in lucene-2.2.0 and lucene-2.3.2.

    Thanks in advance.
    --
    View this message in context:
    http://www.nabble.com/ThreadLocal-in-SegmentReader-tp18306230p18306230.html
    Sent from the Lucene - Java Users mailing list archive at
    Nabble.com.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    --
    View this message in context:
    http://www.nabble.com/ThreadLocal-in-SegmentReader-tp18306230p18314310.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    --
    View this message in context: http://www.nabble.com/ThreadLocal-in-SegmentReader-tp18306230p18317640.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Michael McCandless at Jul 7, 2008 at 6:43 pm
    So now I'm confused: the SegmentReader itself should no longer be
    reachable, assuming you are not holding any references to your
    IndexReader.

    Which means the ThreadLocal instance should no longer be reachable.

    Which means it should be GC'd and everything it's holding should be
    GC'd as well, even if some threads under it are still alive.

    I think?

    So it seems like what we currently do should be working fine, yet it's
    apparently not. Are you certain you don't keep a reference to your
    IndexReader somewhere? Also, are you sure that you waited long enough
    for GC to reclaim these objects?

    Mike

    Roman Puchkovskiy wrote:
    I've tested a little, and it seems that assigning a null is not
    sufficient.
    As expected...
    I don't see other ways how to fix this, but I'm not the Lucene
    developer :)
    Fortunately, there's the work-around with temporary thread.


    Michael McCandless-2 wrote:

    Hmmm, I see... you're right. ThreadLocal is dangerous.

    So how would you recommend fixing it?

    One thing we can do, in SegmentReader.close, is to call
    termVectorsLocal.set(null). We do this eg in FieldsReader.close,
    which uses a ThreadLocal to hold thread-private clones of the
    fieldsStream

    However, that only works if the very same thread that had opened the
    reader also calls close... which likely often is the case but in
    general is not guaranteed and we should not expect/require.

    How about if we set termVectorsLocal itself to null? Will GC then
    "do
    the right thing", ie, recognize (eventually) that this ThreadLocal
    instance is no longer referenced, and clear all Objects for all
    threads that were held in it?

    Mike

    Roman Puchkovskiy wrote:
    Unfortunately, it's not ok sometimes. For instance, when Lucene is
    loaded by
    a web-application from its WEB-INF/lib and SegmentReader is
    initialized
    during the application start-up (i.e. it's initialized in the thread
    which
    will never die), this causes problems with unloading of a
    classloader of the
    web-application. When web-application is undeployed, there's still a
    ThreadLocal in a thread which is external to webapp classloader, and
    this
    ThreadLocal contains an object which references its class, and this
    class
    was loaded through web-app classloader and hence references it... so
    we have
    a chain of hard links from an alive thread to our classloader. In
    the
    result, the classloader cannot be unloaded together will all its
    classes, so
    memory waste is considerable.

    I've found a way to work this around by creating a new thread during
    webapp
    start-up and executing code which eventually initializes Lucene
    indices from
    this thread, so all spawned ThreadLocals go to this short-lived
    thread and
    get garbage-collected shortly after the webapp start-up is finished.
    But
    this does not seem to be a pretty solution. Besides, one has to
    guess that
    ThreadLocals are the problem to invent such a work-around :)


    Michael McCandless-2 wrote:

    Well ... if the thread dies, the value in its ThreadLocal should be
    GC'd.

    If the thread does not die (eg thread pool in an app server) then
    the
    ThreadLocal value remains, but that value is a shallow clone of the
    original TermVectorsReader and should not be consuming that much
    RAM
    per thread.

    So I think it's OK?

    Mike

    Roman Puchkovskiy wrote:
    Hi.

    There's a ThreadLocal field in SegmentReader (it's called
    termVectorsLocal).
    Some value is put to it, but it's never cleared.
    Is it ok? It looks like sometimes this behavior may lead to leaks.

    This is the same in lucene-2.2.0 and lucene-2.3.2.

    Thanks in advance.
    --
    View this message in context:
    http://www.nabble.com/ThreadLocal-in-SegmentReader-tp18306230p18306230.html
    Sent from the Lucene - Java Users mailing list archive at
    Nabble.com.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    --
    View this message in context:
    http://www.nabble.com/ThreadLocal-in-SegmentReader-tp18306230p18314310.html
    Sent from the Lucene - Java Users mailing list archive at
    Nabble.com.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    --
    View this message in context: http://www.nabble.com/ThreadLocal-in-SegmentReader-tp18306230p18317640.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Yonik Seeley at Jul 7, 2008 at 7:31 pm

    On Mon, Jul 7, 2008 at 2:43 PM, Michael McCandless wrote:
    So now I'm confused: the SegmentReader itself should no longer be reachable,
    assuming you are not holding any references to your IndexReader.

    Which means the ThreadLocal instance should no longer be reachable.
    It will still be referenced from the Thread(s) ThreadLocalMap
    The key (the ThreadLocal) will be weakly referenced, but the values
    (now stale) are strongly referenced and won't be actually removed
    until the table is resized (under the Java6 impl at least).
    Nice huh?

    -Yonik

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Michael McCandless at Jul 7, 2008 at 8:56 pm
    Ugh! I'll move this to java-dev to brainstorm fixes...

    Mike

    Yonik Seeley wrote:
    On Mon, Jul 7, 2008 at 2:43 PM, Michael McCandless
    wrote:
    So now I'm confused: the SegmentReader itself should no longer be
    reachable,
    assuming you are not holding any references to your IndexReader.

    Which means the ThreadLocal instance should no longer be reachable.
    It will still be referenced from the Thread(s) ThreadLocalMap
    The key (the ThreadLocal) will be weakly referenced, but the values
    (now stale) are strongly referenced and won't be actually removed
    until the table is resized (under the Java6 impl at least).
    Nice huh?

    -Yonik

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Michael McCandless at Jul 11, 2008 at 10:29 am
    After discussing this on java-dev:

    http://mail-archives.apache.org/mod_mbox/lucene-java-dev/200807.mbox/%3CF5FC94B2-E5C7-40C0-8B73-E12245B91CEE@mikemccandless.com%3E

    it seems that this is not in fact a leak but rather a delayed GC
    issue. The objects are eventually freed, on Sun 1.4, 1.5 and 1.6.

    When a ThreadLocal instance becomes unreferenced & GC'd, Java does not
    immediately reclaim the now-unreferenced objects, if the thread
    remains alive. However, over time, as other ThreadLocals are used
    with that thread, those stale objects do eventually get reclaimed. So
    my feeling for now is it's OK for Lucene to continue to use non-static
    ThreadLocals.

    Mike

    Yonik Seeley wrote:
    On Mon, Jul 7, 2008 at 2:43 PM, Michael McCandless
    wrote:
    So now I'm confused: the SegmentReader itself should no longer be
    reachable,
    assuming you are not holding any references to your IndexReader.

    Which means the ThreadLocal instance should no longer be reachable.
    It will still be referenced from the Thread(s) ThreadLocalMap
    The key (the ThreadLocal) will be weakly referenced, but the values
    (now stale) are strongly referenced and won't be actually removed
    until the table is resized (under the Java6 impl at least).
    Nice huh?

    -Yonik

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Roman Puchkovskiy at Jul 12, 2008 at 4:38 am
    Well, this 'replacement' of the ThreadLocal does not solve the initial
    problem. As there's always at least one ThreadLocal which binds the object
    loaded by the web-app to the Thread which is _not_ loaded by the web-app,
    the classloader never may be unloaded.

    You are right, this is not the 'leak' in the usual sense, but this is not
    the 'delayed GC' problem, this is the 'never-happening-GC' problem when you
    have the classloader which needs to be unloaded.


    Michael McCandless-2 wrote:

    After discussing this on java-dev:


    http://mail-archives.apache.org/mod_mbox/lucene-java-dev/200807.mbox/%3CF5FC94B2-E5C7-40C0-8B73-E12245B91CEE@mikemccandless.com%3E

    it seems that this is not in fact a leak but rather a delayed GC
    issue. The objects are eventually freed, on Sun 1.4, 1.5 and 1.6.

    When a ThreadLocal instance becomes unreferenced & GC'd, Java does not
    immediately reclaim the now-unreferenced objects, if the thread
    remains alive. However, over time, as other ThreadLocals are used
    with that thread, those stale objects do eventually get reclaimed. So
    my feeling for now is it's OK for Lucene to continue to use non-static
    ThreadLocals.

    Mike

    Yonik Seeley wrote:
    On Mon, Jul 7, 2008 at 2:43 PM, Michael McCandless
    wrote:
    So now I'm confused: the SegmentReader itself should no longer be
    reachable,
    assuming you are not holding any references to your IndexReader.

    Which means the ThreadLocal instance should no longer be reachable.
    It will still be referenced from the Thread(s) ThreadLocalMap
    The key (the ThreadLocal) will be weakly referenced, but the values
    (now stale) are strongly referenced and won't be actually removed
    until the table is resized (under the Java6 impl at least).
    Nice huh?

    -Yonik

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    --
    View this message in context: http://www.nabble.com/ThreadLocal-in-SegmentReader-tp18306230p18416048.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-user @
categorieslucene
postedJul 6, '08 at 8:33p
activeJul 12, '08 at 4:38a
posts11
users3
websitelucene.apache.org

People

Translate

site design / logo © 2022 Grokbase