FAQ
My environment:

Lucene 2.3.2 on Linux, Java 1.6.0_07-b06, running under Tomcat 5.5.26

What I'm trying to do seems pretty simple, but is causing a headache
which I can't sleuth out. When I try to build an IndexSearcher using
an IndexReader, the IndexReader.open( String_to_index_dir ) call
sometimes creates a reader with a SegmentReader impl. Other times a
MultiSegmentReader will be created. The SegmentReader impl when used
returns a much reduced or empty result set from searchers; the
MultiSegmentReader returns the right and expected results from the
index.

The indexes I'm using were built initially optimized, but will not
remain so until they are rebuilt from scratch (usually weekly). The
indexes are being constantly updated and searched on in the Tomcat
environment.

I noticed in the guts of the DirectoryIndexReader it will seem to pick
SegmentReader over MultiSegmentReader if the index is optimized. Is
there any way to make this code always return a MultiSegmentReader?
Why is one reader failing over the other? This appears to be a bug
(and a nasty one for me).

Any help or guidance would be much appreciated as I've already spent a
day trying to dig through these mechanics and am a little lost.
Please no, "Upgrade to 2.4 and then see what happens." 2.4 appears to
include a lot of interesting things but also deprecates a lot of API
which I do not have time to dig into or refactor.

thanks,
Micah

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Search Discussions

  • Michael McCandless at Jan 22, 2009 at 10:58 pm
    IndexReader.open opens the latest segments_N file. If that file
    references only 1 segment, a SegmentReader is returned, else a
    MultiSegmentReader.

    I'm confused why you see the SegmentReader impl giving too few results
    -- that should only be returned if your index legitimately has only 1
    segment. Are you sure the SegmentReader impl is not returning the
    right results? Do you have an example index which IndexReader.open
    consistently incorrectly returns a SegmentReader?

    Is an IndexWriter changing the index when you call IndexReader.open?

    Mike

    Micah Jaffe wrote:
    My environment:

    Lucene 2.3.2 on Linux, Java 1.6.0_07-b06, running under Tomcat 5.5.26

    What I'm trying to do seems pretty simple, but is causing a headache
    which I can't sleuth out. When I try to build an IndexSearcher
    using an IndexReader, the IndexReader.open( String_to_index_dir )
    call sometimes creates a reader with a SegmentReader impl. Other
    times a MultiSegmentReader will be created. The SegmentReader impl
    when used returns a much reduced or empty result set from searchers;
    the MultiSegmentReader returns the right and expected results from
    the index.

    The indexes I'm using were built initially optimized, but will not
    remain so until they are rebuilt from scratch (usually weekly). The
    indexes are being constantly updated and searched on in the Tomcat
    environment.

    I noticed in the guts of the DirectoryIndexReader it will seem to
    pick SegmentReader over MultiSegmentReader if the index is
    optimized. Is there any way to make this code always return a
    MultiSegmentReader? Why is one reader failing over the other? This
    appears to be a bug (and a nasty one for me).

    Any help or guidance would be much appreciated as I've already spent
    a day trying to dig through these mechanics and am a little lost.
    Please no, "Upgrade to 2.4 and then see what happens." 2.4 appears
    to include a lot of interesting things but also deprecates a lot of
    API which I do not have time to dig into or refactor.

    thanks,
    Micah

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Micah Jaffe at Jan 23, 2009 at 10:06 pm
    Hi, thanks for responding. A quick follow-up, the problem turned out
    to be more complex than originally explained, and it was not in fact a
    problem with buggy behavior in Lucene. A static nested class created
    a fugly point where I was holding/using the wrong reader at the time
    of failure.

    -M
    On Jan 22, 2009, at 2:57 PM, Michael McCandless wrote:


    IndexReader.open opens the latest segments_N file. If that file
    references only 1 segment, a SegmentReader is returned, else a
    MultiSegmentReader.

    I'm confused why you see the SegmentReader impl giving too few
    results -- that should only be returned if your index legitimately
    has only 1 segment. Are you sure the SegmentReader impl is not
    returning the right results? Do you have an example index which
    IndexReader.open consistently incorrectly returns a SegmentReader?

    Is an IndexWriter changing the index when you call IndexReader.open?

    Mike

    Micah Jaffe wrote:
    My environment:

    Lucene 2.3.2 on Linux, Java 1.6.0_07-b06, running under Tomcat 5.5.26

    What I'm trying to do seems pretty simple, but is causing a
    headache which I can't sleuth out. When I try to build an
    IndexSearcher using an IndexReader, the
    IndexReader.open( String_to_index_dir ) call sometimes creates a
    reader with a SegmentReader impl. Other times a MultiSegmentReader
    will be created. The SegmentReader impl when used returns a much
    reduced or empty result set from searchers; the MultiSegmentReader
    returns the right and expected results from the index.

    The indexes I'm using were built initially optimized, but will not
    remain so until they are rebuilt from scratch (usually weekly).
    The indexes are being constantly updated and searched on in the
    Tomcat environment.

    I noticed in the guts of the DirectoryIndexReader it will seem to
    pick SegmentReader over MultiSegmentReader if the index is
    optimized. Is there any way to make this code always return a
    MultiSegmentReader? Why is one reader failing over the other?
    This appears to be a bug (and a nasty one for me).

    Any help or guidance would be much appreciated as I've already
    spent a day trying to dig through these mechanics and am a little
    lost. Please no, "Upgrade to 2.4 and then see what happens." 2.4
    appears to include a lot of interesting things but also deprecates
    a lot of API which I do not have time to dig into or refactor.

    thanks,
    Micah

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Michael McCandless at Jan 23, 2009 at 10:09 pm
    OK, phew! Thanks for bringing closure.

    Mike

    Micah Jaffe wrote:
    Hi, thanks for responding. A quick follow-up, the problem turned
    out to be more complex than originally explained, and it was not in
    fact a problem with buggy behavior in Lucene. A static nested class
    created a fugly point where I was holding/using the wrong reader at
    the time of failure.

    -M
    On Jan 22, 2009, at 2:57 PM, Michael McCandless wrote:


    IndexReader.open opens the latest segments_N file. If that file
    references only 1 segment, a SegmentReader is returned, else a
    MultiSegmentReader.

    I'm confused why you see the SegmentReader impl giving too few
    results -- that should only be returned if your index legitimately
    has only 1 segment. Are you sure the SegmentReader impl is not
    returning the right results? Do you have an example index which
    IndexReader.open consistently incorrectly returns a SegmentReader?

    Is an IndexWriter changing the index when you call IndexReader.open?

    Mike

    Micah Jaffe wrote:
    My environment:

    Lucene 2.3.2 on Linux, Java 1.6.0_07-b06, running under Tomcat
    5.5.26

    What I'm trying to do seems pretty simple, but is causing a
    headache which I can't sleuth out. When I try to build an
    IndexSearcher using an IndexReader, the
    IndexReader.open( String_to_index_dir ) call sometimes creates a
    reader with a SegmentReader impl. Other times a
    MultiSegmentReader will be created. The SegmentReader impl when
    used returns a much reduced or empty result set from searchers;
    the MultiSegmentReader returns the right and expected results from
    the index.

    The indexes I'm using were built initially optimized, but will not
    remain so until they are rebuilt from scratch (usually weekly).
    The indexes are being constantly updated and searched on in the
    Tomcat environment.

    I noticed in the guts of the DirectoryIndexReader it will seem to
    pick SegmentReader over MultiSegmentReader if the index is
    optimized. Is there any way to make this code always return a
    MultiSegmentReader? Why is one reader failing over the other?
    This appears to be a bug (and a nasty one for me).

    Any help or guidance would be much appreciated as I've already
    spent a day trying to dig through these mechanics and am a little
    lost. Please no, "Upgrade to 2.4 and then see what happens." 2.4
    appears to include a lot of interesting things but also deprecates
    a lot of API which I do not have time to dig into or refactor.

    thanks,
    Micah

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-user @
categorieslucene
postedJan 22, '09 at 7:36p
activeJan 23, '09 at 10:09p
posts4
users2
websitelucene.apache.org

2 users in discussion

Micah Jaffe: 2 posts Michael McCandless: 2 posts

People

Translate

site design / logo © 2022 Grokbase