FAQ
Hi Lucene Gurus:

More of a performance question:

When you pass a Filter to a searcher to do a search, the searcher is
basically doing the full search and then intersect against the bitset given
by the filter. This seems wasteful when there are lotsa hits returned by the
scorer and filter only has only a few docs, e.g.

Running a MatchAllDocsQuery given a Filter with only 1 doc.

Right now Scorers don't know about filters, only the searchers do. And
scorers are doing most of the computation, but it seems scorers should make
use of the filters as the corpus before doing any score computation.

Seems to me this is an area that deserves some attention.

Thoughts?

Thanks

-John

Search Discussions

  • Yonik Seeley at Aug 10, 2007 at 6:27 pm

    On 8/10/07, John Wang wrote:
    Hi Lucene Gurus:

    More of a performance question:

    When you pass a Filter to a searcher to do a search, the searcher is
    basically doing the full search and then intersect against the bitset given
    by the filter. This seems wasteful when there are lotsa hits returned by the
    scorer and filter only has only a few docs, e.g.

    Running a MatchAllDocsQuery given a Filter with only 1 doc.

    Right now Scorers don't know about filters, only the searchers do. And
    scorers are doing most of the computation, but it seems scorers should make
    use of the filters as the corpus before doing any score computation.

    Seems to me this is an area that deserves some attention.
    Skip based on the filter and the query...
    See the comments in FilteredQuery, and see
    https://issues.apache.org/jira/browse/LUCENE-584

    -Yonik

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Paul Elschot at Aug 10, 2007 at 7:34 pm

    On Friday 10 August 2007 20:27, Yonik Seeley wrote:
    On 8/10/07, John Wang wrote:
    Hi Lucene Gurus:

    More of a performance question:

    When you pass a Filter to a searcher to do a search, the searcher is
    basically doing the full search and then intersect against the bitset
    given
    by the filter. This seems wasteful when there are lotsa hits returned by
    the
    scorer and filter only has only a few docs, e.g.

    Running a MatchAllDocsQuery given a Filter with only 1 doc.

    Right now Scorers don't know about filters, only the searchers do. And
    scorers are doing most of the computation, but it seems scorers should
    make
    use of the filters as the corpus before doing any score computation.

    Seems to me this is an area that deserves some attention.
    Yet another way to formulate the need for a common superclass.
    The patch at the issue indicated below is an "extract superclass"
    refactoring.
    And the latest discussion on it is on java-dev today.

    Regards,
    Paul Elschot.
    Skip based on the filter and the query...
    See the comments in FilteredQuery, and see
    https://issues.apache.org/jira/browse/LUCENE-584

    -Yonik

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-user @
categorieslucene
postedAug 10, '07 at 5:48p
activeAug 10, '07 at 7:34p
posts3
users3
websitelucene.apache.org

People

Translate

site design / logo © 2022 Grokbase