FAQ
[ http://issues.apache.org/jira/browse/LUCENE-330?page=comments#action_12368934 ]

Erik Hatcher commented on LUCENE-330:
-------------------------------------

Paul - it is unfortunate that we've let this patch sit for as long as it has. I've just encountered issues with FilteredQuery myself and am looking to apply your patches in hopes they'll address the problem I've encountered with FilteredQuery's nested within a BooleanQuery. There is a comment in some of your code that this doesn't work with BooleanQuery though. Since the code has changed and your patches are no longer easily applied, could you advise on what the latest patches should be and how to go about going from trunk to these patches? Many thanks!
[PATCH] Use filter bits for next() and skipTo() in FilteredQuery
----------------------------------------------------------------

Key: LUCENE-330
URL: http://issues.apache.org/jira/browse/LUCENE-330
Project: Lucene - Java
Type: Improvement
Components: Search
Versions: CVS Nightly - Specify date in submission
Environment: Operating System: other
Platform: Other
Reporter: paul.elschot
Assignee: Lucene Developers
Priority: Minor
Attachments: FilteredQuery.java, FilteredQuery.java, FilteredQuery.java, FilteredQuery.java, FilteredQueryPatch1.txt, IndexSearcherPatch2.txt, SkipFilter.java, SkipFilter.java

This improves performance of FilteredQuery by not calling score()
on documents that do not pass the filter.
This passes the current tests for FilteredQuery, but these tests
have not been adapted/extended.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org

Search Discussions

  • Yonik Seeley (JIRA) at Mar 5, 2006 at 3:05 pm
    [ http://issues.apache.org/jira/browse/LUCENE-330?page=comments#action_12368945 ]

    Yonik Seeley commented on LUCENE-330:
    -------------------------------------

    There are multiple related issues here
    - introduction and use of SkipFilter, which returns DocNrSkipper instead of a BitSet.
    - changes to IndexSearcher.search(...,filter) to create and use a FilteredQuery

    Without taking the full plunge into DocNrSkippers yet, the applicable patch to keep FilteredQuery from
    matching filtered documents would be
    http://issues.apache.org/jira/secure/attachment/12312490/FilteredQueryPatch1.txt
    There is a comment in some of your code that this doesn't work with BooleanQuery though
    The patch I referenced should work. Paul is refering to BooleanScorer which doesn't implement skipTo, as opposed to BooleanScorer2, which does.

    *If* the 1.4 BooleanScorer isn't used, then FilteredQueryPatch1.txt can be made more efficient by using scorer.skipTo() in conjunction with bitset.nextSetBit() (per Paul's comments).

    [PATCH] Use filter bits for next() and skipTo() in FilteredQuery
    ----------------------------------------------------------------

    Key: LUCENE-330
    URL: http://issues.apache.org/jira/browse/LUCENE-330
    Project: Lucene - Java
    Type: Improvement
    Components: Search
    Versions: CVS Nightly - Specify date in submission
    Environment: Operating System: other
    Platform: Other
    Reporter: paul.elschot
    Assignee: Lucene Developers
    Priority: Minor
    Attachments: FilteredQuery.java, FilteredQuery.java, FilteredQuery.java, FilteredQuery.java, FilteredQueryPatch1.txt, IndexSearcherPatch2.txt, SkipFilter.java, SkipFilter.java

    This improves performance of FilteredQuery by not calling score()
    on documents that do not pass the filter.
    This passes the current tests for FilteredQuery, but these tests
    have not been adapted/extended.
    --
    This message is automatically generated by JIRA.
    -
    If you think it was sent incorrectly contact one of the administrators:
    http://issues.apache.org/jira/secure/Administrators.jspa
    -
    For more information on JIRA, see:
    http://www.atlassian.com/software/jira


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Yonik Seeley (JIRA) at Mar 5, 2006 at 3:21 pm
    [ http://issues.apache.org/jira/browse/LUCENE-330?page=comments#action_12368947 ]

    Yonik Seeley commented on LUCENE-330:
    -------------------------------------
    The patch I referenced should work.
    To be more explicit, that patch should work with both BooleanScorer and BooleanScorer2 because it doesn't use skipTo() in next().
    If the optimization was made to use skipTo() in next(), then only BooleanScorer2 would work.
    [PATCH] Use filter bits for next() and skipTo() in FilteredQuery
    ----------------------------------------------------------------

    Key: LUCENE-330
    URL: http://issues.apache.org/jira/browse/LUCENE-330
    Project: Lucene - Java
    Type: Improvement
    Components: Search
    Versions: CVS Nightly - Specify date in submission
    Environment: Operating System: other
    Platform: Other
    Reporter: paul.elschot
    Assignee: Lucene Developers
    Priority: Minor
    Attachments: FilteredQuery.java, FilteredQuery.java, FilteredQuery.java, FilteredQuery.java, FilteredQueryPatch1.txt, IndexSearcherPatch2.txt, SkipFilter.java, SkipFilter.java

    This improves performance of FilteredQuery by not calling score()
    on documents that do not pass the filter.
    This passes the current tests for FilteredQuery, but these tests
    have not been adapted/extended.
    --
    This message is automatically generated by JIRA.
    -
    If you think it was sent incorrectly contact one of the administrators:
    http://issues.apache.org/jira/secure/Administrators.jspa
    -
    For more information on JIRA, see:
    http://www.atlassian.com/software/jira


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Erik Hatcher (JIRA) at Mar 5, 2006 at 3:51 pm
    [ http://issues.apache.org/jira/browse/LUCENE-330?page=comments#action_12368948 ]

    Erik Hatcher commented on LUCENE-330:
    -------------------------------------

    I manually applied that patch (prior to my first comment actually) as automatically applying didn't work. I just committed another test to TestFilteredQuery, which fails with this patch with this error:

    java.lang.IndexOutOfBoundsException: Not a valid hit number: 0
    at org.apache.lucene.search.Hits.hitDoc(Hits.java:134)
    at org.apache.lucene.search.Hits.id(Hits.java:116)
    at org.apache.lucene.search.TestFilteredQuery.testBoolean(TestFilteredQuery.java:139)

    I'm fairly confident I applied the patch correctly, though I suppose its possible I missed something.

    Here's an inlined version of the diff I have locally of FilteredQuery:

    $ svn diff FilteredQuery.java
    Index: FilteredQuery.java
    ===================================================================
    --- FilteredQuery.java (revision 383339)
    +++ FilteredQuery.java (working copy)
    @@ -34,6 +34,7 @@
    * <p>Created: Apr 20, 2004 8:58:29 AM
    *
    * @author Tim Jones
    + * @author Paul Elschot
    * @since 1.4
    * @version $Id$
    * @see CachingWrapperFilter
    @@ -75,22 +76,42 @@
    // return this query
    public Query getQuery() { return FilteredQuery.this; }

    - // return a scorer that overrides the enclosed query's score if
    - // the given hit has been filtered out.
    - public Scorer scorer (IndexReader indexReader) throws IOException {
    + // return a filtering scorer
    + public Scorer scorer (IndexReader indexReader) throws IOException {
    final Scorer scorer = weight.scorer (indexReader);
    final BitSet bitset = filter.bits (indexReader);
    return new Scorer (similarity) {

    - // pass these methods through to the enclosed scorer
    - public boolean next() throws IOException { return scorer.next(); }
    + public boolean next() throws IOException {
    + do {
    + if (! scorer.next()) {
    + return false;
    + }
    + } while (! bitset.get(scorer.doc()));
    + /* When skipTo() is allowed on scorer it should be used here
    + * in combination with bitset.nextSetBit(...)
    + * See the while loop in skipTo() below.
    + */
    + return true;
    + }
    public int doc() { return scorer.doc(); }
    - public boolean skipTo (int i) throws IOException { return scorer.skipTo(i); }

    - // if the document has been filtered out, set score to 0.0
    - public float score() throws IOException {
    - return (bitset.get(scorer.doc())) ? scorer.score() : 0.0f;
    - }
    + public boolean skipTo(int i) throws IOException {
    + if (! scorer.skipTo(i)) {
    + return false;
    + }
    + while (! bitset.get(scorer.doc())) {
    + int nextFiltered = bitset.nextSetBit(scorer.doc() + 1);
    + if (nextFiltered == -1) {
    + return false;
    + } else if (! scorer.skipTo(nextFiltered)) {
    + return false;
    + }
    + }
    + return true;
    + }
    +
    + public float score() throws IOException { return scorer.score(); }

    // add an explanation about whether the document was filtered
    public Explanation explain (int i) throws IOException {

    What am I missing?
    [PATCH] Use filter bits for next() and skipTo() in FilteredQuery
    ----------------------------------------------------------------

    Key: LUCENE-330
    URL: http://issues.apache.org/jira/browse/LUCENE-330
    Project: Lucene - Java
    Type: Improvement
    Components: Search
    Versions: CVS Nightly - Specify date in submission
    Environment: Operating System: other
    Platform: Other
    Reporter: paul.elschot
    Assignee: Lucene Developers
    Priority: Minor
    Attachments: FilteredQuery.java, FilteredQuery.java, FilteredQuery.java, FilteredQuery.java, FilteredQueryPatch1.txt, IndexSearcherPatch2.txt, SkipFilter.java, SkipFilter.java

    This improves performance of FilteredQuery by not calling score()
    on documents that do not pass the filter.
    This passes the current tests for FilteredQuery, but these tests
    have not been adapted/extended.
    --
    This message is automatically generated by JIRA.
    -
    If you think it was sent incorrectly contact one of the administrators:
    http://issues.apache.org/jira/secure/Administrators.jspa
    -
    For more information on JIRA, see:
    http://www.atlassian.com/software/jira


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Yonik Seeley at Mar 5, 2006 at 9:22 pm
    Hmmm, JIRA is down.
    What am I missing?
    Invalid test code.

    System.out.println(hits.id(0));
    System.out.println(hits.id(1));
    assertEquals(0, hits.length());

    If hits.length()==0, then don't try to access any of them.

    -Yonik

    On 3/5/06, Erik Hatcher (JIRA) wrote:
    [ http://issues.apache.org/jira/browse/LUCENE-330?page=comments#action_12368948 ]

    Erik Hatcher commented on LUCENE-330:
    -------------------------------------

    I manually applied that patch (prior to my first comment actually) as automatically applying didn't work. I just committed another test to TestFilteredQuery, which fails with this patch with this error:

    java.lang.IndexOutOfBoundsException: Not a valid hit number: 0
    at org.apache.lucene.search.Hits.hitDoc(Hits.java:134)
    at org.apache.lucene.search.Hits.id(Hits.java:116)
    at org.apache.lucene.search.TestFilteredQuery.testBoolean(TestFilteredQuery.java:139)

    I'm fairly confident I applied the patch correctly, though I suppose its possible I missed something.

    Here's an inlined version of the diff I have locally of FilteredQuery:

    $ svn diff FilteredQuery.java
    Index: FilteredQuery.java
    ===================================================================
    --- FilteredQuery.java (revision 383339)
    +++ FilteredQuery.java (working copy)
    @@ -34,6 +34,7 @@
    * <p>Created: Apr 20, 2004 8:58:29 AM
    *
    * @author Tim Jones
    + * @author Paul Elschot
    * @since 1.4
    * @version $Id$
    * @see CachingWrapperFilter
    @@ -75,22 +76,42 @@
    // return this query
    public Query getQuery() { return FilteredQuery.this; }

    - // return a scorer that overrides the enclosed query's score if
    - // the given hit has been filtered out.
    - public Scorer scorer (IndexReader indexReader) throws IOException {
    + // return a filtering scorer
    + public Scorer scorer (IndexReader indexReader) throws IOException {
    final Scorer scorer = weight.scorer (indexReader);
    final BitSet bitset = filter.bits (indexReader);
    return new Scorer (similarity) {

    - // pass these methods through to the enclosed scorer
    - public boolean next() throws IOException { return scorer.next(); }
    + public boolean next() throws IOException {
    + do {
    + if (! scorer.next()) {
    + return false;
    + }
    + } while (! bitset.get(scorer.doc()));
    + /* When skipTo() is allowed on scorer it should be used here
    + * in combination with bitset.nextSetBit(...)
    + * See the while loop in skipTo() below.
    + */
    + return true;
    + }
    public int doc() { return scorer.doc(); }
    - public boolean skipTo (int i) throws IOException { return scorer.skipTo(i); }

    - // if the document has been filtered out, set score to 0.0
    - public float score() throws IOException {
    - return (bitset.get(scorer.doc())) ? scorer.score() : 0.0f;
    - }
    + public boolean skipTo(int i) throws IOException {
    + if (! scorer.skipTo(i)) {
    + return false;
    + }
    + while (! bitset.get(scorer.doc())) {
    + int nextFiltered = bitset.nextSetBit(scorer.doc() + 1);
    + if (nextFiltered == -1) {
    + return false;
    + } else if (! scorer.skipTo(nextFiltered)) {
    + return false;
    + }
    + }
    + return true;
    + }
    +
    + public float score() throws IOException { return scorer.score(); }

    // add an explanation about whether the document was filtered
    public Explanation explain (int i) throws IOException {

    What am I missing?
    [PATCH] Use filter bits for next() and skipTo() in FilteredQuery
    ----------------------------------------------------------------

    Key: LUCENE-330
    URL: http://issues.apache.org/jira/browse/LUCENE-330
    Project: Lucene - Java
    Type: Improvement
    Components: Search
    Versions: CVS Nightly - Specify date in submission
    Environment: Operating System: other
    Platform: Other
    Reporter: paul.elschot
    Assignee: Lucene Developers
    Priority: Minor
    Attachments: FilteredQuery.java, FilteredQuery.java, FilteredQuery.java, FilteredQuery.java, FilteredQueryPatch1.txt, IndexSearcherPatch2.txt, SkipFilter.java, SkipFilter.java

    This improves performance of FilteredQuery by not calling score()
    on documents that do not pass the filter.
    This passes the current tests for FilteredQuery, but these tests
    have not been adapted/extended.
    --
    This message is automatically generated by JIRA.
    -
    If you think it was sent incorrectly contact one of the administrators:
    http://issues.apache.org/jira/secure/Administrators.jspa
    -
    For more information on JIRA, see:
    http://www.atlassian.com/software/jira


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • paul.elschot (JIRA) at Mar 5, 2006 at 5:02 pm
    [ http://issues.apache.org/jira/browse/LUCENE-330?page=comments#action_12368951 ]

    paul.elschot commented on LUCENE-330:
    -------------------------------------

    (Erik:)
    I've just encountered issues with FilteredQuery ...
    Could you be more specific?
    Since the code has changed and your patches are no longer easily applied, could you advise on what the latest patches should be and how to go about going from trunk to these patches?
    My current version is adapted for gcj 4.0.0 (which can/could not deal with inline abstract classes), so I can't
    provide a fresh patch quickly at the moment.

    Yonik identified all issues, nothing to add there.

    The patch contains my name as @author, could that be removed?
    The ASL 2.0 is somewhat inconclusive as to whether that can be done without me providing a
    fresh patch without that line, but I'd like to have that line removed nonetheless.
    If necessary, I'll provide a patch for this once the tests work again.

    I'll try the new TestFilteredQuery later.

    [PATCH] Use filter bits for next() and skipTo() in FilteredQuery
    ----------------------------------------------------------------

    Key: LUCENE-330
    URL: http://issues.apache.org/jira/browse/LUCENE-330
    Project: Lucene - Java
    Type: Improvement
    Components: Search
    Versions: CVS Nightly - Specify date in submission
    Environment: Operating System: other
    Platform: Other
    Reporter: paul.elschot
    Assignee: Lucene Developers
    Priority: Minor
    Attachments: FilteredQuery.java, FilteredQuery.java, FilteredQuery.java, FilteredQuery.java, FilteredQueryPatch1.txt, IndexSearcherPatch2.txt, SkipFilter.java, SkipFilter.java

    This improves performance of FilteredQuery by not calling score()
    on documents that do not pass the filter.
    This passes the current tests for FilteredQuery, but these tests
    have not been adapted/extended.
    --
    This message is automatically generated by JIRA.
    -
    If you think it was sent incorrectly contact one of the administrators:
    http://issues.apache.org/jira/secure/Administrators.jspa
    -
    For more information on JIRA, see:
    http://www.atlassian.com/software/jira


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Erik Hatcher (JIRA) at Mar 5, 2006 at 5:17 pm
    [ http://issues.apache.org/jira/browse/LUCENE-330?page=comments#action_12368953 ]

    Erik Hatcher commented on LUCENE-330:
    -------------------------------------
    Could you be more specific?
    The new TestFilteredQuery shows the details of the failure, with the stack trace in my last comment provided the patch I supplied. Those are all the specifics I have.
    The patch contains my name as @author, could that be removed?
    Sure, no problem. I simply was true to the patch you provided earlier on in this issue, but I'd be happy to remove it if this patch gets committed.

    [PATCH] Use filter bits for next() and skipTo() in FilteredQuery
    ----------------------------------------------------------------

    Key: LUCENE-330
    URL: http://issues.apache.org/jira/browse/LUCENE-330
    Project: Lucene - Java
    Type: Improvement
    Components: Search
    Versions: CVS Nightly - Specify date in submission
    Environment: Operating System: other
    Platform: Other
    Reporter: paul.elschot
    Assignee: Lucene Developers
    Priority: Minor
    Attachments: FilteredQuery.java, FilteredQuery.java, FilteredQuery.java, FilteredQuery.java, FilteredQueryPatch1.txt, IndexSearcherPatch2.txt, SkipFilter.java, SkipFilter.java

    This improves performance of FilteredQuery by not calling score()
    on documents that do not pass the filter.
    This passes the current tests for FilteredQuery, but these tests
    have not been adapted/extended.
    --
    This message is automatically generated by JIRA.
    -
    If you think it was sent incorrectly contact one of the administrators:
    http://issues.apache.org/jira/secure/Administrators.jspa
    -
    For more information on JIRA, see:
    http://www.atlassian.com/software/jira


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Paul Elschot at Mar 5, 2006 at 6:23 pm
    Erik,

    issues.apache.org is not available here at the moment.
    This test passes:

    public void testBoolean() throws Exception {
    BooleanQuery bq = new BooleanQuery();
    Query query = new FilteredQuery(new MatchAllDocsQuery(),
    new SingleDocTestFilter(0));
    bq.add(query, BooleanClause.Occur.SHOULD);
    query = new FilteredQuery(new MatchAllDocsQuery(),
    new SingleDocTestFilter(1));
    bq.add(query, BooleanClause.Occur.SHOULD);
    Hits hits = searcher.search(bq);
    assertEquals(2, hits.length());
    System.out.println(hits.id(0));
    System.out.println(hits.id(1));
    }

    Regards,
    Paul Elschot

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Erik Hatcher at Mar 5, 2006 at 10:41 pm
    Yes, the SHOULD clauses do pass with an expected and actual result of
    2 documents. However, changing both to MUST expects 0 documents yet
    the actual number is 2.

    Erik

    On Mar 5, 2006, at 1:24 PM, Paul Elschot wrote:

    Erik,

    issues.apache.org is not available here at the moment.
    This test passes:

    public void testBoolean() throws Exception {
    BooleanQuery bq = new BooleanQuery();
    Query query = new FilteredQuery(new MatchAllDocsQuery(),
    new SingleDocTestFilter(0));
    bq.add(query, BooleanClause.Occur.SHOULD);
    query = new FilteredQuery(new MatchAllDocsQuery(),
    new SingleDocTestFilter(1));
    bq.add(query, BooleanClause.Occur.SHOULD);
    Hits hits = searcher.search(bq);
    assertEquals(2, hits.length());
    System.out.println(hits.id(0));
    System.out.println(hits.id(1));
    }

    Regards,
    Paul Elschot

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Erik Hatcher at Mar 5, 2006 at 10:50 pm

    On Mar 5, 2006, at 5:41 PM, Erik Hatcher wrote:
    Yes, the SHOULD clauses do pass with an expected and actual result
    of 2 documents. However, changing both to MUST expects 0 documents
    yet the actual number is 2.
    To clarify, the actual number of 2 documents with both clauses a MUST
    is with the trunk codebase. With the patch from the JIRA issue, an
    exception occurs.

    Erik

    Erik

    On Mar 5, 2006, at 1:24 PM, Paul Elschot wrote:

    Erik,

    issues.apache.org is not available here at the moment.
    This test passes:

    public void testBoolean() throws Exception {
    BooleanQuery bq = new BooleanQuery();
    Query query = new FilteredQuery(new MatchAllDocsQuery(),
    new SingleDocTestFilter(0));
    bq.add(query, BooleanClause.Occur.SHOULD);
    query = new FilteredQuery(new MatchAllDocsQuery(),
    new SingleDocTestFilter(1));
    bq.add(query, BooleanClause.Occur.SHOULD);
    Hits hits = searcher.search(bq);
    assertEquals(2, hits.length());
    System.out.println(hits.id(0));
    System.out.println(hits.id(1));
    }

    Regards,
    Paul Elschot

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Paul Elschot at Mar 6, 2006 at 6:05 pm

    On Sunday 05 March 2006 23:51, Erik Hatcher wrote:
    On Mar 5, 2006, at 5:41 PM, Erik Hatcher wrote:
    Yes, the SHOULD clauses do pass with an expected and actual result
    of 2 documents. However, changing both to MUST expects 0 documents
    yet the actual number is 2.
    To clarify, the actual number of 2 documents with both clauses a MUST
    is with the trunk codebase.

    With the patch from the JIRA issue, an
    exception occurs.
    I tried the FilteredQuery.java patch you posted at the jira issue
    but I cannot get it applied, it fails for both hunks.
    Could you post your version of FilteredQuery.java?

    Regards,
    Paul Elschot

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Erik Hatcher at Mar 6, 2006 at 8:21 pm
    Paul,

    I owe you a big apology for the hassle it appears. I have been
    running from my IDE (IntelliJ) and that test was failing, and I had
    gone and cleaned up lots of modified Lucene source files from
    previous uncommitted patches and ensured my environment was only this
    one change. I had not run it from Ant but just tried it after
    seeing your reference to it. All passed. And so I did a clean build
    from my development environment and the test now passes. I really
    apologize - I've never had things get out of sync like that
    especially after confirming it several times before posting my issue.

    So it appears all is well with that patch. For reference, it is the
    inlined patch in my comments to LUCENE-330.

    I'll commit in a day or so unless there are objections. The rest of
    LUCENE-330 would still be up for discussion, I haven't tried those
    nor in a position knowledgeable enough to comment on them.

    Thank you so much for your contribution(s!) to Lucene, Paul, and also
    with your enduring patience especially with patches that tend to sit
    unapplied for far longer than they should.

    Erik

    On Mar 6, 2006, at 3:04 PM, Paul Elschot wrote:

    Erik,

    With the FilteredQuery that you provided
    and with only the two printing lines removed:

    ===================================================================
    --- TestFilteredQuery.java (revision 383350)
    +++ TestFilteredQuery.java (working copy)
    @@ -136,8 +136,6 @@
    new SingleDocTestFilter(1));
    bq.add(query, BooleanClause.Occur.MUST);
    Hits hits = searcher.search(bq);
    - System.out.println(hits.id(0));
    - System.out.println(hits.id(1));
    assertEquals(0, hits.length());
    }
    }

    ant -Dtestcase=TestFilteredQuery

    passes here.

    With this diff:
    ===================================================================
    --- TestFilteredQuery.java (revision 383350)
    +++ TestFilteredQuery.java (working copy)
    @@ -131,14 +131,12 @@
    BooleanQuery bq = new BooleanQuery();
    Query query = new FilteredQuery(new MatchAllDocsQuery(),
    new SingleDocTestFilter(0));
    - bq.add(query, BooleanClause.Occur.MUST);
    + bq.add(query, BooleanClause.Occur.SHOULD);
    query = new FilteredQuery(new MatchAllDocsQuery(),
    new SingleDocTestFilter(1));
    - bq.add(query, BooleanClause.Occur.MUST);
    + bq.add(query, BooleanClause.Occur.SHOULD);
    Hits hits = searcher.search(bq);
    - System.out.println(hits.id(0));
    - System.out.println(hits.id(1));
    - assertEquals(0, hits.length());
    + assertEquals(2, hits.length());
    }
    }

    the test also passes.

    Regards,
    Paul Elschot

    On Monday 06 March 2006 19:05, you wrote:
    On Sunday 05 March 2006 23:51, Erik Hatcher wrote:
    On Mar 5, 2006, at 5:41 PM, Erik Hatcher wrote:
    Yes, the SHOULD clauses do pass with an expected and actual result
    of 2 documents. However, changing both to MUST expects 0 documents
    yet the actual number is 2.
    To clarify, the actual number of 2 documents with both clauses a
    MUST
    is with the trunk codebase.

    With the patch from the JIRA issue, an
    exception occurs.
    I tried the FilteredQuery.java patch you posted at the jira issue
    but I cannot get it applied, it fails for both hunks.
    Could you post your version of FilteredQuery.java?

    Regards,
    Paul Elschot

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Erik Hatcher (JIRA) at Mar 8, 2006 at 1:02 am
    [ http://issues.apache.org/jira/browse/LUCENE-330?page=comments#action_12369342 ]

    Erik Hatcher commented on LUCENE-330:
    -------------------------------------

    The patch from FilteredQueryPatch1.txt has been applied and committed. Thanks for the fix, Paul!
    [PATCH] Use filter bits for next() and skipTo() in FilteredQuery
    ----------------------------------------------------------------

    Key: LUCENE-330
    URL: http://issues.apache.org/jira/browse/LUCENE-330
    Project: Lucene - Java
    Type: Improvement
    Components: Search
    Versions: CVS Nightly - Specify date in submission
    Environment: Operating System: other
    Platform: Other
    Reporter: paul.elschot
    Assignee: Lucene Developers
    Priority: Minor
    Attachments: FilteredQuery.java, FilteredQuery.java, FilteredQuery.java, FilteredQuery.java, FilteredQueryPatch1.txt, IndexSearcherPatch2.txt, SkipFilter.java, SkipFilter.java

    This improves performance of FilteredQuery by not calling score()
    on documents that do not pass the filter.
    This passes the current tests for FilteredQuery, but these tests
    have not been adapted/extended.
    --
    This message is automatically generated by JIRA.
    -
    If you think it was sent incorrectly contact one of the administrators:
    http://issues.apache.org/jira/secure/Administrators.jspa
    -
    For more information on JIRA, see:
    http://www.atlassian.com/software/jira


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-dev @
categorieslucene
postedMar 5, '06 at 2:07p
activeMar 8, '06 at 1:02a
posts13
users4
websitelucene.apache.org

People

Translate

site design / logo © 2022 Grokbase