FAQ
When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a Scorer which gets cached rather than a bit set
------------------------------------------------------------------------------------------------------------------------------------

Key: LUCENE-1911
URL: https://issues.apache.org/jira/browse/LUCENE-1911
Project: Lucene - Java
Issue Type: Bug
Reporter: Mark Miller
Fix For: 2.9


there is a large performance cost to this.

The old impl for this type of thing, QueryFilter, recommends :

@deprecated use a CachingWrapperFilter with QueryWrapperFilter

see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org

Search Discussions

  • Mark Miller (JIRA) at Sep 16, 2009 at 5:48 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Mark Miller updated LUCENE-1911:
    --------------------------------

    Summary: When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set (was: When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a Scorer which gets cached rather than a bit set)
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Fix For: 2.9


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Mark Miller (JIRA) at Sep 16, 2009 at 5:48 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Mark Miller reassigned LUCENE-1911:
    -----------------------------------

    Assignee: Uwe Schindler
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Mark Miller (JIRA) at Sep 16, 2009 at 5:52 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756133#action_12756133 ]

    Mark Miller commented on LUCENE-1911:
    -------------------------------------

    sounds like Uwe is working on a patch so that you can tell the CachingWrapperFilter to cache the results as a bitset (optionally), which will allow you to duplicated the deprecated QueryFilter.
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Mark Miller (JIRA) at Sep 16, 2009 at 6:00 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Mark Miller updated LUCENE-1911:
    --------------------------------

    Description:
    there is a large performance cost to this.

    The old impl for this type of thing, QueryFilter, recommends :

    @deprecated use a CachingWrapperFilter with QueryWrapperFilter

    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.

    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1

    was:
    there is a large performance cost to this.

    The old impl for this type of thing, QueryFilter, recommends :

    @deprecated use a CachingWrapperFilter with QueryWrapperFilter

    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1

    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Uwe Schindler (JIRA) at Sep 16, 2009 at 6:04 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Uwe Schindler updated LUCENE-1911:
    ----------------------------------

    Attachment: LUCENE-1911.patch

    Here a patch, that implements my proposal.

    Thomas Becker: Could you apply this patch and test if itr resolves your speed problem?
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Mark Miller (JIRA) at Sep 16, 2009 at 6:04 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756138#action_12756138 ]

    Mark Miller commented on LUCENE-1911:
    -------------------------------------

    whether we do an RC or not after this I don't know.

    Technically, if we don't want to - its not required. All thats required is that we get the 3 votes and no -1 votes with valid tech arguments.
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Mark Miller at Sep 17, 2009 at 3:59 pm

    Mark Miller (JIRA) wrote:
    whether we do an RC or not after this I don't know.

    Technically, if we don't want to - its not required. All thats required is that we get the 3 votes and no -1 votes with valid tech arguments.
    Whoops - I was wrong - you can't veto a release. You just need 3 +1's
    from PMC dudes.

    I'm going to make a vote release unless anyone thinks we really need
    another public RC.

    If I can get the votes, I'd like to push ahead.

    Anyone object? This could drag into infinity and not even my new mind
    set could handle that kind of paradox.

    Remember that whole release more often discussion? That was a swell idea.

    Every 6 months would be nice - and then we could name releases after
    animals - uh - just the 6 months actually ...

    --
    - Mark

    http://www.lucidimagination.com




    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Mark Miller (JIRA) at Sep 16, 2009 at 6:10 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756145#action_12756145 ]

    Mark Miller commented on LUCENE-1911:
    -------------------------------------

    Very nice Uwe! Patch looks very good. Nicely takes care of the QueryFilter issue as well.
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Uwe Schindler (JIRA) at Sep 16, 2009 at 6:14 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756148#action_12756148 ]

    Uwe Schindler commented on LUCENE-1911:
    ---------------------------------------

    By the way, the added IOException to the protected method is no BW break, as the method is new in 2.9.
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Mark Miller (JIRA) at Sep 16, 2009 at 6:18 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756151#action_12756151 ]

    Mark Miller commented on LUCENE-1911:
    -------------------------------------

    bq. the protected method

    Its funny, because literally this morning, I was looking at CachingWrapperFilter for other reasons and I saw that method and briefly wondered what it was for - it just return what was passed to it by default. I moved on without thinking too much of it - but it makes perfect sense now ;)
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Uwe Schindler (JIRA) at Sep 16, 2009 at 6:40 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756160#action_12756160 ]

    Uwe Schindler commented on LUCENE-1911:
    ---------------------------------------

    I will post a new patch later with some added tests. I will also add SortedVIntList to the cacheable implementations.

    Maybe in 3.1 we could add a DocIdSet.isCacheable() method (like the proposed isRandomAccess()), that defaults to false. The 4 utils impls of DocIdSet could then return true here. This would be cleaner than the instanceof check.

    Or do somebody want isCacheable to DocIdSet added for 2.9, too?
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Mark Miller (JIRA) at Sep 16, 2009 at 6:52 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756171#action_12756171 ]

    Mark Miller commented on LUCENE-1911:
    -------------------------------------

    Could we commit now Uwe? Its easy to tweak if we have to. Its likely to work and stick like this though - so if we commit, I can make the release artifcats and run tests and all that jazz - then if we are good, it will be a very fast turnaround. Otherwise I have to start all that much later in either case. I'd prefer to be optimistic and get everything ready by tonight.

    I'll leave it up to you though.
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Uwe Schindler (JIRA) at Sep 16, 2009 at 7:02 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756180#action_12756180 ]

    Uwe Schindler commented on LUCENE-1911:
    ---------------------------------------

    I understand you, but:

    After reviewing other DocIdSets, I found more of them, that could be easily cached, like e.g. one of the DocIdSets returned by FieldCacheRangeFilter (the only-FieldCache-non-TermDocs-backed one). You cannot check all of them with instanceof and may some of them twice, which is a bad idea. A simple check with DocIdSet.isCacheable() looks like a good solution. If we default it to false, we have no backwards-problem.

    If we do not want to do this, I will add at least SortedVIntList to the cacheable implementations.
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Mark Miller (JIRA) at Sep 16, 2009 at 7:06 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756181#action_12756181 ]

    Mark Miller commented on LUCENE-1911:
    -------------------------------------

    Okay - no problem - if we already know theres more to do, no point yet. Lets not do it before it makes sense.
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Mark Miller (JIRA) at Sep 16, 2009 at 7:18 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756188#action_12756188 ]

    Mark Miller commented on LUCENE-1911:
    -------------------------------------

    I guess I would lean toward isCacheable myself. Its much cleaner than trying to juggle a list and overrides.

    I guess the downside is that someone could be using a custom one that was cacheable, and so it would be copied. But they can
    update to the new API as a workaround.
    And you get that with the other solution in certain cases too (unless the user updates to use the new API and does the override).
    That also requires tracking which CachingWrapperFilters get passed what Filters and its just much uglier right?

    +1 on isCacheable.

    You could argue to keep the override too - if someone was using a jar that says its not cachaeable, but it is, and can't change the source. A
    Likely scenario I'm sure ;) But worth the method it would seem. It still becomes much less necessary though.
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Michael McCandless (JIRA) at Sep 16, 2009 at 7:34 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756194#action_12756194 ]

    Michael McCandless commented on LUCENE-1911:
    --------------------------------------------

    bq. we could add a DocIdSet.isCacheable() method

    +1 to this, and to keeping docIdSetToCache() for overriding. This is much cleaner than trying to check for every known DocIdSet impl.

    Thanks for fixing this Uwe!
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Paul Elschot at Sep 16, 2009 at 8:08 pm
    I have not yet looked at the the patch, but after the comments at LUCENE-1911
    I think isCacheable() could also return true for OpenBitSet, not only for SortedVIntList.

    Regards,
    Paul Elschot.

    On Wednesday 16 September 2009 21:33:58 Michael McCandless (JIRA) wrote:

    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756194#action_12756194 ]

    Michael McCandless commented on LUCENE-1911:
    --------------------------------------------

    bq. we could add a DocIdSet.isCacheable() method

    +1 to this, and to keeping docIdSetToCache() for overriding. This is much cleaner than trying to check for every known DocIdSet impl.

    Thanks for fixing this Uwe!
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org

  • Uwe Schindler at Sep 16, 2009 at 8:20 pm
    OpenBitSet was the first candidate for this. I reviewed all DocIdSets in
    core and contrib for cacheable. There were some of them.



    -----
    Uwe Schindler
    H.-H.-Meier-Allee 63, D-28213 Bremen
    <http://www.thetaphi.de> http://www.thetaphi.de
    eMail: uwe@thetaphi.de

    _____

    From: Paul Elschot
    Sent: Wednesday, September 16, 2009 10:08 PM
    To: java-dev@lucene.apache.org
    Subject: Re: [jira] Commented: (LUCENE-1911) When using QueryWrapperFilter
    with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that
    creates a Scorer, which gets cached rather than a bit set



    I have not yet looked at the the patch, but after the comments at
    LUCENE-1911
    I think isCacheable() could also return true for OpenBitSet, not only for
    SortedVIntList.





    Regards,
    Paul Elschot.






    On Wednesday 16 September 2009 21:33:58 Michael McCandless (JIRA) wrote:

    [
    https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.pl
    ugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756194#action
    _12756194 ]
    Michael McCandless commented on LUCENE-1911:
    --------------------------------------------

    bq. we could add a DocIdSet.isCacheable() method

    +1 to this, and to keeping docIdSetToCache() for overriding. This is much
    cleaner than trying to check for every known DocIdSet impl.
    Thanks for fixing this Uwe!
    When using QueryWrapperFilter with CachingWrapperFilter,
    QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets
    cached rather than a bit set
    > >
    ----------------------------------------------------------------------------
    ----------------------------------------------------------------------------
    -----
    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because
    its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see
    http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_
    9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org

  • Paul Elschot at Sep 16, 2009 at 8:29 pm

    On Wednesday 16 September 2009 22:19:24 Uwe Schindler wrote:
    OpenBitSet was the first candidate for this. I reviewed all DocIdSets in
    core and contrib for cacheable. There were some of them.
    Thanks. I read your remark on the util implementations only later.

    I should have posted at LUCENE-1911, but somehow I lost the auto login there.
    Anyway, this approach looks good to me.

    Regards,
    Paul Elschot




    -----
    Uwe Schindler
    H.-H.-Meier-Allee 63, D-28213 Bremen
    <http://www.thetaphi.de> http://www.thetaphi.de
    eMail: uwe@thetaphi.de

    _____

    From: Paul Elschot
    Sent: Wednesday, September 16, 2009 10:08 PM
    To: java-dev@lucene.apache.org
    Subject: Re: [jira] Commented: (LUCENE-1911) When using QueryWrapperFilter
    with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that
    creates a Scorer, which gets cached rather than a bit set



    I have not yet looked at the the patch, but after the comments at
    LUCENE-1911
    I think isCacheable() could also return true for OpenBitSet, not only for
    SortedVIntList.





    Regards,
    Paul Elschot.






    On Wednesday 16 September 2009 21:33:58 Michael McCandless (JIRA) wrote:

    [
    https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.pl
    ugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756194#action
    _12756194 ]
    Michael McCandless commented on LUCENE-1911:
    --------------------------------------------

    bq. we could add a DocIdSet.isCacheable() method

    +1 to this, and to keeping docIdSetToCache() for overriding. This is much
    cleaner than trying to check for every known DocIdSet impl.
    Thanks for fixing this Uwe!
    When using QueryWrapperFilter with CachingWrapperFilter,
    QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets
    cached rather than a bit set
    ----------------------------------------------------------------------------
    ----------------------------------------------------------------------------
    -----
    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because
    its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see
    http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_
    9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org






  • Mark Miller (JIRA) at Sep 16, 2009 at 7:58 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756207#action_12756207 ]

    Mark Miller commented on LUCENE-1911:
    -------------------------------------

    Now I regret not pushing for the isCachable route to fix the Highlighter caching issue :(


    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Uwe Schindler (JIRA) at Sep 16, 2009 at 8:08 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756209#action_12756209 ]

    Uwe Schindler commented on LUCENE-1911:
    ---------------------------------------

    What do you mean?
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Mark Miller (JIRA) at Sep 16, 2009 at 8:10 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756210#action_12756210 ]

    Mark Miller commented on LUCENE-1911:
    -------------------------------------

    Remember the CachingTokenFilter issue that first popped with the Highlighter? And you suggested we add the isCachable? I was hoping the RC process was going to be smoother at the time, so I just pushed to have the simple override for now. But looking back with what I know now, I wish we had gone for it.
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Uwe Schindler (JIRA) at Sep 16, 2009 at 8:54 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Uwe Schindler updated LUCENE-1911:
    ----------------------------------

    Attachment: LUCENE-1911.patch

    Here the implementation with isCacheable(). It also has an extra test, checking cacheable on various filter with/without CachingWrapperFilter. The switch was removed from CachingWrapperFilter, as the DocIdSet now says, if it should be cached or not.

    I scanned through all DocIdSets and checked, if cacheable (in general: Do they do disk I/O during iteration or are else very slow). I looked in core and contrib looking for "extends DocIdSet" and "new DocIdSet()" anon inner classes.

    If somebody knows another DocIdSet, that is cacheable, please tell!

    All tests pass.
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch, LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Uwe Schindler (JIRA) at Sep 16, 2009 at 9:16 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Uwe Schindler updated LUCENE-1911:
    ----------------------------------

    Attachment: LUCENE-1911.patch

    Here an updated patch, with a test that QueryWrapperFilter's DocIdSet is cached correctly and the copy to BitSet code works as exspected (TestCachingWrapperFilter does not check this).

    I think this is ready to commit now.
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch, LUCENE-1911.patch, LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Uwe Schindler (JIRA) at Sep 16, 2009 at 9:18 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756232#action_12756232 ]

    Uwe Schindler commented on LUCENE-1911:
    ---------------------------------------

    Mark/Yonik: Do you know if Solr has possible DocIdSet impls that are cacheable and are used for filtering. It would be good to also overide isCacheable() to return true.
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch, LUCENE-1911.patch, LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Yonik Seeley (JIRA) at Sep 16, 2009 at 10:12 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756251#action_12756251 ]

    Yonik Seeley commented on LUCENE-1911:
    --------------------------------------

    bq. Mark/Yonik: Do you know if Solr has possible DocIdSet impls that are cacheable and are used for filtering.

    Doesn't really matter too much - Solr doesn't use CachingWrapperFilter at all. But I'll handle any instances on upgrade of the Lucene libs.
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch, LUCENE-1911.patch, LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Mark Miller (JIRA) at Sep 16, 2009 at 10:14 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756252#action_12756252 ]

    Mark Miller commented on LUCENE-1911:
    -------------------------------------

    There are a couple - including the deprecated Solr version of OpenBitSet itself (only used in tests in Solr right now) - then a couple that use an underlying OpenBitSet -

    but as Yonik says - it doesn't appear it would end up being an issue - CachingWrapperFilter isn't likely ever going to be used with them - nice to have them correct anyhow though.
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch, LUCENE-1911.patch, LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Uwe Schindler (JIRA) at Sep 16, 2009 at 10:30 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Uwe Schindler updated LUCENE-1911:
    ----------------------------------

    Attachment: LUCENE-1911.patch

    - Added CHANGES.txt entry.
    - Added a test to FieldCacheRangeFilter, that the iterators work correct and isCacheable() is false, when the index contains deleted docs and the range contains 0 (in which case TermDocs must be visited to test if a doc is deleted or has value 0 in cache).

    I will go to bed now and commit tomorrow!
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch, LUCENE-1911.patch, LUCENE-1911.patch, LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Mark Miller (JIRA) at Sep 16, 2009 at 10:32 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756263#action_12756263 ]

    Mark Miller commented on LUCENE-1911:
    -------------------------------------

    Patch looks good to me - just looked for any you might have missed and didn't see a one.
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch, LUCENE-1911.patch, LUCENE-1911.patch, LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Uwe Schindler (JIRA) at Sep 16, 2009 at 10:34 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756261#action_12756261 ]

    Uwe Schindler edited comment on LUCENE-1911 at 9/16/09 3:32 PM:
    ----------------------------------------------------------------

    - Added CHANGES.txt entry.
    - Added a test to FieldCacheRangeFilter, that the iterators work correct and isCacheable() is false, when the index contains deleted docs and the range contains 0 (in this case TermDocs must be visited to test if a doc is deleted or has value 0 in cache).

    I will go to bed now and commit tomorrow!

    was (Author: thetaphi):
    - Added CHANGES.txt entry.
    - Added a test to FieldCacheRangeFilter, that the iterators work correct and isCacheable() is false, when the index contains deleted docs and the range contains 0 (in which case TermDocs must be visited to test if a doc is deleted or has value 0 in cache).

    I will go to bed now and commit tomorrow!
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch, LUCENE-1911.patch, LUCENE-1911.patch, LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Uwe Schindler (JIRA) at Sep 16, 2009 at 11:02 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756275#action_12756275 ]

    Uwe Schindler commented on LUCENE-1911:
    ---------------------------------------

    bq. Doesn't really matter too much - Solr doesn't use CachingWrapperFilter at all. But I'll handle any instances on upgrade of the Lucene libs

    Thanks!

    bq. then a couple that use an underlying OpenBitSet

    These are easy, because OpenBitSet has isCacheable()==true -> nothing to do

    bq. but as Yonik says - it doesn't appear it would end up being an issue - CachingWrapperFilter isn't likely ever going to be used with them - nice to have them correct anyhow though.

    Correct. And nothing breaks if you have a cacheable DocIdSet not marked as such. It will simply be copied to an OpenBitSet (with some perf and memory cost on the first call to CachingWrapperFilter.getDocIdSet()).
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch, LUCENE-1911.patch, LUCENE-1911.patch, LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Mark Miller (JIRA) at Sep 16, 2009 at 11:08 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756278#action_12756278 ]

    Mark Miller commented on LUCENE-1911:
    -------------------------------------

    bq. These are easy, because OpenBitSet has isCacheable()==true -> nothing to do

    They (or was it just one?) does it with delegation though - so it will reg as just a DocIdSet (if it mattered).

    I've lost the code right now, but I think it was still cacheable.
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch, LUCENE-1911.patch, LUCENE-1911.patch, LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Uwe Schindler (JIRA) at Sep 16, 2009 at 11:14 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756280#action_12756280 ]

    Uwe Schindler commented on LUCENE-1911:
    ---------------------------------------

    {quote}
    bq. These are easy, because OpenBitSet has isCacheable()==true -> nothing to do

    They (or was it just one?) does it with delegation though - so it will reg as just a DocIdSet (if it mattered).
    {quote}

    OK, I misunderstood. In principle the same like FilteredDocIdSet (see my patch).
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch, LUCENE-1911.patch, LUCENE-1911.patch, LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Thomas Becker (JIRA) at Sep 17, 2009 at 9:21 am
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756432#action_12756432 ]

    Thomas Becker commented on LUCENE-1911:
    ---------------------------------------

    Thanks for the patch. Issue can be closed I guess.

    Attached you'll find a hotspot list for the same request with a patched 2.9-RC4. With the old api it's as fast as 2.4. Maybe a tad faster. With the new api it's a bit slower. Most probably due to not caching sorted resultsets anymore in our implementation.

    Great work guys. Nice patch @Uwe. And big thanks for your support.

    Was a pleasure. Now it's time to find a way to contribute something back to the community.
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch, LUCENE-1911.patch, LUCENE-1911.patch, LUCENE-1911.patch


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Thomas Becker (JIRA) at Sep 17, 2009 at 9:27 am
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Thomas Becker updated LUCENE-1911:
    ----------------------------------

    Attachment: lucene_29_newapi_patched.png

    HotSpots - lucene 2.9-RC4 patched
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch, LUCENE-1911.patch, LUCENE-1911.patch, LUCENE-1911.patch, lucene_29_newapi_patched.png


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Uwe Schindler (JIRA) at Sep 17, 2009 at 10:34 am
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Uwe Schindler updated LUCENE-1911:
    ----------------------------------

    Attachment: LUCENE-1911.patch

    Thanks Thomas!

    Attached is a improved patch, that uses OpenBitSetDISI, a subclass of OpenBitSet, which does the copying of the iterator to the bit set by itsself. Why duplicate the code?

    It also handles the null return value of iterator(), which is allowed. It caches an EMPTY_DOCIDSET in this case.

    I will commit soon!
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch, LUCENE-1911.patch, LUCENE-1911.patch, LUCENE-1911.patch, LUCENE-1911.patch, lucene_29_newapi_patched.png


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Uwe Schindler (JIRA) at Sep 17, 2009 at 12:05 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Uwe Schindler resolved LUCENE-1911.
    -----------------------------------

    Resolution: Fixed

    Committed revision: 816154. Thanks Thomas for reporting this!
    When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
    -------------------------------------------------------------------------------------------------------------------------------------------------------------

    Key: LUCENE-1911
    URL: https://issues.apache.org/jira/browse/LUCENE-1911
    Project: Lucene - Java
    Issue Type: Bug
    Reporter: Mark Miller
    Assignee: Uwe Schindler
    Fix For: 2.9

    Attachments: LUCENE-1911.patch, LUCENE-1911.patch, LUCENE-1911.patch, LUCENE-1911.patch, LUCENE-1911.patch, lucene_29_newapi_patched.png


    there is a large performance cost to this.
    The old impl for this type of thing, QueryFilter, recommends :
    @deprecated use a CachingWrapperFilter with QueryWrapperFilter
    The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
    see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-dev @
categorieslucene
postedSep 16, '09 at 5:42p
activeSep 17, '09 at 3:59p
posts38
users4
websitelucene.apache.org

People

Translate

site design / logo © 2021 Grokbase