FAQ
Hi,

Can anyone please tell as to what is the difference between PrefixFilter and WildcardQuery as far as memory is concerned?

I saw the code of PrefixFilter and it gets TermEnum for all the terms in the index. Won't this consume memory??

I started using PrefixFilter, ConstantScoreQuery instead of WildcardQuery to avoid OOM exception. I am using ChainedFilters to do boolean logic for the filters.

Thanks again for all the help.

-Vasu


---------------------------------
Get your email and more, right on the new Yahoo.com

Search Discussions

  • Yonik Seeley at Oct 17, 2006 at 9:10 pm

    On 10/17/06, vasu shah wrote:
    Can anyone please tell as to what is the difference between PrefixFilter and WildcardQuery as far as memory is concerned?

    I saw the code of PrefixFilter and it gets TermEnum for all the terms in the index. Won't this consume memory??
    It takes time, not memory. TermEnum is like an iterator - it doesn't
    keep everything it iterates over in memory. The memory for filters
    will be for the bitset (1 bit for every document in the index, so a 1M
    doc index will take 125KB of mem per filter).

    Both PrefixQuery and PrefixFilter need to use a TermEnum, the
    difference is what they do with the resulting terms.

    -Yonik
    http://incubator.apache.org/solr Solr, the open-source Lucene search server

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Vasu shah at Oct 17, 2006 at 10:32 pm
    Thanks for the explanation.

    I am using ChainedFilter and it is taking some more time than using just one Filter.

    I read somewhere on the lucene forums that the speed can be increased for Filters if we have a large bitset and then work on it. Is it possible and if yes, how? I would like to know the technique.

    Thanks once again.
    -Vasu

    Yonik Seeley wrote: On 10/17/06, vasu shah wrote:
    Can anyone please tell as to what is the difference between PrefixFilter and WildcardQuery as far as memory is concerned?

    I saw the code of PrefixFilter and it gets TermEnum for all the terms in the index. Won't this consume memory??
    It takes time, not memory. TermEnum is like an iterator - it doesn't
    keep everything it iterates over in memory. The memory for filters
    will be for the bitset (1 bit for every document in the index, so a 1M
    doc index will take 125KB of mem per filter).

    Both PrefixQuery and PrefixFilter need to use a TermEnum, the
    difference is what they do with the resulting terms.

    -Yonik
    http://incubator.apache.org/solr Solr, the open-source Lucene search server

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org




    ---------------------------------
    All-new Yahoo! Mail - Fire up a more powerful email and get things done faster.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-user @
categorieslucene
postedOct 17, '06 at 9:05p
activeOct 17, '06 at 10:32p
posts3
users2
websitelucene.apache.org

2 users in discussion

Vasu shah: 2 posts Yonik Seeley: 1 post

People

Translate

site design / logo © 2022 Grokbase