FAQ
Hi,

I am using multiSearcher as

objMultiSearcher.search(query,null,Integer.MaxValue);

I searched a query, which should & does not return any hit. As I run the
query, the above statement throw OutofMemory exception. I change the
Integer.MaxValue TO 100, then I again searched, it returned 0 hit.

Why its thowing memory exception, even there is not hit found for the query
?

Thanks

Search Discussions

  • Laxmilal Menariya at Dec 28, 2010 at 8:36 am
    Which version you are trying of lucene ?
    On Tue, Dec 28, 2010 at 12:30 PM, Jawahar Lal wrote:

    Hi,

    I am using multiSearcher as

    objMultiSearcher.search(query,null,Integer.MaxValue);

    I searched a query, which should & does not return any hit. As I run the
    query, the above statement throw OutofMemory exception. I change the
    Integer.MaxValue TO 100, then I again searched, it returned 0 hit.

    Why its thowing memory exception, even there is not hit found for the query
    ?

    Thanks


    --
    Thanks,
    Laxmilal Menariya
  • Jawahar Lal at Dec 28, 2010 at 9:05 am
    I am using lucene 3.0.3
    On 28 December 2010 14:06, Laxmilal Menariya wrote:

    Which version you are trying of lucene ?
    On Tue, Dec 28, 2010 at 12:30 PM, Jawahar Lal wrote:

    Hi,

    I am using multiSearcher as

    objMultiSearcher.search(query,null,Integer.MaxValue);

    I searched a query, which should & does not return any hit. As I run the
    query, the above statement throw OutofMemory exception. I change the
    Integer.MaxValue TO 100, then I again searched, it returned 0 hit.

    Why its thowing memory exception, even there is not hit found for the query
    ?

    Thanks


    --
    Thanks,
    Laxmilal Menariya
  • Uwe Schindler at Dec 28, 2010 at 9:55 am
    The TopDocs returning methods are not intended to get all results, so Max Int is not a valid value. The maximum hit count allocates a PriorityQueue with so many slots. To retrieve all hits (unsorted) you have to write your own collector. But that is not the natural use case for a full text engine, which is to return the top n ranking results. For this reason its not allowed to go beyond page 50 or like that in Google (just as example). So choose n as big as how many top ranking results you are interested in. This number is normally not so big, as most users are only interested in top ranking results.

    Uwe



    "Jawahar Lal" <jlal@chambal.com> schrieb:
    I am using lucene 3.0.3
    On 28 December 2010 14:06, Laxmilal Menariya wrote:

    Which version you are trying of lucene ?
    On Tue, Dec 28, 2010 at 12:30 PM, Jawahar Lal wrote:

    Hi,

    I am using multiSearcher as

    objMultiSearcher.search(query,null,Integer.MaxValue);

    I searched a query, which should & does not return any hit. As I
    run the
    query, the above statement throw OutofMemory exception. I change
    the
    Integer.MaxValue TO 100, then I again searched, it returned 0 hit.

    Why its thowing memory exception, even there is not hit found for
    the
    query
    ?

    Thanks


    --
    Thanks,
    Laxmilal Menariya
    --
    Uwe Schindler
    H.-H.-Meier-Allee 63, 28213 Bremen
    http://www.thetaphi.de

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Paul Libbrecht at Dec 28, 2010 at 10:24 am
    I also not that this is a fundamental characteristic of the great performance of Lucene and its related products since it allows cleanly managed resources. "this" is generally called paging.

    paul


    Le 28 déc. 2010 à 10:32, Uwe Schindler a écrit :
    The TopDocs returning methods are not intended to get all results, so Max Int is not a valid value. The maximum hit count allocates a PriorityQueue with so many slots. To retrieve all hits (unsorted) you have to write your own collector. But that is not the natural use case for a full text engine, which is to return the top n ranking results. For this reason its not allowed to go beyond page 50 or like that in Google (just as example). So choose n as big as how many top ranking results you are interested in. This number is normally not so big, as most users are only interested in top ranking results.

    Uwe



    "Jawahar Lal" <jlal@chambal.com> schrieb:
    I am using lucene 3.0.3

    On 28 December 2010 14:06, Laxmilal Menariya <lmenaria@chambal.com>
    wrote:
    Which version you are trying of lucene ?

    On Tue, Dec 28, 2010 at 12:30 PM, Jawahar Lal <jlal@chambal.com>
    wrote:
    Hi,

    I am using multiSearcher as

    objMultiSearcher.search(query,null,Integer.MaxValue);

    I searched a query, which should & does not return any hit. As I
    run the
    query, the above statement throw OutofMemory exception. I change
    the
    Integer.MaxValue TO 100, then I again searched, it returned 0 hit.

    Why its thowing memory exception, even there is not hit found for
    the
    query
    ?

    Thanks


    --
    Thanks,
    Laxmilal Menariya
    --
    Uwe Schindler
    H.-H.-Meier-Allee 63, 28213 Bremen
    http://www.thetaphi.de

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Jawahar Lal at Dec 28, 2010 at 10:34 am
    Thanks all of you.
    On 28 December 2010 15:54, Paul Libbrecht wrote:

    I also not that this is a fundamental characteristic of the great
    performance of Lucene and its related products since it allows cleanly
    managed resources. "this" is generally called paging.

    paul


    Le 28 déc. 2010 à 10:32, Uwe Schindler a écrit :
    The TopDocs returning methods are not intended to get all results, so Max
    Int is not a valid value. The maximum hit count allocates a PriorityQueue
    with so many slots. To retrieve all hits (unsorted) you have to write your
    own collector. But that is not the natural use case for a full text engine,
    which is to return the top n ranking results. For this reason its not
    allowed to go beyond page 50 or like that in Google (just as example). So
    choose n as big as how many top ranking results you are interested in. This
    number is normally not so big, as most users are only interested in top
    ranking results.
    Uwe



    "Jawahar Lal" <jlal@chambal.com> schrieb:
    I am using lucene 3.0.3

    On 28 December 2010 14:06, Laxmilal Menariya <lmenaria@chambal.com>
    wrote:
    Which version you are trying of lucene ?

    On Tue, Dec 28, 2010 at 12:30 PM, Jawahar Lal <jlal@chambal.com>
    wrote:
    Hi,

    I am using multiSearcher as

    objMultiSearcher.search(query,null,Integer.MaxValue);

    I searched a query, which should & does not return any hit. As I
    run the
    query, the above statement throw OutofMemory exception. I change
    the
    Integer.MaxValue TO 100, then I again searched, it returned 0 hit.

    Why its thowing memory exception, even there is not hit found for
    the
    query
    ?

    Thanks


    --
    Thanks,
    Laxmilal Menariya
    --
    Uwe Schindler
    H.-H.-Meier-Allee 63, 28213 Bremen
    http://www.thetaphi.de

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Simon Willnauer at Dec 28, 2010 at 9:13 am
    MultiSearcher uses the given integer as the expected number of results
    and creates a hitqueue for it. Since you are asking for INT_MAX hits
    Lucene creates a hitQueue for either INT_MAX or maxDoc() documents and
    preallocates that memory. So in the worst case it creates an Object
    array with INTEGER_MAX elements - each taking 8 bytes as the object
    reference on a 64 bit VM - unless you have lots of heap space this
    will make your vm barf.

    you should only specify the number of documents you really want no
    blindly get all of them.

    why your query returns 0 docs is a different question.

    simon
    On Tue, Dec 28, 2010 at 8:00 AM, Jawahar Lal wrote:
    Hi,

    I am using multiSearcher as

    objMultiSearcher.search(query,null,Integer.MaxValue);

    I searched a query, which should & does not return any hit. As I run the
    query, the above statement throw OutofMemory exception. I change the
    Integer.MaxValue TO 100, then I again searched, it returned 0 hit.

    Why its thowing memory exception, even there is not hit found for the query
    ?

    Thanks
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-user @
categorieslucene
postedDec 28, '10 at 7:00a
activeDec 28, '10 at 10:34a
posts7
users5
websitelucene.apache.org

People

Translate

site design / logo © 2022 Grokbase