FAQ
Hi,

I have an IndexWriter singleton in my program, and an IndexSearcher
singleton based on a readonly IndexReader singleton. When I use the
IndexWriter to index a large document to lucene, and then, while the program
is still running, use my previously created IndexSearcher to find hits in
that book, they are not found. But if I stop and restart the application,
then they are found.

Andy
--
View this message in context: http://lucene.472066.n3.nabble.com/search-hits-not-returned-until-I-stop-and-restart-application-tp911711p911711.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Search Discussions

  • Steven A Rowe at Jun 21, 2010 at 3:11 pm
    Hi Andy,

    From the API docs for IndexWriter <http://lucene.apache.org/java/3_0_2/api/all/org/apache/lucene/index/IndexWriter.html>:

    [D]ocuments are added with addDocument and removed
    with deleteDocuments(Term) or deleteDocuments(Query).
    A document can be updated with updateDocument (which
    just deletes and then adds the entire document).
    When finished adding, deleting and updating documents,
    close should be called.

    These changes .... are not visible to IndexReader
    until either commit() or close() is called.

    So you gotta call commit() or close(). Once you've done that, you can reduce the (expensive) cost of opening a new IndexReader by calling reopen():

    <http://lucene.apache.org/java/3_0_2/api/all/org/apache/lucene/index/IndexReader.html#reopen%28%29>

    Steve
    -----Original Message-----
    From: andynuss
    Sent: Monday, June 21, 2010 11:02 AM
    To: java-user@lucene.apache.org
    Subject: search hits not returned until I stop and restart application


    Hi,

    I have an IndexWriter singleton in my program, and an IndexSearcher
    singleton based on a readonly IndexReader singleton. When I use the
    IndexWriter to index a large document to lucene, and then, while the
    program is still running, use my previously created IndexSearcher to find
    hits in that book, they are not found. But if I stop and restart the
    application, then they are found.

    Andy
    --
    View this message in context: http://lucene.472066.n3.nabble.com/search-
    hits-not-returned-until-I-stop-and-restart-application-
    tp911711p911711.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Andynuss at Jun 21, 2010 at 5:29 pm
    "So you gotta call commit() or close(). Once you've done that, you can
    reduce the (expensive) cost of opening a new IndexReader by calling
    reopen(): "

    Steve,

    I tried this, and I must have done something wrong.

    After my document set was ingested, I called a function which (1) called the
    IndexWriter singleton commit() function, (2) then called the IndexReader
    singleton reopen() function (no arguments). (My IndexReader is read only.)
    Still didn't find hits in that book. Then I tried (3) creating a new
    IndexSearcher on top of this IndexReader and that also didn't help.

    Wonder what I could be doing wrong.

    Andy
    --
    View this message in context: http://lucene.472066.n3.nabble.com/search-hits-not-returned-until-I-stop-and-restart-application-tp911711p912096.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Steven A Rowe at Jun 21, 2010 at 5:41 pm
    Andy, it sounds like you're doing the right thing.

    Maybe you aren't using the IndexReader instance returned by reopen(), but instead are continuing to use the instance on which you called reopen()? It's tough to figure this kind of thing out without looking at the code.

    For example, what do you mean by "singleton"? (You mentioned this in reference to both IndexWriter and IndexReader.) Is it possible that some part of your code is maintaining a reference to the original IndexReader instance and using it, rather than using the newly opened instance?

    Steve
    -----Original Message-----
    From: andynuss
    Sent: Monday, June 21, 2010 1:29 PM
    To: java-user@lucene.apache.org
    Subject: RE: search hits not returned until I stop and restart application


    "So you gotta call commit() or close(). Once you've done that, you can
    reduce the (expensive) cost of opening a new IndexReader by calling
    reopen(): "

    Steve,

    I tried this, and I must have done something wrong.

    After my document set was ingested, I called a function which (1) called
    the IndexWriter singleton commit() function, (2) then called the
    IndexReader singleton reopen() function (no arguments). (My IndexReader
    is read only.) Still didn't find hits in that book. Then I tried (3)
    creating a new IndexSearcher on top of this IndexReader and that also
    didn't help.

    Wonder what I could be doing wrong.

    Andy
    --
    View this message in context: http://lucene.472066.n3.nabble.com/search-
    hits-not-returned-until-I-stop-and-restart-application-
    tp911711p912096.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Andynuss at Jun 21, 2010 at 6:44 pm
    "Maybe you aren't using the IndexReader instance returned by reopen(), but
    instead are continuing to use the instance on which you called reopen()?
    It's tough to figure this kind of thing out without looking at the code."

    That was it, I was not using the newly (re)opened index. By the way, one
    last question. It doesn't matter for this because I'm indexing one huge
    document at a time, and then committing. But later, I will also be indexing
    very small documents frequently. In that case, it would seem that if I
    index a very small document, I don't want to be thrashing with a commit
    after each one, and then a reopen of the reader and reconstruction of my
    searcher. Do others manage this type of thing with a thread that fires at
    intervals to commit if dirty?
    --
    View this message in context: http://lucene.472066.n3.nabble.com/search-hits-not-returned-until-I-stop-and-restart-application-tp911711p912345.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Steven A Rowe at Jun 21, 2010 at 7:16 pm
    Andy,

    I think batching commits either by time or number of documents is common.

    Do you know about NRT (Near Realtime Search)?: <http://wiki.apache.org/lucene-java/NearRealtimeSearch>. Using IndexWriter.getReader(), you can avoid commits altogether, as well as reducing update->search latency. See IndexWriter.getReader() javadocs for more details: <http://lucene.apache.org/java/3_0_2/api/all/org/apache/lucene/index/IndexWriter.html#getReader%28%29>.

    Depending on requirements, these two strategies can be combined.

    Steve
    -----Original Message-----
    From: andynuss
    Sent: Monday, June 21, 2010 2:44 PM
    To: java-user@lucene.apache.org
    Subject: RE: search hits not returned until I stop and restart application


    "Maybe you aren't using the IndexReader instance returned by reopen(), but
    instead are continuing to use the instance on which you called reopen()?
    It's tough to figure this kind of thing out without looking at the code."

    That was it, I was not using the newly (re)opened index. By the way, one
    last question. It doesn't matter for this because I'm indexing one huge
    document at a time, and then committing. But later, I will also be
    indexing very small documents frequently. In that case, it would seem
    that if I index a very small document, I don't want to be thrashing with a
    commit after each one, and then a reopen of the reader and reconstruction
    of my searcher. Do others manage this type of thing with a thread that
    fires at intervals to commit if dirty?

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-user @
categorieslucene
postedJun 21, '10 at 3:02p
activeJun 21, '10 at 7:16p
posts6
users2
websitelucene.apache.org

2 users in discussion

Steven A Rowe: 3 posts Andynuss: 3 posts

People

Translate

site design / logo © 2022 Grokbase