FAQ
I'm trying what should be the dumbest possible example of concurrency
management with 2.0 in Java with an ordinary FSDirectory.



I create an IndexWriter from a pathname, an IndexReader from the same
pathname, and an IndexSearcher from the reader.



I add one document.



I call isCurrent() on the reader. It says, 'false'.



So, I close the reader and the searcher, and I create a new reader and a
new searcher. I search for the document, and I don't find it.



I must be missing something simple.

Search Discussions

  • Doron Cohen at Jan 10, 2007 at 9:09 pm
    That's strange. Since you don't close the writer usually adding the doc
    would not modify the index (unless adding the doc triggered a merge).

    You may want to check that:
    1. writer and reader really opened against the same path;
    2. reader isCurrent state also before adding the doc and after re-opening;
    3. searched terms vs. added terms - might not be related to concurrency at
    all.

    Finally perhaps post here the code so people can take a look.

    "Benson Margulies" <bim2006@basistech.com> wrote on 10/01/2007 12:45:08:
    I'm trying what should be the dumbest possible example of concurrency
    management with 2.0 in Java with an ordinary FSDirectory.



    I create an IndexWriter from a pathname, an IndexReader from the same
    pathname, and an IndexSearcher from the reader.



    I add one document.



    I call isCurrent() on the reader. It says, 'false'.



    So, I close the reader and the searcher, and I create a new reader and a
    new searcher. I search for the document, and I don't find it.



    I must be missing something simple.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Benson Margulies at Jan 10, 2007 at 10:04 pm
    #2 is a possible issue. I stared at the code some more:

    The test case adds up to :

    Create all the objects.

    Add three docs.
    Add a fourth doc.

    Do a query aimed at the fourth doc.

    isCurrent() returns false.

    Close reader/searcher, open reader/searcher, numDocs() in the reader
    returns 3. Not 4.

    However, reading your message carefully, I realize that I'm probably
    fundamentally misguided. There's no flush() API on a writer, so, of
    course, the only possible way for a reader to see current contents is if
    the writer gets closed and reopened. I keep trying to cook up some
    scheme in which this is not true, but, with the stock classes, it now
    seems self-evident to me that it has to be true.

    I'll put in the requisite code, and slink away.


    -----Original Message-----
    From: Doron Cohen
    Sent: Wednesday, January 10, 2007 4:07 PM
    To: java-user@lucene.apache.org
    Subject: Re: isCurrent says no, but contents still invisible

    That's strange. Since you don't close the writer usually adding the doc
    would not modify the index (unless adding the doc triggered a merge).

    You may want to check that:
    1. writer and reader really opened against the same path;
    2. reader isCurrent state also before adding the doc and after
    re-opening;
    3. searched terms vs. added terms - might not be related to concurrency
    at
    all.

    Finally perhaps post here the code so people can take a look.

    "Benson Margulies" <bim2006@basistech.com> wrote on 10/01/2007 12:45:08:
    I'm trying what should be the dumbest possible example of concurrency
    management with 2.0 in Java with an ordinary FSDirectory.



    I create an IndexWriter from a pathname, an IndexReader from the same
    pathname, and an IndexSearcher from the reader.



    I add one document.



    I call isCurrent() on the reader. It says, 'false'.



    So, I close the reader and the searcher, and I create a new reader and a
    new searcher. I search for the document, and I don't find it.



    I must be missing something simple.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Erick Erickson at Jan 10, 2007 at 10:22 pm
    And don't forget that you need to close and re-open the reader to pick up
    the changes.......

    Erick@BeenThereDoneThat <G>.
    On 1/10/07, Benson Margulies wrote:

    #2 is a possible issue. I stared at the code some more:

    The test case adds up to :

    Create all the objects.

    Add three docs.
    Add a fourth doc.

    Do a query aimed at the fourth doc.

    isCurrent() returns false.

    Close reader/searcher, open reader/searcher, numDocs() in the reader
    returns 3. Not 4.

    However, reading your message carefully, I realize that I'm probably
    fundamentally misguided. There's no flush() API on a writer, so, of
    course, the only possible way for a reader to see current contents is if
    the writer gets closed and reopened. I keep trying to cook up some
    scheme in which this is not true, but, with the stock classes, it now
    seems self-evident to me that it has to be true.

    I'll put in the requisite code, and slink away.


    -----Original Message-----
    From: Doron Cohen
    Sent: Wednesday, January 10, 2007 4:07 PM
    To: java-user@lucene.apache.org
    Subject: Re: isCurrent says no, but contents still invisible

    That's strange. Since you don't close the writer usually adding the doc
    would not modify the index (unless adding the doc triggered a merge).

    You may want to check that:
    1. writer and reader really opened against the same path;
    2. reader isCurrent state also before adding the doc and after
    re-opening;
    3. searched terms vs. added terms - might not be related to concurrency
    at
    all.

    Finally perhaps post here the code so people can take a look.

    "Benson Margulies" <bim2006@basistech.com> wrote on 10/01/2007 12:45:08:
    I'm trying what should be the dumbest possible example of concurrency
    management with 2.0 in Java with an ordinary FSDirectory.



    I create an IndexWriter from a pathname, an IndexReader from the same
    pathname, and an IndexSearcher from the reader.



    I add one document.



    I call isCurrent() on the reader. It says, 'false'.



    So, I close the reader and the searcher, and I create a new reader and a
    new searcher. I search for the document, and I don't find it.



    I must be missing something simple.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Benson Margulies at Jan 10, 2007 at 10:23 pm
    Yea, that part I got.

    -----Original Message-----
    From: Erick Erickson
    Sent: Wednesday, January 10, 2007 5:22 PM
    To: java-user@lucene.apache.org
    Subject: Re: isCurrent says no, but contents still invisible

    And don't forget that you need to close and re-open the reader to pick
    up
    the changes.......

    Erick@BeenThereDoneThat <G>.
    On 1/10/07, Benson Margulies wrote:

    #2 is a possible issue. I stared at the code some more:

    The test case adds up to :

    Create all the objects.

    Add three docs.
    Add a fourth doc.

    Do a query aimed at the fourth doc.

    isCurrent() returns false.

    Close reader/searcher, open reader/searcher, numDocs() in the reader
    returns 3. Not 4.

    However, reading your message carefully, I realize that I'm probably
    fundamentally misguided. There's no flush() API on a writer, so, of
    course, the only possible way for a reader to see current contents is if
    the writer gets closed and reopened. I keep trying to cook up some
    scheme in which this is not true, but, with the stock classes, it now
    seems self-evident to me that it has to be true.

    I'll put in the requisite code, and slink away.


    -----Original Message-----
    From: Doron Cohen
    Sent: Wednesday, January 10, 2007 4:07 PM
    To: java-user@lucene.apache.org
    Subject: Re: isCurrent says no, but contents still invisible

    That's strange. Since you don't close the writer usually adding the doc
    would not modify the index (unless adding the doc triggered a merge).

    You may want to check that:
    1. writer and reader really opened against the same path;
    2. reader isCurrent state also before adding the doc and after
    re-opening;
    3. searched terms vs. added terms - might not be related to
    concurrency
    at
    all.

    Finally perhaps post here the code so people can take a look.

    "Benson Margulies" <bim2006@basistech.com> wrote on 10/01/2007
    12:45:08:
    I'm trying what should be the dumbest possible example of
    concurrency
    management with 2.0 in Java with an ordinary FSDirectory.



    I create an IndexWriter from a pathname, an IndexReader from the
    same
    pathname, and an IndexSearcher from the reader.



    I add one document.



    I call isCurrent() on the reader. It says, 'false'.



    So, I close the reader and the searcher, and I create a new reader
    and
    a
    new searcher. I search for the document, and I don't find it.



    I must be missing something simple.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Benson Margulies at Jan 10, 2007 at 10:27 pm
    Oh, boy, what a mistake. I thought I was being clever by creating a
    Directory object. All that did was prevent the writer from ever quite
    flushing because I wasn't closing THAT.

    -----Original Message-----
    From: Erick Erickson
    Sent: Wednesday, January 10, 2007 5:22 PM
    To: java-user@lucene.apache.org
    Subject: Re: isCurrent says no, but contents still invisible

    And don't forget that you need to close and re-open the reader to pick
    up
    the changes.......

    Erick@BeenThereDoneThat <G>.
    On 1/10/07, Benson Margulies wrote:

    #2 is a possible issue. I stared at the code some more:

    The test case adds up to :

    Create all the objects.

    Add three docs.
    Add a fourth doc.

    Do a query aimed at the fourth doc.

    isCurrent() returns false.

    Close reader/searcher, open reader/searcher, numDocs() in the reader
    returns 3. Not 4.

    However, reading your message carefully, I realize that I'm probably
    fundamentally misguided. There's no flush() API on a writer, so, of
    course, the only possible way for a reader to see current contents is if
    the writer gets closed and reopened. I keep trying to cook up some
    scheme in which this is not true, but, with the stock classes, it now
    seems self-evident to me that it has to be true.

    I'll put in the requisite code, and slink away.


    -----Original Message-----
    From: Doron Cohen
    Sent: Wednesday, January 10, 2007 4:07 PM
    To: java-user@lucene.apache.org
    Subject: Re: isCurrent says no, but contents still invisible

    That's strange. Since you don't close the writer usually adding the doc
    would not modify the index (unless adding the doc triggered a merge).

    You may want to check that:
    1. writer and reader really opened against the same path;
    2. reader isCurrent state also before adding the doc and after
    re-opening;
    3. searched terms vs. added terms - might not be related to
    concurrency
    at
    all.

    Finally perhaps post here the code so people can take a look.

    "Benson Margulies" <bim2006@basistech.com> wrote on 10/01/2007
    12:45:08:
    I'm trying what should be the dumbest possible example of
    concurrency
    management with 2.0 in Java with an ordinary FSDirectory.



    I create an IndexWriter from a pathname, an IndexReader from the
    same
    pathname, and an IndexSearcher from the reader.



    I add one document.



    I call isCurrent() on the reader. It says, 'false'.



    So, I close the reader and the searcher, and I create a new reader
    and
    a
    new searcher. I search for the document, and I don't find it.



    I must be missing something simple.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Doron Cohen at Jan 10, 2007 at 10:55 pm

    "Benson Margulies" <bim2006@basistech.com> wrote on 10/01/2007 14:26:42:

    Oh, boy, what a mistake. I thought I was being clever by creating a
    Directory object. All that did was prevent the writer from ever quite
    flushing because I wasn't closing THAT.
    No need to close the directory object for that.
    Closing the IndexWriter will do.

    I think you will find "Lucene In Action" book very useful in understanding
    the life-cycle of an index.

    Or you may want to start with the javadocs intro at
    http://lucene.apache.org/java/docs/api (see the bottom of that page) and
    explore the API from there.
    In particular:
    http://lucene.apache.org/java/docs/api/org/apache/lucene/index/IndexWriter.html#addDocument(org.apache.lucene.document.Document)
    http://lucene.apache.org/java/docs/api/org/apache/lucene/index/IndexWriter.html#close()

    Also relevant are FAQ entries:
    http://wiki.apache.org/jakarta-lucene/LuceneFAQ#head-6c56b0449d114826586940dcc6fe51582676a36e
    (concurrency)
    http://wiki.apache.org/jakarta-lucene/LuceneFAQ#head-3558e5121806fb4fce80fc022d889484a9248b71
    (why-no-hits)




    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Benson Margulies at Jan 10, 2007 at 11:43 pm
    If all is well with the lifecycle, should IndexReader.numDocs() return
    an accurate count?



    -----Original Message-----
    From: Doron Cohen
    Sent: Wednesday, January 10, 2007 5:54 PM
    To: java-user@lucene.apache.org
    Subject: RE: isCurrent says no, but contents still invisible

    "Benson Margulies" <bim2006@basistech.com> wrote on 10/01/2007 14:26:42:
    Oh, boy, what a mistake. I thought I was being clever by creating a
    Directory object. All that did was prevent the writer from ever quite
    flushing because I wasn't closing THAT.
    No need to close the directory object for that.
    Closing the IndexWriter will do.

    I think you will find "Lucene In Action" book very useful in
    understanding
    the life-cycle of an index.

    Or you may want to start with the javadocs intro at
    http://lucene.apache.org/java/docs/api (see the bottom of that page) and
    explore the API from there.
    In particular:
    http://lucene.apache.org/java/docs/api/org/apache/lucene/index/IndexWrit
    er.html#addDocument(org.apache.lucene.document.Document)
    http://lucene.apache.org/java/docs/api/org/apache/lucene/index/IndexWrit
    er.html#close()

    Also relevant are FAQ entries:
    http://wiki.apache.org/jakarta-lucene/LuceneFAQ#head-6c56b0449d114826586
    940dcc6fe51582676a36e
    (concurrency)
    http://wiki.apache.org/jakarta-lucene/LuceneFAQ#head-3558e5121806fb4fce8
    0fc022d889484a9248b71
    (why-no-hits)




    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Benson Margulies at Jan 10, 2007 at 11:48 pm
    OK, all is well. I had a truly embarrassing logic error that I
    introduced while getting all the closing and opening straightened out.
    Thanks for the patience.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-user @
categorieslucene
postedJan 10, '07 at 8:45p
activeJan 10, '07 at 11:48p
posts9
users3
websitelucene.apache.org

People

Translate

site design / logo © 2022 Grokbase