FAQ
Hi,

We're considering to use the new IndexWriter.deleteDocuments call rather
than the IndexReader.delete call. Are there any performance
improvements that this may provide, other than the benefit of not having
to switch between readers/writers?

We've looked at LUCENE-565, but there's no clear view of performance
enhancements over the old IndexReader call.

Cheers,
Andreas

--
ATLASSIAN
Our products help over 7,000 organisations in more than 88 countries to collaborate. http://www.atlassian.com/


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Search Discussions

  • Doron Cohen at Aug 3, 2007 at 5:46 am

    Andreas Knecht wrote:

    We're considering to use the new IndexWriter.deleteDocuments call rather
    than the IndexReader.delete call. Are there any performance
    improvements that this may provide, other than the benefit of not having
    to switch between readers/writers?

    We've looked at LUCENE-565, but there's no clear view of performance
    enhancements over the old IndexReader call.
    I think Yonik's comment in 565 holds here -
    http://issues.apache.org/jira/browse/LUCENE-565#action_12432155
    - if your application is buffering deletes/updates and
    then batch the deletes you probably won't see a large
    improvement. But if your application does not buffer
    the deletes and does not batch them, then I believe
    moving to IndexWriter.delete() (and update()) should
    buy you performance improvement, because IndexWriter
    would now buffer the deletes for you.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Mark Miller at Aug 3, 2007 at 10:29 am
    Also, IndexWriter probably buffers better than you would. If you buffer
    a delete with IndexWriter and then add a document that would be removed
    by that delete right after, when the buffered deletes are flushed, your
    latest doc will not be removed. Its unlikely your own buffer system
    would work so well.

    - Mark

    Doron Cohen wrote:
    Andreas Knecht wrote:

    We're considering to use the new IndexWriter.deleteDocuments call rather
    than the IndexReader.delete call. Are there any performance
    improvements that this may provide, other than the benefit of not having
    to switch between readers/writers?

    We've looked at LUCENE-565, but there's no clear view of performance
    enhancements over the old IndexReader call.
    I think Yonik's comment in 565 holds here -
    http://issues.apache.org/jira/browse/LUCENE-565#action_12432155
    - if your application is buffering deletes/updates and
    then batch the deletes you probably won't see a large
    improvement. But if your application does not buffer
    the deletes and does not batch them, then I believe
    moving to IndexWriter.delete() (and update()) should
    buy you performance improvement, because IndexWriter
    would now buffer the deletes for you.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Mike Klaas at Aug 3, 2007 at 6:28 pm

    On 3-Aug-07, at 3:27 AM, Mark Miller wrote:

    Also, IndexWriter probably buffers better than you would. If you
    buffer a delete with IndexWriter and then add a document that would
    be removed by that delete right after, when the buffered deletes
    are flushed, your latest doc will not be removed. Its unlikely your
    own buffer system would work so well.
    Is it? ISTM that any sane buffering system would account for that case.

    -Mike

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Mark Miller at Aug 3, 2007 at 7:47 pm
    Heh. I suppose I'll defer to your judgment. In my mind, the simple
    system to make is to just buffer the adds, buffer the deletes - later
    apply the adds, apply the deletes (or the reverse). I am sure something
    in Solr would have a more sophisticated process, but my guess was about
    what the new Lucene user would throw together. But again, I'll defer to
    your judgment anytime.

    - Mark

    Mike Klaas wrote:
    On 3-Aug-07, at 3:27 AM, Mark Miller wrote:

    Also, IndexWriter probably buffers better than you would. If you
    buffer a delete with IndexWriter and then add a document that would
    be removed by that delete right after, when the buffered deletes are
    flushed, your latest doc will not be removed. Its unlikely your own
    buffer system would work so well.
    Is it? ISTM that any sane buffering system would account for that case.

    -Mike

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-user @
categorieslucene
postedAug 3, '07 at 4:27a
activeAug 3, '07 at 7:47p
posts5
users4
websitelucene.apache.org

People

Translate

site design / logo © 2022 Grokbase