FAQ
Hello:

Currently, I have a large index being build in sections. The indexing
program has multiple threads which it uses to optimize time; each thread
makes its own, separate index to avoid threads fighting over resources. At
the end of the program, the indexes are merged into a single index.

...I've read both that merging deletes and does not delete the indexes
merged. It looks like they're not being deleted, since when I check on the
sizes of each index, I see the following:

SIZE------INDEX NAME/DIRECTORY
361M term2gm/WorkerThread0
357M term2gm/WorkerThread1
449M term2gm/WorkerThread2
404M term2gm/WorkerThread3
359M term2gm/WorkerThread4
274M term2gm/WorkerThread5
428M term2gm/WorkerThread6
306M term2gm/WorkerThread7
317M term2gm/WorkerThread8
309M term2gm/WorkerThread9

2.8G term2gm/finalindex_term2gm

The final index is the merged index at the end of the program. Is it safe
for me to delete the WorkerThread# indexes, or do I need to keep them so
that my final, merged index is maintained?

Also, if I merge indexes later, will the original indexes be kept, or
deleted, after the merge? Like I said, I've read it both ways, and I want to
be sure of how it is with Lucene 2.4.1 / Lucene 2.9

Thanks!
Kylie

--
The Circle of the Dragon -- dragon history and mystery
http://www.blackdrago.com/index.html

Online Resume and Portfolio
http://www.kyliemccormick.com/

"Light, seeking light, doth the light of light beguile!"
-- William Shakespeare's Love's Labor's Lost

Search Discussions

  • Chris Lu at Oct 26, 2009 at 11:52 pm
    Pretty sure you can delete the small indexes after the merge.

    BTW: How long does your indexing and merging take respectively?

    --

    --
    Chris Lu
    -------------------------
    Instant Scalable Full-Text Search On Any Database/Application
    site: http://www.dbsight.net
    demo: http://search.dbsight.com
    Lucene Database Search in 3 minutes: http://wiki.dbsight.com/index.php?title=Create_Lucene_Database_Search_in_3_minutes
    DBSight customer, a shopping comparison site, (anonymous per request) got 2.6 Million Euro funding!



    K. M. McCormick wrote:
    Hello:

    Currently, I have a large index being build in sections. The indexing
    program has multiple threads which it uses to optimize time; each thread
    makes its own, separate index to avoid threads fighting over resources. At
    the end of the program, the indexes are merged into a single index.

    ...I've read both that merging deletes and does not delete the indexes
    merged. It looks like they're not being deleted, since when I check on the
    sizes of each index, I see the following:

    SIZE------INDEX NAME/DIRECTORY
    361M term2gm/WorkerThread0
    357M term2gm/WorkerThread1
    449M term2gm/WorkerThread2
    404M term2gm/WorkerThread3
    359M term2gm/WorkerThread4
    274M term2gm/WorkerThread5
    428M term2gm/WorkerThread6
    306M term2gm/WorkerThread7
    317M term2gm/WorkerThread8
    309M term2gm/WorkerThread9

    2.8G term2gm/finalindex_term2gm

    The final index is the merged index at the end of the program. Is it safe
    for me to delete the WorkerThread# indexes, or do I need to keep them so
    that my final, merged index is maintained?

    Also, if I merge indexes later, will the original indexes be kept, or
    deleted, after the merge? Like I said, I've read it both ways, and I want to
    be sure of how it is with Lucene 2.4.1 / Lucene 2.9

    Thanks!
    Kylie
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: [email protected]
    For additional commands, e-mail: [email protected]

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-user @
categorieslucene
postedOct 26, '09 at 7:32p
activeOct 26, '09 at 11:52p
posts2
users2
websitelucene.apache.org

2 users in discussion

Chris Lu: 1 post K. M. McCormick: 1 post

People

Translate

site design / logo © 2023 Grokbase