FAQ
Hi Lucene gurus,
I am newbie and i have a question on transfering index directories across
multiple machines. Whenever i update/add any new documents to the existing
index, then it is generating new .cfs file. Because of this , i ended up
doing complete transfer (instead of diff) as the file names got changed. Is
there any way to make the .cfs file name as fixed. I would like to know if
anyone successfully synced index directories using rsync.

Thanks for ur time
varma

Search Discussions

  • Tom at Aug 26, 2007 at 1:53 am
    Tom Roberts is out of the office until 3rd September 2007 and will get back to you on his return.

    http://www.luxonline.org.uk
    http://www.lux.org.uk




    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Grant Ingersoll at Aug 26, 2007 at 10:40 am
    Yes, see Solr. It takes care of all this, and then some, for you.

    -Grant
    On Aug 25, 2007, at 9:52 PM, varma d wrote:

    Hi Lucene gurus,
    I am newbie and i have a question on transfering index
    directories across
    multiple machines. Whenever i update/add any new documents to the
    existing
    index, then it is generating new .cfs file. Because of this , i
    ended up
    doing complete transfer (instead of diff) as the file names got
    changed. Is
    there any way to make the .cfs file name as fixed. I would like to
    know if
    anyone successfully synced index directories using rsync.

    Thanks for ur time
    varma
    --------------------------
    Grant Ingersoll
    http://lucene.grantingersoll.com

    Lucene Helpful Hints:
    http://wiki.apache.org/lucene-java/BasicsOfPerformance
    http://wiki.apache.org/lucene-java/LuceneFAQ



    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Pof at Jun 8, 2009 at 11:44 pm
    Is their a way to address this without Solr? I was looking at Solr today and
    it looks like a lot of piss-farting around just to backup an index. I also
    found their explaination on
    http://wiki.apache.org/solr/CollectionDistribution poorly explained, so I am
    a bit confused with the whole process.


    Grant Ingersoll-6 wrote:
    Yes, see Solr. It takes care of all this, and then some, for you.

    -Grant
    On Aug 25, 2007, at 9:52 PM, varma d wrote:

    Hi Lucene gurus,
    I am newbie and i have a question on transfering index
    directories across
    multiple machines. Whenever i update/add any new documents to the
    existing
    index, then it is generating new .cfs file. Because of this , i
    ended up
    doing complete transfer (instead of diff) as the file names got
    changed. Is
    there any way to make the .cfs file name as fixed. I would like to
    know if
    anyone successfully synced index directories using rsync.

    Thanks for ur time
    varma
    --------------------------
    Grant Ingersoll
    http://lucene.grantingersoll.com

    Lucene Helpful Hints:
    http://wiki.apache.org/lucene-java/BasicsOfPerformance
    http://wiki.apache.org/lucene-java/LuceneFAQ



    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    --
    View this message in context: http://www.nabble.com/How-to-distribute-lucene-using-rsync-tp12331421p23934073.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Ian Lea at Jun 9, 2009 at 9:13 am
    Unless you optimize it or are doing weird things with merge factors
    you won't get completely new files every time you update an index.
    Some files will change, or be created, or deleted, and some won't.
    Then you can just copy them wherever you want using rsync or whatever
    you like. We use rsync, mainly on non compound indexes although also
    on compound, and it works well.

    $ rsync -av --delete /local/index/dir-or-tree desthost::lucene


    --
    Ian.


    On Tue, Jun 9, 2009 at 12:44 AM, pofwrote:
    Is their a way to address this without Solr? I was looking at Solr today and
    it looks like a lot of piss-farting around just to backup an index. I also
    found their explaination on
    http://wiki.apache.org/solr/CollectionDistribution poorly explained, so I am
    a bit confused with the whole process.


    Grant Ingersoll-6 wrote:
    Yes, see Solr.  It takes care of all this, and then some, for you.

    -Grant
    On Aug 25, 2007, at 9:52 PM, varma d wrote:

    Hi Lucene gurus,
    I am newbie and i have a question on transfering index
    directories across
    multiple machines. Whenever i update/add any new documents to the
    existing
    index, then it is generating new .cfs file. Because of this , i
    ended up
    doing complete transfer (instead of diff) as the file names got
    changed. Is
    there any way to make the .cfs file name as fixed. I would like to
    know if
    anyone successfully synced index directories using rsync.

    Thanks for ur time
    varma
    --------------------------
    Grant Ingersoll
    http://lucene.grantingersoll.com

    Lucene Helpful Hints:
    http://wiki.apache.org/lucene-java/BasicsOfPerformance
    http://wiki.apache.org/lucene-java/LuceneFAQ



    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    --
    View this message in context: http://www.nabble.com/How-to-distribute-lucene-using-rsync-tp12331421p23934073.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
  • Michael McCandless at Jun 9, 2009 at 9:19 am
    Note that when using rsync, you must first close the IndexWriter, else
    the copy can be corrupt.

    If having to close IndexWriter (and stop indexing) is a hassle, then
    you should use SnapshotDeletionPolicy; it was created exactly for this
    reason (to take a backup of the index even while further indexing is
    proceeding).

    Mike

    On Mon, Jun 8, 2009 at 7:44 PM, pofwrote:
    Is their a way to address this without Solr? I was looking at Solr today and
    it looks like a lot of piss-farting around just to backup an index. I also
    found their explaination on
    http://wiki.apache.org/solr/CollectionDistribution poorly explained, so I am
    a bit confused with the whole process.


    Grant Ingersoll-6 wrote:
    Yes, see Solr.  It takes care of all this, and then some, for you.

    -Grant
    On Aug 25, 2007, at 9:52 PM, varma d wrote:

    Hi Lucene gurus,
    I am newbie and i have a question on transfering index
    directories across
    multiple machines. Whenever i update/add any new documents to the
    existing
    index, then it is generating new .cfs file. Because of this , i
    ended up
    doing complete transfer (instead of diff) as the file names got
    changed. Is
    there any way to make the .cfs file name as fixed. I would like to
    know if
    anyone successfully synced index directories using rsync.

    Thanks for ur time
    varma
    --------------------------
    Grant Ingersoll
    http://lucene.grantingersoll.com

    Lucene Helpful Hints:
    http://wiki.apache.org/lucene-java/BasicsOfPerformance
    http://wiki.apache.org/lucene-java/LuceneFAQ



    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

    --
    View this message in context: http://www.nabble.com/How-to-distribute-lucene-using-rsync-tp12331421p23934073.html
    Sent from the Lucene - Java Users mailing list archive at Nabble.com.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-user-help@lucene.apache.org

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-user @
categorieslucene
postedAug 26, '07 at 1:52a
activeJun 9, '09 at 9:19a
posts6
users6
websitelucene.apache.org

People

Translate

site design / logo © 2022 Grokbase