FAQ
I use SolrCloud. Let's assume that I want to move all indexes from one
place to another. There maybe two reasons for that:

First one is that: I will close all my system and I will use new machines
with previous indexes (if it is a must they may have same network topology)
at anywhere else after some time later.
Second one is that: I know that SolrCloud handles failures but I will back
up my indexes for a disaster event.

How can I back up my indexes? I know that I can start up new nodes and I
can close the old ones so I can move my indexes to other machines. However
how can I do such kind of backup (should I just copy data folder of Solr
nodes and put them to new Solr nodes after I change Zookeeper
configuration)?

What folks do?

Search Discussions

  • Timothy Potter at Apr 25, 2013 at 10:32 pm
    Hi Furkan,

    So here's what I do (not saying this is the best method, but
    definitely works great albeit with a little work on my part)

    The replication handler (which must be enabled for Solr cloud)
    supports a backup command, e.g.
    .../replication?command=backup&location=/mnt/backups
    From what I've heard, this is the "safe" way to backup an index in Solr.
    Every X minutes, I send a hard commit and then send this "backup"
    command to only 1 replica per shard as you don't need multiple
    replicas of the same shard backing up the same thing. I chose to send
    this to the leaders but that is definitely not required.

    The actual backup runs in the background asynchronously, so the
    command returns to your client immediately. So there's not a good way
    to know when the backup is done ... so I "poll" the replication
    handler's details action, e.g. .../replication?command=details, which
    will return a completed on date when your backup is done. So my
    backup tool simply waits until all shards report as being completed.
    Then I move the backup offsite, which for us is S3.

    Incidentally, I wrote my backup driver in Java b/c the SolrJ library
    gives you access to all the cluster state information you need to pick
    one replica per shard to start the backup on, e.g. something like
    this:

         private static final Map<String,String>
    getShardLeaders(CloudSolrServer solr, String collection) throws
    Exception {
             Map<String,String> leaders = new TreeMap<String,String>();
             ZkStateReader zkStateReader = solr.getZkStateReader();
             for (Slice slice :
    zkStateReader.getClusterState().getSlices(collection)) {
                 leaders.put(slice.getName(),
    zkStateReader.getLeaderUrl(collection, slice.getName(), ZK_TIMEOUT));
             }
             return leaders;
         }
    On Thu, Apr 25, 2013 at 3:22 PM, Furkan KAMACI wrote:
    I use SolrCloud. Let's assume that I want to move all indexes from one
    place to another. There maybe two reasons for that:

    First one is that: I will close all my system and I will use new machines
    with previous indexes (if it is a must they may have same network topology)
    at anywhere else after some time later.
    Second one is that: I know that SolrCloud handles failures but I will back
    up my indexes for a disaster event.

    How can I back up my indexes? I know that I can start up new nodes and I
    can close the old ones so I can move my indexes to other machines. However
    how can I do such kind of backup (should I just copy data folder of Solr
    nodes and put them to new Solr nodes after I change Zookeeper
    configuration)?

    What folks do?
  • Otis Gospodnetic at Apr 26, 2013 at 12:27 am
    You can use the index backup command that's part of index replication,
    check the Wiki.

    Otis
    Solr & ElasticSearch Support
    http://sematext.com/
    On Apr 25, 2013 5:23 PM, "Furkan KAMACI" wrote:

    I use SolrCloud. Let's assume that I want to move all indexes from one
    place to another. There maybe two reasons for that:

    First one is that: I will close all my system and I will use new machines
    with previous indexes (if it is a must they may have same network topology)
    at anywhere else after some time later.
    Second one is that: I know that SolrCloud handles failures but I will back
    up my indexes for a disaster event.

    How can I back up my indexes? I know that I can start up new nodes and I
    can close the old ones so I can move my indexes to other machines. However
    how can I do such kind of backup (should I just copy data folder of Solr
    nodes and put them to new Solr nodes after I change Zookeeper
    configuration)?

    What folks do?
  • Furkan KAMACI at May 19, 2013 at 2:17 pm
    Hi Otis;

    You said:

    "which will return a completed on date when your backup is done"

    which field is that?

    2013/4/26 Otis Gospodnetic <otis.gospodnetic@gmail.com>
    You can use the index backup command that's part of index replication,
    check the Wiki.

    Otis
    Solr & ElasticSearch Support
    http://sematext.com/
    On Apr 25, 2013 5:23 PM, "Furkan KAMACI" wrote:

    I use SolrCloud. Let's assume that I want to move all indexes from one
    place to another. There maybe two reasons for that:

    First one is that: I will close all my system and I will use new machines
    with previous indexes (if it is a must they may have same network topology)
    at anywhere else after some time later.
    Second one is that: I know that SolrCloud handles failures but I will back
    up my indexes for a disaster event.

    How can I back up my indexes? I know that I can start up new nodes and I
    can close the old ones so I can move my indexes to other machines. However
    how can I do such kind of backup (should I just copy data folder of Solr
    nodes and put them to new Solr nodes after I change Zookeeper
    configuration)?

    What folks do?
  • Otis Gospodnetic at May 20, 2013 at 12:00 am
    Hm, did I really say that? What was the context? Because I don't see
    that in my response below....

    Otis
    --
    Search Analytics - http://sematext.com/search-analytics/index.html
    SOLR Performance Monitoring - http://sematext.com/spm/index.html




    On Sun, May 19, 2013 at 10:16 AM, Furkan KAMACI wrote:
    Hi Otis;

    You said:

    "which will return a completed on date when your backup is done"

    which field is that?

    2013/4/26 Otis Gospodnetic <otis.gospodnetic@gmail.com>
    You can use the index backup command that's part of index replication,
    check the Wiki.

    Otis
    Solr & ElasticSearch Support
    http://sematext.com/
    On Apr 25, 2013 5:23 PM, "Furkan KAMACI" wrote:

    I use SolrCloud. Let's assume that I want to move all indexes from one
    place to another. There maybe two reasons for that:

    First one is that: I will close all my system and I will use new machines
    with previous indexes (if it is a must they may have same network topology)
    at anywhere else after some time later.
    Second one is that: I know that SolrCloud handles failures but I will back
    up my indexes for a disaster event.

    How can I back up my indexes? I know that I can start up new nodes and I
    can close the old ones so I can move my indexes to other machines. However
    how can I do such kind of backup (should I just copy data folder of Solr
    nodes and put them to new Solr nodes after I change Zookeeper
    configuration)?

    What folks do?
  • Furkan KAMACI at May 20, 2013 at 7:47 am
    Ooops, you didn't say it OK. It is at Timothy's answer.

    2013/5/20 Otis Gospodnetic <otis.gospodnetic@gmail.com>
    Hm, did I really say that? What was the context? Because I don't see
    that in my response below....

    Otis
    --
    Search Analytics - http://sematext.com/search-analytics/index.html
    SOLR Performance Monitoring - http://sematext.com/spm/index.html




    On Sun, May 19, 2013 at 10:16 AM, Furkan KAMACI wrote:
    Hi Otis;

    You said:

    "which will return a completed on date when your backup is done"

    which field is that?

    2013/4/26 Otis Gospodnetic <otis.gospodnetic@gmail.com>
    You can use the index backup command that's part of index replication,
    check the Wiki.

    Otis
    Solr & ElasticSearch Support
    http://sematext.com/
    On Apr 25, 2013 5:23 PM, "Furkan KAMACI" wrote:

    I use SolrCloud. Let's assume that I want to move all indexes from one
    place to another. There maybe two reasons for that:

    First one is that: I will close all my system and I will use new
    machines
    with previous indexes (if it is a must they may have same network topology)
    at anywhere else after some time later.
    Second one is that: I know that SolrCloud handles failures but I will back
    up my indexes for a disaster event.

    How can I back up my indexes? I know that I can start up new nodes
    and I
    can close the old ones so I can move my indexes to other machines. However
    how can I do such kind of backup (should I just copy data folder of
    Solr
    nodes and put them to new Solr nodes after I change Zookeeper
    configuration)?

    What folks do?
  • Furkan KAMACI at May 20, 2013 at 8:55 am
    OK, I found it, no problem.

    2013/5/20 Furkan KAMACI <furkankamaci@gmail.com>
    Ooops, you didn't say it OK. It is at Timothy's answer.


    2013/5/20 Otis Gospodnetic <otis.gospodnetic@gmail.com>
    Hm, did I really say that? What was the context? Because I don't see
    that in my response below....

    Otis
    --
    Search Analytics - http://sematext.com/search-analytics/index.html
    SOLR Performance Monitoring - http://sematext.com/spm/index.html





    On Sun, May 19, 2013 at 10:16 AM, Furkan KAMACI <furkankamaci@gmail.com>
    wrote:
    Hi Otis;

    You said:

    "which will return a completed on date when your backup is done"

    which field is that?

    2013/4/26 Otis Gospodnetic <otis.gospodnetic@gmail.com>
    You can use the index backup command that's part of index replication,
    check the Wiki.

    Otis
    Solr & ElasticSearch Support
    http://sematext.com/
    On Apr 25, 2013 5:23 PM, "Furkan KAMACI" <furkankamaci@gmail.com>
    wrote:
    I use SolrCloud. Let's assume that I want to move all indexes from
    one
    place to another. There maybe two reasons for that:

    First one is that: I will close all my system and I will use new
    machines
    with previous indexes (if it is a must they may have same network topology)
    at anywhere else after some time later.
    Second one is that: I know that SolrCloud handles failures but I will back
    up my indexes for a disaster event.

    How can I back up my indexes? I know that I can start up new nodes
    and I
    can close the old ones so I can move my indexes to other machines. However
    how can I do such kind of backup (should I just copy data folder of
    Solr
    nodes and put them to new Solr nodes after I change Zookeeper
    configuration)?

    What folks do?
  • Upayavira at May 20, 2013 at 8:55 am
    From looking at the ReplicationHandler code, it looks like if you hit it
    with a 'details' request, it'll show you the details of the most recent
    backup, including file count, status and completion time.

    Upayavira
    On Mon, May 20, 2013, at 08:46 AM, Furkan KAMACI wrote:
    Ooops, you didn't say it OK. It is at Timothy's answer.

    2013/5/20 Otis Gospodnetic <otis.gospodnetic@gmail.com>
    Hm, did I really say that? What was the context? Because I don't see
    that in my response below....

    Otis
    --
    Search Analytics - http://sematext.com/search-analytics/index.html
    SOLR Performance Monitoring - http://sematext.com/spm/index.html





    On Sun, May 19, 2013 at 10:16 AM, Furkan KAMACI <furkankamaci@gmail.com>
    wrote:
    Hi Otis;

    You said:

    "which will return a completed on date when your backup is done"

    which field is that?

    2013/4/26 Otis Gospodnetic <otis.gospodnetic@gmail.com>
    You can use the index backup command that's part of index replication,
    check the Wiki.

    Otis
    Solr & ElasticSearch Support
    http://sematext.com/
    On Apr 25, 2013 5:23 PM, "Furkan KAMACI" <furkankamaci@gmail.com>
    wrote:
    I use SolrCloud. Let's assume that I want to move all indexes from one
    place to another. There maybe two reasons for that:

    First one is that: I will close all my system and I will use new
    machines
    with previous indexes (if it is a must they may have same network topology)
    at anywhere else after some time later.
    Second one is that: I know that SolrCloud handles failures but I will back
    up my indexes for a disaster event.

    How can I back up my indexes? I know that I can start up new nodes
    and I
    can close the old ones so I can move my indexes to other machines. However
    how can I do such kind of backup (should I just copy data folder of
    Solr
    nodes and put them to new Solr nodes after I change Zookeeper
    configuration)?

    What folks do?

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupsolr-user @
categorieslucene
postedApr 25, '13 at 9:23p
activeMay 20, '13 at 8:55a
posts8
users4
websitelucene.apache.org...

People

Translate

site design / logo © 2019 Grokbase