Grokbase › Groups › Hadoop › scm-users › February 2013
FAQ
Hi Pankaj,

We recommend you make a backup of the CM embedded database (e.g., with
pg_dump). Doing so will let you rebuild the machine on another host.
While the CM server is unavailable, Hadoop will continue running as before.

-- Philip

On Thu, Jan 31, 2013 at 4:57 PM, Mark Grover wrote:

Moving to [email protected]
(bcc: [email protected])
On Thu, Jan 31, 2013 at 4:56 PM, Pankaj Gupta wrote:

Hi,

I'm just setting up a cluster using Cloudera Manager using the embedded
database option. I'm wondering about how to handle failure of Cloudera
Manager machine. Supposing that Cloudera Manager machine is completely lost
and I set up a new one and point the agents to it, would the new manager be
able to get relevant information from agents? If not then which files from
the embedded database do I need to take snapshots of? Basically what's the
recommended way of recovering from Cloudera Manager failure.

Thanks in Advance,
Pankaj

--

Search Discussions

  • David Montgomery at Feb 1, 2013 at 6:27 am ⇧
    Hi,

    So e.g. you back up the database once an hour and place e.g. on s3. If you
    boot a new manager node then restore the backup?

    then back to normal?

    Thanks

    On Fri, Feb 1, 2013 at 2:22 PM, Philip Zeyliger wrote:

    Hi Pankaj,

    We recommend you make a backup of the CM embedded database (e.g., with
    pg_dump). Doing so will let you rebuild the machine on another host.
    While the CM server is unavailable, Hadoop will continue running as before.

    -- Philip

    On Thu, Jan 31, 2013 at 4:57 PM, Mark Grover wrote:

    Moving to [email protected]
    (bcc: [email protected])
    On Thu, Jan 31, 2013 at 4:56 PM, Pankaj Gupta wrote:

    Hi,

    I'm just setting up a cluster using Cloudera Manager using the embedded
    database option. I'm wondering about how to handle failure of Cloudera
    Manager machine. Supposing that Cloudera Manager machine is completely lost
    and I set up a new one and point the agents to it, would the new manager be
    able to get relevant information from agents? If not then which files from
    the embedded database do I need to take snapshots of? Basically what's the
    recommended way of recovering from Cloudera Manager failure.

    Thanks in Advance,
    Pankaj

    --


  • Philip Zeyliger at Feb 1, 2013 at 6:33 am ⇧
    Yes. If the hostname of the CM server changes, edit
    /etc/cloudera-scm-agent/config.ini, and do a "service cloudera-scm-agent
    restart" to have the agents pick up the new hostname. All of the
    non-transient CM state is in that database.

    -- Philip

    On Thu, Jan 31, 2013 at 10:27 PM, David Montgomery wrote:

    Hi,

    So e.g. you back up the database once an hour and place e.g. on s3. If
    you boot a new manager node then restore the backup?

    then back to normal?

    Thanks

    On Fri, Feb 1, 2013 at 2:22 PM, Philip Zeyliger wrote:

    Hi Pankaj,

    We recommend you make a backup of the CM embedded database (e.g., with
    pg_dump). Doing so will let you rebuild the machine on another host.
    While the CM server is unavailable, Hadoop will continue running as before.

    -- Philip

    On Thu, Jan 31, 2013 at 4:57 PM, Mark Grover wrote:

    Moving to [email protected]
    (bcc: [email protected])
    On Thu, Jan 31, 2013 at 4:56 PM, Pankaj Gupta wrote:

    Hi,

    I'm just setting up a cluster using Cloudera Manager using the embedded
    database option. I'm wondering about how to handle failure of Cloudera
    Manager machine. Supposing that Cloudera Manager machine is completely lost
    and I set up a new one and point the agents to it, would the new manager be
    able to get relevant information from agents? If not then which files from
    the embedded database do I need to take snapshots of? Basically what's the
    recommended way of recovering from Cloudera Manager failure.

    Thanks in Advance,
    Pankaj

    --


  • James Hogarth at Feb 1, 2013 at 11:13 am ⇧

    On Friday, 1 February 2013 06:32:39 UTC, Philip Zeyliger wrote:
    Yes. If the hostname of the CM server changes, edit
    /etc/cloudera-scm-agent/config.ini, and do a "service cloudera-scm-agent
    restart" to have the agents pick up the new hostname. All of the
    non-transient CM state is in that database.

    If you use an external rather than embedded database it's feasible to
    provide for a more highly available solution via postgres replication and
    pooling via a tool like pgpool-II

    You could, in theory, have a standby cloudera manager server pointed to
    that and ready to start the services on and change DNS/IPs etc to point to
    that...

    It would of course be important to have suitable fencing or other methods
    (use stuff like pacemaker etc) to ensure services did not run on both nodes
    at the same time and avoid split-brain situations and so on ...

    You're talking about the cloudera manager 'machine' being completely
    lost... if you virtualize SCM that of course can provide additional
    resilience and flexibility too...

    James

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedFeb 1, '13 at 6:23a
activeFeb 1, '13 at 11:13a
posts4
users3
websitecloudera.com
irc#hadoop

3 users in discussion

Philip Zeyliger: 2 posts James Hogarth: 1 post David Montgomery: 1 post

Content

People

Support

Translate

site design / logo © 2023 Grokbase