FAQ
Hi Guys,

Best way take CM config file backup.

I my case. I am using HDFS,HBASE service currently using. We are managing
cluster via. CM.

I seen download client-config.zip option in CM. I downloaded after unzipped
i seen some config is there. But not fully configuration I believe.

After CM handling different way configuration management change done via CM.

when i login my cluste node. The process for example datanode has own
config. file It's located in /run/cloudera-scm-agent/ directory.

150:~# ls -ld /proc/10709/cwd
lrwxrwxrwx 1 hdfs hdfs 0 Apr 13 07:48 /proc/10709/cwd ->
/run/cloudera-scm-agent/process/8140-hdfs-DATANODE

When something i changes done. CM create new number
example.
Before chnages:
8140-hdfs-DATANODE
After Changes:
8144-hdfs-DATANODE

Please guide me.

My CM-version=4.0.4

-Dhanasekaran.
Did I learn something today? If not, I wasted it.

Search Discussions

  • Philip Zeyliger at Apr 13, 2013 at 1:27 pm
    bcc: cdh-user

    Hi there,

    When possible, please avoid double-posting to multiple lists. In this
    case, scm-users is the right list.

    The best way to take a backup of all of your CM configuration is to use the
    API. http://yourhost:7180/api/v3/cm/deployment . That would work in 4.5.
    (And, with v2 of the API, on 4.1.) However, since you're on 4.0, you
    should visit http://yourhost:7180/cmf/exportCLI?download=false instead.

    Alternatively, you can back up the entire CM database with a database tool,
    e.g., mysql_dump or pg_dump. See
    http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/4.5.1/Cloudera-Manager-Enterprise-Edition-Installation-Guide/cmeeig_topic_5_6.html
    for
    an example of how to use pg_dump. (I've pasted it in below.)

    Cheers,

    -- Philip


    ============================================

    Backing Up the PostgreSQL Database

    It's important that you periodically back up the external or embedded
    PostgreSQL database that Cloudera Manager uses to store configuration
    information.

    To back up the embedded PostgreSQL database, you can simply backup the
    /var/lib/cloudera-scm-server-db directory.

    To back up the external PostgreSQL database, you can use the pg_dump
    utility.

    *To use the pg_dump utility:*

    1. Log in to the host where the Cloudera Manager Server is installed.
    2. Run the following command as root:


    cat /etc/cloudera-scm-server/db.properties.
    The db.properties file contains:
    # Auto-generated by scm_prepare_database.sh
    # Mon Jul 27 22:36:36 PDT 2011
    com.cloudera.cmf.db.type=postgresql
    com.cloudera.cmf.db.host=localhost:7432com.cloudera.cmf.db.name=scm
    com.cloudera.cmf.db.user=scm
    com.cloudera.cmf.db.password=NnYfWIjlbk

    3. Run the following command as root using the parameters from the
    preceding step:


    # pg_dump -h localhost -p 7432 -U scm >
    /tmp/scm_server_db_backup.$(date +%Y%m%d)

    4. Enter the password specified for the
    com.cloudera.cmf.db.password property
    on the last line of the db.properties file. If you are using the
    embedded database, Cloudera Manager generated the password for you during
    installation. If you are using an external database, enter the appropriate
    information for your database.

    For more information about using the pg_dump utility, see the pg_dump
    reference <http://www.postgresql.org/docs/8.4/static/app-pgdump.html>.



    On Sat, Apr 13, 2013 at 4:59 AM, Dhanasekaran Anbalagan
    wrote:
    Hi Guys,

    Best way take CM config file backup.

    I my case. I am using HDFS,HBASE service currently using. We are managing
    cluster via. CM.

    I seen download client-config.zip option in CM. I downloaded
    after unzipped i seen some config is there. But not fully configuration
    I believe.

    After CM handling different way configuration management change done via
    CM.

    when i login my cluste node. The process for example datanode has own
    config. file It's located in /run/cloudera-scm-agent/ directory.

    150:~# ls -ld /proc/10709/cwd
    lrwxrwxrwx 1 hdfs hdfs 0 Apr 13 07:48 /proc/10709/cwd ->
    /run/cloudera-scm-agent/process/8140-hdfs-DATANODE

    When something i changes done. CM create new number
    example.
    Before chnages:
    8140-hdfs-DATANODE
    After Changes:
    8144-hdfs-DATANODE

    Please guide me.

    My CM-version=4.0.4

    -Dhanasekaran.
    Did I learn something today? If not, I wasted it.
  • Dhanasekaran Anbalagan at Apr 13, 2013 at 11:42 pm
    HI Philip,

    Thanks for give good tip for me.

    can you tell me. How to import already taken backup apply to CM?
    I tried http://cloudera.github.io/cm_api/apidocs/v1/index.html I didn't
    fine any api for this.
    also share me api import for V1 and V3

    Please guide me.

    -Dhanasekaran.

    Did I learn something today? If not, I wasted it.

    On Sat, Apr 13, 2013 at 6:57 PM, Philip Zeyliger wrote:

    bcc: cdh-user

    Hi there,

    When possible, please avoid double-posting to multiple lists. In this
    case, scm-users is the right list.

    The best way to take a backup of all of your CM configuration is to use
    the API. http://yourhost:7180/api/v3/cm/deployment . That would work in
    4.5. (And, with v2 of the API, on 4.1.) However, since you're on 4.0, you
    should visit http://yourhost:7180/cmf/exportCLI?download=false instead.

    Alternatively, you can back up the entire CM database with a database
    tool, e.g., mysql_dump or pg_dump. See
    http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/4.5.1/Cloudera-Manager-Enterprise-Edition-Installation-Guide/cmeeig_topic_5_6.html for
    an example of how to use pg_dump. (I've pasted it in below.)

    Cheers,

    -- Philip


    ============================================

    Backing Up the PostgreSQL Database

    It's important that you periodically back up the external or embedded
    PostgreSQL database that Cloudera Manager uses to store configuration
    information.

    To back up the embedded PostgreSQL database, you can simply backup the
    /var/lib/cloudera-scm-server-db directory.

    To back up the external PostgreSQL database, you can use the pg_dump
    utility.

    *To use the pg_dump utility:*

    1. Log in to the host where the Cloudera Manager Server is installed.
    2. Run the following command as root:



    cat /etc/cloudera-scm-server/db.properties.
    The db.properties file contains:
    # Auto-generated by scm_prepare_database.sh
    # Mon Jul 27 22:36:36 PDT 2011
    com.cloudera.cmf.db.type=postgresql
    com.cloudera.cmf.db.host=localhost:7432com.cloudera.cmf.db.name=scm
    com.cloudera.cmf.db.user=scm
    com.cloudera.cmf.db.password=NnYfWIjlbk

    3. Run the following command as root using the parameters from the
    preceding step:



    # pg_dump -h localhost -p 7432 -U scm > /tmp/scm_server_db_backup.$(date +%Y%m%d)

    4. Enter the password specified for the com.cloudera.cmf.db.password property
    on the last line of the db.properties file. If you are using the
    embedded database, Cloudera Manager generated the password for you during
    installation. If you are using an external database, enter the appropriate
    information for your database.

    For more information about using the pg_dump utility, see the pg_dump
    reference <http://www.postgresql.org/docs/8.4/static/app-pgdump.html>.



    On Sat, Apr 13, 2013 at 4:59 AM, Dhanasekaran Anbalagan <
    bugcy013@gmail.com> wrote:
    Hi Guys,

    Best way take CM config file backup.

    I my case. I am using HDFS,HBASE service currently using. We are managing
    cluster via. CM.

    I seen download client-config.zip option in CM. I downloaded
    after unzipped i seen some config is there. But not fully configuration
    I believe.

    After CM handling different way configuration management change done via
    CM.

    when i login my cluste node. The process for example datanode has own
    config. file It's located in /run/cloudera-scm-agent/ directory.

    150:~# ls -ld /proc/10709/cwd
    lrwxrwxrwx 1 hdfs hdfs 0 Apr 13 07:48 /proc/10709/cwd ->
    /run/cloudera-scm-agent/process/8140-hdfs-DATANODE

    When something i changes done. CM create new number
    example.
    Before chnages:
    8140-hdfs-DATANODE
    After Changes:
    8144-hdfs-DATANODE

    Please guide me.

    My CM-version=4.0.4

    -Dhanasekaran.
    Did I learn something today? If not, I wasted it.
  • Philip Zeyliger at Apr 15, 2013 at 8:03 pm
    Hi Dhanasekaran,

    In v2 and higher, you POST to /cm/deployment. In older versions, you can
    import from the web UI. See page 72 of
    http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/4.0.4/PDF/CM-4.0-enterprise-help-guide.pdf.
    Underneath the covers, it's just a POST, so you can script it if necessary
    for CM4.0.

    Cheers,

    -- Philip


    On Sat, Apr 13, 2013 at 4:42 PM, Dhanasekaran Anbalagan
    wrote:
    HI Philip,

    Thanks for give good tip for me.

    can you tell me. How to import already taken backup apply to CM?
    I tried http://cloudera.github.io/cm_api/apidocs/v1/index.html I didn't
    fine any api for this.
    also share me api import for V1 and V3

    Please guide me.

    -Dhanasekaran.

    Did I learn something today? If not, I wasted it.

    On Sat, Apr 13, 2013 at 6:57 PM, Philip Zeyliger wrote:

    bcc: cdh-user

    Hi there,

    When possible, please avoid double-posting to multiple lists. In this
    case, scm-users is the right list.

    The best way to take a backup of all of your CM configuration is to use
    the API. http://yourhost:7180/api/v3/cm/deployment . That would work
    in 4.5. (And, with v2 of the API, on 4.1.) However, since you're on 4.0,
    you should visit http://yourhost:7180/cmf/exportCLI?download=falseinstead.

    Alternatively, you can back up the entire CM database with a database
    tool, e.g., mysql_dump or pg_dump. See
    http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/4.5.1/Cloudera-Manager-Enterprise-Edition-Installation-Guide/cmeeig_topic_5_6.html for
    an example of how to use pg_dump. (I've pasted it in below.)

    Cheers,

    -- Philip


    ============================================

    Backing Up the PostgreSQL Database

    It's important that you periodically back up the external or embedded
    PostgreSQL database that Cloudera Manager uses to store configuration
    information.

    To back up the embedded PostgreSQL database, you can simply backup the
    /var/lib/cloudera-scm-server-db directory.

    To back up the external PostgreSQL database, you can use the pg_dump
    utility.

    *To use the pg_dump utility:*

    1. Log in to the host where the Cloudera Manager Server is installed.
    2. Run the following command as root:




    cat /etc/cloudera-scm-server/db.properties.
    The db.properties file contains:
    # Auto-generated by scm_prepare_database.sh
    # Mon Jul 27 22:36:36 PDT 2011
    com.cloudera.cmf.db.type=postgresql
    com.cloudera.cmf.db.host=localhost:7432com.cloudera.cmf.db.name=scm
    com.cloudera.cmf.db.user=scm
    com.cloudera.cmf.db.password=NnYfWIjlbk

    3. Run the following command as root using the parameters from the
    preceding step:




    # pg_dump -h localhost -p 7432 -U scm > /tmp/scm_server_db_backup.$(date +%Y%m%d)

    4. Enter the password specified for the com.cloudera.cmf.db.password property
    on the last line of the db.properties file. If you are using the
    embedded database, Cloudera Manager generated the password for you during
    installation. If you are using an external database, enter the appropriate
    information for your database.

    For more information about using the pg_dump utility, see the pg_dump
    reference <http://www.postgresql.org/docs/8.4/static/app-pgdump.html>.



    On Sat, Apr 13, 2013 at 4:59 AM, Dhanasekaran Anbalagan <
    bugcy013@gmail.com> wrote:
    Hi Guys,

    Best way take CM config file backup.

    I my case. I am using HDFS,HBASE service currently using. We are
    managing cluster via. CM.

    I seen download client-config.zip option in CM. I downloaded
    after unzipped i seen some config is there. But not fully configuration
    I believe.

    After CM handling different way configuration management change done via
    CM.

    when i login my cluste node. The process for example datanode has own
    config. file It's located in /run/cloudera-scm-agent/ directory.

    150:~# ls -ld /proc/10709/cwd
    lrwxrwxrwx 1 hdfs hdfs 0 Apr 13 07:48 /proc/10709/cwd ->
    /run/cloudera-scm-agent/process/8140-hdfs-DATANODE

    When something i changes done. CM create new number
    example.
    Before chnages:
    8140-hdfs-DATANODE
    After Changes:
    8144-hdfs-DATANODE

    Please guide me.

    My CM-version=4.0.4

    -Dhanasekaran.
    Did I learn something today? If not, I wasted it.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedApr 13, '13 at 12:00p
activeApr 15, '13 at 8:03p
posts4
users2
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase