Re: How to migrate CM out of the Namenode box

Harsh J

at Aug 30, 2013 at 4:15 pm

Hello,

Look instead at /etc/cloudera-scm-server/db.mgmt.properties for the
Management services' DB passwords.

On Fri, Aug 30, 2013 at 8:30 AM, Rex Zhen wrote:
thank you for your suggestion.

which account i can use to run pg_dump in the original? I tried the one in
/etc/cloudera-scm-server/db.properties, but it can only backup database scm,
i can not use that dump amon/hmon/...

# pg_dump -h localhost -U scm -p 7432 amon > amon.sql
Password:
pg_dump: SQL command failed
pg_dump: Error message from server: ERROR: permission denied for relation
cmon_schema_version
pg_dump: The command was: LOCK TABLE public.cmon_schema_version IN ACCESS
SHARE MODE

any idea?

On Thursday, August 15, 2013 2:47:55 PM UTC-7, Philip Zeyliger wrote:

Hi Rex,

Use "pg_dump" and "pg_restore" which are Postgres utilities, against the
database as specified by /etc/cloudera-scm-server/db.properties.

Strictly speaking, there's no need to turn off the cluster, though it
won't hurt. I'd shut down the management services and CM, take a database
dump, start CM on the new machine and restore the DB. Then update the agent
configs and restart them.

-- Philip

On Thu, Aug 15, 2013 at 2:45 PM, Rex Zhen wrote:

This is the plan, let me know if there is anything wrong here.

1. stop the cluster and all the cluster services.
2. make a backup for the embedded postgresql database for CM.
3. Install a CM in a new box, point the agent to the cluster.
4. Restore the database back to CM.

I have one more question, how to backup and restore embedded database
here?

On Thursday, August 15, 2013 11:11:31 AM UTC-7, Rex Zhen wrote:

Hi,

For some reason I install CM in the same box as name node at the
beginning, and I realize that it is not a good idea when I have HA deployed
in the cluster.

Is there a way to migrate the CM out to another dedicate box or event a
virtualized box?

To unsubscribe from this group and stop receiving emails from it, send an
email to scm-users+...@cloudera.org.

To unsubscribe from this group and stop receiving emails from it, send an
email to scm-users+unsubscribe@cloudera.org.

--
Harsh J

To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.

reply | permalink

Rex Zhen

at Sep 3, 2013 at 6:51 pm

⇧

great, thank you very much. I can customize the CM to use standard
postgresql instance on a remote box when I set up the CM at the beginning.
But I can not see where can I set the database scm to that remote box, is
that scm database have to be local with embedded postgresql?

On Friday, August 30, 2013 9:15:09 AM UTC-7, Harsh J wrote:

Hello,

Look instead at /etc/cloudera-scm-server/db.mgmt.properties for the
Management services' DB passwords.

On Fri, Aug 30, 2013 at 8:30 AM, Rex Zhen <rex....@gmail.com <javascript:>>
wrote:

thank you for your suggestion.

which account i can use to run pg_dump in the original? I tried the one in
/etc/cloudera-scm-server/db.properties, but it can only backup database scm,
i can not use that dump amon/hmon/...

# pg_dump -h localhost -U scm -p 7432 amon > amon.sql
Password:
pg_dump: SQL command failed
pg_dump: Error message from server: ERROR: permission denied for relation
cmon_schema_version
pg_dump: The command was: LOCK TABLE public.cmon_schema_version IN ACCESS
SHARE MODE

any idea?

On Thursday, August 15, 2013 2:47:55 PM UTC-7, Philip Zeyliger wrote:

Hi Rex,

Use "pg_dump" and "pg_restore" which are Postgres utilities, against

the

database as specified by /etc/cloudera-scm-server/db.properties.

Strictly speaking, there's no need to turn off the cluster, though it
won't hurt. I'd shut down the management services and CM, take a

database

dump, start CM on the new machine and restore the DB. Then update the

agent

configs and restart them.

-- Philip

On Thu, Aug 15, 2013 at 2:45 PM, Rex Zhen wrote:

This is the plan, let me know if there is anything wrong here.

1. stop the cluster and all the cluster services.
2. make a backup for the embedded postgresql database for CM.
3. Install a CM in a new box, point the agent to the cluster.
4. Restore the database back to CM.

I have one more question, how to backup and restore embedded database
here?

On Thursday, August 15, 2013 11:11:31 AM UTC-7, Rex Zhen wrote:

Hi,

For some reason I install CM in the same box as name node at the
beginning, and I realize that it is not a good idea when I have HA

deployed

in the cluster.

Is there a way to migrate the CM out to another dedicate box or event

a

virtualized box?

To unsubscribe from this group and stop receiving emails from it, send

an

email to scm-users+...@cloudera.org.

To unsubscribe from this group and stop receiving emails from it, send an
email to scm-users+...@cloudera.org <javascript:>.

--
Harsh J

To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.

reply | permalink

Darren Lo

at Sep 3, 2013 at 6:53 pm

⇧

Hi Rex,

/etc/cloudera-scm-server/db.properties is used to control the primary
database of CM, which holds basic information such as host and services
configuration.

Thanks,
Darren

On Tue, Sep 3, 2013 at 11:51 AM, Rex Zhen wrote:

great, thank you very much. I can customize the CM to use standard
postgresql instance on a remote box when I set up the CM at the beginning.
But I can not see where can I set the database scm to that remote box, is
that scm database have to be local with embedded postgresql?

On Friday, August 30, 2013 9:15:09 AM UTC-7, Harsh J wrote:

Hello,

Look instead at /etc/cloudera-scm-server/db.**mgmt.properties for the
Management services' DB passwords.

On Fri, Aug 30, 2013 at 8:30 AM, Rex Zhen wrote:
thank you for your suggestion.

which account i can use to run pg_dump in the original? I tried the one in
/etc/cloudera-scm-server/db.**properties, but it can only backup
database scm,
i can not use that dump amon/hmon/...

# pg_dump -h localhost -U scm -p 7432 amon > amon.sql
Password:
pg_dump: SQL command failed
pg_dump: Error message from server: ERROR: permission denied for relation
cmon_schema_version
pg_dump: The command was: LOCK TABLE public.cmon_schema_version IN ACCESS
SHARE MODE

any idea?

On Thursday, August 15, 2013 2:47:55 PM UTC-7, Philip Zeyliger wrote:

Hi Rex,

Use "pg_dump" and "pg_restore" which are Postgres utilities, against

the

database as specified by /etc/cloudera-scm-server/db.**properties.

Strictly speaking, there's no need to turn off the cluster, though it
won't hurt. I'd shut down the management services and CM, take a

database

dump, start CM on the new machine and restore the DB. Then update the

agent

configs and restart them.

-- Philip

On Thu, Aug 15, 2013 at 2:45 PM, Rex Zhen wrote:

This is the plan, let me know if there is anything wrong here.

1. stop the cluster and all the cluster services.
2. make a backup for the embedded postgresql database for CM.
3. Install a CM in a new box, point the agent to the cluster.
4. Restore the database back to CM.

I have one more question, how to backup and restore embedded database
here?

On Thursday, August 15, 2013 11:11:31 AM UTC-7, Rex Zhen wrote:

Hi,

For some reason I install CM in the same box as name node at the
beginning, and I realize that it is not a good idea when I have HA

deployed

in the cluster.

Is there a way to migrate the CM out to another dedicate box or

event a

virtualized box?

To unsubscribe from this group and stop receiving emails from it,

send an

email to scm-users+...@cloudera.org.

To unsubscribe from this group and stop receiving emails from it, send an
email to scm-users+...@**cloudera.org.

--
Harsh J

To unsubscribe from this group and stop receiving emails from it, send an
email to scm-users+unsubscribe@cloudera.org.

--
Thanks,
Darren

To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.

reply | permalink

Rex Zhen

at Sep 3, 2013 at 7:08 pm

⇧

um, i noticed that.

another reason i asked that is, the db.mgmt.properties file is not reflect
the real information for now. as all my amon/smon/hmon/rman/nav are located
romotly, but the remote database information is not shown here.

On Friday, August 30, 2013 9:15:09 AM UTC-7, Harsh J wrote:

Hello,

Look instead at /etc/cloudera-scm-server/db.mgmt.properties for the
Management services' DB passwords.

On Fri, Aug 30, 2013 at 8:30 AM, Rex Zhen <rex....@gmail.com <javascript:>>
wrote:

thank you for your suggestion.

which account i can use to run pg_dump in the original? I tried the one in
/etc/cloudera-scm-server/db.properties, but it can only backup database scm,
i can not use that dump amon/hmon/...

# pg_dump -h localhost -U scm -p 7432 amon > amon.sql
Password:
pg_dump: SQL command failed
pg_dump: Error message from server: ERROR: permission denied for relation
cmon_schema_version
pg_dump: The command was: LOCK TABLE public.cmon_schema_version IN ACCESS
SHARE MODE

any idea?

On Thursday, August 15, 2013 2:47:55 PM UTC-7, Philip Zeyliger wrote:

Hi Rex,

Use "pg_dump" and "pg_restore" which are Postgres utilities, against

the

database as specified by /etc/cloudera-scm-server/db.properties.

Strictly speaking, there's no need to turn off the cluster, though it
won't hurt. I'd shut down the management services and CM, take a

database

dump, start CM on the new machine and restore the DB. Then update the

agent

configs and restart them.

-- Philip

On Thu, Aug 15, 2013 at 2:45 PM, Rex Zhen wrote:

This is the plan, let me know if there is anything wrong here.

1. stop the cluster and all the cluster services.
2. make a backup for the embedded postgresql database for CM.
3. Install a CM in a new box, point the agent to the cluster.
4. Restore the database back to CM.

I have one more question, how to backup and restore embedded database
here?

On Thursday, August 15, 2013 11:11:31 AM UTC-7, Rex Zhen wrote:

Hi,

For some reason I install CM in the same box as name node at the
beginning, and I realize that it is not a good idea when I have HA

deployed

in the cluster.

Is there a way to migrate the CM out to another dedicate box or event

a

virtualized box?

To unsubscribe from this group and stop receiving emails from it, send

an

email to scm-users+...@cloudera.org.

To unsubscribe from this group and stop receiving emails from it, send an
email to scm-users+...@cloudera.org <javascript:>.

--
Harsh J

To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.

reply | permalink

Darren Lo

at Sep 3, 2013 at 7:40 pm

⇧

Hi Rex,

db.mgmt.properties file stores the embedded db information for mgmt
databases. The configuration actually used by CM is located in the CM
database, and can be configured via the CM UI as properties of the
Management service. You can think of this file as a kind of "default
account info for management services", which are not necessarily the values
in use by CM.

To tell CM that the primary database is on another box, edit
/etc/cloudera-scm-server/db.properties.

Thanks,
Darren

On Tue, Sep 3, 2013 at 12:08 PM, Rex Zhen wrote:

um, i noticed that.

another reason i asked that is, the db.mgmt.properties file is not reflect
the real information for now. as all my amon/smon/hmon/rman/nav are located
romotly, but the remote database information is not shown here.

On Friday, August 30, 2013 9:15:09 AM UTC-7, Harsh J wrote:

Hello,

Look instead at /etc/cloudera-scm-server/db.**mgmt.properties for the
Management services' DB passwords.

On Fri, Aug 30, 2013 at 8:30 AM, Rex Zhen wrote:
thank you for your suggestion.

which account i can use to run pg_dump in the original? I tried the one in
/etc/cloudera-scm-server/db.**properties, but it can only backup
database scm,
i can not use that dump amon/hmon/...

# pg_dump -h localhost -U scm -p 7432 amon > amon.sql
Password:
pg_dump: SQL command failed
pg_dump: Error message from server: ERROR: permission denied for relation
cmon_schema_version
pg_dump: The command was: LOCK TABLE public.cmon_schema_version IN ACCESS
SHARE MODE

any idea?

On Thursday, August 15, 2013 2:47:55 PM UTC-7, Philip Zeyliger wrote:

Hi Rex,

Use "pg_dump" and "pg_restore" which are Postgres utilities, against

the

database as specified by /etc/cloudera-scm-server/db.**properties.

Strictly speaking, there's no need to turn off the cluster, though it
won't hurt. I'd shut down the management services and CM, take a

database

dump, start CM on the new machine and restore the DB. Then update the

agent

configs and restart them.

-- Philip

On Thu, Aug 15, 2013 at 2:45 PM, Rex Zhen wrote:

This is the plan, let me know if there is anything wrong here.

1. stop the cluster and all the cluster services.
2. make a backup for the embedded postgresql database for CM.
3. Install a CM in a new box, point the agent to the cluster.
4. Restore the database back to CM.

I have one more question, how to backup and restore embedded database
here?

On Thursday, August 15, 2013 11:11:31 AM UTC-7, Rex Zhen wrote:

Hi,

For some reason I install CM in the same box as name node at the
beginning, and I realize that it is not a good idea when I have HA

deployed

in the cluster.

Is there a way to migrate the CM out to another dedicate box or

event a

virtualized box?

To unsubscribe from this group and stop receiving emails from it,

send an

email to scm-users+...@cloudera.org.

To unsubscribe from this group and stop receiving emails from it, send an
email to scm-users+...@**cloudera.org.

--
Harsh J

To unsubscribe from this group and stop receiving emails from it, send an
email to scm-users+unsubscribe@cloudera.org.

--
Thanks,
Darren

To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.

reply | permalink

Rex Zhen

at Sep 3, 2013 at 9:29 pm

⇧

got it,

On Tuesday, September 3, 2013 12:40:28 PM UTC-7, Darren Lo wrote:

Hi Rex,

db.mgmt.properties file stores the embedded db information for mgmt
databases. The configuration actually used by CM is located in the CM
database, and can be configured via the CM UI as properties of the
Management service. You can think of this file as a kind of "default
account info for management services", which are not necessarily the values
in use by CM.

To tell CM that the primary database is on another box, edit
/etc/cloudera-scm-server/db.properties.

Thanks,
Darren

On Tue, Sep 3, 2013 at 12:08 PM, Rex Zhen <rex....@gmail.com <javascript:>

wrote:
um, i noticed that.

another reason i asked that is, the db.mgmt.properties file is not
reflect the real information for now. as all my amon/smon/hmon/rman/nav are
located romotly, but the remote database information is not shown here.

On Friday, August 30, 2013 9:15:09 AM UTC-7, Harsh J wrote:

Hello,

Look instead at /etc/cloudera-scm-server/db.**mgmt.properties for the
Management services' DB passwords.

On Fri, Aug 30, 2013 at 8:30 AM, Rex Zhen wrote:
thank you for your suggestion.

which account i can use to run pg_dump in the original? I tried the one in
/etc/cloudera-scm-server/db.**properties, but it can only backup
database scm,
i can not use that dump amon/hmon/...

# pg_dump -h localhost -U scm -p 7432 amon > amon.sql
Password:
pg_dump: SQL command failed
pg_dump: Error message from server: ERROR: permission denied for relation
cmon_schema_version
pg_dump: The command was: LOCK TABLE public.cmon_schema_version IN ACCESS
SHARE MODE

any idea?

On Thursday, August 15, 2013 2:47:55 PM UTC-7, Philip Zeyliger wrote:

Hi Rex,

Use "pg_dump" and "pg_restore" which are Postgres utilities, against

the

database as specified by /etc/cloudera-scm-server/db.**properties.

Strictly speaking, there's no need to turn off the cluster, though it
won't hurt. I'd shut down the management services and CM, take a

database

dump, start CM on the new machine and restore the DB. Then update

the agent

configs and restart them.

-- Philip

On Thu, Aug 15, 2013 at 2:45 PM, Rex Zhen wrote:

This is the plan, let me know if there is anything wrong here.

1. stop the cluster and all the cluster services.
2. make a backup for the embedded postgresql database for CM.
3. Install a CM in a new box, point the agent to the cluster.
4. Restore the database back to CM.

I have one more question, how to backup and restore embedded

database

here?

On Thursday, August 15, 2013 11:11:31 AM UTC-7, Rex Zhen wrote:

Hi,

For some reason I install CM in the same box as name node at the
beginning, and I realize that it is not a good idea when I have HA

deployed

in the cluster.

Is there a way to migrate the CM out to another dedicate box or

event a

virtualized box?

To unsubscribe from this group and stop receiving emails from it,

send an

email to scm-users+...@cloudera.org.

To unsubscribe from this group and stop receiving emails from it, send an
email to scm-users+...@**cloudera.org.

--
Harsh J

To unsubscribe from this group and stop receiving emails from it, send
an email to scm-users+...@cloudera.org <javascript:>.

--
Thanks,
Darren

To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.

reply | permalink

group	scm-users @
categories	hadoop
posted	Aug 30, '13 at 3:00a
active	Sep 3, '13 at 9:29p
posts	7
users	3
website	cloudera.com
irc	#hadoop

responses	expanded	Hotkey: s
font	variable	Hotkey: f
user style	avatars	Hotkey: a

Re: How to migrate CM out of the Namenode box

Search Discussions

Related Discussions

3 users in discussion