FAQ
Hello, I am trying to set up a Hadoop/Hbase cluster on EC2. I have to use
Hadoop 1.0.4 and HBase 0.94, so I followed this guide:
http://rogueleaderr.com/post/32768181371/set-up-and-run-a-fully-distributed-hadoop-hbase-cluster

I realize the instructions are depreciated, but I know the latest Cloudera
CDH doesn't run Hadoop 1.0.4.

I installed the correct versions of Hadoop and Hbase from the Apache
archives and set up the cluster using Whirr and all went well. Haddop and
Hbase are running - however, the configurations and start/stop scripts
weren't working well, so I thought installing the Cloudera Manager would
help in that regard.

Anyway, I give the install process the IPs of my boxes (I have 1 master and
3 slaves) and confirming that they were reachable by pings, I started the
installation process- all was going well until the process was waiting for
a heartbeat and each of my boxes failed to give one and then the install
failed.

I figured maybe the internal ip's of the EC2 instances was the problem,
changed all the hostnames to their FQDNs and still the same issue happened.

My log file is as follows:

31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO No
command line vars
[31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Missing
database jar: /usr/share/java/mysql-connector-java.jar (normal, if you're
not using this database type)
[31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Missing
database jar: /usr/share/java/oracle-connector-java.jar (normal, if you're
not using this database type)
[31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Found
database jar: /usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar
[31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Agent
starting as pid 18298 user root(0) group root(0).
[31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Created
/run/cloudera-scm-agent
[31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
Chmod'ing /run/cloudera-scm-agent to 0751
[31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Agent
is running on a public cloud. Public hostname:
ec2-54-242-227-124.compute-1.amazonaws.com
[31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Created
/run/cloudera-scm-agent/cgroups
[31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
Chmod'ing /run/cloudera-scm-agent/cgroups to 0751
[31/Jul/2013 17:53:27 +0000] 18298 MainThread cgroups INFO Created
/run/cloudera-scm-agent/cgroups/memory
[31/Jul/2013 17:53:27 +0000] 18298 MainThread cgroups INFO Created
/run/cloudera-scm-agent/cgroups/cpu
[31/Jul/2013 17:53:27 +0000] 18298 MainThread cgroups INFO Created
/run/cloudera-scm-agent/cgroups/cpuacct
[31/Jul/2013 17:53:27 +0000] 18298 MainThread cgroups INFO Created
/run/cloudera-scm-agent/cgroups/blkio
[31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Found
cgroups capabilities: {'has_memory': True, 'default_memory_limit_in_bytes':
9223372036854775807, 'default_memory_soft_limit_in_bytes':
9223372036854775807, 'writable_cgroup_dot_procs': True,
'default_cpu_rt_runtime_us': 950000, 'has_cpu': True,
'default_blkio_weight': 1000, 'default_cpu_shares': 1024, 'has_cpuacct':
True, 'has_blkio': True}
[31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Setting
up supervisord event monitor.
[31/Jul/2013 17:53:27 +0000] 18298 MainThread filesystem_map INFO
Monitored nodev filesystem types: ['nfs', 'nfs4', 'tmpfs']
[31/Jul/2013 17:53:27 +0000] 18298 MainThread filesystem_map INFO Using
timeout of 2.000000
[31/Jul/2013 17:53:27 +0000] 18298 MainThread filesystem_map INFO Using
join timeout of 0.100000
[31/Jul/2013 17:53:27 +0000] 18298 MainThread filesystem_map INFO Using
tolerance of 60.000000
[31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Using
metrics_url_timeout_seconds of 30.000000
[31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Using
task_metrics_timeout_seconds of 5.000000
[31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Using
max_collection_wait_seconds of 10.000000
[31/Jul/2013 17:53:27 +0000] 18298 MainThread metrics INFO
Importing tasktracker metric schema from file
/usr/lib/cmf/agent/src/cmf/monitor/tasktracker/schema.json
[31/Jul/2013 17:53:27 +0000] 18298 MainThread dns_names INFO Using
timeout of 2.000000
[31/Jul/2013 17:53:27 +0000] 18298 MainThread __init__ INFO
Importing metric schema from file
/usr/lib/cmf/agent/src/cmf/monitor/schema.json
[31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
Supervised processes will inherit the following environment: {'LANG':
'en_US.UTF-8', 'TERM': 'vt100', 'CMF_CONF_DIR': '/etc/cloudera-scm-agent',
'SHLVL': '1', 'PWD': '/', 'PATH':
'/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games'}
[31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
Supervised processes will add the following to their environment:
{'CDH_OOZIE_HOME': '/usr/lib/oozie', 'CDH_MR2_HOME':
'/usr/lib/hadoop-mapreduce', 'CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper',
'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME': '/usr/share/cmf',
'CDH_IMPALA_HOME': '/usr/lib/impala', 'CLOUDERA_MYSQL_CONNECTOR_JAR':
'/usr/share/java/mysql-connector-java.jar', 'CDH_YARN_HOME':
'/usr/lib/hadoop-yarn', 'CMF_PACKAGE_DIR': '/usr/lib/cmf/service',
'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs', 'CDH_SOLR_HOME': '/usr/lib/solr',
'CDH_PIG_HOME': '/usr/lib/pig', 'CDH_SQOOP2_HOME': '/usr/lib/sqoop2',
'CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CM_STATUS_CODES': u'STATUS_NONE
HDFS_DFS_DIR_NOT_EMPTY JOBTRACKER_IN_STANDBY_MODE', 'CDH_MR1_HOME':
'/usr/lib/hadoop-0.20-mapreduce', 'CLOUDERA_ORACLE_CONNECTOR_JAR':
'/usr/share/java/oracle-connector-java.jar', 'CDH_HUE_HOME':
'/usr/share/hue', 'CDH_HIVE_HOME': '/usr/lib/hive', 'CDH_HTTPFS_HOME':
'/usr/lib/hadoop-httpfs', 'CDH_HADOOP_HOME': '/usr/lib/hadoop',
'JSVC_HOME': '/usr/libexec/bigtop-utils', 'HIVE_DEFAULT_XML':
'/etc/hive/conf.dist/hive-default.xml', 'CDH_FLUME_HOME':
'/usr/lib/flume-ng', 'CLOUDERA_POSTGRESQL_JDBC_JAR':
'/usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar', 'TOMCAT_HOME':
'/usr/lib/bigtop-tomcat', 'CDH_HBASE_HOME': '/usr/lib/hbase',
'CDH_HCAT_HOME': '/usr/lib/hcatalog'}
[31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO To
override these variables, use /etc/cloudera-scm-agent/config.ini.
Environment variables for CDH locations are not used when CDH is installed
from parcels.
[31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Created
/run/cloudera-scm-agent/process
[31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
Chmod'ing /run/cloudera-scm-agent/process to 0751
[31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Created
/run/cloudera-scm-agent/supervisor
[31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
Chmod'ing /run/cloudera-scm-agent/supervisor to 0751
[31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Created
/run/cloudera-scm-agent/supervisor/include
[31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
Chmod'ing /run/cloudera-scm-agent/supervisor/include to 0751
[31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Failed
to connect to previous supervisor.
[31/Jul/2013 17:53:29 +0000] 18298 MainThread agent INFO Trying
to connect to newly launched supervisor (Attempt 1)
[31/Jul/2013 17:53:29 +0000] 18298 MainThread agent INFO
Successfully connected to supervisor
[31/Jul/2013 17:53:29 +0000] 18298 MainThread _cplogging INFO
[31/Jul/2013:17:53:29] ENGINE Bus STARTING
[31/Jul/2013 17:53:29 +0000] 18298 MainThread _cplogging INFO
[31/Jul/2013:17:53:29] ENGINE Started monitor thread '_TimeoutMonitor'.
[31/Jul/2013 17:53:29 +0000] 18298 MainThread _cplogging INFO
[31/Jul/2013:17:53:29] ENGINE Serving on ip-10-77-74-189.ec2.internal:9000
[31/Jul/2013 17:53:29 +0000] 18298 MainThread _cplogging INFO
[31/Jul/2013:17:53:29] ENGINE Bus STARTED
[31/Jul/2013 17:53:29 +0000] 18298 MainThread __init__ INFO New
monitor: (<cmf.monitor.host.hostmonitor object@0x7f4e98496790>,)
[31/Jul/2013 17:53:29 +0000] 18298 MainThread agent WARNING Setting
default socket timeout to 30!
[31/Jul/2013 17:53:29 +0000] 18298 MonitorDaemon-Scheduler __init__
INFO Monitor ready to report: (<cmf.monitor.host.HostMonitor object at
0x7f4e98496790>,)
[31/Jul/2013 17:53:29 +0000] 18298 MainThread agent INFO Created
/opt/cloudera/parcels
[31/Jul/2013 17:53:29 +0000] 18298 MainThread agent INFO
Chowning /opt/cloudera/parcels to root (0) root (0)
[31/Jul/2013 17:53:29 +0000] 18298 MainThread agent INFO
Chmod'ing /opt/cloudera/parcels to 0755
[31/Jul/2013 17:53:29 +0000] 18298 MainThread parcel INFO Agent
creates users/groups and applies file permissions: True
[31/Jul/2013 17:53:29 +0000] 18298 MainThread agent INFO Created
/opt/cloudera/parcel-cache
[31/Jul/2013 17:53:29 +0000] 18298 MainThread agent INFO
Chowning /opt/cloudera/parcel-cache to root (0) root (0)
[31/Jul/2013 17:53:29 +0000] 18298 MainThread agent INFO
Chmod'ing /opt/cloudera/parcel-cache to 0755
[31/Jul/2013 17:53:29 +0000] 18298 MainThread downloader INFO
Downloader path: /opt/cloudera/parcel-cache
[31/Jul/2013 17:53:29 +0000] 18298 MainThread parcel_cache INFO Using
/opt/cloudera/parcel-cache for parcel cache
[31/Jul/2013 17:53:34 +0000] 18298 Monitor-HostMonitor network_interfaces
INFO NIC iface eth0 doesn't support ETHTOOL (95)
[31/Jul/2013 18:00:43 +0000] 18298 MainThread agent INFO
Stopping agent...
[31/Jul/2013 18:00:43 +0000] 18298 MainThread agent INFO No
cgrouped processes running; unmounting any cgroup roots
[31/Jul/2013 18:00:43 +0000] 18298 MainThread agent INFO No
processes are being managed; Supervisor will shutdown.
[31/Jul/2013 18:00:43 +0000] 18298 MainThread agent INFO
Shutting down supervisord, pid 18318
[31/Jul/2013 18:00:43 +0000] 18298 MainThread agent INFO
Successfully shutdown supervisord, pid 18318
[31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
[31/Jul/2013:18:00:43] ENGINE Bus STOPPING
[31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
[31/Jul/2013:18:00:43] ENGINE HTTP Server
cherrypy._cpwsgi_server.CPWSGIServer(('ip-10-77-74-189.ec2.internal',
9000)) shut down
[31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
[31/Jul/2013:18:00:43] ENGINE Stopped thread '_TimeoutMonitor'.
[31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
[31/Jul/2013:18:00:43] ENGINE Bus STOPPED
[31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
[31/Jul/2013:18:00:43] ENGINE Bus STOPPING
[31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
[31/Jul/2013:18:00:43] ENGINE HTTP Server
cherrypy._cpwsgi_server.CPWSGIServer(('ip-10-77-74-189.ec2.internal',
9000)) already shut down
[31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
[31/Jul/2013:18:00:43] ENGINE No thread running for None.
[31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
[31/Jul/2013:18:00:43] ENGINE Bus STOPPED
[31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
[31/Jul/2013:18:00:43] ENGINE Bus EXITING
[31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
[31/Jul/2013:18:00:43] ENGINE Bus EXITED
[31/Jul/2013 18:00:43 +0000] 18298 MainThread agent INFO Agent
exiting; caught signal 15
[31/Jul/2013 18:00:43 +0000] 19950 MainThread agent INFO No
command line vars
[31/Jul/2013 18:00:43 +0000] 19950 MainThread agent INFO Missing
database jar: /usr/share/java/mysql-connector-java.jar (normal, if you're
not using this database type)
[31/Jul/2013 18:00:43 +0000] 19950 MainThread agent INFO Missing
database jar: /usr/share/java/oracle-connector-java.jar (normal, if you're
not using this database type)
[31/Jul/2013 18:00:43 +0000] 19950 MainThread agent INFO Found
database jar: /usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar
[31/Jul/2013 18:00:43 +0000] 19950 MainThread agent INFO Agent
starting as pid 19950 user root(0) group root(0).
[31/Jul/2013 18:00:43 +0000] 19950 MainThread agent INFO
Re-using pre-existing directory: /run/cloudera-scm-agent
[31/Jul/2013 18:00:43 +0000] 19950 MainThread agent INFO Agent
is running on a public cloud. Public hostname:
ec2-54-242-227-124.compute-1.amazonaws.com
[31/Jul/2013 18:00:43 +0000] 19950 MainThread agent INFO
Re-using pre-existing directory: /run/cloudera-scm-agent/cgroups
[31/Jul/2013 18:00:44 +0000] 19950 MainThread cgroups INFO Reusing
/run/cloudera-scm-agent/cgroups/memory
[31/Jul/2013 18:00:44 +0000] 19950 MainThread cgroups INFO Reusing
/run/cloudera-scm-agent/cgroups/cpu
[31/Jul/2013 18:00:44 +0000] 19950 MainThread cgroups INFO Reusing
/run/cloudera-scm-agent/cgroups/cpuacct
[31/Jul/2013 18:00:44 +0000] 19950 MainThread cgroups INFO Reusing
/run/cloudera-scm-agent/cgroups/blkio
[31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO Found
cgroups capabilities: {'has_memory': True, 'default_memory_limit_in_bytes':
9223372036854775807, 'default_memory_soft_limit_in_bytes':
9223372036854775807, 'writable_cgroup_dot_procs': True,
'default_cpu_rt_runtime_us': 950000, 'has_cpu': True,
'default_blkio_weight': 1000, 'default_cpu_shares': 1024, 'has_cpuacct':
True, 'has_blkio': True}
[31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO Setting
up supervisord event monitor.
[31/Jul/2013 18:00:44 +0000] 19950 MainThread filesystem_map INFO
Monitored nodev filesystem types: ['nfs', 'nfs4', 'tmpfs']
[31/Jul/2013 18:00:44 +0000] 19950 MainThread filesystem_map INFO Using
timeout of 2.000000
[31/Jul/2013 18:00:44 +0000] 19950 MainThread filesystem_map INFO Using
join timeout of 0.100000
[31/Jul/2013 18:00:44 +0000] 19950 MainThread filesystem_map INFO Using
tolerance of 60.000000
[31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO Using
metrics_url_timeout_seconds of 30.000000
[31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO Using
task_metrics_timeout_seconds of 5.000000
[31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO Using
max_collection_wait_seconds of 10.000000
[31/Jul/2013 18:00:44 +0000] 19950 MainThread metrics INFO
Importing tasktracker metric schema from file
/usr/lib/cmf/agent/src/cmf/monitor/tasktracker/schema.json
[31/Jul/2013 18:00:44 +0000] 19950 MainThread dns_names INFO Using
timeout of 2.000000
[31/Jul/2013 18:00:44 +0000] 19950 MainThread __init__ INFO
Importing metric schema from file
/usr/lib/cmf/agent/src/cmf/monitor/schema.json
[31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO
Supervised processes will inherit the following environment: {'LANG':
'en_US.UTF-8', 'TERM': 'vt100', 'CMF_CONF_DIR': '/etc/cloudera-scm-agent',
'SHLVL': '1', 'PWD': '/', 'PATH':
'/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games'}
[31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO
Supervised processes will add the following to their environment:
{'CDH_OOZIE_HOME': '/usr/lib/oozie', 'CDH_MR2_HOME':
'/usr/lib/hadoop-mapreduce', 'CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper',
'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME': '/usr/share/cmf',
'CDH_IMPALA_HOME': '/usr/lib/impala', 'CLOUDERA_MYSQL_CONNECTOR_JAR':
'/usr/share/java/mysql-connector-java.jar', 'CDH_YARN_HOME':
'/usr/lib/hadoop-yarn', 'CMF_PACKAGE_DIR': '/usr/lib/cmf/service',
'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs', 'CDH_SOLR_HOME': '/usr/lib/solr',
'CDH_PIG_HOME': '/usr/lib/pig', 'CDH_SQOOP2_HOME': '/usr/lib/sqoop2',
'CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CM_STATUS_CODES': u'STATUS_NONE
HDFS_DFS_DIR_NOT_EMPTY JOBTRACKER_IN_STANDBY_MODE', 'CDH_MR1_HOME':
'/usr/lib/hadoop-0.20-mapreduce', 'CLOUDERA_ORACLE_CONNECTOR_JAR':
'/usr/share/java/oracle-connector-java.jar', 'CDH_HUE_HOME':
'/usr/share/hue', 'CDH_HIVE_HOME': '/usr/lib/hive', 'CDH_HTTPFS_HOME':
'/usr/lib/hadoop-httpfs', 'CDH_HADOOP_HOME': '/usr/lib/hadoop',
'JSVC_HOME': '/usr/libexec/bigtop-utils', 'HIVE_DEFAULT_XML':
'/etc/hive/conf.dist/hive-default.xml', 'CDH_FLUME_HOME':
'/usr/lib/flume-ng', 'CLOUDERA_POSTGRESQL_JDBC_JAR':
'/usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar', 'TOMCAT_HOME':
'/usr/lib/bigtop-tomcat', 'CDH_HBASE_HOME': '/usr/lib/hbase',
'CDH_HCAT_HOME': '/usr/lib/hcatalog'}
[31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO To
override these variables, use /etc/cloudera-scm-agent/config.ini.
Environment variables for CDH locations are not used when CDH is installed
from parcels.
[31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO
Re-using pre-existing directory: /run/cloudera-scm-agent/process
[31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO
Re-using pre-existing directory: /run/cloudera-scm-agent/supervisor
[31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO
Re-using pre-existing directory: /run/cloudera-scm-agent/supervisor/include
[31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO Failed
to connect to previous supervisor.
[31/Jul/2013 18:00:46 +0000] 19950 MainThread agent INFO Trying
to connect to newly launched supervisor (Attempt 1)
[31/Jul/2013 18:00:46 +0000] 19950 MainThread agent INFO
Successfully connected to supervisor
[31/Jul/2013 18:00:46 +0000] 19950 MainThread _cplogging INFO
[31/Jul/2013:18:00:46] ENGINE Bus STARTING
[31/Jul/2013 18:00:46 +0000] 19950 MainThread _cplogging INFO
[31/Jul/2013:18:00:46] ENGINE Started monitor thread '_TimeoutMonitor'.
[31/Jul/2013 18:00:46 +0000] 19950 MainThread _cplogging INFO
[31/Jul/2013:18:00:46] ENGINE Serving on ip-10-77-74-189.ec2.internal:9000
[31/Jul/2013 18:00:46 +0000] 19950 MainThread _cplogging INFO
[31/Jul/2013:18:00:46] ENGINE Bus STARTED
[31/Jul/2013 18:00:46 +0000] 19950 MainThread __init__ INFO New
monitor: (<cmf.monitor.host.hostmonitor object@0x2b86f50>,)
[31/Jul/2013 18:00:46 +0000] 19950 MainThread agent WARNING Setting
default socket timeout to 30!
[31/Jul/2013 18:00:46 +0000] 19950 MonitorDaemon-Scheduler __init__
INFO Monitor ready to report: (<cmf.monitor.host.HostMonitor object at
0x2b86f50>,)
[31/Jul/2013 18:00:46 +0000] 19950 MainThread parcel INFO Agent
creates users/groups and applies file permissions: True
[31/Jul/2013 18:00:46 +0000] 19950 MainThread downloader INFO
Downloader path: /opt/cloudera/parcel-cache
[31/Jul/2013 18:00:46 +0000] 19950 MainThread parcel_cache INFO Using
/opt/cloudera/parcel-cache for parcel cache
[31/Jul/2013 18:00:51 +0000] 19950 Monitor-HostMonitor network_interfaces
INFO NIC iface eth0 doesn't support ETHTOOL (95)
[31/Jul/2013 18:16:32 +0000] 19950 MainThread agent INFO
Stopping agent...
[31/Jul/2013 18:16:32 +0000] 19950 MainThread agent INFO No
cgrouped processes running; unmounting any cgroup roots
[31/Jul/2013 18:16:32 +0000] 19950 MainThread agent INFO No
processes are being managed; Supervisor will shutdown.
[31/Jul/2013 18:16:32 +0000] 19950 MainThread agent INFO
Shutting down supervisord, pid 19969
[31/Jul/2013 18:16:32 +0000] 19950 MainThread agent INFO
Successfully shutdown supervisord, pid 19969
[31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
[31/Jul/2013:18:16:32] ENGINE Bus STOPPING
[31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
[31/Jul/2013:18:16:32] ENGINE HTTP Server
cherrypy._cpwsgi_server.CPWSGIServer(('ip-10-77-74-189.ec2.internal',
9000)) shut down
[31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
[31/Jul/2013:18:16:32] ENGINE Stopped thread '_TimeoutMonitor'.
[31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
[31/Jul/2013:18:16:32] ENGINE Bus STOPPED
[31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
[31/Jul/2013:18:16:32] ENGINE Bus STOPPING
[31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
[31/Jul/2013:18:16:32] ENGINE HTTP Server
cherrypy._cpwsgi_server.CPWSGIServer(('ip-10-77-74-189.ec2.internal',
9000)) already shut down
[31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
[31/Jul/2013:18:16:32] ENGINE No thread running for None.
[31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
[31/Jul/2013:18:16:32] ENGINE Bus STOPPED
[31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
[31/Jul/2013:18:16:32] ENGINE Bus EXITING
[31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
[31/Jul/2013:18:16:32] ENGINE Bus EXITED
[31/Jul/2013 18:16:32 +0000] 19950 MainThread agent INFO Agent
exiting; caught signal 15
[31/Jul/2013 18:16:32 +0000] 23193 MainThread agent INFO No
command line vars
[31/Jul/2013 18:16:32 +0000] 23193 MainThread agent INFO Missing
database jar: /usr/share/java/mysql-connector-java.jar (normal, if you're
not using this database type)
[31/Jul/2013 18:16:32 +0000] 23193 MainThread agent INFO Missing
database jar: /usr/share/java/oracle-connector-java.jar (normal, if you're
not using this database type)
[31/Jul/2013 18:16:32 +0000] 23193 MainThread agent INFO Found
database jar: /usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar
[31/Jul/2013 18:16:32 +0000] 23193 MainThread agent INFO Agent
starting as pid 23193 user root(0) group root(0).
[31/Jul/2013 18:16:32 +0000] 23193 MainThread agent INFO
Re-using pre-existing directory: /run/cloudera-scm-agent
[31/Jul/2013 18:16:32 +0000] 23193 MainThread agent INFO Agent
is running on a public cloud. Public hostname:
ec2-54-242-227-124.compute-1.amazonaws.com
[31/Jul/2013 18:16:32 +0000] 23193 MainThread agent INFO
Re-using pre-existing directory: /run/cloudera-scm-agent/cgroups
[31/Jul/2013 18:16:33 +0000] 23193 MainThread cgroups INFO Reusing
/run/cloudera-scm-agent/cgroups/memory
[31/Jul/2013 18:16:33 +0000] 23193 MainThread cgroups INFO Reusing
/run/cloudera-scm-agent/cgroups/cpu
[31/Jul/2013 18:16:33 +0000] 23193 MainThread cgroups INFO Reusing
/run/cloudera-scm-agent/cgroups/cpuacct
[31/Jul/2013 18:16:33 +0000] 23193 MainThread cgroups INFO Reusing
/run/cloudera-scm-agent/cgroups/blkio
[31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO Found
cgroups capabilities: {'has_memory': True, 'default_memory_limit_in_bytes':
9223372036854775807, 'default_memory_soft_limit_in_bytes':
9223372036854775807, 'writable_cgroup_dot_procs': True,
'default_cpu_rt_runtime_us': 950000, 'has_cpu': True,
'default_blkio_weight': 1000, 'default_cpu_shares': 1024, 'has_cpuacct':
True, 'has_blkio': True}
[31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO Setting
up supervisord event monitor.
[31/Jul/2013 18:16:33 +0000] 23193 MainThread filesystem_map INFO
Monitored nodev filesystem types: ['nfs', 'nfs4', 'tmpfs']
[31/Jul/2013 18:16:33 +0000] 23193 MainThread filesystem_map INFO Using
timeout of 2.000000
[31/Jul/2013 18:16:33 +0000] 23193 MainThread filesystem_map INFO Using
join timeout of 0.100000
[31/Jul/2013 18:16:33 +0000] 23193 MainThread filesystem_map INFO Using
tolerance of 60.000000
[31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO Using
metrics_url_timeout_seconds of 30.000000
[31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO Using
task_metrics_timeout_seconds of 5.000000
[31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO Using
max_collection_wait_seconds of 10.000000
[31/Jul/2013 18:16:33 +0000] 23193 MainThread metrics INFO
Importing tasktracker metric schema from file
/usr/lib/cmf/agent/src/cmf/monitor/tasktracker/schema.json
[31/Jul/2013 18:16:33 +0000] 23193 MainThread dns_names INFO Using
timeout of 2.000000
[31/Jul/2013 18:16:33 +0000] 23193 MainThread __init__ INFO
Importing metric schema from file
/usr/lib/cmf/agent/src/cmf/monitor/schema.json
[31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO
Supervised processes will inherit the following environment: {'LANG':
'en_US.UTF-8', 'TERM': 'vt100', 'CMF_CONF_DIR': '/etc/cloudera-scm-agent',
'SHLVL': '1', 'PWD': '/', 'PATH':
'/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games'}
[31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO
Supervised processes will add the following to their environment:
{'CDH_OOZIE_HOME': '/usr/lib/oozie', 'CDH_MR2_HOME':
'/usr/lib/hadoop-mapreduce', 'CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper',
'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME': '/usr/share/cmf',
'CDH_IMPALA_HOME': '/usr/lib/impala', 'CLOUDERA_MYSQL_CONNECTOR_JAR':
'/usr/share/java/mysql-connector-java.jar', 'CDH_YARN_HOME':
'/usr/lib/hadoop-yarn', 'CMF_PACKAGE_DIR': '/usr/lib/cmf/service',
'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs', 'CDH_SOLR_HOME': '/usr/lib/solr',
'CDH_PIG_HOME': '/usr/lib/pig', 'CDH_SQOOP2_HOME': '/usr/lib/sqoop2',
'CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CM_STATUS_CODES': u'STATUS_NONE
HDFS_DFS_DIR_NOT_EMPTY JOBTRACKER_IN_STANDBY_MODE', 'CDH_MR1_HOME':
'/usr/lib/hadoop-0.20-mapreduce', 'CLOUDERA_ORACLE_CONNECTOR_JAR':
'/usr/share/java/oracle-connector-java.jar', 'CDH_HUE_HOME':
'/usr/share/hue', 'CDH_HIVE_HOME': '/usr/lib/hive', 'CDH_HTTPFS_HOME':
'/usr/lib/hadoop-httpfs', 'CDH_HADOOP_HOME': '/usr/lib/hadoop',
'JSVC_HOME': '/usr/libexec/bigtop-utils', 'HIVE_DEFAULT_XML':
'/etc/hive/conf.dist/hive-default.xml', 'CDH_FLUME_HOME':
'/usr/lib/flume-ng', 'CLOUDERA_POSTGRESQL_JDBC_JAR':
'/usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar', 'TOMCAT_HOME':
'/usr/lib/bigtop-tomcat', 'CDH_HBASE_HOME': '/usr/lib/hbase',
'CDH_HCAT_HOME': '/usr/lib/hcatalog'}
[31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO To
override these variables, use /etc/cloudera-scm-agent/config.ini.
Environment variables for CDH locations are not used when CDH is installed
from parcels.
[31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO
Re-using pre-existing directory: /run/cloudera-scm-agent/process
[31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO
Re-using pre-existing directory: /run/cloudera-scm-agent/supervisor
[31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO
Re-using pre-existing directory: /run/cloudera-scm-agent/supervisor/include
[31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO Failed
to connect to previous supervisor.
[31/Jul/2013 18:16:35 +0000] 23193 MainThread agent INFO Trying
to connect to newly launched supervisor (Attempt 1)
[31/Jul/2013 18:16:35 +0000] 23193 MainThread agent INFO
Successfully connected to supervisor
[31/Jul/2013 18:16:35 +0000] 23193 MainThread _cplogging INFO
[31/Jul/2013:18:16:35] ENGINE Bus STARTING
[31/Jul/2013 18:16:35 +0000] 23193 MainThread _cplogging INFO
[31/Jul/2013:18:16:35] ENGINE Started monitor thread '_TimeoutMonitor'.
[31/Jul/2013 18:16:35 +0000] 23193 MainThread _cplogging INFO
[31/Jul/2013:18:16:35] ENGINE Serving on
ec2-54-242-227-124.compute-1.amazonaws.com:9000
[31/Jul/2013 18:16:35 +0000] 23193 MainThread _cplogging INFO
[31/Jul/2013:18:16:35] ENGINE Bus STARTED
[31/Jul/2013 18:16:35 +0000] 23193 MainThread __init__ INFO New
monitor: (<cmf.monitor.host.hostmonitor object@0x3230f50>,)
[31/Jul/2013 18:16:35 +0000] 23193 MainThread agent WARNING Setting
default socket timeout to 30!
[31/Jul/2013 18:16:35 +0000] 23193 MonitorDaemon-Scheduler __init__
INFO Monitor ready to report: (<cmf.monitor.host.HostMonitor object at
0x3230f50>,)
[31/Jul/2013 18:16:35 +0000] 23193 MainThread parcel INFO Agent
creates users/groups and applies file permissions: True
[31/Jul/2013 18:16:35 +0000] 23193 MainThread downloader INFO
Downloader path: /opt/cloudera/parcel-cache
[31/Jul/2013 18:16:35 +0000] 23193 MainThread parcel_cache INFO Using
/opt/cloudera/parcel-cache for parcel cache
[31/Jul/2013 18:16:40 +0000] 23193 Monitor-HostMonitor network_interfaces
INFO NIC iface eth0 doesn't support ETHTOOL (95)



Any help would be greatly appreciated!

Thanks,
Ryan

Search Discussions

  • Ryan Alexander at Jul 31, 2013 at 6:49 pm
    Oh and all ports are open for all the boxes, forgot to mention that.
    On Wednesday, July 31, 2013 1:46:58 PM UTC-5, Ryan Alexander wrote:

    Hello, I am trying to set up a Hadoop/Hbase cluster on EC2. I have to use
    Hadoop 1.0.4 and HBase 0.94, so I followed this guide:

    http://rogueleaderr.com/post/32768181371/set-up-and-run-a-fully-distributed-hadoop-hbase-cluster

    I realize the instructions are depreciated, but I know the latest Cloudera
    CDH doesn't run Hadoop 1.0.4.

    I installed the correct versions of Hadoop and Hbase from the Apache
    archives and set up the cluster using Whirr and all went well. Haddop and
    Hbase are running - however, the configurations and start/stop scripts
    weren't working well, so I thought installing the Cloudera Manager would
    help in that regard.

    Anyway, I give the install process the IPs of my boxes (I have 1 master
    and 3 slaves) and confirming that they were reachable by pings, I started
    the installation process- all was going well until the process was waiting
    for a heartbeat and each of my boxes failed to give one and then the
    install failed.

    I figured maybe the internal ip's of the EC2 instances was the problem,
    changed all the hostnames to their FQDNs and still the same issue happened.

    My log file is as follows:

    31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO No
    command line vars
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Missing database jar: /usr/share/java/mysql-connector-java.jar (normal, if
    you're not using this database type)
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Missing database jar: /usr/share/java/oracle-connector-java.jar (normal, if
    you're not using this database type)
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Found
    database jar: /usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Agent
    starting as pid 18298 user root(0) group root(0).
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Created /run/cloudera-scm-agent
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Chmod'ing /run/cloudera-scm-agent to 0751
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Agent
    is running on a public cloud. Public hostname:
    ec2-54-242-227-124.compute-1.amazonaws.com
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Created /run/cloudera-scm-agent/cgroups
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Chmod'ing /run/cloudera-scm-agent/cgroups to 0751
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread cgroups INFO
    Created /run/cloudera-scm-agent/cgroups/memory
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread cgroups INFO
    Created /run/cloudera-scm-agent/cgroups/cpu
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread cgroups INFO
    Created /run/cloudera-scm-agent/cgroups/cpuacct
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread cgroups INFO
    Created /run/cloudera-scm-agent/cgroups/blkio
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Found
    cgroups capabilities: {'has_memory': True, 'default_memory_limit_in_bytes':
    9223372036854775807, 'default_memory_soft_limit_in_bytes':
    9223372036854775807, 'writable_cgroup_dot_procs': True,
    'default_cpu_rt_runtime_us': 950000, 'has_cpu': True,
    'default_blkio_weight': 1000, 'default_cpu_shares': 1024, 'has_cpuacct':
    True, 'has_blkio': True}
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Setting up supervisord event monitor.
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread filesystem_map INFO
    Monitored nodev filesystem types: ['nfs', 'nfs4', 'tmpfs']
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread filesystem_map INFO
    Using timeout of 2.000000
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread filesystem_map INFO
    Using join timeout of 0.100000
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread filesystem_map INFO
    Using tolerance of 60.000000
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Using
    metrics_url_timeout_seconds of 30.000000
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Using
    task_metrics_timeout_seconds of 5.000000
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Using
    max_collection_wait_seconds of 10.000000
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread metrics INFO
    Importing tasktracker metric schema from file
    /usr/lib/cmf/agent/src/cmf/monitor/tasktracker/schema.json
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread dns_names INFO Using
    timeout of 2.000000
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread __init__ INFO
    Importing metric schema from file
    /usr/lib/cmf/agent/src/cmf/monitor/schema.json
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Supervised processes will inherit the following environment: {'LANG':
    'en_US.UTF-8', 'TERM': 'vt100', 'CMF_CONF_DIR': '/etc/cloudera-scm-agent',
    'SHLVL': '1', 'PWD': '/', 'PATH':
    '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games'}
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Supervised processes will add the following to their environment:
    {'CDH_OOZIE_HOME': '/usr/lib/oozie', 'CDH_MR2_HOME':
    '/usr/lib/hadoop-mapreduce', 'CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper',
    'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME': '/usr/share/cmf',
    'CDH_IMPALA_HOME': '/usr/lib/impala', 'CLOUDERA_MYSQL_CONNECTOR_JAR':
    '/usr/share/java/mysql-connector-java.jar', 'CDH_YARN_HOME':
    '/usr/lib/hadoop-yarn', 'CMF_PACKAGE_DIR': '/usr/lib/cmf/service',
    'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs', 'CDH_SOLR_HOME': '/usr/lib/solr',
    'CDH_PIG_HOME': '/usr/lib/pig', 'CDH_SQOOP2_HOME': '/usr/lib/sqoop2',
    'CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CM_STATUS_CODES': u'STATUS_NONE
    HDFS_DFS_DIR_NOT_EMPTY JOBTRACKER_IN_STANDBY_MODE', 'CDH_MR1_HOME':
    '/usr/lib/hadoop-0.20-mapreduce', 'CLOUDERA_ORACLE_CONNECTOR_JAR':
    '/usr/share/java/oracle-connector-java.jar', 'CDH_HUE_HOME':
    '/usr/share/hue', 'CDH_HIVE_HOME': '/usr/lib/hive', 'CDH_HTTPFS_HOME':
    '/usr/lib/hadoop-httpfs', 'CDH_HADOOP_HOME': '/usr/lib/hadoop',
    'JSVC_HOME': '/usr/libexec/bigtop-utils', 'HIVE_DEFAULT_XML':
    '/etc/hive/conf.dist/hive-default.xml', 'CDH_FLUME_HOME':
    '/usr/lib/flume-ng', 'CLOUDERA_POSTGRESQL_JDBC_JAR':
    '/usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar', 'TOMCAT_HOME':
    '/usr/lib/bigtop-tomcat', 'CDH_HBASE_HOME': '/usr/lib/hbase',
    'CDH_HCAT_HOME': '/usr/lib/hcatalog'}
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO To
    override these variables, use /etc/cloudera-scm-agent/config.ini.
    Environment variables for CDH locations are not used when CDH is installed
    from parcels.
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Created /run/cloudera-scm-agent/process
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Chmod'ing /run/cloudera-scm-agent/process to 0751
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Created /run/cloudera-scm-agent/supervisor
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Chmod'ing /run/cloudera-scm-agent/supervisor to 0751
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Created /run/cloudera-scm-agent/supervisor/include
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Chmod'ing /run/cloudera-scm-agent/supervisor/include to 0751
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Failed
    to connect to previous supervisor.
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread agent INFO Trying
    to connect to newly launched supervisor (Attempt 1)
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread agent INFO
    Successfully connected to supervisor
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:17:53:29] ENGINE Bus STARTING
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:17:53:29] ENGINE Started monitor thread '_TimeoutMonitor'.
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:17:53:29] ENGINE Serving on ip-10-77-74-189.ec2.internal:9000
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:17:53:29] ENGINE Bus STARTED
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread __init__ INFO New
    monitor: (<cmf.monitor.host.hostmonitor object@0x7f4e98496790>,)
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread agent WARNING
    Setting default socket timeout to 30!
    [31/Jul/2013 17:53:29 +0000] 18298 MonitorDaemon-Scheduler __init__
    INFO Monitor ready to report: (<cmf.monitor.host.HostMonitor object at
    0x7f4e98496790>,)
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread agent INFO
    Created /opt/cloudera/parcels
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread agent INFO
    Chowning /opt/cloudera/parcels to root (0) root (0)
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread agent INFO
    Chmod'ing /opt/cloudera/parcels to 0755
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread parcel INFO Agent
    creates users/groups and applies file permissions: True
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread agent INFO
    Created /opt/cloudera/parcel-cache
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread agent INFO
    Chowning /opt/cloudera/parcel-cache to root (0) root (0)
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread agent INFO
    Chmod'ing /opt/cloudera/parcel-cache to 0755
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread downloader INFO
    Downloader path: /opt/cloudera/parcel-cache
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread parcel_cache INFO Using
    /opt/cloudera/parcel-cache for parcel cache
    [31/Jul/2013 17:53:34 +0000] 18298 Monitor-HostMonitor network_interfaces
    INFO NIC iface eth0 doesn't support ETHTOOL (95)
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread agent INFO
    Stopping agent...
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread agent INFO No
    cgrouped processes running; unmounting any cgroup roots
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread agent INFO No
    processes are being managed; Supervisor will shutdown.
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread agent INFO
    Shutting down supervisord, pid 18318
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread agent INFO
    Successfully shutdown supervisord, pid 18318
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:18:00:43] ENGINE Bus STOPPING
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:18:00:43] ENGINE HTTP Server
    cherrypy._cpwsgi_server.CPWSGIServer(('ip-10-77-74-189.ec2.internal',
    9000)) shut down
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:18:00:43] ENGINE Stopped thread '_TimeoutMonitor'.
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:18:00:43] ENGINE Bus STOPPED
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:18:00:43] ENGINE Bus STOPPING
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:18:00:43] ENGINE HTTP Server
    cherrypy._cpwsgi_server.CPWSGIServer(('ip-10-77-74-189.ec2.internal',
    9000)) already shut down
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:18:00:43] ENGINE No thread running for None.
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:18:00:43] ENGINE Bus STOPPED
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:18:00:43] ENGINE Bus EXITING
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:18:00:43] ENGINE Bus EXITED
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread agent INFO Agent
    exiting; caught signal 15
    [31/Jul/2013 18:00:43 +0000] 19950 MainThread agent INFO No
    command line vars
    [31/Jul/2013 18:00:43 +0000] 19950 MainThread agent INFO
    Missing database jar: /usr/share/java/mysql-connector-java.jar (normal, if
    you're not using this database type)
    [31/Jul/2013 18:00:43 +0000] 19950 MainThread agent INFO
    Missing database jar: /usr/share/java/oracle-connector-java.jar (normal, if
    you're not using this database type)
    [31/Jul/2013 18:00:43 +0000] 19950 MainThread agent INFO Found
    database jar: /usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar
    [31/Jul/2013 18:00:43 +0000] 19950 MainThread agent INFO Agent
    starting as pid 19950 user root(0) group root(0).
    [31/Jul/2013 18:00:43 +0000] 19950 MainThread agent INFO
    Re-using pre-existing directory: /run/cloudera-scm-agent
    [31/Jul/2013 18:00:43 +0000] 19950 MainThread agent INFO Agent
    is running on a public cloud. Public hostname:
    ec2-54-242-227-124.compute-1.amazonaws.com
    [31/Jul/2013 18:00:43 +0000] 19950 MainThread agent INFO
    Re-using pre-existing directory: /run/cloudera-scm-agent/cgroups
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread cgroups INFO
    Reusing /run/cloudera-scm-agent/cgroups/memory
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread cgroups INFO
    Reusing /run/cloudera-scm-agent/cgroups/cpu
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread cgroups INFO
    Reusing /run/cloudera-scm-agent/cgroups/cpuacct
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread cgroups INFO
    Reusing /run/cloudera-scm-agent/cgroups/blkio
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO Found
    cgroups capabilities: {'has_memory': True, 'default_memory_limit_in_bytes':
    9223372036854775807, 'default_memory_soft_limit_in_bytes':
    9223372036854775807, 'writable_cgroup_dot_procs': True,
    'default_cpu_rt_runtime_us': 950000, 'has_cpu': True,
    'default_blkio_weight': 1000, 'default_cpu_shares': 1024, 'has_cpuacct':
    True, 'has_blkio': True}
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO
    Setting up supervisord event monitor.
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread filesystem_map INFO
    Monitored nodev filesystem types: ['nfs', 'nfs4', 'tmpfs']
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread filesystem_map INFO
    Using timeout of 2.000000
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread filesystem_map INFO
    Using join timeout of 0.100000
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread filesystem_map INFO
    Using tolerance of 60.000000
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO Using
    metrics_url_timeout_seconds of 30.000000
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO Using
    task_metrics_timeout_seconds of 5.000000
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO Using
    max_collection_wait_seconds of 10.000000
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread metrics INFO
    Importing tasktracker metric schema from file
    /usr/lib/cmf/agent/src/cmf/monitor/tasktracker/schema.json
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread dns_names INFO Using
    timeout of 2.000000
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread __init__ INFO
    Importing metric schema from file
    /usr/lib/cmf/agent/src/cmf/monitor/schema.json
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO
    Supervised processes will inherit the following environment: {'LANG':
    'en_US.UTF-8', 'TERM': 'vt100', 'CMF_CONF_DIR': '/etc/cloudera-scm-agent',
    'SHLVL': '1', 'PWD': '/', 'PATH':
    '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games'}
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO
    Supervised processes will add the following to their environment:
    {'CDH_OOZIE_HOME': '/usr/lib/oozie', 'CDH_MR2_HOME':
    '/usr/lib/hadoop-mapreduce', 'CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper',
    'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME': '/usr/share/cmf',
    'CDH_IMPALA_HOME': '/usr/lib/impala', 'CLOUDERA_MYSQL_CONNECTOR_JAR':
    '/usr/share/java/mysql-connector-java.jar', 'CDH_YARN_HOME':
    '/usr/lib/hadoop-yarn', 'CMF_PACKAGE_DIR': '/usr/lib/cmf/service',
    'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs', 'CDH_SOLR_HOME': '/usr/lib/solr',
    'CDH_PIG_HOME': '/usr/lib/pig', 'CDH_SQOOP2_HOME': '/usr/lib/sqoop2',
    'CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CM_STATUS_CODES': u'STATUS_NONE
    HDFS_DFS_DIR_NOT_EMPTY JOBTRACKER_IN_STANDBY_MODE', 'CDH_MR1_HOME':
    '/usr/lib/hadoop-0.20-mapreduce', 'CLOUDERA_ORACLE_CONNECTOR_JAR':
    '/usr/share/java/oracle-connector-java.jar', 'CDH_HUE_HOME':
    '/usr/share/hue', 'CDH_HIVE_HOME': '/usr/lib/hive', 'CDH_HTTPFS_HOME':
    '/usr/lib/hadoop-httpfs', 'CDH_HADOOP_HOME': '/usr/lib/hadoop',
    'JSVC_HOME': '/usr/libexec/bigtop-utils', 'HIVE_DEFAULT_XML':
    '/etc/hive/conf.dist/hive-default.xml', 'CDH_FLUME_HOME':
    '/usr/lib/flume-ng', 'CLOUDERA_POSTGRESQL_JDBC_JAR':
    '/usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar', 'TOMCAT_HOME':
    '/usr/lib/bigtop-tomcat', 'CDH_HBASE_HOME': '/usr/lib/hbase',
    'CDH_HCAT_HOME': '/usr/lib/hcatalog'}
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO To
    override these variables, use /etc/cloudera-scm-agent/config.ini.
    Environment variables for CDH locations are not used when CDH is installed
    from parcels.
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO
    Re-using pre-existing directory: /run/cloudera-scm-agent/process
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO
    Re-using pre-existing directory: /run/cloudera-scm-agent/supervisor
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO
    Re-using pre-existing directory: /run/cloudera-scm-agent/supervisor/include
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO Failed
    to connect to previous supervisor.
    [31/Jul/2013 18:00:46 +0000] 19950 MainThread agent INFO Trying
    to connect to newly launched supervisor (Attempt 1)
    [31/Jul/2013 18:00:46 +0000] 19950 MainThread agent INFO
    Successfully connected to supervisor
    [31/Jul/2013 18:00:46 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:00:46] ENGINE Bus STARTING
    [31/Jul/2013 18:00:46 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:00:46] ENGINE Started monitor thread '_TimeoutMonitor'.
    [31/Jul/2013 18:00:46 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:00:46] ENGINE Serving on ip-10-77-74-189.ec2.internal:9000
    [31/Jul/2013 18:00:46 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:00:46] ENGINE Bus STARTED
    [31/Jul/2013 18:00:46 +0000] 19950 MainThread __init__ INFO New
    monitor: (<cmf.monitor.host.hostmonitor object@0x2b86f50>,)
    [31/Jul/2013 18:00:46 +0000] 19950 MainThread agent WARNING
    Setting default socket timeout to 30!
    [31/Jul/2013 18:00:46 +0000] 19950 MonitorDaemon-Scheduler __init__
    INFO Monitor ready to report: (<cmf.monitor.host.HostMonitor object at
    0x2b86f50>,)
    [31/Jul/2013 18:00:46 +0000] 19950 MainThread parcel INFO Agent
    creates users/groups and applies file permissions: True
    [31/Jul/2013 18:00:46 +0000] 19950 MainThread downloader INFO
    Downloader path: /opt/cloudera/parcel-cache
    [31/Jul/2013 18:00:46 +0000] 19950 MainThread parcel_cache INFO Using
    /opt/cloudera/parcel-cache for parcel cache
    [31/Jul/2013 18:00:51 +0000] 19950 Monitor-HostMonitor network_interfaces
    INFO NIC iface eth0 doesn't support ETHTOOL (95)
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread agent INFO
    Stopping agent...
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread agent INFO No
    cgrouped processes running; unmounting any cgroup roots
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread agent INFO No
    processes are being managed; Supervisor will shutdown.
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread agent INFO
    Shutting down supervisord, pid 19969
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread agent INFO
    Successfully shutdown supervisord, pid 19969
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:16:32] ENGINE Bus STOPPING
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:16:32] ENGINE HTTP Server
    cherrypy._cpwsgi_server.CPWSGIServer(('ip-10-77-74-189.ec2.internal',
    9000)) shut down
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:16:32] ENGINE Stopped thread '_TimeoutMonitor'.
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:16:32] ENGINE Bus STOPPED
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:16:32] ENGINE Bus STOPPING
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:16:32] ENGINE HTTP Server
    cherrypy._cpwsgi_server.CPWSGIServer(('ip-10-77-74-189.ec2.internal',
    9000)) already shut down
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:16:32] ENGINE No thread running for None.
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:16:32] ENGINE Bus STOPPED
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:16:32] ENGINE Bus EXITING
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:16:32] ENGINE Bus EXITED
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread agent INFO Agent
    exiting; caught signal 15
    [31/Jul/2013 18:16:32 +0000] 23193 MainThread agent INFO No
    command line vars
    [31/Jul/2013 18:16:32 +0000] 23193 MainThread agent INFO
    Missing database jar: /usr/share/java/mysql-connector-java.jar (normal, if
    you're not using this database type)
    [31/Jul/2013 18:16:32 +0000] 23193 MainThread agent INFO
    Missing database jar: /usr/share/java/oracle-connector-java.jar (normal, if
    you're not using this database type)
    [31/Jul/2013 18:16:32 +0000] 23193 MainThread agent INFO Found
    database jar: /usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar
    [31/Jul/2013 18:16:32 +0000] 23193 MainThread agent INFO Agent
    starting as pid 23193 user root(0) group root(0).
    [31/Jul/2013 18:16:32 +0000] 23193 MainThread agent INFO
    Re-using pre-existing directory: /run/cloudera-scm-agent
    [31/Jul/2013 18:16:32 +0000] 23193 MainThread agent INFO Agent
    is running on a public cloud. Public hostname:
    ec2-54-242-227-124.compute-1.amazonaws.com
    [31/Jul/2013 18:16:32 +0000] 23193 MainThread agent INFO
    Re-using pre-existing directory: /run/cloudera-scm-agent/cgroups
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread cgroups INFO
    Reusing /run/cloudera-scm-agent/cgroups/memory
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread cgroups INFO
    Reusing /run/cloudera-scm-agent/cgroups/cpu
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread cgroups INFO
    Reusing /run/cloudera-scm-agent/cgroups/cpuacct
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread cgroups INFO
    Reusing /run/cloudera-scm-agent/cgroups/blkio
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO Found
    cgroups capabilities: {'has_memory': True, 'default_memory_limit_in_bytes':
    9223372036854775807, 'default_memory_soft_limit_in_bytes':
    9223372036854775807, 'writable_cgroup_dot_procs': True,
    'default_cpu_rt_runtime_us': 950000, 'has_cpu': True,
    'default_blkio_weight': 1000, 'default_cpu_shares': 1024, 'has_cpuacct':
    True, 'has_blkio': True}
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO
    Setting up supervisord event monitor.
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread filesystem_map INFO
    Monitored nodev filesystem types: ['nfs', 'nfs4', 'tmpfs']
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread filesystem_map INFO
    Using timeout of 2.000000
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread filesystem_map INFO
    Using join timeout of 0.100000
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread filesystem_map INFO
    Using tolerance of 60.000000
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO Using
    metrics_url_timeout_seconds of 30.000000
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO Using
    task_metrics_timeout_seconds of 5.000000
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO Using
    max_collection_wait_seconds of 10.000000
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread metrics INFO
    Importing tasktracker metric schema from file
    /usr/lib/cmf/agent/src/cmf/monitor/tasktracker/schema.json
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread dns_names INFO Using
    timeout of 2.000000
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread __init__ INFO
    Importing metric schema from file
    /usr/lib/cmf/agent/src/cmf/monitor/schema.json
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO
    Supervised processes will inherit the following environment: {'LANG':
    'en_US.UTF-8', 'TERM': 'vt100', 'CMF_CONF_DIR': '/etc/cloudera-scm-agent',
    'SHLVL': '1', 'PWD': '/', 'PATH':
    '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games'}
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO
    Supervised processes will add the following to their environment:
    {'CDH_OOZIE_HOME': '/usr/lib/oozie', 'CDH_MR2_HOME':
    '/usr/lib/hadoop-mapreduce', 'CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper',
    'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME': '/usr/share/cmf',
    'CDH_IMPALA_HOME': '/usr/lib/impala', 'CLOUDERA_MYSQL_CONNECTOR_JAR':
    '/usr/share/java/mysql-connector-java.jar', 'CDH_YARN_HOME':
    '/usr/lib/hadoop-yarn', 'CMF_PACKAGE_DIR': '/usr/lib/cmf/service',
    'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs', 'CDH_SOLR_HOME': '/usr/lib/solr',
    'CDH_PIG_HOME': '/usr/lib/pig', 'CDH_SQOOP2_HOME': '/usr/lib/sqoop2',
    'CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CM_STATUS_CODES': u'STATUS_NONE
    HDFS_DFS_DIR_NOT_EMPTY JOBTRACKER_IN_STANDBY_MODE', 'CDH_MR1_HOME':
    '/usr/lib/hadoop-0.20-mapreduce', 'CLOUDERA_ORACLE_CONNECTOR_JAR':
    '/usr/share/java/oracle-connector-java.jar', 'CDH_HUE_HOME':
    '/usr/share/hue', 'CDH_HIVE_HOME': '/usr/lib/hive', 'CDH_HTTPFS_HOME':
    '/usr/lib/hadoop-httpfs', 'CDH_HADOOP_HOME': '/usr/lib/hadoop',
    'JSVC_HOME': '/usr/libexec/bigtop-utils', 'HIVE_DEFAULT_XML':
    '/etc/hive/conf.dist/hive-default.xml', 'CDH_FLUME_HOME':
    '/usr/lib/flume-ng', 'CLOUDERA_POSTGRESQL_JDBC_JAR':
    '/usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar', 'TOMCAT_HOME':
    '/usr/lib/bigtop-tomcat', 'CDH_HBASE_HOME': '/usr/lib/hbase',
    'CDH_HCAT_HOME': '/usr/lib/hcatalog'}
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO To
    override these variables, use /etc/cloudera-scm-agent/config.ini.
    Environment variables for CDH locations are not used when CDH is installed
    from parcels.
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO
    Re-using pre-existing directory: /run/cloudera-scm-agent/process
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO
    Re-using pre-existing directory: /run/cloudera-scm-agent/supervisor
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO
    Re-using pre-existing directory: /run/cloudera-scm-agent/supervisor/include
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO Failed
    to connect to previous supervisor.
    [31/Jul/2013 18:16:35 +0000] 23193 MainThread agent INFO Trying
    to connect to newly launched supervisor (Attempt 1)
    [31/Jul/2013 18:16:35 +0000] 23193 MainThread agent INFO
    Successfully connected to supervisor
    [31/Jul/2013 18:16:35 +0000] 23193 MainThread _cplogging INFO
    [31/Jul/2013:18:16:35] ENGINE Bus STARTING
    [31/Jul/2013 18:16:35 +0000] 23193 MainThread _cplogging INFO
    [31/Jul/2013:18:16:35] ENGINE Started monitor thread '_TimeoutMonitor'.
    [31/Jul/2013 18:16:35 +0000] 23193 MainThread _cplogging INFO
    [31/Jul/2013:18:16:35] ENGINE Serving on
    ec2-54-242-227-124.compute-1.amazonaws.com:9000
    [31/Jul/2013 18:16:35 +0000] 23193 MainThread _cplogging INFO
    [31/Jul/2013:18:16:35] ENGINE Bus STARTED
    [31/Jul/2013 18:16:35 +0000] 23193 MainThread __init__ INFO New
    monitor: (<cmf.monitor.host.hostmonitor object@0x3230f50>,)
    [31/Jul/2013 18:16:35 +0000] 23193 MainThread agent WARNING
    Setting default socket timeout to 30!
    [31/Jul/2013 18:16:35 +0000] 23193 MonitorDaemon-Scheduler __init__
    INFO Monitor ready to report: (<cmf.monitor.host.HostMonitor object at
    0x3230f50>,)
    [31/Jul/2013 18:16:35 +0000] 23193 MainThread parcel INFO Agent
    creates users/groups and applies file permissions: True
    [31/Jul/2013 18:16:35 +0000] 23193 MainThread downloader INFO
    Downloader path: /opt/cloudera/parcel-cache
    [31/Jul/2013 18:16:35 +0000] 23193 MainThread parcel_cache INFO Using
    /opt/cloudera/parcel-cache for parcel cache
    [31/Jul/2013 18:16:40 +0000] 23193 Monitor-HostMonitor network_interfaces
    INFO NIC iface eth0 doesn't support ETHTOOL (95)



    Any help would be greatly appreciated!

    Thanks,
    Ryan
  • Ryan Alexander at Jul 31, 2013 at 6:49 pm
    Oh and all ports for all the boxes are open, forgot to mention that
  • James Chen at Jan 10, 2014 at 10:53 pm
    Any answer to this question? I have the same problem...

    Thanks.

    James

    On Wednesday, July 31, 2013 11:46:58 AM UTC-7, Ryan Alexander wrote:

    Hello, I am trying to set up a Hadoop/Hbase cluster on EC2. I have to use
    Hadoop 1.0.4 and HBase 0.94, so I followed this guide:

    http://rogueleaderr.com/post/32768181371/set-up-and-run-a-fully-distributed-hadoop-hbase-cluster

    I realize the instructions are depreciated, but I know the latest Cloudera
    CDH doesn't run Hadoop 1.0.4.

    I installed the correct versions of Hadoop and Hbase from the Apache
    archives and set up the cluster using Whirr and all went well. Haddop and
    Hbase are running - however, the configurations and start/stop scripts
    weren't working well, so I thought installing the Cloudera Manager would
    help in that regard.

    Anyway, I give the install process the IPs of my boxes (I have 1 master
    and 3 slaves) and confirming that they were reachable by pings, I started
    the installation process- all was going well until the process was waiting
    for a heartbeat and each of my boxes failed to give one and then the
    install failed.

    I figured maybe the internal ip's of the EC2 instances was the problem,
    changed all the hostnames to their FQDNs and still the same issue happened.

    My log file is as follows:

    31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO No
    command line vars
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Missing database jar: /usr/share/java/mysql-connector-java.jar (normal, if
    you're not using this database type)
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Missing database jar: /usr/share/java/oracle-connector-java.jar (normal, if
    you're not using this database type)
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Found
    database jar: /usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Agent
    starting as pid 18298 user root(0) group root(0).
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Created /run/cloudera-scm-agent
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Chmod'ing /run/cloudera-scm-agent to 0751
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Agent
    is running on a public cloud. Public hostname:
    ec2-54-242-227-124.compute-1.amazonaws.com
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Created /run/cloudera-scm-agent/cgroups
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Chmod'ing /run/cloudera-scm-agent/cgroups to 0751
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread cgroups INFO
    Created /run/cloudera-scm-agent/cgroups/memory
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread cgroups INFO
    Created /run/cloudera-scm-agent/cgroups/cpu
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread cgroups INFO
    Created /run/cloudera-scm-agent/cgroups/cpuacct
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread cgroups INFO
    Created /run/cloudera-scm-agent/cgroups/blkio
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Found
    cgroups capabilities: {'has_memory': True, 'default_memory_limit_in_bytes':
    9223372036854775807, 'default_memory_soft_limit_in_bytes':
    9223372036854775807, 'writable_cgroup_dot_procs': True,
    'default_cpu_rt_runtime_us': 950000, 'has_cpu': True,
    'default_blkio_weight': 1000, 'default_cpu_shares': 1024, 'has_cpuacct':
    True, 'has_blkio': True}
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Setting up supervisord event monitor.
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread filesystem_map INFO
    Monitored nodev filesystem types: ['nfs', 'nfs4', 'tmpfs']
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread filesystem_map INFO
    Using timeout of 2.000000
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread filesystem_map INFO
    Using join timeout of 0.100000
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread filesystem_map INFO
    Using tolerance of 60.000000
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Using
    metrics_url_timeout_seconds of 30.000000
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Using
    task_metrics_timeout_seconds of 5.000000
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Using
    max_collection_wait_seconds of 10.000000
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread metrics INFO
    Importing tasktracker metric schema from file
    /usr/lib/cmf/agent/src/cmf/monitor/tasktracker/schema.json
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread dns_names INFO Using
    timeout of 2.000000
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread __init__ INFO
    Importing metric schema from file
    /usr/lib/cmf/agent/src/cmf/monitor/schema.json
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Supervised processes will inherit the following environment: {'LANG':
    'en_US.UTF-8', 'TERM': 'vt100', 'CMF_CONF_DIR': '/etc/cloudera-scm-agent',
    'SHLVL': '1', 'PWD': '/', 'PATH':
    '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games'}
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Supervised processes will add the following to their environment:
    {'CDH_OOZIE_HOME': '/usr/lib/oozie', 'CDH_MR2_HOME':
    '/usr/lib/hadoop-mapreduce', 'CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper',
    'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME': '/usr/share/cmf',
    'CDH_IMPALA_HOME': '/usr/lib/impala', 'CLOUDERA_MYSQL_CONNECTOR_JAR':
    '/usr/share/java/mysql-connector-java.jar', 'CDH_YARN_HOME':
    '/usr/lib/hadoop-yarn', 'CMF_PACKAGE_DIR': '/usr/lib/cmf/service',
    'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs', 'CDH_SOLR_HOME': '/usr/lib/solr',
    'CDH_PIG_HOME': '/usr/lib/pig', 'CDH_SQOOP2_HOME': '/usr/lib/sqoop2',
    'CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CM_STATUS_CODES': u'STATUS_NONE
    HDFS_DFS_DIR_NOT_EMPTY JOBTRACKER_IN_STANDBY_MODE', 'CDH_MR1_HOME':
    '/usr/lib/hadoop-0.20-mapreduce', 'CLOUDERA_ORACLE_CONNECTOR_JAR':
    '/usr/share/java/oracle-connector-java.jar', 'CDH_HUE_HOME':
    '/usr/share/hue', 'CDH_HIVE_HOME': '/usr/lib/hive', 'CDH_HTTPFS_HOME':
    '/usr/lib/hadoop-httpfs', 'CDH_HADOOP_HOME': '/usr/lib/hadoop',
    'JSVC_HOME': '/usr/libexec/bigtop-utils', 'HIVE_DEFAULT_XML':
    '/etc/hive/conf.dist/hive-default.xml', 'CDH_FLUME_HOME':
    '/usr/lib/flume-ng', 'CLOUDERA_POSTGRESQL_JDBC_JAR':
    '/usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar', 'TOMCAT_HOME':
    '/usr/lib/bigtop-tomcat', 'CDH_HBASE_HOME': '/usr/lib/hbase',
    'CDH_HCAT_HOME': '/usr/lib/hcatalog'}
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO To
    override these variables, use /etc/cloudera-scm-agent/config.ini.
    Environment variables for CDH locations are not used when CDH is installed
    from parcels.
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Created /run/cloudera-scm-agent/process
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Chmod'ing /run/cloudera-scm-agent/process to 0751
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Created /run/cloudera-scm-agent/supervisor
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Chmod'ing /run/cloudera-scm-agent/supervisor to 0751
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Created /run/cloudera-scm-agent/supervisor/include
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO
    Chmod'ing /run/cloudera-scm-agent/supervisor/include to 0751
    [31/Jul/2013 17:53:27 +0000] 18298 MainThread agent INFO Failed
    to connect to previous supervisor.
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread agent INFO Trying
    to connect to newly launched supervisor (Attempt 1)
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread agent INFO
    Successfully connected to supervisor
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:17:53:29] ENGINE Bus STARTING
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:17:53:29] ENGINE Started monitor thread '_TimeoutMonitor'.
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:17:53:29] ENGINE Serving on ip-10-77-74-189.ec2.internal:9000
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:17:53:29] ENGINE Bus STARTED
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread __init__ INFO New
    monitor: (<cmf.monitor.host.hostmonitor object@0x7f4e98496790>,)
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread agent WARNING
    Setting default socket timeout to 30!
    [31/Jul/2013 17:53:29 +0000] 18298 MonitorDaemon-Scheduler __init__
    INFO Monitor ready to report: (<cmf.monitor.host.HostMonitor object at
    0x7f4e98496790>,)
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread agent INFO
    Created /opt/cloudera/parcels
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread agent INFO
    Chowning /opt/cloudera/parcels to root (0) root (0)
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread agent INFO
    Chmod'ing /opt/cloudera/parcels to 0755
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread parcel INFO Agent
    creates users/groups and applies file permissions: True
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread agent INFO
    Created /opt/cloudera/parcel-cache
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread agent INFO
    Chowning /opt/cloudera/parcel-cache to root (0) root (0)
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread agent INFO
    Chmod'ing /opt/cloudera/parcel-cache to 0755
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread downloader INFO
    Downloader path: /opt/cloudera/parcel-cache
    [31/Jul/2013 17:53:29 +0000] 18298 MainThread parcel_cache INFO Using
    /opt/cloudera/parcel-cache for parcel cache
    [31/Jul/2013 17:53:34 +0000] 18298 Monitor-HostMonitor network_interfaces
    INFO NIC iface eth0 doesn't support ETHTOOL (95)
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread agent INFO
    Stopping agent...
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread agent INFO No
    cgrouped processes running; unmounting any cgroup roots
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread agent INFO No
    processes are being managed; Supervisor will shutdown.
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread agent INFO
    Shutting down supervisord, pid 18318
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread agent INFO
    Successfully shutdown supervisord, pid 18318
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:18:00:43] ENGINE Bus STOPPING
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:18:00:43] ENGINE HTTP Server
    cherrypy._cpwsgi_server.CPWSGIServer(('ip-10-77-74-189.ec2.internal',
    9000)) shut down
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:18:00:43] ENGINE Stopped thread '_TimeoutMonitor'.
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:18:00:43] ENGINE Bus STOPPED
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:18:00:43] ENGINE Bus STOPPING
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:18:00:43] ENGINE HTTP Server
    cherrypy._cpwsgi_server.CPWSGIServer(('ip-10-77-74-189.ec2.internal',
    9000)) already shut down
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:18:00:43] ENGINE No thread running for None.
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:18:00:43] ENGINE Bus STOPPED
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:18:00:43] ENGINE Bus EXITING
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread _cplogging INFO
    [31/Jul/2013:18:00:43] ENGINE Bus EXITED
    [31/Jul/2013 18:00:43 +0000] 18298 MainThread agent INFO Agent
    exiting; caught signal 15
    [31/Jul/2013 18:00:43 +0000] 19950 MainThread agent INFO No
    command line vars
    [31/Jul/2013 18:00:43 +0000] 19950 MainThread agent INFO
    Missing database jar: /usr/share/java/mysql-connector-java.jar (normal, if
    you're not using this database type)
    [31/Jul/2013 18:00:43 +0000] 19950 MainThread agent INFO
    Missing database jar: /usr/share/java/oracle-connector-java.jar (normal, if
    you're not using this database type)
    [31/Jul/2013 18:00:43 +0000] 19950 MainThread agent INFO Found
    database jar: /usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar
    [31/Jul/2013 18:00:43 +0000] 19950 MainThread agent INFO Agent
    starting as pid 19950 user root(0) group root(0).
    [31/Jul/2013 18:00:43 +0000] 19950 MainThread agent INFO
    Re-using pre-existing directory: /run/cloudera-scm-agent
    [31/Jul/2013 18:00:43 +0000] 19950 MainThread agent INFO Agent
    is running on a public cloud. Public hostname:
    ec2-54-242-227-124.compute-1.amazonaws.com
    [31/Jul/2013 18:00:43 +0000] 19950 MainThread agent INFO
    Re-using pre-existing directory: /run/cloudera-scm-agent/cgroups
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread cgroups INFO
    Reusing /run/cloudera-scm-agent/cgroups/memory
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread cgroups INFO
    Reusing /run/cloudera-scm-agent/cgroups/cpu
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread cgroups INFO
    Reusing /run/cloudera-scm-agent/cgroups/cpuacct
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread cgroups INFO
    Reusing /run/cloudera-scm-agent/cgroups/blkio
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO Found
    cgroups capabilities: {'has_memory': True, 'default_memory_limit_in_bytes':
    9223372036854775807, 'default_memory_soft_limit_in_bytes':
    9223372036854775807, 'writable_cgroup_dot_procs': True,
    'default_cpu_rt_runtime_us': 950000, 'has_cpu': True,
    'default_blkio_weight': 1000, 'default_cpu_shares': 1024, 'has_cpuacct':
    True, 'has_blkio': True}
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO
    Setting up supervisord event monitor.
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread filesystem_map INFO
    Monitored nodev filesystem types: ['nfs', 'nfs4', 'tmpfs']
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread filesystem_map INFO
    Using timeout of 2.000000
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread filesystem_map INFO
    Using join timeout of 0.100000
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread filesystem_map INFO
    Using tolerance of 60.000000
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO Using
    metrics_url_timeout_seconds of 30.000000
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO Using
    task_metrics_timeout_seconds of 5.000000
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO Using
    max_collection_wait_seconds of 10.000000
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread metrics INFO
    Importing tasktracker metric schema from file
    /usr/lib/cmf/agent/src/cmf/monitor/tasktracker/schema.json
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread dns_names INFO Using
    timeout of 2.000000
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread __init__ INFO
    Importing metric schema from file
    /usr/lib/cmf/agent/src/cmf/monitor/schema.json
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO
    Supervised processes will inherit the following environment: {'LANG':
    'en_US.UTF-8', 'TERM': 'vt100', 'CMF_CONF_DIR': '/etc/cloudera-scm-agent',
    'SHLVL': '1', 'PWD': '/', 'PATH':
    '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games'}
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO
    Supervised processes will add the following to their environment:
    {'CDH_OOZIE_HOME': '/usr/lib/oozie', 'CDH_MR2_HOME':
    '/usr/lib/hadoop-mapreduce', 'CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper',
    'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME': '/usr/share/cmf',
    'CDH_IMPALA_HOME': '/usr/lib/impala', 'CLOUDERA_MYSQL_CONNECTOR_JAR':
    '/usr/share/java/mysql-connector-java.jar', 'CDH_YARN_HOME':
    '/usr/lib/hadoop-yarn', 'CMF_PACKAGE_DIR': '/usr/lib/cmf/service',
    'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs', 'CDH_SOLR_HOME': '/usr/lib/solr',
    'CDH_PIG_HOME': '/usr/lib/pig', 'CDH_SQOOP2_HOME': '/usr/lib/sqoop2',
    'CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CM_STATUS_CODES': u'STATUS_NONE
    HDFS_DFS_DIR_NOT_EMPTY JOBTRACKER_IN_STANDBY_MODE', 'CDH_MR1_HOME':
    '/usr/lib/hadoop-0.20-mapreduce', 'CLOUDERA_ORACLE_CONNECTOR_JAR':
    '/usr/share/java/oracle-connector-java.jar', 'CDH_HUE_HOME':
    '/usr/share/hue', 'CDH_HIVE_HOME': '/usr/lib/hive', 'CDH_HTTPFS_HOME':
    '/usr/lib/hadoop-httpfs', 'CDH_HADOOP_HOME': '/usr/lib/hadoop',
    'JSVC_HOME': '/usr/libexec/bigtop-utils', 'HIVE_DEFAULT_XML':
    '/etc/hive/conf.dist/hive-default.xml', 'CDH_FLUME_HOME':
    '/usr/lib/flume-ng', 'CLOUDERA_POSTGRESQL_JDBC_JAR':
    '/usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar', 'TOMCAT_HOME':
    '/usr/lib/bigtop-tomcat', 'CDH_HBASE_HOME': '/usr/lib/hbase',
    'CDH_HCAT_HOME': '/usr/lib/hcatalog'}
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO To
    override these variables, use /etc/cloudera-scm-agent/config.ini.
    Environment variables for CDH locations are not used when CDH is installed
    from parcels.
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO
    Re-using pre-existing directory: /run/cloudera-scm-agent/process
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO
    Re-using pre-existing directory: /run/cloudera-scm-agent/supervisor
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO
    Re-using pre-existing directory: /run/cloudera-scm-agent/supervisor/include
    [31/Jul/2013 18:00:44 +0000] 19950 MainThread agent INFO Failed
    to connect to previous supervisor.
    [31/Jul/2013 18:00:46 +0000] 19950 MainThread agent INFO Trying
    to connect to newly launched supervisor (Attempt 1)
    [31/Jul/2013 18:00:46 +0000] 19950 MainThread agent INFO
    Successfully connected to supervisor
    [31/Jul/2013 18:00:46 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:00:46] ENGINE Bus STARTING
    [31/Jul/2013 18:00:46 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:00:46] ENGINE Started monitor thread '_TimeoutMonitor'.
    [31/Jul/2013 18:00:46 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:00:46] ENGINE Serving on ip-10-77-74-189.ec2.internal:9000
    [31/Jul/2013 18:00:46 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:00:46] ENGINE Bus STARTED
    [31/Jul/2013 18:00:46 +0000] 19950 MainThread __init__ INFO New
    monitor: (<cmf.monitor.host.hostmonitor object@0x2b86f50>,)
    [31/Jul/2013 18:00:46 +0000] 19950 MainThread agent WARNING
    Setting default socket timeout to 30!
    [31/Jul/2013 18:00:46 +0000] 19950 MonitorDaemon-Scheduler __init__
    INFO Monitor ready to report: (<cmf.monitor.host.HostMonitor object at
    0x2b86f50>,)
    [31/Jul/2013 18:00:46 +0000] 19950 MainThread parcel INFO Agent
    creates users/groups and applies file permissions: True
    [31/Jul/2013 18:00:46 +0000] 19950 MainThread downloader INFO
    Downloader path: /opt/cloudera/parcel-cache
    [31/Jul/2013 18:00:46 +0000] 19950 MainThread parcel_cache INFO Using
    /opt/cloudera/parcel-cache for parcel cache
    [31/Jul/2013 18:00:51 +0000] 19950 Monitor-HostMonitor network_interfaces
    INFO NIC iface eth0 doesn't support ETHTOOL (95)
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread agent INFO
    Stopping agent...
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread agent INFO No
    cgrouped processes running; unmounting any cgroup roots
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread agent INFO No
    processes are being managed; Supervisor will shutdown.
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread agent INFO
    Shutting down supervisord, pid 19969
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread agent INFO
    Successfully shutdown supervisord, pid 19969
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:16:32] ENGINE Bus STOPPING
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:16:32] ENGINE HTTP Server
    cherrypy._cpwsgi_server.CPWSGIServer(('ip-10-77-74-189.ec2.internal',
    9000)) shut down
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:16:32] ENGINE Stopped thread '_TimeoutMonitor'.
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:16:32] ENGINE Bus STOPPED
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:16:32] ENGINE Bus STOPPING
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:16:32] ENGINE HTTP Server
    cherrypy._cpwsgi_server.CPWSGIServer(('ip-10-77-74-189.ec2.internal',
    9000)) already shut down
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:16:32] ENGINE No thread running for None.
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:16:32] ENGINE Bus STOPPED
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:16:32] ENGINE Bus EXITING
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread _cplogging INFO
    [31/Jul/2013:18:16:32] ENGINE Bus EXITED
    [31/Jul/2013 18:16:32 +0000] 19950 MainThread agent INFO Agent
    exiting; caught signal 15
    [31/Jul/2013 18:16:32 +0000] 23193 MainThread agent INFO No
    command line vars
    [31/Jul/2013 18:16:32 +0000] 23193 MainThread agent INFO
    Missing database jar: /usr/share/java/mysql-connector-java.jar (normal, if
    you're not using this database type)
    [31/Jul/2013 18:16:32 +0000] 23193 MainThread agent INFO
    Missing database jar: /usr/share/java/oracle-connector-java.jar (normal, if
    you're not using this database type)
    [31/Jul/2013 18:16:32 +0000] 23193 MainThread agent INFO Found
    database jar: /usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar
    [31/Jul/2013 18:16:32 +0000] 23193 MainThread agent INFO Agent
    starting as pid 23193 user root(0) group root(0).
    [31/Jul/2013 18:16:32 +0000] 23193 MainThread agent INFO
    Re-using pre-existing directory: /run/cloudera-scm-agent
    [31/Jul/2013 18:16:32 +0000] 23193 MainThread agent INFO Agent
    is running on a public cloud. Public hostname:
    ec2-54-242-227-124.compute-1.amazonaws.com
    [31/Jul/2013 18:16:32 +0000] 23193 MainThread agent INFO
    Re-using pre-existing directory: /run/cloudera-scm-agent/cgroups
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread cgroups INFO
    Reusing /run/cloudera-scm-agent/cgroups/memory
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread cgroups INFO
    Reusing /run/cloudera-scm-agent/cgroups/cpu
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread cgroups INFO
    Reusing /run/cloudera-scm-agent/cgroups/cpuacct
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread cgroups INFO
    Reusing /run/cloudera-scm-agent/cgroups/blkio
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO Found
    cgroups capabilities: {'has_memory': True, 'default_memory_limit_in_bytes':
    9223372036854775807, 'default_memory_soft_limit_in_bytes':
    9223372036854775807, 'writable_cgroup_dot_procs': True,
    'default_cpu_rt_runtime_us': 950000, 'has_cpu': True,
    'default_blkio_weight': 1000, 'default_cpu_shares': 1024, 'has_cpuacct':
    True, 'has_blkio': True}
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO
    Setting up supervisord event monitor.
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread filesystem_map INFO
    Monitored nodev filesystem types: ['nfs', 'nfs4', 'tmpfs']
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread filesystem_map INFO
    Using timeout of 2.000000
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread filesystem_map INFO
    Using join timeout of 0.100000
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread filesystem_map INFO
    Using tolerance of 60.000000
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO Using
    metrics_url_timeout_seconds of 30.000000
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO Using
    task_metrics_timeout_seconds of 5.000000
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO Using
    max_collection_wait_seconds of 10.000000
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread metrics INFO
    Importing tasktracker metric schema from file
    /usr/lib/cmf/agent/src/cmf/monitor/tasktracker/schema.json
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread dns_names INFO Using
    timeout of 2.000000
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread __init__ INFO
    Importing metric schema from file
    /usr/lib/cmf/agent/src/cmf/monitor/schema.json
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO
    Supervised processes will inherit the following environment: {'LANG':
    'en_US.UTF-8', 'TERM': 'vt100', 'CMF_CONF_DIR': '/etc/cloudera-scm-agent',
    'SHLVL': '1', 'PWD': '/', 'PATH':
    '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games'}
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO
    Supervised processes will add the following to their environment:
    {'CDH_OOZIE_HOME': '/usr/lib/oozie', 'CDH_MR2_HOME':
    '/usr/lib/hadoop-mapreduce', 'CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper',
    'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME': '/usr/share/cmf',
    'CDH_IMPALA_HOME': '/usr/lib/impala', 'CLOUDERA_MYSQL_CONNECTOR_JAR':
    '/usr/share/java/mysql-connector-java.jar', 'CDH_YARN_HOME':
    '/usr/lib/hadoop-yarn', 'CMF_PACKAGE_DIR': '/usr/lib/cmf/service',
    'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs', 'CDH_SOLR_HOME': '/usr/lib/solr',
    'CDH_PIG_HOME': '/usr/lib/pig', 'CDH_SQOOP2_HOME': '/usr/lib/sqoop2',
    'CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CM_STATUS_CODES': u'STATUS_NONE
    HDFS_DFS_DIR_NOT_EMPTY JOBTRACKER_IN_STANDBY_MODE', 'CDH_MR1_HOME':
    '/usr/lib/hadoop-0.20-mapreduce', 'CLOUDERA_ORACLE_CONNECTOR_JAR':
    '/usr/share/java/oracle-connector-java.jar', 'CDH_HUE_HOME':
    '/usr/share/hue', 'CDH_HIVE_HOME': '/usr/lib/hive', 'CDH_HTTPFS_HOME':
    '/usr/lib/hadoop-httpfs', 'CDH_HADOOP_HOME': '/usr/lib/hadoop',
    'JSVC_HOME': '/usr/libexec/bigtop-utils', 'HIVE_DEFAULT_XML':
    '/etc/hive/conf.dist/hive-default.xml', 'CDH_FLUME_HOME':
    '/usr/lib/flume-ng', 'CLOUDERA_POSTGRESQL_JDBC_JAR':
    '/usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar', 'TOMCAT_HOME':
    '/usr/lib/bigtop-tomcat', 'CDH_HBASE_HOME': '/usr/lib/hbase',
    'CDH_HCAT_HOME': '/usr/lib/hcatalog'}
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO To
    override these variables, use /etc/cloudera-scm-agent/config.ini.
    Environment variables for CDH locations are not used when CDH is installed
    from parcels.
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO
    Re-using pre-existing directory: /run/cloudera-scm-agent/process
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO
    Re-using pre-existing directory: /run/cloudera-scm-agent/supervisor
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO
    Re-using pre-existing directory: /run/cloudera-scm-agent/supervisor/include
    [31/Jul/2013 18:16:33 +0000] 23193 MainThread agent INFO Failed
    to connect to previous supervisor.
    [31/Jul/2013 18:16:35 +0000] 23193 MainThread agent INFO Trying
    to connect to newly launched supervisor (Attempt 1)
    [31/Jul/2013 18:16:35 +0000] 23193 MainThread agent INFO
    Successfully connected to supervisor
    [31/Jul/2013 18:16:35 +0000] 23193 MainThread _cplogging INFO
    [31/Jul/2013:18:16:35] ENGINE Bus STARTING
    [31/Jul/2013 18:16:35 +0000] 23193 MainThread _cplogging INFO
    [31/Jul/2013:18:16:35] ENGINE Started monitor thread '_TimeoutMonitor'.
    [31/Jul/2013 18:16:35 +0000] 23193 MainThread _cplogging INFO
    [31/Jul/2013:18:16:35] ENGINE Serving on
    ec2-54-242-227-124.compute-1.amazonaws.com:9000
    [31/Jul/2013 18:16:35 +0000] 23193 MainThread _cplogging INFO
    [31/Jul/2013:18:16:35] ENGINE Bus STARTED
    [31/Jul/2013 18:16:35 +0000] 23193 MainThread __init__ INFO New
    monitor: (<cmf.monitor.host.hostmonitor object@0x3230f50>,)
    [31/Jul/2013 18:16:35 +0000] 23193 MainThread agent WARNING
    Setting default socket timeout to 30!
    [31/Jul/2013 18:16:35 +0000] 23193 MonitorDaemon-Scheduler __init__
    INFO Monitor ready to report: (<cmf.monitor.host.HostMonitor object at
    0x3230f50>,)
    [31/Jul/2013 18:16:35 +0000] 23193 MainThread parcel INFO Agent
    creates users/groups and applies file permissions: True
    [31/Jul/2013 18:16:35 +0000] 23193 MainThread downloader INFO
    Downloader path: /opt/cloudera/parcel-cache
    [31/Jul/2013 18:16:35 +0000] 23193 MainThread parcel_cache INFO Using
    /opt/cloudera/parcel-cache for parcel cache
    [31/Jul/2013 18:16:40 +0000] 23193 Monitor-HostMonitor network_interfaces
    INFO NIC iface eth0 doesn't support ETHTOOL (95)



    Any help would be greatly appreciated!

    Thanks,
    Ryan
    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedJul 31, '13 at 6:47p
activeJan 10, '14 at 10:53p
posts4
users2
websitecloudera.com
irc#hadoop

2 users in discussion

Ryan Alexander: 3 posts James Chen: 1 post

People

Translate

site design / logo © 2022 Grokbase