Cloudera Manager - Installation failed. Failed to receive heartbeat from agent.

On Fri, Jan 25, 2013 at 2:53 PM, Barry wrote:

Looks like the web server on the agent machine can't start. Here is a
snippet from the agent log file:

[25/Jan/2013 11:46:06 +0000] 9335 MainThread _cplogging INFO
[25/Jan/2013:11:46:06] ENGINE Bus STARTING
[25/Jan/2013 11:46:06 +0000] 9335 MainThread _cplogging INFO
[25/Jan/2013:11:46:06] ENGINE Started monitor thread
'_TimeoutMonitor'.
[25/Jan/2013 11:46:12 +0000] 9335 HTTPServer Thread-3 _cplogging
ERROR [25/Jan/2013:11:46:12] ENGINE Error in HTTP
server: shutting down
Traceback (most recent call last):
File
"/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/CherryPy-3.2.2-py2.7.egg/cherrypy/process/servers.py",

line 187, in _start_http_thread
self.httpserver.start()
File
"/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/CherryPy-3.2.2-py2.7.egg/cherrypy/wsgiserver/wsgiserver
2.py", line 1824, in start
raise socket.error(msg)
error: No socket could be created

[25/Jan/2013 11:46:12 +0000] 9335 HTTPServer Thread-3 _cplogging
INFO [25/Jan/2013:11:46:12] ENGINE Bus STOPPING
[25/Jan/2013 11:46:12 +0000] 9335 HTTPServer Thread-3 _cplogging
INFO [25/Jan/2013:11:46:12] ENGINE HTTP Server ch
errypy._cpwsgi_server.CPWSGIServer(('hit-nxdomain.opendns.com', 9000))
already shut down
[25/Jan/2013 11:46:12 +0000] 9335 HTTPServer Thread-3 _cplogging
INFO [25/Jan/2013:11:46:12] ENGINE Stopped thread
'_TimeoutMonitor'.
[25/Jan/2013 11:46:12 +0000] 9335 HTTPServer Thread-3 _cplogging
INFO [25/Jan/2013:11:46:12] ENGINE Bus STOPPED
[25/Jan/2013 11:46:12 +0000] 9335 HTTPServer Thread-3 _cplogging
INFO [25/Jan/2013:11:46:12] ENGINE Bus EXITING
[25/Jan/2013 11:46:12 +0000] 9335 HTTPServer Thread-3 _cplogging
INFO [25/Jan/2013:11:46:12] ENGINE Bus EXITED

Any thoughts on what is going on?

Barry

On Saturday, 26 January 2013 18:55:51 UTC, Adam Smieszny wrote:

Can you try the following:
validate that no firewall is running, SElinux is not enabled, and what
processes are listening on your server

$> service iptables status

$> cat /etc/selinux/enforce

$> netstat -a -t --numeric-ports -p


Thanks,
Adam




On Fri, Jan 25, 2013 at 2:53 PM, Barry <[email protected] <javascript:>>wrote:


--
Adam Smieszny
Cloudera | Systems Engineer | http://www.linkedin.com/in/adamsmieszny
917.830.4156

On Thu, May 16, 2013 at 1:14 AM, Rory Marland wrote:

Hi Barry,

IP Tables are off on all systems "iptables: Firewall is not running."

Selinux is in a state "SELINUX=disabled"

Netstat results:

Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign
Address State PID/Program name
tcp 0 0 *:7432
*:* LISTEN 1798/postgres
tcp 0 0 localhost:9001
*:* LISTEN 8945/python2.6
tcp 0 0 *:7180
*:* LISTEN 1826/java
tcp 0 0 *:7182
*:* LISTEN 1826/java
tcp 0 0 *:111
*:* LISTEN 1426/rpcbind
tcp 0 0 *:22
*:* LISTEN 1546/sshd
tcp 0 0 *:54102
*:* LISTEN 1444/rpc.statd
tcp 0 0 cloudera-node1:43423
cloudera-node6:22 ESTABLISHED 1826/java
tcp 0 0 localhost:7432
localhost:58248 ESTABLISHED 1845/postgres
tcp 0 0 cloudera-node1:7180 webvpn.domain.co.uk:44786
ESTABLISHED 1826/java
tcp 0 0 localhost:58245
localhost:7432 ESTABLISHED 1826/java
tcp 0 0 localhost:58247
localhost:7432 ESTABLISHED 1826/java
tcp 0 0 localhost:7432
localhost:58246 ESTABLISHED 1843/postgres
tcp 0 484 cloudera-node1:22 webvpn.domain.co.uk:16062
ESTABLISHED 16171/sshd
tcp 0 0 cloudera-node1:35210
cloudera-node2:22 ESTABLISHED 1826/java
tcp 0 0 localhost:58246
localhost:7432 ESTABLISHED 1826/java
tcp 0 0 cloudera-node1:49604
cloudera-node4:22 ESTABLISHED 1826/java
tcp 0 0 localhost:58248
localhost:7432 ESTABLISHED 1826/java
tcp 0 0 localhost:7432
localhost:58244 ESTABLISHED 1841/postgres
tcp 0 0 cloudera-node1:49608
cloudera-node4:22 ESTABLISHED 1826/java
tcp 0 0 localhost:7432
localhost:58247 ESTABLISHED 1844/postgres
tcp 0 0 localhost:7432
localhost:58245 ESTABLISHED 1842/postgres
tcp 0 0 cloudera-node1:49007
cloudera-node5:22 ESTABLISHED 1826/java
tcp 0 0 localhost:58244
localhost:7432 ESTABLISHED 1826/java
tcp 0 0 cloudera-node1:38716
cloudera-node3:22 ESTABLISHED 1826/java
tcp 0 0 *:7432
*:* LISTEN 1798/postgres
tcp 0 0 *:33514
*:* LISTEN 1444/rpc.statd
tcp 0 0 *:111
*:* LISTEN 1426/rpcbind
tcp 0 0 *:22
*:* LISTEN 1546/sshd


Thanks

On Thu, May 16, 2013 at 7:22 AM, Philip Zeyliger wrote:

Hi Barry,
9000)) already shut down

Can you try to do "nc -l hit-nxdomain.opendnx.com 9000" to see if that
port is open? The agent tries to bind to port 9000 and 9001; if that
doesn't work, it won't start. "error: No socket could be created" is the
underlying error.

-- Philip