FAQ
HI,

I have installed CDH4 using Cloudera Manager Enterprise Edition - 4.5. All
my services and role instances status are "Good" except the Hostmonitor
instance in the Management Services which is showing "Bad".
The error log for the instance gives the following error message:

ERROR com.cloudera.enterprise.DebugServer

Unable to start debug server
java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:383)
at java.net.ServerSocket.bind(ServerSocket.java:328)
at java.net.ServerSocket.(SocketConnector.java:80)
at org.mortbay.jetty.bio.SocketConnector.open(SocketConnector.java:73)
at org.mortbay.jetty.AbstractConnector.doStart(AbstractConnector.java:283)
at org.mortbay.jetty.bio.SocketConnector.doStart(SocketConnector.java:147)
at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at org.mortbay.jetty.Server.doStart(Server.java:235)
at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at com.cloudera.enterprise.DebugServer.startService(DebugServer.java:259)
at com.cloudera.enterprise.EnterpriseService.start(EnterpriseService.java:71)
at com.cloudera.enterprise.EnterpriseService.start(EnterpriseService.java:68)
at com.cloudera.cmon.firehose.Main.main(Main.java:363)


The "cloudera-scm-agent" log file gives the following error message:

*[05/Mar/2013 09:32:11 +0000] 6010 MonitorDaemon-Reporter throttling_logger
ERROR (1 skipped) Error sending messages to firehose:
mgmt1-HOSTMONITOR-536206c9dccb73d5aad7fca5b1fa5deb
Traceback (most recent call last):
File "/usr/lib64/cmf/agent/src/cmf/monitor/firehose.py", line 67, in send
self._requestor.request('sendAgentMessages', dict(messages=messages))
File
"/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py",
line 145, in request
return self.issue_request(call_request, message_name, request_datum)
File
"/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py",
line 256, in issue_request
call_response = self.transceiver.transceive(call_request)
File
"/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py",
line 485, in transceive
result = self.read_framed_message()
File
"/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py",
line 489, in read_framed_message
response = self.conn.getresponse()
File "/usr/lib64/python2.6/httplib.py", line 990, in getresponse
response.begin()
File "/usr/lib64/python2.6/httplib.py", line 391, in begin
version, status, reason = self._read_status()
File "/usr/lib64/python2.6/httplib.py", line 355, in _read_status
raise BadStatusLine(line)
BadStatusLine*

Has anyone faced a similar issue? Can some one help me in understanding the
problem involved here and help me rectify this?

Thanks,
Gaurav

Search Discussions

  • Philip Zeyliger at Mar 5, 2013 at 4:35 pm
    Hi Gaurav,

    The host monitor is trying to bind on ports 8085, 9994, and 9995. If you
    have some other process on those ports, it'll fail to start. You can
    change the configuration to bind to a different port, or use "sudo lsof -P
    -n | grep LISTEN | grep 8085" to figure out who's listening on the relevant
    port.

    Cheers,

    -- Philip

    ---------- Forwarded message ----------
    From: Gaurav Dasgupta <gdsayshi@gmail.com>
    Date: Tue, Mar 5, 2013 at 8:30 AM
    Subject: Hostmonitor health showing bad
    To: scm-users <scm-users@cloudera.org>


    HI,

    I have installed CDH4 using Cloudera Manager Enterprise Edition - 4.5. All
    my services and role instances status are "Good" except the Hostmonitor
    instance in the Management Services which is showing "Bad".
    The error log for the instance gives the following error message:

    ERROR com.cloudera.enterprise.DebugServer

    Unable to start debug server
    java.net.BindException: Address already in use
    at java.net.PlainSocketImpl.socketBind(Native Method)
    at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:383)
    at java.net.ServerSocket.bind(ServerSocket.java:328)
    at java.net.ServerSocket.(SocketConnector.java:80)
    at org.mortbay.jetty.bio.SocketConnector.open(SocketConnector.java:73)
    at org.mortbay.jetty.AbstractConnector.doStart(AbstractConnector.java:283)
    at org.mortbay.jetty.bio.SocketConnector.doStart(SocketConnector.java:147)
    at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
    at org.mortbay.jetty.Server.doStart(Server.java:235)
    at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
    at com.cloudera.enterprise.DebugServer.startService(DebugServer.java:259)
    at com.cloudera.enterprise.EnterpriseService.start(EnterpriseService.java:71)
    at com.cloudera.enterprise.EnterpriseService.start(EnterpriseService.java:68)
    at com.cloudera.cmon.firehose.Main.main(Main.java:363)


    The "cloudera-scm-agent" log file gives the following error message:

    *[05/Mar/2013 09:32:11 +0000] 6010 MonitorDaemon-Reporter throttling_logger
    ERROR (1 skipped) Error sending messages to firehose:
    mgmt1-HOSTMONITOR-536206c9dccb73d5aad7fca5b1fa5deb
    Traceback (most recent call last):
    File "/usr/lib64/cmf/agent/src/cmf/monitor/firehose.py", line 67, in send
    self._requestor.request('sendAgentMessages', dict(messages=messages))
    File
    "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py",
    line 145, in request
    return self.issue_request(call_request, message_name, request_datum)
    File
    "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py",
    line 256, in issue_request
    call_response = self.transceiver.transceive(call_request)
    File
    "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py",
    line 485, in transceive
    result = self.read_framed_message()
    File
    "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py",
    line 489, in read_framed_message
    response = self.conn.getresponse()
    File "/usr/lib64/python2.6/httplib.py", line 990, in getresponse
    response.begin()
    File "/usr/lib64/python2.6/httplib.py", line 391, in begin
    version, status, reason = self._read_status()
    File "/usr/lib64/python2.6/httplib.py", line 355, in _read_status
    raise BadStatusLine(line)
    BadStatusLine*

    Has anyone faced a similar issue? Can some one help me in understanding the
    problem involved here and help me rectify this?

    Thanks,
    Gaurav
  • Gaurav Dasgupta at Mar 5, 2013 at 5:54 pm
    HI Philip,

    Thanks for replying. I have checked if any other processes listening to the
    ports. But there's nothing coming out. The ports are not bind to any
    process. Can there be any other reason for the issue?

    Thanks,
    Gaurav
  • Philip Zeyliger at Mar 5, 2013 at 6:37 pm
    Hi Gaurav,

    Does CM tell you why the host monitor is bad? If you click through, there
    is usually a health check or two that explains it. The full host monitor
    log would be useful too.

    -- Philip

    On Tue, Mar 5, 2013 at 9:54 AM, Gaurav Dasgupta wrote:

    HI Philip,

    Thanks for replying. I have checked if any other processes listening to
    the ports. But there's nothing coming out. The ports are not bind to any
    process. Can there be any other reason for the issue?

    Thanks,
    Gaurav

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedMar 5, '13 at 4:30p
activeMar 5, '13 at 6:37p
posts4
users2
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase