FAQ
Installination fails and ratehr concered. I am using digi ocean. I have
all ports open. How do I resovle? All machines are distributed the same
pub and private keys


' Installation failed. Failed to receive heartbeat from agent.

     Ensure that the host's hostname is configured properly.
     Ensure that port 7182 is accessible on the Cloudera Manager Server
(check firewall rules).
     Ensure that ports 9000 and 9001 are free on the host being added.
     Check agent logs in /var/log/cloudera-scm-agent/ on the host being
added (some of the logs can be found in the installation details).

Below is the first error in cloudera-scm-agent.log on the host I added.


[20/Apr/2014 03:41:41 +0000] 2974 MainThread agent INFO
Chmod'ing /run/cloudera-scm-agent/supervisor to 0751
[20/Apr/2014 03:41:41 +0000] 2974 MainThread agent INFO Created
/run/cloudera-scm-agent/supervisor/include
[20/Apr/2014 03:41:41 +0000] 2974 MainThread agent INFO
Chmod'ing /run/cloudera-scm-agent/supervisor/include to 0751
[20/Apr/2014 03:41:41 +0000] 2974 MainThread agent ERROR Failed
to connect to previous supervisor.
Traceback (most recent call last):
   File "/usr/lib/cmf/agent/src/cmf/agent.py", line 1230, in
find_or_start_supervisor
     self.configure_supervisor_clients()
   File "/usr/lib/cmf/agent/src/cmf/agent.py", line 1453, in
configure_supervisor_clients
     supervisor_options.realize(args=["-c",
os.path.join(self.supervisor_dir, "supervisord.conf")])
   File
"/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py",
line 1559, in realize
     Options.realize(self, *arg, **kw)
   File
"/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py",
line 309, in realize
     self.process_config()
   File
"/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py",
line 317, in process_config
     self.process_config_file(do_usage)
   File
"/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py",
line 352, in process_config_file
     self.usage(str(msg))
   File
"/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py",
line 140, in usage
     self.exit(2)
SystemExit: 2
[20/Apr/2014 03:41:41 +0000] 2974 MainThread tmpfs INFO
Successfully mounted tmpfs at /run/cloudera-scm-agent/process
[20/Apr/2014 03:41:42 +0000] 2974 MainThread agent INFO Trying
to connect to newly launched supervisor (Attempt 1)
[20/Apr/2014 03:41:42 +0000] 2974 MainThread agent INFO
Successfully connected to supervisor
[20/Apr/2014 03:41:42 +0000] 2974 MainThread _cplogging INFO
[20/Apr/2014:03:41:42] ENGINE Bus STARTING
[20/Apr/2014 03:41:42 +0000] 2974 MainThread _cplogging INFO
[20/Apr/2014:03:41:42] ENGINE Started monitor thread '_TimeoutMonitor'.
[20/Apr/2014 03:41:43 +0000] 2974 MainThread _cplogging INFO
[20/Apr/2014:03:41:43] ENGINE Serving on localhost:9000
[20/Apr/2014 03:41:43 +0000] 2974 MainThread _cplogging INFO
[20/Apr/2014:03:41:43] ENGINE Bus STARTED
[20/Apr/2014 03:41:43 +0000] 2974 MainThread __init__ INFO New










2 +0000] 4445 MainThread agent INFO Re-using pre-existing directory:
/run/cloudera-scm-agent/process
[20/Apr/2014 03:46:52 +0000] 4445 MainThread agent INFO Re-using
pre-existing directory: /run/cloudera-scm-agent/supervisor
[20/Apr/2014 03:46:52 +0000] 4445 MainThread agent INFO Re-using
pre-existing directory: /run/cloudera-scm-agent/supervisor/include
[20/Apr/2014 03:46:52 +0000] 4445 MainThread agent ERROR Failed
to connect to previous supervisor.
Traceback (most recent call last):
File "/usr/lib/cmf/agent/src/cmf/agent.py", line 1231, in
find_or_start_supervisor
self.get_supervisor_process_info()
File "/usr/lib/cmf/agent/src/cmf/agent.py", line 1418, in
get_supervisor_process_info
self.identifier = self.supervisor_client.supervisor.getIdentification()
File "/usr/lib/python2.7/xmlrpclib.py", line 1224, in __call__
return self.__send(self.__name, args)
File "/usr/lib/python2.7/xmlrpclib.py", line 1578, in __request
verbose=self.__verbose
File
"/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/xmlrpc.py",
line 460, in request
self.connection.request('POST', handler, request_body, self.headers)
File "/usr/lib/python2.7/httplib.py", line 958, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python2.7/httplib.py", line 992, in _send_request
self.endheaders(body)
File "/usr/lib/python2.7/httplib.py", line 954, in endheaders
self._send_output(message_body)
File "/usr/lib/python2.7/httplib.py", line 814, in _send_output
self.send(msg)
File "/usr/lib/python2.7/httplib.py", line 776, in send
self.connect()
File "/usr/lib/python2.7/httplib.py", line 757, in connect
self.timeout, self.source_address)
File "/usr/lib/python2.7/socket.py", line 571, in create_connection
raise err
error: [Errno 111] Connection refused
[20/Apr/2014 03:46:52 +0000] 4445 MainThread tmpfs INFO Reusing mounted
tmpfs at /run/cloudera-scm-agent/process
[20/Apr/2014 03:46:53 +0000] 4445 MainThread agent INFO Trying to connect
to newly launched supervisor (Attempt 1)
[20/Apr/2014 03:46:53 +0000] 4445 MainThread agent INFO Successfully
connected to supervisor
[20/Apr/2014 03:46:53 +0000] 4445 MainThread _cplogging INFO
[20/Apr/2014:03:46:53] ENGINE Bus STARTING

To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.

Search Discussions

  • Todd Grayson at Apr 20, 2014 at 2:31 pm
    Please avoid submitting blank subject e-mails to the list.

    Both forward and reverse lookup must be working correctly.

      From cloudera manager navigate to Hosts and then click the grey button
    to run the host inspector. When that completes, review the results,
    anything in yellow must be resolved before things will work properly.
    You can expand the list of issues for any issues found by clicking on
    the ">" in the results text.

      From what your describing, forward and reverse look up need to be check
    to be consistent and returning the FQDN of the hosts.

    python -c "import socket; print socket.getfqdn(); print
    socket.gethostbyname(socket.getfqdn())"

    We document this here:
    http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/latest/Cloudera-Manager-Installation-Guide/cmig_cm_requirements.html?scroll=cmig_topic_4_3_3_unique_1

    Todd
    On 4/20/14 1:55 AM, David Montgomery wrote:
    Installination fails and ratehr concered. I am using digi ocean. I
    have all ports open. How do I resovle? All machines are distributed
    the same pub and private keys


    ' Installation failed. Failed to receive heartbeat from agent.

    Ensure that the host's hostname is configured properly.
    Ensure that port 7182 is accessible on the Cloudera Manager Server
    (check firewall rules).
    Ensure that ports 9000 and 9001 are free on the host being added.
    Check agent logs in /var/log/cloudera-scm-agent/ on the host being
    added (some of the logs can be found in the installation details).

    Below is the first error in cloudera-scm-agent.log on the host I added.


    [20/Apr/2014 03:41:41 +0000] 2974 MainThread agent INFO Chmod'ing
    /run/cloudera-scm-agent/supervisor to 0751
    [20/Apr/2014 03:41:41 +0000] 2974 MainThread agent INFO Created
    /run/cloudera-scm-agent/supervisor/include
    [20/Apr/2014 03:41:41 +0000] 2974 MainThread agent INFO Chmod'ing
    /run/cloudera-scm-agent/supervisor/include to 0751
    [20/Apr/2014 03:41:41 +0000] 2974 MainThread agent ERROR Failed to
    connect to previous supervisor.
    Traceback (most recent call last):
    File "/usr/lib/cmf/agent/src/cmf/agent.py", line 1230, in
    find_or_start_supervisor
    self.configure_supervisor_clients()
    File "/usr/lib/cmf/agent/src/cmf/agent.py", line 1453, in
    configure_supervisor_clients
    supervisor_options.realize(args=["-c",
    os.path.join(self.supervisor_dir, "supervisord.conf")])
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py",
    line 1559, in realize
    Options.realize(self, *arg, **kw)
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py",
    line 309, in realize
    self.process_config()
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py",
    line 317, in process_config
    self.process_config_file(do_usage)
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py",
    line 352, in process_config_file
    self.usage(str(msg))
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py",
    line 140, in usage
    self.exit(2)
    SystemExit: 2
    [20/Apr/2014 03:41:41 +0000] 2974 MainThread tmpfs INFO
    Successfully mounted tmpfs at /run/cloudera-scm-agent/process
    [20/Apr/2014 03:41:42 +0000] 2974 MainThread agent INFO Trying to
    connect to newly launched supervisor (Attempt 1)
    [20/Apr/2014 03:41:42 +0000] 2974 MainThread agent INFO
    Successfully connected to supervisor
    [20/Apr/2014 03:41:42 +0000] 2974 MainThread _cplogging INFO
    [20/Apr/2014:03:41:42] ENGINE Bus STARTING
    [20/Apr/2014 03:41:42 +0000] 2974 MainThread _cplogging INFO
    [20/Apr/2014:03:41:42] ENGINE Started monitor thread '_TimeoutMonitor'.
    [20/Apr/2014 03:41:43 +0000] 2974 MainThread _cplogging INFO
    [20/Apr/2014:03:41:43] ENGINE Serving on localhost:9000
    [20/Apr/2014 03:41:43 +0000] 2974 MainThread _cplogging INFO
    [20/Apr/2014:03:41:43] ENGINE Bus STARTED
    [20/Apr/2014 03:41:43 +0000] 2974 MainThread __init__ INFO New










    2 +0000] 4445 MainThread agent INFO Re-using pre-existing directory:
    /run/cloudera-scm-agent/process
    [20/Apr/2014 03:46:52 +0000] 4445 MainThread agent INFO Re-using
    pre-existing directory: /run/cloudera-scm-agent/supervisor
    [20/Apr/2014 03:46:52 +0000] 4445 MainThread agent INFO Re-using
    pre-existing directory: /run/cloudera-scm-agent/supervisor/include
    [20/Apr/2014 03:46:52 +0000] 4445 MainThread agent ERROR
    Failed to connect to previous supervisor.
    Traceback (most recent call last):
    File "/usr/lib/cmf/agent/src/cmf/agent.py", line 1231, in
    find_or_start_supervisor
    self.get_supervisor_process_info()
    File "/usr/lib/cmf/agent/src/cmf/agent.py", line 1418, in
    get_supervisor_process_info
    self.identifier = self.supervisor_client.supervisor.getIdentification()
    File "/usr/lib/python2.7/xmlrpclib.py", line 1224, in __call__
    return self.__send(self.__name, args)
    File "/usr/lib/python2.7/xmlrpclib.py", line 1578, in __request
    verbose=self.__verbose
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/xmlrpc.py",
    line 460, in request
    self.connection.request('POST', handler, request_body, self.headers)
    File "/usr/lib/python2.7/httplib.py", line 958, in request
    self._send_request(method, url, body, headers)
    File "/usr/lib/python2.7/httplib.py", line 992, in _send_request
    self.endheaders(body)
    File "/usr/lib/python2.7/httplib.py", line 954, in endheaders
    self._send_output(message_body)
    File "/usr/lib/python2.7/httplib.py", line 814, in _send_output
    self.send(msg)
    File "/usr/lib/python2.7/httplib.py", line 776, in send
    self.connect()
    File "/usr/lib/python2.7/httplib.py", line 757, in connect
    self.timeout, self.source_address)
    File "/usr/lib/python2.7/socket.py", line 571, in create_connection
    raise err
    error: [Errno 111] Connection refused
    [20/Apr/2014 03:46:52 +0000] 4445 MainThread tmpfs INFO Reusing
    mounted tmpfs at /run/cloudera-scm-agent/process
    [20/Apr/2014 03:46:53 +0000] 4445 MainThread agent INFO Trying to
    connect to newly launched supervisor (Attempt 1)
    [20/Apr/2014 03:46:53 +0000] 4445 MainThread agent INFO Successfully
    connected to supervisor
    [20/Apr/2014 03:46:53 +0000] 4445 MainThread _cplogging INFO
    [20/Apr/2014:03:46:53] ENGINE Bus STARTING





    To unsubscribe from this group and stop receiving emails from it, send
    an email to scm-users+unsubscribe@cloudera.org
    .

    --
    Todd Grayson
    Cloudera Support
    Customer Operations Engineering
    tgrayson@cloudera.com

    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.
  • Todd Grayson at Apr 20, 2014 at 4:16 pm
    Lets maintain the discussion on the list.

    Now that you made the changes you indicated of setting up IP and
    hostname in /etc/hosts.

    Does your system know itself by that name. You installed the agents
    when that configuration was not in place. It might be worth verifying
    what is in the agent configuration file. Look at
    /etc/cloudera-scm-agent on the cluster nodes you are managing from
    cloudera manager.

    What is the python script returning now that I provided, below?

    What do you see in the network initialization script for your network
    interface your system is starting up with?

    Thanks
    Todd


    On 4/20/14 8:31 AM, Todd Grayson wrote:
    Please avoid submitting blank subject e-mails to the list.

    Both forward and reverse lookup must be working correctly.

    From cloudera manager navigate to Hosts and then click the grey button
    to run the host inspector. When that completes, review the results,
    anything in yellow must be resolved before things will work properly.
    You can expand the list of issues for any issues found by clicking on
    the ">" in the results text.

    From what your describing, forward and reverse look up need to be
    check to be consistent and returning the FQDN of the hosts.

    python -c "import socket; print socket.getfqdn(); print
    socket.gethostbyname(socket.getfqdn())"

    We document this here:
    http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/latest/Cloudera-Manager-Installation-Guide/cmig_cm_requirements.html?scroll=cmig_topic_4_3_3_unique_1

    Todd
    On 4/20/14 1:55 AM, David Montgomery wrote:
    Installination fails and ratehr concered. I am using digi ocean. I
    have all ports open. How do I resovle? All machines are distributed
    the same pub and private keys


    ' Installation failed. Failed to receive heartbeat from agent.

    Ensure that the host's hostname is configured properly.
    Ensure that port 7182 is accessible on the Cloudera Manager
    Server (check firewall rules).
    Ensure that ports 9000 and 9001 are free on the host being added.
    Check agent logs in /var/log/cloudera-scm-agent/ on the host
    being added (some of the logs can be found in the installation details).

    Below is the first error in cloudera-scm-agent.log on the host I added.


    [20/Apr/2014 03:41:41 +0000] 2974 MainThread agent INFO Chmod'ing
    /run/cloudera-scm-agent/supervisor to 0751
    [20/Apr/2014 03:41:41 +0000] 2974 MainThread agent INFO Created
    /run/cloudera-scm-agent/supervisor/include
    [20/Apr/2014 03:41:41 +0000] 2974 MainThread agent INFO Chmod'ing
    /run/cloudera-scm-agent/supervisor/include to 0751
    [20/Apr/2014 03:41:41 +0000] 2974 MainThread agent ERROR Failed to
    connect to previous supervisor.
    Traceback (most recent call last):
    File "/usr/lib/cmf/agent/src/cmf/agent.py", line 1230, in
    find_or_start_supervisor
    self.configure_supervisor_clients()
    File "/usr/lib/cmf/agent/src/cmf/agent.py", line 1453, in
    configure_supervisor_clients
    supervisor_options.realize(args=["-c",
    os.path.join(self.supervisor_dir, "supervisord.conf")])
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py",
    line 1559, in realize
    Options.realize(self, *arg, **kw)
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py",
    line 309, in realize
    self.process_config()
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py",
    line 317, in process_config
    self.process_config_file(do_usage)
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py",
    line 352, in process_config_file
    self.usage(str(msg))
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py",
    line 140, in usage
    self.exit(2)
    SystemExit: 2
    [20/Apr/2014 03:41:41 +0000] 2974 MainThread tmpfs INFO
    Successfully mounted tmpfs at /run/cloudera-scm-agent/process
    [20/Apr/2014 03:41:42 +0000] 2974 MainThread agent INFO Trying to
    connect to newly launched supervisor (Attempt 1)
    [20/Apr/2014 03:41:42 +0000] 2974 MainThread agent INFO
    Successfully connected to supervisor
    [20/Apr/2014 03:41:42 +0000] 2974 MainThread _cplogging INFO
    [20/Apr/2014:03:41:42] ENGINE Bus STARTING
    [20/Apr/2014 03:41:42 +0000] 2974 MainThread _cplogging INFO
    [20/Apr/2014:03:41:42] ENGINE Started monitor thread '_TimeoutMonitor'.
    [20/Apr/2014 03:41:43 +0000] 2974 MainThread _cplogging INFO
    [20/Apr/2014:03:41:43] ENGINE Serving on localhost:9000
    [20/Apr/2014 03:41:43 +0000] 2974 MainThread _cplogging INFO
    [20/Apr/2014:03:41:43] ENGINE Bus STARTED
    [20/Apr/2014 03:41:43 +0000] 2974 MainThread __init__ INFO New










    2 +0000] 4445 MainThread agent INFO Re-using pre-existing directory:
    /run/cloudera-scm-agent/process
    [20/Apr/2014 03:46:52 +0000] 4445 MainThread agent INFO Re-using
    pre-existing directory: /run/cloudera-scm-agent/supervisor
    [20/Apr/2014 03:46:52 +0000] 4445 MainThread agent INFO Re-using
    pre-existing directory: /run/cloudera-scm-agent/supervisor/include
    [20/Apr/2014 03:46:52 +0000] 4445 MainThread agent ERROR
    Failed to connect to previous supervisor.
    Traceback (most recent call last):
    File "/usr/lib/cmf/agent/src/cmf/agent.py", line 1231, in
    find_or_start_supervisor
    self.get_supervisor_process_info()
    File "/usr/lib/cmf/agent/src/cmf/agent.py", line 1418, in
    get_supervisor_process_info
    self.identifier =
    self.supervisor_client.supervisor.getIdentification()
    File "/usr/lib/python2.7/xmlrpclib.py", line 1224, in __call__
    return self.__send(self.__name, args)
    File "/usr/lib/python2.7/xmlrpclib.py", line 1578, in __request
    verbose=self.__verbose
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/xmlrpc.py",
    line 460, in request
    self.connection.request('POST', handler, request_body, self.headers)
    File "/usr/lib/python2.7/httplib.py", line 958, in request
    self._send_request(method, url, body, headers)
    File "/usr/lib/python2.7/httplib.py", line 992, in _send_request
    self.endheaders(body)
    File "/usr/lib/python2.7/httplib.py", line 954, in endheaders
    self._send_output(message_body)
    File "/usr/lib/python2.7/httplib.py", line 814, in _send_output
    self.send(msg)
    File "/usr/lib/python2.7/httplib.py", line 776, in send
    self.connect()
    File "/usr/lib/python2.7/httplib.py", line 757, in connect
    self.timeout, self.source_address)
    File "/usr/lib/python2.7/socket.py", line 571, in create_connection
    raise err
    error: [Errno 111] Connection refused
    [20/Apr/2014 03:46:52 +0000] 4445 MainThread tmpfs INFO Reusing
    mounted tmpfs at /run/cloudera-scm-agent/process
    [20/Apr/2014 03:46:53 +0000] 4445 MainThread agent INFO Trying to
    connect to newly launched supervisor (Attempt 1)
    [20/Apr/2014 03:46:53 +0000] 4445 MainThread agent INFO
    Successfully connected to supervisor
    [20/Apr/2014 03:46:53 +0000] 4445 MainThread _cplogging INFO
    [20/Apr/2014:03:46:53] ENGINE Bus STARTING





    To unsubscribe from this group and stop receiving emails from it,
    send an email to scm-users+unsubscribe@cloudera.org
    --
    Todd Grayson
    Cloudera Support
    Customer Operations Engineering
    tgrayson@cloudera.com

    --
    Todd Grayson
    Cloudera Support
    Customer Operations Engineering
    tgrayson@cloudera.com

    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.
  • David Montgomery at Apr 21, 2014 at 11:59 am
    Below is the output.


    python -c "import socket; print socket.getfqdn(); print
    socket.gethostbyname(socket.getfqdn())"
    localhost
    127.0.0.1


    Anyway to fix I really do t get it


    Heres what is in /etc/cloudera-scm-agent.



    [General]
    # Hostname of the CM server.
    server_host=107.170.224.152

    # Port that the CM server is listening on.
    server_port=7182

    ## It should not normally be necessary to modify these.
    # Port that the CM agent should listen on.
    # listening_port=9000

    # IP Address that the CM agent should listen on.
    # listening_ip=

    # Hostname that the CM agent reports as its hostname. If unset, will be
    # obtained in code through something like this:
    #
    # python -c 'import socket; \
    # print socket.getfqdn(), \
    # socket.gethostbyname(socket.getfqdn())'
    #
    # listening_hostname=










    On Mon, Apr 21, 2014 at 12:16 AM, Todd Grayson wrote:

    Lets maintain the discussion on the list.

    Now that you made the changes you indicated of setting up IP and hostname
    in /etc/hosts.

    Does your system know itself by that name. You installed the agents when
    that configuration was not in place. It might be worth verifying what is
    in the agent configuration file. Look at /etc/cloudera-scm-agent on the
    cluster nodes you are managing from cloudera manager.

    What is the python script returning now that I provided, below?

    What do you see in the network initialization script for your network
    interface your system is starting up with?

    Thanks
    Todd




    On 4/20/14 8:31 AM, Todd Grayson wrote:

    Please avoid submitting blank subject e-mails to the list.

    Both forward and reverse lookup must be working correctly.

    From cloudera manager navigate to Hosts and then click the grey button to
    run the host inspector. When that completes, review the results, anything
    in yellow must be resolved before things will work properly. You can expand
    the list of issues for any issues found by clicking on the ">" in the
    results text.

    From what your describing, forward and reverse look up need to be check to
    be consistent and returning the FQDN of the hosts.

    python -c "import socket; print socket.getfqdn(); print
    socket.gethostbyname(socket.getfqdn())"

    We document this here:

    http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/latest/Cloudera-Manager-Installation-Guide/cmig_cm_requirements.html?scroll=cmig_topic_4_3_3_unique_1

    Todd

    On 4/20/14 1:55 AM, David Montgomery wrote:

    Installination fails and ratehr concered. I am using digi ocean. I
    have all ports open. How do I resovle? All machines are distributed the
    same pub and private keys


    ' Installation failed. Failed to receive heartbeat from agent.

    Ensure that the host's hostname is configured properly.
    Ensure that port 7182 is accessible on the Cloudera Manager Server
    (check firewall rules).
    Ensure that ports 9000 and 9001 are free on the host being added.
    Check agent logs in /var/log/cloudera-scm-agent/ on the host being
    added (some of the logs can be found in the installation details).

    Below is the first error in cloudera-scm-agent.log on the host I added.


    [20/Apr/2014 03:41:41 +0000] 2974 MainThread agent INFO
    Chmod'ing /run/cloudera-scm-agent/supervisor to 0751
    [20/Apr/2014 03:41:41 +0000] 2974 MainThread agent INFO Created
    /run/cloudera-scm-agent/supervisor/include
    [20/Apr/2014 03:41:41 +0000] 2974 MainThread agent INFO
    Chmod'ing /run/cloudera-scm-agent/supervisor/include to 0751
    [20/Apr/2014 03:41:41 +0000] 2974 MainThread agent ERROR Failed
    to connect to previous supervisor.
    Traceback (most recent call last):
    File "/usr/lib/cmf/agent/src/cmf/agent.py", line 1230, in
    find_or_start_supervisor
    self.configure_supervisor_clients()
    File "/usr/lib/cmf/agent/src/cmf/agent.py", line 1453, in
    configure_supervisor_clients
    supervisor_options.realize(args=["-c",
    os.path.join(self.supervisor_dir, "supervisord.conf")])
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py",
    line 1559, in realize
    Options.realize(self, *arg, **kw)
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py",
    line 309, in realize
    self.process_config()
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py",
    line 317, in process_config
    self.process_config_file(do_usage)
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py",
    line 352, in process_config_file
    self.usage(str(msg))
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py",
    line 140, in usage
    self.exit(2)
    SystemExit: 2
    [20/Apr/2014 03:41:41 +0000] 2974 MainThread tmpfs INFO
    Successfully mounted tmpfs at /run/cloudera-scm-agent/process
    [20/Apr/2014 03:41:42 +0000] 2974 MainThread agent INFO Trying
    to connect to newly launched supervisor (Attempt 1)
    [20/Apr/2014 03:41:42 +0000] 2974 MainThread agent INFO
    Successfully connected to supervisor
    [20/Apr/2014 03:41:42 +0000] 2974 MainThread _cplogging INFO
    [20/Apr/2014:03:41:42] ENGINE Bus STARTING
    [20/Apr/2014 03:41:42 +0000] 2974 MainThread _cplogging INFO
    [20/Apr/2014:03:41:42] ENGINE Started monitor thread '_TimeoutMonitor'.
    [20/Apr/2014 03:41:43 +0000] 2974 MainThread _cplogging INFO
    [20/Apr/2014:03:41:43] ENGINE Serving on localhost:9000
    [20/Apr/2014 03:41:43 +0000] 2974 MainThread _cplogging INFO
    [20/Apr/2014:03:41:43] ENGINE Bus STARTED
    [20/Apr/2014 03:41:43 +0000] 2974 MainThread __init__ INFO New










    2 +0000] 4445 MainThread agent INFO Re-using pre-existing directory:
    /run/cloudera-scm-agent/process
    [20/Apr/2014 03:46:52 +0000] 4445 MainThread agent INFO Re-using
    pre-existing directory: /run/cloudera-scm-agent/supervisor
    [20/Apr/2014 03:46:52 +0000] 4445 MainThread agent INFO Re-using
    pre-existing directory: /run/cloudera-scm-agent/supervisor/include
    [20/Apr/2014 03:46:52 +0000] 4445 MainThread agent ERROR
    Failed to connect to previous supervisor.
    Traceback (most recent call last):
    File "/usr/lib/cmf/agent/src/cmf/agent.py", line 1231, in
    find_or_start_supervisor
    self.get_supervisor_process_info()
    File "/usr/lib/cmf/agent/src/cmf/agent.py", line 1418, in
    get_supervisor_process_info
    self.identifier = self.supervisor_client.supervisor.getIdentification()
    File "/usr/lib/python2.7/xmlrpclib.py", line 1224, in __call__
    return self.__send(self.__name, args)
    File "/usr/lib/python2.7/xmlrpclib.py", line 1578, in __request
    verbose=self.__verbose
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/xmlrpc.py",
    line 460, in request
    self.connection.request('POST', handler, request_body, self.headers)
    File "/usr/lib/python2.7/httplib.py", line 958, in request
    self._send_request(method, url, body, headers)
    File "/usr/lib/python2.7/httplib.py", line 992, in _send_request
    self.endheaders(body)
    File "/usr/lib/python2.7/httplib.py", line 954, in endheaders
    self._send_output(message_body)
    File "/usr/lib/python2.7/httplib.py", line 814, in _send_output
    self.send(msg)
    File "/usr/lib/python2.7/httplib.py", line 776, in send
    self.connect()
    File "/usr/lib/python2.7/httplib.py", line 757, in connect
    self.timeout, self.source_address)
    File "/usr/lib/python2.7/socket.py", line 571, in create_connection
    raise err
    error: [Errno 111] Connection refused
    [20/Apr/2014 03:46:52 +0000] 4445 MainThread tmpfs INFO Reusing mounted
    tmpfs at /run/cloudera-scm-agent/process
    [20/Apr/2014 03:46:53 +0000] 4445 MainThread agent INFO Trying to
    connect to newly launched supervisor (Attempt 1)
    [20/Apr/2014 03:46:53 +0000] 4445 MainThread agent INFO Successfully
    connected to supervisor
    [20/Apr/2014 03:46:53 +0000] 4445 MainThread _cplogging INFO
    [20/Apr/2014:03:46:53] ENGINE Bus STARTING





    To unsubscribe from this group and stop receiving emails from it, send
    an email to scm-users+unsubscribe@cloudera.org.



    --
    Todd Grayson
    Cloudera Support
    Customer Operations engineeringtgrayson@cloudera.com



    --
    Todd Grayson
    Cloudera Support
    Customer Operations engineeringtgrayson@cloudera.com
    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.
  • David Montgomery at Apr 21, 2014 at 11:52 pm
    Resolved.....changed /etc/hostname and restarted the service. Thanks

    On Mon, Apr 21, 2014 at 7:59 PM, David Montgomery wrote:

    Below is the output.


    python -c "import socket; print socket.getfqdn(); print
    socket.gethostbyname(socket.getfqdn())"
    localhost
    127.0.0.1


    Anyway to fix I really do t get it


    Heres what is in /etc/cloudera-scm-agent.



    [General]
    # Hostname of the CM server.
    server_host=107.170.224.152

    # Port that the CM server is listening on.
    server_port=7182

    ## It should not normally be necessary to modify these.
    # Port that the CM agent should listen on.
    # listening_port=9000

    # IP Address that the CM agent should listen on.
    # listening_ip=

    # Hostname that the CM agent reports as its hostname. If unset, will be
    # obtained in code through something like this:
    #
    # python -c 'import socket; \
    # print socket.getfqdn(), \
    # socket.gethostbyname(socket.getfqdn())'
    #
    # listening_hostname=










    On Mon, Apr 21, 2014 at 12:16 AM, Todd Grayson wrote:

    Lets maintain the discussion on the list.

    Now that you made the changes you indicated of setting up IP and hostname
    in /etc/hosts.

    Does your system know itself by that name. You installed the agents when
    that configuration was not in place. It might be worth verifying what is
    in the agent configuration file. Look at /etc/cloudera-scm-agent on the
    cluster nodes you are managing from cloudera manager.

    What is the python script returning now that I provided, below?

    What do you see in the network initialization script for your network
    interface your system is starting up with?

    Thanks
    Todd




    On 4/20/14 8:31 AM, Todd Grayson wrote:

    Please avoid submitting blank subject e-mails to the list.

    Both forward and reverse lookup must be working correctly.

    From cloudera manager navigate to Hosts and then click the grey button to
    run the host inspector. When that completes, review the results, anything
    in yellow must be resolved before things will work properly. You can expand
    the list of issues for any issues found by clicking on the ">" in the
    results text.

    From what your describing, forward and reverse look up need to be check
    to be consistent and returning the FQDN of the hosts.

    python -c "import socket; print socket.getfqdn(); print
    socket.gethostbyname(socket.getfqdn())"

    We document this here:

    http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/latest/Cloudera-Manager-Installation-Guide/cmig_cm_requirements.html?scroll=cmig_topic_4_3_3_unique_1

    Todd

    On 4/20/14 1:55 AM, David Montgomery wrote:

    Installination fails and ratehr concered. I am using digi ocean. I
    have all ports open. How do I resovle? All machines are distributed the
    same pub and private keys


    ' Installation failed. Failed to receive heartbeat from agent.

    Ensure that the host's hostname is configured properly.
    Ensure that port 7182 is accessible on the Cloudera Manager Server
    (check firewall rules).
    Ensure that ports 9000 and 9001 are free on the host being added.
    Check agent logs in /var/log/cloudera-scm-agent/ on the host being
    added (some of the logs can be found in the installation details).

    Below is the first error in cloudera-scm-agent.log on the host I added.


    [20/Apr/2014 03:41:41 +0000] 2974 MainThread agent INFO
    Chmod'ing /run/cloudera-scm-agent/supervisor to 0751
    [20/Apr/2014 03:41:41 +0000] 2974 MainThread agent INFO
    Created /run/cloudera-scm-agent/supervisor/include
    [20/Apr/2014 03:41:41 +0000] 2974 MainThread agent INFO
    Chmod'ing /run/cloudera-scm-agent/supervisor/include to 0751
    [20/Apr/2014 03:41:41 +0000] 2974 MainThread agent ERROR Failed
    to connect to previous supervisor.
    Traceback (most recent call last):
    File "/usr/lib/cmf/agent/src/cmf/agent.py", line 1230, in
    find_or_start_supervisor
    self.configure_supervisor_clients()
    File "/usr/lib/cmf/agent/src/cmf/agent.py", line 1453, in
    configure_supervisor_clients
    supervisor_options.realize(args=["-c",
    os.path.join(self.supervisor_dir, "supervisord.conf")])
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py",
    line 1559, in realize
    Options.realize(self, *arg, **kw)
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py",
    line 309, in realize
    self.process_config()
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py",
    line 317, in process_config
    self.process_config_file(do_usage)
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py",
    line 352, in process_config_file
    self.usage(str(msg))
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/options.py",
    line 140, in usage
    self.exit(2)
    SystemExit: 2
    [20/Apr/2014 03:41:41 +0000] 2974 MainThread tmpfs INFO
    Successfully mounted tmpfs at /run/cloudera-scm-agent/process
    [20/Apr/2014 03:41:42 +0000] 2974 MainThread agent INFO Trying
    to connect to newly launched supervisor (Attempt 1)
    [20/Apr/2014 03:41:42 +0000] 2974 MainThread agent INFO
    Successfully connected to supervisor
    [20/Apr/2014 03:41:42 +0000] 2974 MainThread _cplogging INFO
    [20/Apr/2014:03:41:42] ENGINE Bus STARTING
    [20/Apr/2014 03:41:42 +0000] 2974 MainThread _cplogging INFO
    [20/Apr/2014:03:41:42] ENGINE Started monitor thread '_TimeoutMonitor'.
    [20/Apr/2014 03:41:43 +0000] 2974 MainThread _cplogging INFO
    [20/Apr/2014:03:41:43] ENGINE Serving on localhost:9000
    [20/Apr/2014 03:41:43 +0000] 2974 MainThread _cplogging INFO
    [20/Apr/2014:03:41:43] ENGINE Bus STARTED
    [20/Apr/2014 03:41:43 +0000] 2974 MainThread __init__ INFO New










    2 +0000] 4445 MainThread agent INFO Re-using pre-existing directory:
    /run/cloudera-scm-agent/process
    [20/Apr/2014 03:46:52 +0000] 4445 MainThread agent INFO Re-using
    pre-existing directory: /run/cloudera-scm-agent/supervisor
    [20/Apr/2014 03:46:52 +0000] 4445 MainThread agent INFO Re-using
    pre-existing directory: /run/cloudera-scm-agent/supervisor/include
    [20/Apr/2014 03:46:52 +0000] 4445 MainThread agent ERROR
    Failed to connect to previous supervisor.
    Traceback (most recent call last):
    File "/usr/lib/cmf/agent/src/cmf/agent.py", line 1231, in
    find_or_start_supervisor
    self.get_supervisor_process_info()
    File "/usr/lib/cmf/agent/src/cmf/agent.py", line 1418, in
    get_supervisor_process_info
    self.identifier = self.supervisor_client.supervisor.getIdentification()
    File "/usr/lib/python2.7/xmlrpclib.py", line 1224, in __call__
    return self.__send(self.__name, args)
    File "/usr/lib/python2.7/xmlrpclib.py", line 1578, in __request
    verbose=self.__verbose
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/xmlrpc.py",
    line 460, in request
    self.connection.request('POST', handler, request_body, self.headers)
    File "/usr/lib/python2.7/httplib.py", line 958, in request
    self._send_request(method, url, body, headers)
    File "/usr/lib/python2.7/httplib.py", line 992, in _send_request
    self.endheaders(body)
    File "/usr/lib/python2.7/httplib.py", line 954, in endheaders
    self._send_output(message_body)
    File "/usr/lib/python2.7/httplib.py", line 814, in _send_output
    self.send(msg)
    File "/usr/lib/python2.7/httplib.py", line 776, in send
    self.connect()
    File "/usr/lib/python2.7/httplib.py", line 757, in connect
    self.timeout, self.source_address)
    File "/usr/lib/python2.7/socket.py", line 571, in create_connection
    raise err
    error: [Errno 111] Connection refused
    [20/Apr/2014 03:46:52 +0000] 4445 MainThread tmpfs INFO Reusing mounted
    tmpfs at /run/cloudera-scm-agent/process
    [20/Apr/2014 03:46:53 +0000] 4445 MainThread agent INFO Trying to
    connect to newly launched supervisor (Attempt 1)
    [20/Apr/2014 03:46:53 +0000] 4445 MainThread agent INFO Successfully
    connected to supervisor
    [20/Apr/2014 03:46:53 +0000] 4445 MainThread _cplogging INFO
    [20/Apr/2014:03:46:53] ENGINE Bus STARTING





    To unsubscribe from this group and stop receiving emails from it, send
    an email to scm-users+unsubscribe@cloudera.org.



    --
    Todd Grayson
    Cloudera Support
    Customer Operations engineeringtgrayson@cloudera.com



    --
    Todd Grayson
    Cloudera Support
    Customer Operations engineeringtgrayson@cloudera.com
    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedApr 20, '14 at 7:55a
activeApr 21, '14 at 11:52p
posts5
users2
websitecloudera.com
irc#hadoop

2 users in discussion

David Montgomery: 3 posts Todd Grayson: 2 posts

People

Translate

site design / logo © 2022 Grokbase