FAQ
Running CM 4.0.3 (it's a new install) on CentOS 6. It installs all the
packages on the data node (2nd server) but then looking @ the details it
says can't start the scm agent:
"
BEGIN /sbin/service cloudera-scm-agent status | grep running
END (1)
BEGIN /sbin/service cloudera-scm-agent start
Starting cloudera-scm-agent: [FAILED]
END (1)
scm agent could not be started, giving up
"
This is the error i see in the cloudera-scm-agent.out file:

"/usr/lib64/cmf/agent/src/cmf/agent.py:24: DeprecationWarning: the sha
module is deprecated; use the hashlib module instead
import sha
/usr/lib64/cmf/agent/src/cmf/monitor/host/network_interfaces.py:11:
DeprecationWarning: the sets module is deprecated
from sets import Set
[25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Using
directory: /var/run/cloudera-scm-agent
[25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Using
supervisor binary path:
/usr/lib64/cmf/agent/src/cmf/../../build/env/bin/supervisord
[25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Logging
to /var/log/cloudera-scm-agent/cloudera-scm-agent.log
Error: could not find config file
/var/run/cloudera-scm-agent/supervisor/supervisord.conf
For help, use /usr/lib64/cmf/agent/src/cmf/agent.py -h
/usr/lib64/cmf/agent/src/cmf/agent.py:382: DeprecationWarning:
psutil.used_phymem is deprecated; use psutil.phymem_usage instead
really_used = psutil.used_phymem() - cached - buffers
/usr/lib64/cmf/agent/src/cmf/agent.py:384: DeprecationWarning:
psutil.total_virtmem is deprecated; use psutil.virtmem_usage instead
stats["virtual_memory_total"] = psutil.total_virtmem()
/usr/lib64/cmf/agent/src/cmf/agent.py:385: DeprecationWarning:
psutil.used_virtmem is deprecated; use psutil.virtmem_usage instead
stats["virtual_memory_used"] = psutil.used_virtmem()
/usr/lib64/cmf/agent/src/cmf/monitor/host/__init__.py:89:
DeprecationWarning: psutil.used_phymem is deprecated; use
psutil.phymem_usage instead
really_used = psutil.used_phymem() - cached - buffers
"
I tried looking for that directory on the agent would be installed on, but
it's not there. But i see the other folders (hdfs, hive, oozie, etc...)
there. I'm logging in through CM as root on that box. Any ideas?

Search Discussions

  • Philip Langdale at Jul 26, 2012 at 5:32 pm
    What are the permissions and ownerships on the directories?

    /var/run/cloudera-scm-agent/supervisor/

    --phil


    On 26 July 2012 10:29, ansonism wrote:

    /var/run/cloudera-scm-agent/supervisor/
  • Ansonism at Jul 26, 2012 at 5:35 pm
    drwxr-x--x -- owned by root.


    On Thursday, July 26, 2012 1:32:19 PM UTC-4, Philip Langdale wrote:

    What are the permissions and ownerships on the directories?

    /var/run/cloudera-scm-agent/supervisor/

    --phil


    On 26 July 2012 10:29, ansonism wrote:

    /var/run/cloudera-scm-agent/supervisor/
  • Ansonism at Jul 26, 2012 at 6:06 pm
    This is on the server which is the CM server.
    The server which needs agent as well, does not have that directory.
    On Thursday, July 26, 2012 1:29:35 PM UTC-4, ansonism wrote:

    Running CM 4.0.3 (it's a new install) on CentOS 6. It installs all the
    packages on the data node (2nd server) but then looking @ the details it
    says can't start the scm agent:
    "
    BEGIN /sbin/service cloudera-scm-agent status | grep running
    END (1)
    BEGIN /sbin/service cloudera-scm-agent start
    Starting cloudera-scm-agent: [FAILED]
    END (1)
    scm agent could not be started, giving up
    "
    This is the error i see in the cloudera-scm-agent.out file:

    "/usr/lib64/cmf/agent/src/cmf/agent.py:24: DeprecationWarning: the sha
    module is deprecated; use the hashlib module instead
    import sha
    /usr/lib64/cmf/agent/src/cmf/monitor/host/network_interfaces.py:11:
    DeprecationWarning: the sets module is deprecated
    from sets import Set
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Using
    directory: /var/run/cloudera-scm-agent
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Using
    supervisor binary path:
    /usr/lib64/cmf/agent/src/cmf/../../build/env/bin/supervisord
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Logging
    to /var/log/cloudera-scm-agent/cloudera-scm-agent.log
    Error: could not find config file
    /var/run/cloudera-scm-agent/supervisor/supervisord.conf
    For help, use /usr/lib64/cmf/agent/src/cmf/agent.py -h
    /usr/lib64/cmf/agent/src/cmf/agent.py:382: DeprecationWarning:
    psutil.used_phymem is deprecated; use psutil.phymem_usage instead
    really_used = psutil.used_phymem() - cached - buffers
    /usr/lib64/cmf/agent/src/cmf/agent.py:384: DeprecationWarning:
    psutil.total_virtmem is deprecated; use psutil.virtmem_usage instead
    stats["virtual_memory_total"] = psutil.total_virtmem()
    /usr/lib64/cmf/agent/src/cmf/agent.py:385: DeprecationWarning:
    psutil.used_virtmem is deprecated; use psutil.virtmem_usage instead
    stats["virtual_memory_used"] = psutil.used_virtmem()
    /usr/lib64/cmf/agent/src/cmf/monitor/host/__init__.py:89:
    DeprecationWarning: psutil.used_phymem is deprecated; use
    psutil.phymem_usage instead
    really_used = psutil.used_phymem() - cached - buffers
    "
    I tried looking for that directory on the agent would be installed on, but
    it's not there. But i see the other folders (hdfs, hive, oozie, etc...)
    there. I'm logging in through CM as root on that box. Any ideas?
  • Philip Langdale at Jul 26, 2012 at 6:43 pm
    Hmm. What happens if you manually create /var/run/cloudera-scm-agent and
    retry?

    --phil


    On 26 July 2012 11:06, ansonism wrote:

    This is on the server which is the CM server.
    The server which needs agent as well, does not have that directory.

    On Thursday, July 26, 2012 1:29:35 PM UTC-4, ansonism wrote:

    Running CM 4.0.3 (it's a new install) on CentOS 6. It installs all the
    packages on the data node (2nd server) but then looking @ the details it
    says can't start the scm agent:
    "
    BEGIN /sbin/service cloudera-scm-agent status | grep running
    END (1)
    BEGIN /sbin/service cloudera-scm-agent start
    Starting cloudera-scm-agent: [60G[ [0;31mFAILED [0;39m]
    END (1)
    scm agent could not be started, giving up
    "
    This is the error i see in the cloudera-scm-agent.out file:

    "/usr/lib64/cmf/agent/src/cmf/**agent.py:24: DeprecationWarning: the sha
    module is deprecated; use the hashlib module instead
    import sha
    /usr/lib64/cmf/agent/src/cmf/**monitor/host/network_**interfaces.py:11:
    DeprecationWarning: the sets module is deprecated
    from sets import Set
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Using
    directory: /var/run/cloudera-scm-agent
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Using
    supervisor binary path: /usr/lib64/cmf/agent/src/cmf/.**
    ./../build/env/bin/supervisord
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    Logging to /var/log/cloudera-scm-agent/**cloudera-scm-agent.log
    Error: could not find config file /var/run/cloudera-scm-agent/**
    supervisor/supervisord.conf
    For help, use /usr/lib64/cmf/agent/src/cmf/**agent.py -h
    /usr/lib64/cmf/agent/src/cmf/**agent.py:382: DeprecationWarning:
    psutil.used_phymem is deprecated; use psutil.phymem_usage instead
    really_used = psutil.used_phymem() - cached - buffers
    /usr/lib64/cmf/agent/src/cmf/**agent.py:384: DeprecationWarning:
    psutil.total_virtmem is deprecated; use psutil.virtmem_usage instead
    stats["virtual_memory_total"] = psutil.total_virtmem()
    /usr/lib64/cmf/agent/src/cmf/**agent.py:385: DeprecationWarning:
    psutil.used_virtmem is deprecated; use psutil.virtmem_usage instead
    stats["virtual_memory_used"] = psutil.used_virtmem()
    /usr/lib64/cmf/agent/src/cmf/**monitor/host/__init__.py:89:
    DeprecationWarning: psutil.used_phymem is deprecated; use
    psutil.phymem_usage instead
    really_used = psutil.used_phymem() - cached - buffers
    "
    I tried looking for that directory on the agent would be installed on,
    but it's not there. But i see the other folders (hdfs, hive, oozie,
    etc...) there. I'm logging in through CM as root on that box. Any ideas?
  • Ansonism at Jul 26, 2012 at 6:48 pm
    Yeah, i created that directory as well, and still nothing. I even tried to
    copy over the directory from the server (the agent folder) and still
    nothing.
    On Thursday, July 26, 2012 1:29:35 PM UTC-4, ansonism wrote:

    Running CM 4.0.3 (it's a new install) on CentOS 6. It installs all the
    packages on the data node (2nd server) but then looking @ the details it
    says can't start the scm agent:
    "
    BEGIN /sbin/service cloudera-scm-agent status | grep running
    END (1)
    BEGIN /sbin/service cloudera-scm-agent start
    Starting cloudera-scm-agent: [FAILED]
    END (1)
    scm agent could not be started, giving up
    "
    This is the error i see in the cloudera-scm-agent.out file:

    "/usr/lib64/cmf/agent/src/cmf/agent.py:24: DeprecationWarning: the sha
    module is deprecated; use the hashlib module instead
    import sha
    /usr/lib64/cmf/agent/src/cmf/monitor/host/network_interfaces.py:11:
    DeprecationWarning: the sets module is deprecated
    from sets import Set
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Using
    directory: /var/run/cloudera-scm-agent
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Using
    supervisor binary path:
    /usr/lib64/cmf/agent/src/cmf/../../build/env/bin/supervisord
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Logging
    to /var/log/cloudera-scm-agent/cloudera-scm-agent.log
    Error: could not find config file
    /var/run/cloudera-scm-agent/supervisor/supervisord.conf
    For help, use /usr/lib64/cmf/agent/src/cmf/agent.py -h
    /usr/lib64/cmf/agent/src/cmf/agent.py:382: DeprecationWarning:
    psutil.used_phymem is deprecated; use psutil.phymem_usage instead
    really_used = psutil.used_phymem() - cached - buffers
    /usr/lib64/cmf/agent/src/cmf/agent.py:384: DeprecationWarning:
    psutil.total_virtmem is deprecated; use psutil.virtmem_usage instead
    stats["virtual_memory_total"] = psutil.total_virtmem()
    /usr/lib64/cmf/agent/src/cmf/agent.py:385: DeprecationWarning:
    psutil.used_virtmem is deprecated; use psutil.virtmem_usage instead
    stats["virtual_memory_used"] = psutil.used_virtmem()
    /usr/lib64/cmf/agent/src/cmf/monitor/host/__init__.py:89:
    DeprecationWarning: psutil.used_phymem is deprecated; use
    psutil.phymem_usage instead
    really_used = psutil.used_phymem() - cached - buffers
    "
    I tried looking for that directory on the agent would be installed on, but
    it's not there. But i see the other folders (hdfs, hive, oozie, etc...)
    there. I'm logging in through CM as root on that box. Any ideas?
  • Philip Langdale at Jul 26, 2012 at 6:54 pm
    Can you look in the cloudera-scm-agent.log file (as opposed to do the .out
    file). That
    should have more information.

    --phil


    On 26 July 2012 11:48, ansonism wrote:

    Yeah, i created that directory as well, and still nothing. I even tried
    to copy over the directory from the server (the agent folder) and still
    nothing.

    On Thursday, July 26, 2012 1:29:35 PM UTC-4, ansonism wrote:

    Running CM 4.0.3 (it's a new install) on CentOS 6. It installs all the
    packages on the data node (2nd server) but then looking @ the details it
    says can't start the scm agent:
    "
    BEGIN /sbin/service cloudera-scm-agent status | grep running
    END (1)
    BEGIN /sbin/service cloudera-scm-agent start
    Starting cloudera-scm-agent: [60G[ [0;31mFAILED [0;39m]
    END (1)
    scm agent could not be started, giving up
    "
    This is the error i see in the cloudera-scm-agent.out file:

    "/usr/lib64/cmf/agent/src/cmf/**agent.py:24: DeprecationWarning: the sha
    module is deprecated; use the hashlib module instead
    import sha
    /usr/lib64/cmf/agent/src/cmf/**monitor/host/network_**interfaces.py:11:
    DeprecationWarning: the sets module is deprecated
    from sets import Set
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Using
    directory: /var/run/cloudera-scm-agent
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Using
    supervisor binary path: /usr/lib64/cmf/agent/src/cmf/.**
    ./../build/env/bin/supervisord
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    Logging to /var/log/cloudera-scm-agent/**cloudera-scm-agent.log
    Error: could not find config file /var/run/cloudera-scm-agent/**
    supervisor/supervisord.conf
    For help, use /usr/lib64/cmf/agent/src/cmf/**agent.py -h
    /usr/lib64/cmf/agent/src/cmf/**agent.py:382: DeprecationWarning:
    psutil.used_phymem is deprecated; use psutil.phymem_usage instead
    really_used = psutil.used_phymem() - cached - buffers
    /usr/lib64/cmf/agent/src/cmf/**agent.py:384: DeprecationWarning:
    psutil.total_virtmem is deprecated; use psutil.virtmem_usage instead
    stats["virtual_memory_total"] = psutil.total_virtmem()
    /usr/lib64/cmf/agent/src/cmf/**agent.py:385: DeprecationWarning:
    psutil.used_virtmem is deprecated; use psutil.virtmem_usage instead
    stats["virtual_memory_used"] = psutil.used_virtmem()
    /usr/lib64/cmf/agent/src/cmf/**monitor/host/__init__.py:89:
    DeprecationWarning: psutil.used_phymem is deprecated; use
    psutil.phymem_usage instead
    really_used = psutil.used_phymem() - cached - buffers
    "
    I tried looking for that directory on the agent would be installed on,
    but it's not there. But i see the other folders (hdfs, hive, oozie,
    etc...) there. I'm logging in through CM as root on that box. Any ideas?
  • Ansonism at Jul 26, 2012 at 9:16 pm
    It doesn't create a .log file. But I just realized that I looked @ the
    wrong server for the .out file.

    This is what's in the log file:

    /usr/lib64/cmf/agent/src/cmf/agent.py:24: DeprecationWarning: the sha
    module is deprecated; use the hashlib module instead
    import sha
    /usr/lib64/cmf/agent/src/cmf/monitor/host/network_interfaces.py:11:
    DeprecationWarning: the sets module is deprecated
    from sets import Set
    [26/Jul/2012 16:59:55 +0000] 10867 MainThread agent ERROR Could
    not determine hostname or ip address; proceeding.
    Traceback (most recent call last):
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1110, in
    parse_arguments
    ip_address = socket.gethostbyname(fqdn)
    gaierror: [Errno -2] Name or service not known
    usage: agent.py [-h] [--agent_dir AGENT_DIR]
    [--agent_httpd_port AGENT_HTTPD_PORT] --package_dir
    PACKAGE_DIR [--supervisord_path SUPERVISORD_PATH]
    [--supervisord_httpd_port SUPERVISORD_HTTPD_PORT]
    [--standalone STANDALONE] [--master MASTER]
    [--environment ENVIRONMENT] [--host_id HOST_ID]
    [--disable_supervisord_events] [--disable_eager_heartbeats]
    --hostname HOSTNAME --ip_address IP_ADDRESS [--use_tls]
    [--client_key_file CLIENT_KEY_FILE]
    [--client_cert_file CLIENT_CERT_FILE]
    [--verify_cert_file VERIFY_CERT_FILE]
    [--client_keypw_file CLIENT_KEYPW_FILE] [--logfile LOGFILE]
    [--optional_token]
    agent.py: error: argument --hostname is required
    [26/Jul/2012 16:59:55 +0000] 10867 Dummy-1 agent INFO Stopping
    agent...


    I checked the hostname file and that looks good. So not sure what is
    happening exactly. I happened to have check what it was doing through the
    details log and this is what I'm seeing:



    This is what I see it's trying to do in the config:

    BEGIN grep server_host=10.16.10.15 /etc/cloudera-scm-agent/config.ini
    END (1)
    BEGIN sed -e 's/\(server_host=\).*/\110.16.10.15/' -i
    /etc/cloudera-scm-agent/config.ini
    END (0)
    scm agent configured


    I'm not a sed export by \1 i'm assuming is still ok / good.

    On Thursday, July 26, 2012 2:54:46 PM UTC-4, Philip Langdale wrote:

    Can you look in the cloudera-scm-agent.log file (as opposed to do the .out
    file). That
    should have more information.

    --phil


    On 26 July 2012 11:48, ansonism wrote:

    Yeah, i created that directory as well, and still nothing. I even tried
    to copy over the directory from the server (the agent folder) and still
    nothing.

    On Thursday, July 26, 2012 1:29:35 PM UTC-4, ansonism wrote:

    Running CM 4.0.3 (it's a new install) on CentOS 6. It installs all the
    packages on the data node (2nd server) but then looking @ the details it
    says can't start the scm agent:
    "
    BEGIN /sbin/service cloudera-scm-agent status | grep running
    END (1)
    BEGIN /sbin/service cloudera-scm-agent start
    Starting cloudera-scm-agent: [60G[ [0;31mFAILED [0;39m]
    END (1)
    scm agent could not be started, giving up
    "
    This is the error i see in the cloudera-scm-agent.out file:

    "/usr/lib64/cmf/agent/src/cmf/**agent.py:24: DeprecationWarning: the
    sha module is deprecated; use the hashlib module instead
    import sha
    /usr/lib64/cmf/agent/src/cmf/**monitor/host/network_**interfaces.py:11:
    DeprecationWarning: the sets module is deprecated
    from sets import Set
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Using
    directory: /var/run/cloudera-scm-agent
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Using
    supervisor binary path: /usr/lib64/cmf/agent/src/cmf/.**
    ./../build/env/bin/supervisord
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    Logging to /var/log/cloudera-scm-agent/**cloudera-scm-agent.log
    Error: could not find config file /var/run/cloudera-scm-agent/**
    supervisor/supervisord.conf
    For help, use /usr/lib64/cmf/agent/src/cmf/**agent.py -h
    /usr/lib64/cmf/agent/src/cmf/**agent.py:382: DeprecationWarning:
    psutil.used_phymem is deprecated; use psutil.phymem_usage instead
    really_used = psutil.used_phymem() - cached - buffers
    /usr/lib64/cmf/agent/src/cmf/**agent.py:384: DeprecationWarning:
    psutil.total_virtmem is deprecated; use psutil.virtmem_usage instead
    stats["virtual_memory_total"] = psutil.total_virtmem()
    /usr/lib64/cmf/agent/src/cmf/**agent.py:385: DeprecationWarning:
    psutil.used_virtmem is deprecated; use psutil.virtmem_usage instead
    stats["virtual_memory_used"] = psutil.used_virtmem()
    /usr/lib64/cmf/agent/src/cmf/**monitor/host/__init__.py:89:
    DeprecationWarning: psutil.used_phymem is deprecated; use
    psutil.phymem_usage instead
    really_used = psutil.used_phymem() - cached - buffers
    "
    I tried looking for that directory on the agent would be installed on,
    but it's not there. But i see the other folders (hdfs, hive, oozie,
    etc...) there. I'm logging in through CM as root on that box. Any ideas?
  • Philip Langdale at Jul 26, 2012 at 9:22 pm
    The issue is that the gethostbyname() call is failing. which indicates a
    pretty fundamental
    problem with how the system identifies itself.

    What do the 'hostname' and 'hostname -f' commands return? What is the
    contents of your
    /etc/hosts file.

    --phil


    On 26 July 2012 14:10, ansonism wrote:

    It doesn't create a .log file. But I just realized that I looked @ the
    wrong server for the .out file.

    This is what's in the log file:

    /usr/lib64/cmf/agent/src/cmf/agent.py:24: DeprecationWarning: the sha
    module is deprecated; use the hashlib module instead
    import sha
    /usr/lib64/cmf/agent/src/cmf/monitor/host/network_interfaces.py:11:
    DeprecationWarning: the sets module is deprecated
    from sets import Set
    [26/Jul/2012 16:59:55 +0000] 10867 MainThread agent ERROR Could
    not determine hostname or ip address; proceeding.
    Traceback (most recent call last):
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1110, in
    parse_arguments
    ip_address = socket.gethostbyname(fqdn)
    gaierror: [Errno -2] Name or service not known
    usage: agent.py [-h] [--agent_dir AGENT_DIR]
    [--agent_httpd_port AGENT_HTTPD_PORT] --package_dir
    PACKAGE_DIR [--supervisord_path SUPERVISORD_PATH]
    [--supervisord_httpd_port SUPERVISORD_HTTPD_PORT]
    [--standalone STANDALONE] [--master MASTER]
    [--environment ENVIRONMENT] [--host_id HOST_ID]
    [--disable_supervisord_events] [--disable_eager_heartbeats]
    --hostname HOSTNAME --ip_address IP_ADDRESS [--use_tls]
    [--client_key_file CLIENT_KEY_FILE]
    [--client_cert_file CLIENT_CERT_FILE]
    [--verify_cert_file VERIFY_CERT_FILE]
    [--client_keypw_file CLIENT_KEYPW_FILE] [--logfile LOGFILE]
    [--optional_token]
    agent.py: error: argument --hostname is required
    [26/Jul/2012 16:59:55 +0000] 10867 Dummy-1 agent INFO Stopping
    agent...


    I checked the hostname file and that looks good. So not sure what is
    happening exactly. I happened to have check what it was doing through the
    details log and this is what I'm seeing:



    This is what I see it's trying to do in the config:

    BEGIN grep server_host=10.16.10.15 /etc/cloudera-scm-agent/config.ini
    END (1)
    BEGIN sed -e 's/\(server_host=\).*/\110.16.10.15/' -i
    /etc/cloudera-scm-agent/config.ini
    END (0)
    scm agent configured


    I'm not a sed export by \1 i'm assuming is still ok / good.

    On Thursday, July 26, 2012 2:54:46 PM UTC-4, Philip Langdale wrote:

    Can you look in the cloudera-scm-agent.log file (as opposed to do the
    .out file). That
    should have more information.

    --phil


    On 26 July 2012 11:48, ansonism wrote:

    Yeah, i created that directory as well, and still nothing. I even tried
    to copy over the directory from the server (the agent folder) and still
    nothing.

    On Thursday, July 26, 2012 1:29:35 PM UTC-4, ansonism wrote:

    Running CM 4.0.3 (it's a new install) on CentOS 6. It installs all the
    packages on the data node (2nd server) but then looking @ the details it
    says can't start the scm agent:
    "
    BEGIN /sbin/service cloudera-scm-agent status | grep running
    END (1)
    BEGIN /sbin/service cloudera-scm-agent start
    Starting cloudera-scm-agent: [60G[ [0;31mFAILED [0;39m]
    END (1)
    scm agent could not be started, giving up
    "
    This is the error i see in the cloudera-scm-agent.out file:

    "/usr/lib64/cmf/agent/src/cmf/****agent.py:24: DeprecationWarning: the
    sha module is deprecated; use the hashlib module instead
    import sha
    /usr/lib64/cmf/agent/src/cmf/**m**onitor/host/network_**interfaces**.py:11:
    DeprecationWarning: the sets module is deprecated
    from sets import Set
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    Using directory: /var/run/cloudera-scm-agent
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    Using supervisor binary path: /usr/lib64/cmf/agent/src/cmf/.****
    ./../build/env/bin/supervisord
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    Logging to /var/log/cloudera-scm-agent/**cl**oudera-scm-agent.log
    Error: could not find config file /var/run/cloudera-scm-agent/**su**
    pervisor/supervisord.conf
    For help, use /usr/lib64/cmf/agent/src/cmf/**a**gent.py -h
    /usr/lib64/cmf/agent/src/cmf/**a**gent.py:382: DeprecationWarning:
    psutil.used_phymem is deprecated; use psutil.phymem_usage instead
    really_used = psutil.used_phymem() - cached - buffers
    /usr/lib64/cmf/agent/src/cmf/**a**gent.py:384: DeprecationWarning:
    psutil.total_virtmem is deprecated; use psutil.virtmem_usage instead
    stats["virtual_memory_total"] = psutil.total_virtmem()
    /usr/lib64/cmf/agent/src/cmf/**a**gent.py:385: DeprecationWarning:
    psutil.used_virtmem is deprecated; use psutil.virtmem_usage instead
    stats["virtual_memory_used"] = psutil.used_virtmem()
    /usr/lib64/cmf/agent/src/cmf/**m**onitor/host/__init__.py:89:
    DeprecationWarning: psutil.used_phymem is deprecated; use
    psutil.phymem_usage instead
    really_used = psutil.used_phymem() - cached - buffers
    "
    I tried looking for that directory on the agent would be installed on,
    but it's not there. But i see the other folders (hdfs, hive, oozie,
    etc...) there. I'm logging in through CM as root on that box. Any ideas?
  • Ansonism at Jul 26, 2012 at 9:32 pm
    hostname comes back w/ the host.
    hostname -f comes back an Unknown Host.

    this is what's in the host file:

    127.0.0.1 localhost localhost.localdomain localhost4
    localhost4.localdomain4
    ::1 localhost localhost.localdomain localhost6
    localhost6.localdomain6
    On Thursday, July 26, 2012 5:22:25 PM UTC-4, Philip Langdale wrote:

    The issue is that the gethostbyname() call is failing. which indicates a
    pretty fundamental
    problem with how the system identifies itself.

    What do the 'hostname' and 'hostname -f' commands return? What is the
    contents of your
    /etc/hosts file.

    --phil


    On 26 July 2012 14:10, ansonism wrote:

    It doesn't create a .log file. But I just realized that I looked @ the
    wrong server for the .out file.

    This is what's in the log file:

    /usr/lib64/cmf/agent/src/cmf/agent.py:24: DeprecationWarning: the sha
    module is deprecated; use the hashlib module instead
    import sha
    /usr/lib64/cmf/agent/src/cmf/monitor/host/network_interfaces.py:11:
    DeprecationWarning: the sets module is deprecated
    from sets import Set
    [26/Jul/2012 16:59:55 +0000] 10867 MainThread agent ERROR Could
    not determine hostname or ip address; proceeding.
    Traceback (most recent call last):
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1110, in
    parse_arguments
    ip_address = socket.gethostbyname(fqdn)
    gaierror: [Errno -2] Name or service not known
    usage: agent.py [-h] [--agent_dir AGENT_DIR]
    [--agent_httpd_port AGENT_HTTPD_PORT] --package_dir
    PACKAGE_DIR [--supervisord_path SUPERVISORD_PATH]
    [--supervisord_httpd_port SUPERVISORD_HTTPD_PORT]
    [--standalone STANDALONE] [--master MASTER]
    [--environment ENVIRONMENT] [--host_id HOST_ID]
    [--disable_supervisord_events]
    [--disable_eager_heartbeats]
    --hostname HOSTNAME --ip_address IP_ADDRESS [--use_tls]
    [--client_key_file CLIENT_KEY_FILE]
    [--client_cert_file CLIENT_CERT_FILE]
    [--verify_cert_file VERIFY_CERT_FILE]
    [--client_keypw_file CLIENT_KEYPW_FILE] [--logfile
    LOGFILE]
    [--optional_token]
    agent.py: error: argument --hostname is required
    [26/Jul/2012 16:59:55 +0000] 10867 Dummy-1 agent INFO Stopping
    agent...


    I checked the hostname file and that looks good. So not sure what is
    happening exactly. I happened to have check what it was doing through the
    details log and this is what I'm seeing:



    This is what I see it's trying to do in the config:

    BEGIN grep server_host=10.16.10.15 /etc/cloudera-scm-agent/config.ini
    END (1)
    BEGIN sed -e 's/\(server_host=\).*/\110.16.10.15/' -i
    /etc/cloudera-scm-agent/config.ini
    END (0)
    scm agent configured


    I'm not a sed export by \1 i'm assuming is still ok / good.

    On Thursday, July 26, 2012 2:54:46 PM UTC-4, Philip Langdale wrote:

    Can you look in the cloudera-scm-agent.log file (as opposed to do the
    .out file). That
    should have more information.

    --phil


    On 26 July 2012 11:48, ansonism wrote:

    Yeah, i created that directory as well, and still nothing. I even
    tried to copy over the directory from the server (the agent folder) and
    still nothing.

    On Thursday, July 26, 2012 1:29:35 PM UTC-4, ansonism wrote:

    Running CM 4.0.3 (it's a new install) on CentOS 6. It installs all
    the packages on the data node (2nd server) but then looking @ the details
    it says can't start the scm agent:
    "
    BEGIN /sbin/service cloudera-scm-agent status | grep running
    END (1)
    BEGIN /sbin/service cloudera-scm-agent start
    Starting cloudera-scm-agent: [60G[ [0;31mFAILED [0;39m]
    END (1)
    scm agent could not be started, giving up
    "
    This is the error i see in the cloudera-scm-agent.out file:

    "/usr/lib64/cmf/agent/src/cmf/****agent.py:24: DeprecationWarning:
    the sha module is deprecated; use the hashlib module instead
    import sha
    /usr/lib64/cmf/agent/src/cmf/**m**onitor/host/network_**interfaces**.py:11:
    DeprecationWarning: the sets module is deprecated
    from sets import Set
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    Using directory: /var/run/cloudera-scm-agent
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    Using supervisor binary path: /usr/lib64/cmf/agent/src/cmf/.****
    ./../build/env/bin/supervisord
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    Logging to /var/log/cloudera-scm-agent/**cl**oudera-scm-agent.log
    Error: could not find config file /var/run/cloudera-scm-agent/**su**
    pervisor/supervisord.conf
    For help, use /usr/lib64/cmf/agent/src/cmf/**a**gent.py -h
    /usr/lib64/cmf/agent/src/cmf/**a**gent.py:382: DeprecationWarning:
    psutil.used_phymem is deprecated; use psutil.phymem_usage instead
    really_used = psutil.used_phymem() - cached - buffers
    /usr/lib64/cmf/agent/src/cmf/**a**gent.py:384: DeprecationWarning:
    psutil.total_virtmem is deprecated; use psutil.virtmem_usage instead
    stats["virtual_memory_total"] = psutil.total_virtmem()
    /usr/lib64/cmf/agent/src/cmf/**a**gent.py:385: DeprecationWarning:
    psutil.used_virtmem is deprecated; use psutil.virtmem_usage instead
    stats["virtual_memory_used"] = psutil.used_virtmem()
    /usr/lib64/cmf/agent/src/cmf/**m**onitor/host/__init__.py:89:
    DeprecationWarning: psutil.used_phymem is deprecated; use
    psutil.phymem_usage instead
    really_used = psutil.used_phymem() - cached - buffers
    "
    I tried looking for that directory on the agent would be installed on,
    but it's not there. But i see the other folders (hdfs, hive, oozie,
    etc...) there. I'm logging in through CM as root on that box. Any ideas?
  • Ansonism at Jul 26, 2012 at 9:46 pm
    Oh geez...thanks. The resolv.conf file was not good.
    Thanks for all the help.
    On Thursday, July 26, 2012 5:22:25 PM UTC-4, Philip Langdale wrote:

    The issue is that the gethostbyname() call is failing. which indicates a
    pretty fundamental
    problem with how the system identifies itself.

    What do the 'hostname' and 'hostname -f' commands return? What is the
    contents of your
    /etc/hosts file.

    --phil


    On 26 July 2012 14:10, ansonism wrote:

    It doesn't create a .log file. But I just realized that I looked @ the
    wrong server for the .out file.

    This is what's in the log file:

    /usr/lib64/cmf/agent/src/cmf/agent.py:24: DeprecationWarning: the sha
    module is deprecated; use the hashlib module instead
    import sha
    /usr/lib64/cmf/agent/src/cmf/monitor/host/network_interfaces.py:11:
    DeprecationWarning: the sets module is deprecated
    from sets import Set
    [26/Jul/2012 16:59:55 +0000] 10867 MainThread agent ERROR Could
    not determine hostname or ip address; proceeding.
    Traceback (most recent call last):
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1110, in
    parse_arguments
    ip_address = socket.gethostbyname(fqdn)
    gaierror: [Errno -2] Name or service not known
    usage: agent.py [-h] [--agent_dir AGENT_DIR]
    [--agent_httpd_port AGENT_HTTPD_PORT] --package_dir
    PACKAGE_DIR [--supervisord_path SUPERVISORD_PATH]
    [--supervisord_httpd_port SUPERVISORD_HTTPD_PORT]
    [--standalone STANDALONE] [--master MASTER]
    [--environment ENVIRONMENT] [--host_id HOST_ID]
    [--disable_supervisord_events]
    [--disable_eager_heartbeats]
    --hostname HOSTNAME --ip_address IP_ADDRESS [--use_tls]
    [--client_key_file CLIENT_KEY_FILE]
    [--client_cert_file CLIENT_CERT_FILE]
    [--verify_cert_file VERIFY_CERT_FILE]
    [--client_keypw_file CLIENT_KEYPW_FILE] [--logfile
    LOGFILE]
    [--optional_token]
    agent.py: error: argument --hostname is required
    [26/Jul/2012 16:59:55 +0000] 10867 Dummy-1 agent INFO Stopping
    agent...


    I checked the hostname file and that looks good. So not sure what is
    happening exactly. I happened to have check what it was doing through the
    details log and this is what I'm seeing:



    This is what I see it's trying to do in the config:

    BEGIN grep server_host=10.16.10.15 /etc/cloudera-scm-agent/config.ini
    END (1)
    BEGIN sed -e 's/\(server_host=\).*/\110.16.10.15/' -i
    /etc/cloudera-scm-agent/config.ini
    END (0)
    scm agent configured


    I'm not a sed export by \1 i'm assuming is still ok / good.

    On Thursday, July 26, 2012 2:54:46 PM UTC-4, Philip Langdale wrote:

    Can you look in the cloudera-scm-agent.log file (as opposed to do the
    .out file). That
    should have more information.

    --phil


    On 26 July 2012 11:48, ansonism wrote:

    Yeah, i created that directory as well, and still nothing. I even
    tried to copy over the directory from the server (the agent folder) and
    still nothing.

    On Thursday, July 26, 2012 1:29:35 PM UTC-4, ansonism wrote:

    Running CM 4.0.3 (it's a new install) on CentOS 6. It installs all
    the packages on the data node (2nd server) but then looking @ the details
    it says can't start the scm agent:
    "
    BEGIN /sbin/service cloudera-scm-agent status | grep running
    END (1)
    BEGIN /sbin/service cloudera-scm-agent start
    Starting cloudera-scm-agent: [60G[ [0;31mFAILED [0;39m]
    END (1)
    scm agent could not be started, giving up
    "
    This is the error i see in the cloudera-scm-agent.out file:

    "/usr/lib64/cmf/agent/src/cmf/****agent.py:24: DeprecationWarning:
    the sha module is deprecated; use the hashlib module instead
    import sha
    /usr/lib64/cmf/agent/src/cmf/**m**onitor/host/network_**interfaces**.py:11:
    DeprecationWarning: the sets module is deprecated
    from sets import Set
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    Using directory: /var/run/cloudera-scm-agent
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    Using supervisor binary path: /usr/lib64/cmf/agent/src/cmf/.****
    ./../build/env/bin/supervisord
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    Logging to /var/log/cloudera-scm-agent/**cl**oudera-scm-agent.log
    Error: could not find config file /var/run/cloudera-scm-agent/**su**
    pervisor/supervisord.conf
    For help, use /usr/lib64/cmf/agent/src/cmf/**a**gent.py -h
    /usr/lib64/cmf/agent/src/cmf/**a**gent.py:382: DeprecationWarning:
    psutil.used_phymem is deprecated; use psutil.phymem_usage instead
    really_used = psutil.used_phymem() - cached - buffers
    /usr/lib64/cmf/agent/src/cmf/**a**gent.py:384: DeprecationWarning:
    psutil.total_virtmem is deprecated; use psutil.virtmem_usage instead
    stats["virtual_memory_total"] = psutil.total_virtmem()
    /usr/lib64/cmf/agent/src/cmf/**a**gent.py:385: DeprecationWarning:
    psutil.used_virtmem is deprecated; use psutil.virtmem_usage instead
    stats["virtual_memory_used"] = psutil.used_virtmem()
    /usr/lib64/cmf/agent/src/cmf/**m**onitor/host/__init__.py:89:
    DeprecationWarning: psutil.used_phymem is deprecated; use
    psutil.phymem_usage instead
    really_used = psutil.used_phymem() - cached - buffers
    "
    I tried looking for that directory on the agent would be installed on,
    but it's not there. But i see the other folders (hdfs, hive, oozie,
    etc...) there. I'm logging in through CM as root on that box. Any ideas?
  • Ansonism at Jul 26, 2012 at 7:11 pm
    There's nothing much in there. This is on the CM server agent directory:

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Adding
    env vars that start with CMF_AGENT_

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    explicitly passing env var to supervisee: HIVE_DEFAULT_XML:
    /usr/lib/hive/conf/hive-default.xml

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    explicitly passing env var to supervisee: MGMT_HOME: /usr/share/cmf

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    explicitly passing env var to supervisee: CDH_HTTPFS_HOME:
    /usr/lib/hadoop-httpfs

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    explicitly passing env var to supervisee: CDH_HBASE_HOME: /usr/lib/hbase

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    explicitly passing env var to supervisee: CDH_HADOOP_HOME: /usr/lib/hadoop

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    explicitly passing env var to supervisee: CDH_HUE_HOME: /usr/share/hue

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    explicitly passing env var to supervisee: TOMCAT_HOME:
    /usr/lib/bigtop-tomcat

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    explicitly passing env var to supervisee: CDH_HIVE_HOME: /usr/lib/hive

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    explicitly passing env var to supervisee: CDH_YARN_HOME:
    /usr/lib/hadoop-yarn

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    explicitly passing env var to supervisee: CDH_OOZIE_HOME: /usr/lib/oozie

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    explicitly passing env var to supervisee: CDH_ZOOKEEPER_HOME:
    /usr/lib/zookeeper

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    explicitly passing env var to supervisee: CDH_HUE_PLUGINS_HOME:
    /usr/lib/hadoop

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    explicitly passing env var to supervisee: JSVC_HOME:
    /usr/libexec/bigtop-utils

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    explicitly passing env var to supervisee: CDH_HDFS_HOME:
    /usr/lib/hadoop-hdfs

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    explicitly passing env var to supervisee: CDH_MR1_HOME:
    /usr/lib/hadoop-0.20-mapreduce

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    explicitly passing env var to supervisee: CDH_MR2_HOME:
    /usr/lib/hadoop-mapreduce

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    explicitly passing env var to supervisee: CDH_HADOOP_BIN: /usr/bin/hadoop

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO No
    command line vars

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Agent
    starting as pid 9438 user root(0) group root(0).

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Created
    /var/run/cloudera-scm-agent

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    Chmod'ing /var/run/cloudera-scm-agent to 0751

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Setting
    up eager heartbeat event monitor.

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Using
    metrics_url_timeout_seconds of 30.000000

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Using
    task_metrics_timeout_seconds of 5.000000

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Using
    max_collection_wait_seconds of 10.000000

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread metrics INFO
    Importing tasktracker metric schema from file
    /usr/lib64/cmf/agent/src/cmf/monitor/tasktracker/schema.json

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    Supervised processes will inherit the following environment:
    {'CMF_AGENT_HIVE_DEFAULT_XML': '/usr/lib/hive/conf/hive-default.xml',
    'CMF_AGENT_MGMT_HOME': '/usr/share/cmf', 'CMF_AGENT_CDH_HTTPFS_HOME':
    '/usr/lib/hadoop-httpfs', 'PATH': '/sbin:/usr/sbin:/bin:/usr/bin',
    'CMF_AGENT_CDH_HBASE_HOME': '/usr/lib/hbase', 'LANG': 'en_US.UTF-8',
    'TERM': 'vt100', 'CMF_AGENT_CDH_HADOOP_HOME': '/usr/lib/hadoop',
    'CMF_AGENT_CDH_HUE_HOME': '/usr/share/hue', 'SHLVL': '1',
    'CMF_AGENT_TOMCAT_HOME': '/usr/lib/bigtop-tomcat',
    'CMF_AGENT_CDH_HIVE_HOME': '/usr/lib/hive', 'CLOUDERA_MYSQL_CONNECTOR_JAR':
    '/usr/share/java/mysql-connector-java.jar', 'CMF_AGENT_CDH_YARN_HOME':
    '/usr/lib/hadoop-yarn', 'CMF_AGENT_CDH_OOZIE_HOME': '/usr/lib/oozie',
    'CMF_AGENT_CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper',
    'CMF_AGENT_CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CMF_AGENT_JSVC_HOME':
    '/usr/libexec/bigtop-utils', 'CMF_AGENT_CDH_HDFS_HOME':
    '/usr/lib/hadoop-hdfs', 'CMF_AGENT_CDH_MR1_HOME':
    '/usr/lib/hadoop-0.20-mapreduce', 'CMF_AGENT_CDH_MR2_HOME':
    '/usr/lib/hadoop-mapreduce', 'PWD': '/', 'CMF_AGENT_CDH_HADOOP_BIN':
    '/usr/bin/hadoop'}

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    Supervised processes will add the following their environment environment:
    {'CDH_OOZIE_HOME': '/usr/lib/oozie', 'JSVC_HOME':
    '/usr/libexec/bigtop-utils', 'CMF_PACKAGE_DIR': '/usr/lib64/cmf/service',
    'CDH_HUE_HOME': '/usr/share/hue', 'MGMT_HOME': '/usr/share/cmf',
    'CDH_HIVE_HOME': '/usr/lib/hive', 'CDH_HTTPFS_HOME':
    '/usr/lib/hadoop-httpfs', 'CLOUDERA_MYSQL_CONNECTOR_JAR':
    '/usr/share/java/mysql-connector-java.jar', 'CDH_YARN_HOME':
    '/usr/lib/hadoop-yarn', 'HIVE_DEFAULT_XML':
    '/usr/lib/hive/conf/hive-default.xml', 'CDH_HUE_PLUGINS_HOME':
    '/usr/lib/hadoop', 'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'CDH_MR2_HOME':
    '/usr/lib/hadoop-mapreduce', 'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs',
    'CDH_MR1_HOME': '/usr/lib/hadoop-0.20-mapreduce', 'TOMCAT_HOME':
    '/usr/lib/bigtop-tomcat', 'CDH_HBASE_HOME': '/usr/lib/hbase',
    'CDH_HADOOP_HOME': '/usr/lib/hadoop', 'CDH_ZOOKEEPER_HOME':
    '/usr/lib/zookeeper'}

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Created
    /var/run/cloudera-scm-agent/process

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    Chmod'ing /var/run/cloudera-scm-agent/process to 0751

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Created
    /var/run/cloudera-scm-agent/supervisor

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    Chmod'ing /var/run/cloudera-scm-agent/supervisor to 0751

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Created
    /var/run/cloudera-scm-agent/supervisor/include

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO
    Chmod'ing /var/run/cloudera-scm-agent/supervisor/include to 0751

    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Failed
    to connect to previous supervisor.

    [25/Jul/2012 17:23:19 +0000] 9438 MainThread agent INFO Trying
    to connect to newly launched supervisor (Attempt 1)

    [25/Jul/2012 17:23:20 +0000] 9438 MainThread agent INFO
    Successfully connected to supervisor

    [25/Jul/2012 17:23:20 +0000] 9438 MainThread _cplogging INFO
    [25/Jul/2012:17:23:20] ENGINE Bus STARTING

    [25/Jul/2012 17:23:20 +0000] 9438 MainThread _cplogging INFO
    [25/Jul/2012:17:23:20] ENGINE Started monitor thread '_TimeoutMonitor'.

    [25/Jul/2012 17:23:20 +0000] 9438 MainThread _cplogging INFO
    [25/Jul/2012:17:23:20] ENGINE Serving on server1Main:9000

    [25/Jul/2012 17:23:20 +0000] 9438 MainThread _cplogging INFO
    [25/Jul/2012:17:23:20] ENGINE Bus STARTED

    [25/Jul/2012 17:23:20 +0000] 9438 MainThread agent WARNING Setting
    default socket timeout to 30!

    [25/Jul/2012 17:23:20 +0000] 9438 Monitor-HostMonitor network_interfaces
    WARNING Interface 'eth2' reported unknown speed '65535'

    [25/Jul/2012 17:23:20 +0000] 9438 Monitor-HostMonitor network_interfaces
    WARNING Interface 'eth2' reported unknown duplex mode '255'

    [25/Jul/2012 17:23:20 +0000] 9438 Monitor-HostMonitor network_interfaces
    WARNING Interface 'eth3' reported unknown speed '65535'

    [25/Jul/2012 17:23:20 +0000] 9438 Monitor-HostMonitor network_interfaces
    WARNING Interface 'eth3' reported unknown duplex mode '255'

    [25/Jul/2012 17:23:20 +0000] 9438 Monitor-HostMonitor network_interfaces
    WARNING Interface 'em4' reported unknown speed '65535'

    [25/Jul/2012 17:23:20 +0000] 9438 Monitor-HostMonitor network_interfaces
    WARNING Interface 'em4' reported unknown duplex mode '255'

    [25/Jul/2012 17:23:20 +0000] 9438 Monitor-HostMonitor network_interfaces
    WARNING Interface 'p2p1' reported unknown speed '4294967295'

    [25/Jul/2012 17:23:20 +0000] 9438 Monitor-HostMonitor network_interfaces
    WARNING Interface 'p2p1' reported unknown duplex mode '255'

    [25/Jul/2012 17:23:20 +0000] 9438 Monitor-HostMonitor network_interfaces
    WARNING Interface 'p2p2' reported unknown speed '4294967295'

    [25/Jul/2012 17:23:20 +0000] 9438 Monitor-HostMonitor network_interfaces
    WARNING Interface 'p2p2' reported unknown duplex mode '255'

    [25/Jul/2012 18:43:35 +0000] 9438 CP Server Thread-4 _cplogging INFO
    192.168.100.70 - - [25/Jul/2012:18:43:35] "GET
    /heartbeat?token=45c678dafe40130122108ff4ea1e0c76 HTTP/1.1" 200 2 ""
    "NING/1.0"

    [25/Jul/2012 18:43:35 +0000] 9438 MainThread agent INFO
    Activating Process 1-host-inspector

    [25/Jul/2012 18:43:35 +0000] 9438 MainThread agent INFO Created
    /var/run/cloudera-scm-agent/process/1-host-inspector

    [25/Jul/2012 18:43:35 +0000] 9438 MainThread agent INFO Chowning
    /var/run/cloudera-scm-agent/process/1-host-inspector to root (0) root (0)

    [25/Jul/2012 18:43:35 +0000] 9438 MainThread agent INFO
    Chmod'ing /var/run/cloudera-scm-agent/process/1-host-inspector to 0751

    [25/Jul/2012 18:43:35 +0000] 9438 MainThread util INFO
    Extracted 1 files and 0 dirs to
    /var/run/cloudera-scm-agent/process/1-host-inspector.

    [25/Jul/2012 18:43:35 +0000] 9438 MainThread agent INFO Created
    /var/run/cloudera-scm-agent/process/1-host-inspector/logs

    [25/Jul/2012 18:43:35 +0000] 9438 MainThread agent INFO Chowning
    /var/run/cloudera-scm-agent/process/1-host-inspector/logs to root (0) root
    (0)

    [25/Jul/2012 18:43:35 +0000] 9438 MainThread agent INFO
    Chmod'ing /var/run/cloudera-scm-agent/process/1-host-inspector/logs to 0751

    [25/Jul/2012 18:43:35 +0000] 9438 MainThread agent INFO
    Triggering supervisord update.

    [25/Jul/2012 18:43:38 +0000] 9438 CP Server Thread-5 _cplogging INFO
    12.25.12.10 - - [25/Jul/2012:18:43:38] "GET
    /process/1-host-inspector/files/inspector?token=45c678dafe40130122108ff4ea1e0c76
    HTTP/1.1" 200 1017 "" "Java/1.6.0_31"

    [25/Jul/2012 18:43:52 +0000] 9438 MainThread agent INFO Process
    with same id has changed: 1-host-inspector.

    [25/Jul/2012 18:43:52 +0000] 9438 MainThread agent INFO
    Deactivating process 1-host-inspector



    Again, there's no directory created on the data node (even when i create
    the directory, there's no log files there).
    On Thursday, July 26, 2012 1:29:35 PM UTC-4, ansonism wrote:

    Running CM 4.0.3 (it's a new install) on CentOS 6. It installs all the
    packages on the data node (2nd server) but then looking @ the details it
    says can't start the scm agent:
    "
    BEGIN /sbin/service cloudera-scm-agent status | grep running
    END (1)
    BEGIN /sbin/service cloudera-scm-agent start
    Starting cloudera-scm-agent: [FAILED]
    END (1)
    scm agent could not be started, giving up
    "
    This is the error i see in the cloudera-scm-agent.out file:

    "/usr/lib64/cmf/agent/src/cmf/agent.py:24: DeprecationWarning: the sha
    module is deprecated; use the hashlib module instead
    import sha
    /usr/lib64/cmf/agent/src/cmf/monitor/host/network_interfaces.py:11:
    DeprecationWarning: the sets module is deprecated
    from sets import Set
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Using
    directory: /var/run/cloudera-scm-agent
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Using
    supervisor binary path:
    /usr/lib64/cmf/agent/src/cmf/../../build/env/bin/supervisord
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Logging
    to /var/log/cloudera-scm-agent/cloudera-scm-agent.log
    Error: could not find config file
    /var/run/cloudera-scm-agent/supervisor/supervisord.conf
    For help, use /usr/lib64/cmf/agent/src/cmf/agent.py -h
    /usr/lib64/cmf/agent/src/cmf/agent.py:382: DeprecationWarning:
    psutil.used_phymem is deprecated; use psutil.phymem_usage instead
    really_used = psutil.used_phymem() - cached - buffers
    /usr/lib64/cmf/agent/src/cmf/agent.py:384: DeprecationWarning:
    psutil.total_virtmem is deprecated; use psutil.virtmem_usage instead
    stats["virtual_memory_total"] = psutil.total_virtmem()
    /usr/lib64/cmf/agent/src/cmf/agent.py:385: DeprecationWarning:
    psutil.used_virtmem is deprecated; use psutil.virtmem_usage instead
    stats["virtual_memory_used"] = psutil.used_virtmem()
    /usr/lib64/cmf/agent/src/cmf/monitor/host/__init__.py:89:
    DeprecationWarning: psutil.used_phymem is deprecated; use
    psutil.phymem_usage instead
    really_used = psutil.used_phymem() - cached - buffers
    "
    I tried looking for that directory on the agent would be installed on, but
    it's not there. But i see the other folders (hdfs, hive, oozie, etc...)
    there. I'm logging in through CM as root on that box. Any ideas?
  • Ansonism at Jul 26, 2012 at 9:12 pm
    Sorry i would looking in the wrong server the .out file: This is what it
    is:


    /usr/lib64/cmf/agent/src/cmf/agent.py:24: DeprecationWarning: the sha
    module is deprecated; use the hashlib module instead
    import sha
    /usr/lib64/cmf/agent/src/cmf/monitor/host/network_interfaces.py:11:
    DeprecationWarning: the sets module is deprecated
    from sets import Set
    [26/Jul/2012 16:59:55 +0000] 10867 MainThread agent ERROR Could
    not determine hostname or ip address; proceeding.
    Traceback (most recent call last):
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1110, in
    parse_arguments
    ip_address = socket.gethostbyname(fqdn)
    gaierror: [Errno -2] Name or service not known
    usage: agent.py [-h] [--agent_dir AGENT_DIR]
    [--agent_httpd_port AGENT_HTTPD_PORT] --package_dir
    PACKAGE_DIR [--supervisord_path SUPERVISORD_PATH]
    [--supervisord_httpd_port SUPERVISORD_HTTPD_PORT]
    [--standalone STANDALONE] [--master MASTER]
    [--environment ENVIRONMENT] [--host_id HOST_ID]
    [--disable_supervisord_events] [--disable_eager_heartbeats]
    --hostname HOSTNAME --ip_address IP_ADDRESS [--use_tls]
    [--client_key_file CLIENT_KEY_FILE]
    [--client_cert_file CLIENT_CERT_FILE]
    [--verify_cert_file VERIFY_CERT_FILE]
    [--client_keypw_file CLIENT_KEYPW_FILE] [--logfile LOGFILE]
    [--optional_token]
    agent.py: error: argument --hostname is required
    [26/Jul/2012 16:59:55 +0000] 10867 Dummy-1 agent INFO Stopping
    agent...

    Not sure what's going on there. I checked the hostname file and that's
    good.
    On Thursday, July 26, 2012 1:29:35 PM UTC-4, ansonism wrote:

    Running CM 4.0.3 (it's a new install) on CentOS 6. It installs all the
    packages on the data node (2nd server) but then looking @ the details it
    says can't start the scm agent:
    "
    BEGIN /sbin/service cloudera-scm-agent status | grep running
    END (1)
    BEGIN /sbin/service cloudera-scm-agent start
    Starting cloudera-scm-agent: [FAILED]
    END (1)
    scm agent could not be started, giving up
    "
    This is the error i see in the cloudera-scm-agent.out file:

    "/usr/lib64/cmf/agent/src/cmf/agent.py:24: DeprecationWarning: the sha
    module is deprecated; use the hashlib module instead
    import sha
    /usr/lib64/cmf/agent/src/cmf/monitor/host/network_interfaces.py:11:
    DeprecationWarning: the sets module is deprecated
    from sets import Set
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Using
    directory: /var/run/cloudera-scm-agent
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Using
    supervisor binary path:
    /usr/lib64/cmf/agent/src/cmf/../../build/env/bin/supervisord
    [25/Jul/2012 17:23:18 +0000] 9438 MainThread agent INFO Logging
    to /var/log/cloudera-scm-agent/cloudera-scm-agent.log
    Error: could not find config file
    /var/run/cloudera-scm-agent/supervisor/supervisord.conf
    For help, use /usr/lib64/cmf/agent/src/cmf/agent.py -h
    /usr/lib64/cmf/agent/src/cmf/agent.py:382: DeprecationWarning:
    psutil.used_phymem is deprecated; use psutil.phymem_usage instead
    really_used = psutil.used_phymem() - cached - buffers
    /usr/lib64/cmf/agent/src/cmf/agent.py:384: DeprecationWarning:
    psutil.total_virtmem is deprecated; use psutil.virtmem_usage instead
    stats["virtual_memory_total"] = psutil.total_virtmem()
    /usr/lib64/cmf/agent/src/cmf/agent.py:385: DeprecationWarning:
    psutil.used_virtmem is deprecated; use psutil.virtmem_usage instead
    stats["virtual_memory_used"] = psutil.used_virtmem()
    /usr/lib64/cmf/agent/src/cmf/monitor/host/__init__.py:89:
    DeprecationWarning: psutil.used_phymem is deprecated; use
    psutil.phymem_usage instead
    really_used = psutil.used_phymem() - cached - buffers
    "
    I tried looking for that directory on the agent would be installed on, but
    it's not there. But i see the other folders (hdfs, hive, oozie, etc...)
    there. I'm logging in through CM as root on that box. Any ideas?

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedJul 26, '12 at 5:29p
activeJul 26, '12 at 9:46p
posts13
users2
websitecloudera.com
irc#hadoop

2 users in discussion

Ansonism: 9 posts Philip Langdale: 4 posts

People

Translate

site design / logo © 2022 Grokbase