FAQ
I've setup a clean install of Cloudera Manager 4.5 on Ubuntu 12.04 LTS.
When trying to start a tasktracker I get the following error:

execve() arg 3 contains a non-string value

And in the UI:

*Start this TaskTracker<http://ec2-54-234-68-242.compute-1.amazonaws.com:7180/cmf/command/80/details>
*NoneMar 8, 2013 9:20:34 PM UTC** Finished , Mar 8, 2013 9:20:59 PM UTC

Supervisor returned FATAL: Error found before invoking supervisord: execve() arg 3 contains a non-string value



From the cloudera-scm-agent.log:


Traceback (most recent call last):
File "/usr/lib/cmf/agent/src/cmf/agent.py", line 902, in
handle_heartbeat_response
new_process.activate()
File "/usr/lib/cmf/agent/src/cmf/agent.py", line 1685, in activate
self.write_process_conf()
File "/usr/lib/cmf/agent/src/cmf/agent.py", line 1776, in
write_process_conf
"source_parcel_environment", env))
File "/usr/lib/cmf/agent/src/cmf/util.py", line 334, in source
stdout=subprocess.PIPE, env=caller_env)
File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1249, in _execute_child
raise child_exception
TypeError: execve() arg 3 contains a non-string value


I did some debugging and found that in the call to subprocess.Popen the env
dict argument contains a mix of unicode and non-unicode strings. For
instance check out the key HADOOP_TASKTRACKER_OPTS below:

{'CDH_OOZIE_HOME': '/usr/lib/oozie', 'CMF_CONF_DIR':
'/etc/cloudera-scm-agent', 'CMF_PACKAGE_DIR': '/usr/lib/cmf/service',
'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME': '/usr/share/cmf',
'CDH_IMPALA_HOME': '/usr/lib/impala', 'CMF_AGENT_CDH_HTTPFS_HOME':
'/usr/lib/hadoop-httpfs', 'CDH_HIVE_HOME': '/usr/lib/hive', *u'HADOOP_TASKTRACKER_OPTS':
u'-Xmx729845102 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled
-Dhadoop.event.appender=,EventCatcher',* 'CMF_AGENT_MGMT_HOME':
'/usr/share/cmf', 'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs',
'CMF_AGENT_HIVE_DEFAULT_XML': '/usr/lib/hive/conf/hive-default.xml',
'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin',
'CMF_AGENT_CDH_FLUME_HOME': '/usr/lib/flume-ng', 'CDH_HUE_PLUGINS_HOME':
'/usr/lib/hadoop', 'CMF_AGENT_CDH_HBASE_HOME': '/usr/lib/hbase',
'CM_STATUS_CODES': u'STATUS_NONE HDFS_DFS_DIR_NOT_EMPTY', 'LANG':
'en_US.UTF-8', 'TERM': 'xterm-256color', 'CLOUDERA_ORACLE_CONNECTOR_JAR':
None, u'CDH_VERSION': u'4', 'CMF_AGENT_CDH_HADOOP_HOME': '/usr/lib/hadoop',
'CMF_AGENT_CDH_HUE_HOME': '/usr/share/hue', 'SHLVL': '1', 'JSVC_HOME':
'/usr/libexec/bigtop-utils', 'CMF_AGENT_TOMCAT_HOME':
'/usr/lib/bigtop-tomcat', 'JAVA_HOME':
'/usr/lib/jvm/default-java','CLOUDERA_POSTGRESQL_JDBC_JAR': None,
'HIVE_DEFAULT_XML': '/usr/lib/hive/conf/hive-default.xml',
u'HADOOP_SECURITY_LOGGER': u'INFO,RFAS', u'HADOOP_LOG_DIR':
u'/var/log/hadoop-0.20-mapreduce', u'HADOOP_ROOT_LOGGER': u'INFO,RFA',
'CMF_AGENT_CDH_HIVE_HOME': '/usr/lib/hive', 'CMF_AGENT_JAVA_HOME':
'/usr/lib/jvm/default-java', 'CDH_YARN_HOME': '/usr/lib/hadoop-yarn',
'CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper', u'HADOOP_AUDIT_LOGGER':
u'INFO,RFAAUDIT', 'CDH_HUE_HOME': '/usr/share/hue',
'CLOUDERA_MYSQL_CONNECTOR_JAR': '/usr/share/java/mysql-co
nnector-java.jar', 'CONF_DIR':
u'/run/cloudera-scm-agent/process/50-mapreduce-TASKTRACKER',
'CMF_AGENT_JSVC_HOME': '/usr/libexe c/bigtop-utils', 'SCM_DEFINES_SCRIPTS':
'', 'CMF_AGENT_CDH_OOZIE_HOME': '/usr/lib/oozie', 'CDH_MR1_HOME':
'/usr/lib/hadoop-0.20-mapreduce', 'TOMCAT_HOME': '/usr/lib/bigtop-tomcat',
'CMF_AGENT_CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper', 'CMF_AGENT_CDH_HUE_
PLUGINS_HOME': '/usr/lib/hadoop', 'CMF_AGENT_CDH_YARN_HOME':
'/usr/lib/hadoop-yarn', 'CDH_HBASE_HOME': '/usr/lib/hbase', 'CMF_A
GENT_CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs', 'CDH_HTTPFS_HOME':
'/usr/lib/hadoop-httpfs', u'HADOOP_LOGFILE': u'hadoop-cmf-mapre
duce1-TASKTRACKER-ip-10-60-147-9.ec2.internal.log.out', 'CDH_HADOOP_HOME':
'/usr/lib/hadoop', 'CMF_AGENT_CDH_MR1_HOME': '/usr/l
ib/hadoop-0.20-mapreduce', 'CMF_AGENT_CDH_MR2_HOME':
'/usr/lib/hadoop-mapreduce', 'PWD': '/', 'CDH_MR2_HOME': '/usr/lib/hadoop-
mapreduce', 'CMF_AGENT_CDH_HADOOP_BIN': '/usr/bin/hadoop', 'PARCELS_ROOT':
'/opt/cloudera/parcels', 'CMF_AGENT_CDH_IMPALA_HOME' : '/usr/lib/impala',
'CDH_FLUME_HOME': '/usr/lib/flume-ng'}

I was able to work around this by modifying
/usr/lib/cmf/agent/src/cmf/util.py to convert the dictionary key/values to
strings before calling subprocess.Popen. But I'm wondering if there's a
better way to handle this. I'm not explicitly setting any of the values in
the dictionary above from any of my configs. Is there some place I should
check in my config to see where the encoding may be mixed?

Again this is on a clean install using Ubuntu 12.04, Cloudera Manager 4.5
and CDH 4.2.

Thanks!
--Russell

Search Discussions

  • Adar Dembo at Mar 9, 2013 at 1:40 am
    This is the problem:

    'CLOUDERA_POSTGRESQL_JDBC_JAR': None

    We've fixed this internally, though I don't know what future CM release
    it'll ship with. For now, you need to satisfy the following glob in
    /etc/default/cloudera-scm-agent:

    #
    # Locate the Postgresql driver jar file.
    #
    export CLOUDERA_POSTGRESQL_JDBC_JAR=$(find
    /usr/share/cmf/lib/postgresql-*jdbc*.jar 2> /dev/null | tail -n 1)

    Did you install the cloudera-manager-daemons package on every node? I think
    that package contains the postgres JDBC JAR.


    On Fri, Mar 8, 2013 at 4:12 PM, Russell Cardullo
    wrote:
    I've setup a clean install of Cloudera Manager 4.5 on Ubuntu 12.04 LTS.
    When trying to start a tasktracker I get the following error:

    execve() arg 3 contains a non-string value

    And in the UI:

    *Start this TaskTracker<http://ec2-54-234-68-242.compute-1.amazonaws.com:7180/cmf/command/80/details>
    *NoneMar 8, 2013 9:20:34 PM UTC** Finished , Mar 8, 2013 9:20:59 PM UTC

    Supervisor returned FATAL: Error found before invoking supervisord: execve() arg 3 contains a non-string value



    From the cloudera-scm-agent.log:


    Traceback (most recent call last):
    File "/usr/lib/cmf/agent/src/cmf/**agent.py", line 902, in
    handle_heartbeat_response
    new_process.activate()
    File "/usr/lib/cmf/agent/src/cmf/**agent.py", line 1685, in activate
    self.write_process_conf()
    File "/usr/lib/cmf/agent/src/cmf/**agent.py", line 1776, in
    write_process_conf
    "source_parcel_environment", env))
    File "/usr/lib/cmf/agent/src/cmf/**util.py", line 334, in source
    stdout=subprocess.PIPE, env=caller_env)
    File "/usr/lib/python2.7/**subprocess.py", line 679, in __init__
    errread, errwrite)
    File "/usr/lib/python2.7/**subprocess.py", line 1249, in _execute_child
    raise child_exception
    TypeError: execve() arg 3 contains a non-string value


    I did some debugging and found that in the call to subprocess.Popen the
    env dict argument contains a mix of unicode and non-unicode strings. For
    instance check out the key HADOOP_TASKTRACKER_OPTS below:

    {'CDH_OOZIE_HOME': '/usr/lib/oozie', 'CMF_CONF_DIR':
    '/etc/cloudera-scm-agent', 'CMF_PACKAGE_DIR': '/usr/lib/cmf/service',
    'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME': '/usr/share/cmf',
    'CDH_IMPALA_HOME': '/usr/lib/impala', 'CMF_AGENT_CDH_HTTPFS_HOME':
    '/usr/lib/hadoop-httpfs', 'CDH_HIVE_HOME': '/usr/lib/hive', *u'HADOOP_TASKTRACKER_OPTS':
    u'-Xmx729845102 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
    -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled
    -Dhadoop.event.appender=,EventCatcher',* 'CMF_AGENT_MGMT_HOME':
    '/usr/share/cmf', 'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs',
    'CMF_AGENT_HIVE_DEFAULT_XML': '/usr/lib/hive/conf/hive-default.xml',
    'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin',
    'CMF_AGENT_CDH_FLUME_HOME': '/usr/lib/flume-ng', 'CDH_HUE_PLUGINS_HOME':
    '/usr/lib/hadoop', 'CMF_AGENT_CDH_HBASE_HOME': '/usr/lib/hbase',
    'CM_STATUS_CODES': u'STATUS_NONE HDFS_DFS_DIR_NOT_EMPTY', 'LANG':
    'en_US.UTF-8', 'TERM': 'xterm-256color', 'CLOUDERA_ORACLE_CONNECTOR_JAR':
    None, u'CDH_VERSION': u'4', 'CMF_AGENT_CDH_HADOOP_HOME': '/usr/lib/hadoop',
    'CMF_AGENT_CDH_HUE_HOME': '/usr/share/hue', 'SHLVL': '1', 'JSVC_HOME':
    '/usr/libexec/bigtop-utils', 'CMF_AGENT_TOMCAT_HOME':
    '/usr/lib/bigtop-tomcat', 'JAVA_HOME':
    '/usr/lib/jvm/default-java','CLOUDERA_POSTGRESQL_JDBC_JAR': None,
    'HIVE_DEFAULT_XML': '/usr/lib/hive/conf/hive-default.xml',
    u'HADOOP_SECURITY_LOGGER': u'INFO,RFAS', u'HADOOP_LOG_DIR':
    u'/var/log/hadoop-0.20-mapreduce', u'HADOOP_ROOT_LOGGER': u'INFO,RFA',
    'CMF_AGENT_CDH_HIVE_HOME': '/usr/lib/hive', 'CMF_AGENT_JAVA_HOME':
    '/usr/lib/jvm/default-java', 'CDH_YARN_HOME': '/usr/lib/hadoop-yarn',
    'CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper', u'HADOOP_AUDIT_LOGGER':
    u'INFO,RFAAUDIT', 'CDH_HUE_HOME': '/usr/share/hue',
    'CLOUDERA_MYSQL_CONNECTOR_JAR': '/usr/share/java/mysql-co
    nnector-java.jar', 'CONF_DIR':
    u'/run/cloudera-scm-agent/process/50-mapreduce-TASKTRACKER',
    'CMF_AGENT_JSVC_HOME': '/usr/libexe c/bigtop-utils', 'SCM_DEFINES_SCRIPTS':
    '', 'CMF_AGENT_CDH_OOZIE_HOME': '/usr/lib/oozie', 'CDH_MR1_HOME':
    '/usr/lib/hadoop-0.20-mapreduce', 'TOMCAT_HOME': '/usr/lib/bigtop-tomcat',
    'CMF_AGENT_CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper', 'CMF_AGENT_CDH_HUE_
    PLUGINS_HOME': '/usr/lib/hadoop', 'CMF_AGENT_CDH_YARN_HOME':
    '/usr/lib/hadoop-yarn', 'CDH_HBASE_HOME': '/usr/lib/hbase', 'CMF_A
    GENT_CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs', 'CDH_HTTPFS_HOME':
    '/usr/lib/hadoop-httpfs', u'HADOOP_LOGFILE': u'hadoop-cmf-mapre
    duce1-TASKTRACKER-ip-10-60-147-9.ec2.internal.log.out', 'CDH_HADOOP_HOME':
    '/usr/lib/hadoop', 'CMF_AGENT_CDH_MR1_HOME': '/usr/l
    ib/hadoop-0.20-mapreduce', 'CMF_AGENT_CDH_MR2_HOME':
    '/usr/lib/hadoop-mapreduce', 'PWD': '/', 'CDH_MR2_HOME': '/usr/lib/hadoop-
    mapreduce', 'CMF_AGENT_CDH_HADOOP_BIN': '/usr/bin/hadoop', 'PARCELS_ROOT':
    '/opt/cloudera/parcels', 'CMF_AGENT_CDH_IMPALA_HOME' : '/usr/lib/impala',
    'CDH_FLUME_HOME': '/usr/lib/flume-ng'}

    I was able to work around this by modifying /usr/lib/cmf/agent/**
    src/cmf/util.py to convert the dictionary key/values to strings before
    calling subprocess.Popen. But I'm wondering if there's a better way to
    handle this. I'm not explicitly setting any of the values in the
    dictionary above from any of my configs. Is there some place I should
    check in my config to see where the encoding may be mixed?

    Again this is on a clean install using Ubuntu 12.04, Cloudera Manager 4.5
    and CDH 4.2.

    Thanks!
    --Russell
  • Russell Cardullo at Mar 10, 2013 at 3:15 pm
    Thanks, your explanation makes sense as later I had to install the Postgres
    driver package to get Hive working. I'll test my provisioning again with
    the driver install first to see if that resolves the issue.
    On Friday, March 8, 2013 5:40:49 PM UTC-8, Adar Dembo wrote:

    This is the problem:

    'CLOUDERA_POSTGRESQL_JDBC_JAR': None

    We've fixed this internally, though I don't know what future CM release
    it'll ship with. For now, you need to satisfy the following glob in
    /etc/default/cloudera-scm-agent:

    #
    # Locate the Postgresql driver jar file.
    #
    export CLOUDERA_POSTGRESQL_JDBC_JAR=$(find
    /usr/share/cmf/lib/postgresql-*jdbc*.jar 2> /dev/null | tail -n 1)

    Did you install the cloudera-manager-daemons package on every node? I
    think that package contains the postgres JDBC JAR.


    On Fri, Mar 8, 2013 at 4:12 PM, Russell Cardullo <russell...@gmail.com<javascript:>
    wrote:
    I've setup a clean install of Cloudera Manager 4.5 on Ubuntu 12.04 LTS.
    When trying to start a tasktracker I get the following error:

    execve() arg 3 contains a non-string value

    And in the UI:

    *Start this TaskTracker<http://ec2-54-234-68-242.compute-1.amazonaws.com:7180/cmf/command/80/details>
    * NoneMar 8, 2013 9:20:34 PM UTC ** Finished , Mar 8, 2013 9:20:59 PM UTC

    Supervisor returned FATAL: Error found before invoking supervisord: execve() arg 3 contains a non-string value



    From the cloudera-scm-agent.log:


    Traceback (most recent call last):
    File "/usr/lib/cmf/agent/src/cmf/**agent.py", line 902, in
    handle_heartbeat_response
    new_process.activate()
    File "/usr/lib/cmf/agent/src/cmf/**agent.py", line 1685, in activate
    self.write_process_conf()
    File "/usr/lib/cmf/agent/src/cmf/**agent.py", line 1776, in
    write_process_conf
    "source_parcel_environment", env))
    File "/usr/lib/cmf/agent/src/cmf/**util.py", line 334, in source
    stdout=subprocess.PIPE, env=caller_env)
    File "/usr/lib/python2.7/**subprocess.py", line 679, in __init__
    errread, errwrite)
    File "/usr/lib/python2.7/**subprocess.py", line 1249, in _execute_child
    raise child_exception
    TypeError: execve() arg 3 contains a non-string value


    I did some debugging and found that in the call to subprocess.Popen the
    env dict argument contains a mix of unicode and non-unicode strings. For
    instance check out the key HADOOP_TASKTRACKER_OPTS below:

    {'CDH_OOZIE_HOME': '/usr/lib/oozie', 'CMF_CONF_DIR':
    '/etc/cloudera-scm-agent', 'CMF_PACKAGE_DIR': '/usr/lib/cmf/service',
    'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME': '/usr/share/cmf',
    'CDH_IMPALA_HOME': '/usr/lib/impala', 'CMF_AGENT_CDH_HTTPFS_HOME':
    '/usr/lib/hadoop-httpfs', 'CDH_HIVE_HOME': '/usr/lib/hive', *u'HADOOP_TASKTRACKER_OPTS':
    u'-Xmx729845102 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
    -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled
    -Dhadoop.event.appender=,EventCatcher',* 'CMF_AGENT_MGMT_HOME':
    '/usr/share/cmf', 'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs',
    'CMF_AGENT_HIVE_DEFAULT_XML': '/usr/lib/hive/conf/hive-default.xml',
    'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin',
    'CMF_AGENT_CDH_FLUME_HOME': '/usr/lib/flume-ng', 'CDH_HUE_PLUGINS_HOME':
    '/usr/lib/hadoop', 'CMF_AGENT_CDH_HBASE_HOME': '/usr/lib/hbase',
    'CM_STATUS_CODES': u'STATUS_NONE HDFS_DFS_DIR_NOT_EMPTY', 'LANG':
    'en_US.UTF-8', 'TERM': 'xterm-256color', 'CLOUDERA_ORACLE_CONNECTOR_JAR':
    None, u'CDH_VERSION': u'4', 'CMF_AGENT_CDH_HADOOP_HOME': '/usr/lib/hadoop',
    'CMF_AGENT_CDH_HUE_HOME': '/usr/share/hue', 'SHLVL': '1', 'JSVC_HOME':
    '/usr/libexec/bigtop-utils', 'CMF_AGENT_TOMCAT_HOME':
    '/usr/lib/bigtop-tomcat', 'JAVA_HOME':
    '/usr/lib/jvm/default-java','CLOUDERA_POSTGRESQL_JDBC_JAR': None,
    'HIVE_DEFAULT_XML': '/usr/lib/hive/conf/hive-default.xml',
    u'HADOOP_SECURITY_LOGGER': u'INFO,RFAS', u'HADOOP_LOG_DIR':
    u'/var/log/hadoop-0.20-mapreduce', u'HADOOP_ROOT_LOGGER': u'INFO,RFA',
    'CMF_AGENT_CDH_HIVE_HOME': '/usr/lib/hive', 'CMF_AGENT_JAVA_HOME':
    '/usr/lib/jvm/default-java', 'CDH_YARN_HOME': '/usr/lib/hadoop-yarn',
    'CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper', u'HADOOP_AUDIT_LOGGER':
    u'INFO,RFAAUDIT', 'CDH_HUE_HOME': '/usr/share/hue',
    'CLOUDERA_MYSQL_CONNECTOR_JAR': '/usr/share/java/mysql-co
    nnector-java.jar', 'CONF_DIR':
    u'/run/cloudera-scm-agent/process/50-mapreduce-TASKTRACKER',
    'CMF_AGENT_JSVC_HOME': '/usr/libexe c/bigtop-utils', 'SCM_DEFINES_SCRIPTS':
    '', 'CMF_AGENT_CDH_OOZIE_HOME': '/usr/lib/oozie', 'CDH_MR1_HOME':
    '/usr/lib/hadoop-0.20-mapreduce', 'TOMCAT_HOME': '/usr/lib/bigtop-tomcat',
    'CMF_AGENT_CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper', 'CMF_AGENT_CDH_HUE_
    PLUGINS_HOME': '/usr/lib/hadoop', 'CMF_AGENT_CDH_YARN_HOME':
    '/usr/lib/hadoop-yarn', 'CDH_HBASE_HOME': '/usr/lib/hbase', 'CMF_A
    GENT_CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs', 'CDH_HTTPFS_HOME':
    '/usr/lib/hadoop-httpfs', u'HADOOP_LOGFILE': u'hadoop-cmf-mapre
    duce1-TASKTRACKER-ip-10-60-147-9.ec2.internal.log.out', 'CDH_HADOOP_HOME':
    '/usr/lib/hadoop', 'CMF_AGENT_CDH_MR1_HOME': '/usr/l
    ib/hadoop-0.20-mapreduce', 'CMF_AGENT_CDH_MR2_HOME':
    '/usr/lib/hadoop-mapreduce', 'PWD': '/', 'CDH_MR2_HOME': '/usr/lib/hadoop-
    mapreduce', 'CMF_AGENT_CDH_HADOOP_BIN': '/usr/bin/hadoop', 'PARCELS_ROOT':
    '/opt/cloudera/parcels', 'CMF_AGENT_CDH_IMPALA_HOME' : '/usr/lib/impala',
    'CDH_FLUME_HOME': '/usr/lib/flume-ng'}

    I was able to work around this by modifying /usr/lib/cmf/agent/**
    src/cmf/util.py to convert the dictionary key/values to strings before
    calling subprocess.Popen. But I'm wondering if there's a better way to
    handle this. I'm not explicitly setting any of the values in the
    dictionary above from any of my configs. Is there some place I should
    check in my config to see where the encoding may be mixed?

    Again this is on a clean install using Ubuntu 12.04, Cloudera Manager 4.5
    and CDH 4.2.

    Thanks!
    --Russell

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedMar 9, '13 at 12:12a
activeMar 10, '13 at 3:15p
posts3
users2
websitecloudera.com
irc#hadoop

2 users in discussion

Russell Cardullo: 2 posts Adar Dembo: 1 post

People

Translate

site design / logo © 2022 Grokbase