FAQ
Hi there,

I am running Cloudera Manager Free Edition and would like to create a
prototype Hadoop environment where everything is running on a single
computer.

On the first run when all the CDH services are configured and started I am
running into an error message stating that the TaskTracker cannot be
started.


Supervisor returned FATAL: + set -x
+ perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER#g' /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/core-site.xml /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/hdfs-site.xml /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/mapred-site.xml
+ '[' -e /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/topology.py ']'
+ export 'HADOOP_OPTS=-Djava.net.preferIPv4Stack=true '
+ HADOOP_OPTS='-Djava.net.preferIPv4Stack=true '
+ acquire_kerberos_tgt mapred.keytab
+ '[' -z mapred.keytab ']'
+ '[' -n '' ']'
+ '[' tasktracker = jobtracker ']'
+ '[' tasktracker = tasktracker ']'
+ chmod 0644 /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/mapred-site.xml /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/hdfs-site.xml /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/core-site.xml /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/log4j.properties
++ tr -d '\n'
++ find /usr/share/cmf/lib/plugins -name 'tt-instrumentation-*.jar'
+ ADD_TO_CP=/usr/share/cmf/lib/plugins/tt-instrumentation-4.0.4.jar
+ '[' -z '/usr/share/cmf/lib/plugins/event-publish-4.0.4-shaded.jar:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/*:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/*::' ']'
+ export 'HADOOP_CLASSPATH=/usr/share/cmf/lib/plugins/tt-instrumentation-4.0.4.jar:/usr/share/cmf/lib/plugins/event-publish-4.0.4-shaded.jar:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/*:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/*::'
+ HADOOP_CLASSPATH='/usr/share/cmf/lib/plugins/tt-instrumentation-4.0.4.jar:/usr/share/cmf/lib/plugins/event-publish-4.0.4-shaded.jar:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/*:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/*::'
+ exec /usr/lib/hadoop-0.20-mapreduce/bin/hadoop --config /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER tasktracker
Exploring the log files lead to this part:


2012-10-19 09:31:40,034 INFO org.apache.hadoop.util.HostsFileReader:
Setting the includes file to
/var/run/cloudera-scm-agent/process/24-mapreduce-JOBTRACKER/mapred_hosts_allow.txt
2012-10-19 09:31:40,034 INFO org.apache.hadoop.util.HostsFileReader:
Setting the excludes file to
/var/run/cloudera-scm-agent/process/24-mapreduce-JOBTRACKER/mapred_hosts_exclude.txt
2012-10-19 09:31:40,034 INFO org.apache.hadoop.util.HostsFileReader:
Refreshing hosts (include/exclude) list
2012-10-19 09:31:40,035 INFO org.apache.hadoop.util.HostsFileReader:
Adding localhost.localdomain to the list of hosts from
/var/run/cloudera-scm-agent/process/24-mapreduce-JOBTRACKER/mapred_hosts_allow.txt
2012-10-19 09:31:40,036 INFO org.apache.hadoop.util.HostsFileReader:
Adding 127.0.0.1 to the list of hosts from
/var/run/cloudera-scm-agent/process/24-mapreduce-JOBTRACKER/mapred_hosts_allow.txt
2012-10-19 09:31:40,036 INFO org.apache.hadoop.mapred.JobTracker:
Decommissioning 0 nodes
2012-10-19 09:31:40,048 INFO org.apache.hadoop.ipc.Server: IPC Server
Responder: starting
2012-10-19 09:31:40,049 INFO org.apache.hadoop.ipc.Server: IPC Server
listener on 8021: starting
2012-10-19 09:31:40,066 INFO org.apache.hadoop.mapred.JobTracker: Starting
RUNNING
2012-10-19 09:31:40,403 ERROR
org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
as:mapred (auth:SIMPLE)
cause:org.apache.hadoop.mapred.DisallowedTaskTrackerException: Tasktracker
denied communication with jobtracker:
tracker_localhost:localhost/127.0.0.1:32833
2012-10-19 09:31:40,404 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 3 on 8021, call
heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@6b248979, true, true,
true, -1), rpc version=2, client version=32, methodsFingerPrint=-159967141
from 127.0.0.1:54999: error:
org.apache.hadoop.mapred.DisallowedTaskTrackerException: Tasktracker denied
communication with jobtracker: tracker_localhost:localhost/127.0.0.1:32833
org.apache.hadoop.mapred.DisallowedTaskTrackerException: Tasktracker
denied communication with jobtracker:
tracker_localhost:localhost/127.0.0.1:32833
at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:2857)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.ipc.WritableRpcEngine$Server$WritableRpcInvoker.call(WritableRpcEngine.java:474)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:898)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1693)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1689)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1687)
Any ideas why the services won't start or why the communication between the
job and the task tracker would be denied? I am running CentOS 6.3 and
selinux is disabled.

Thanks,
- Thomas

Search Discussions

  • Chris at Oct 19, 2012 at 10:27 pm
    Go to your main MapReduce page and go over to the "Actions" tab on the top
    right, click it, go down to "Refresh Job Tracker" and click that. After it
    finishes, try again.

    I had the same problem on multiple occasions, and this fixed it.

    I hope this helps.

    -Chris


    On Fri, Oct 19, 2012 at 1:48 PM, Thomas Emge wrote:

    Hi there,

    I am running Cloudera Manager Free Edition and would like to create a
    prototype Hadoop environment where everything is running on a single
    computer.

    On the first run when all the CDH services are configured and started I am
    running into an error message stating that the TaskTracker cannot be
    started.


    Supervisor returned FATAL: + set -x
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER#g' /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/core-site.xml /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/hdfs-site.xml /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/mapred-site.xml
    + '[' -e /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/topology.py ']'
    + export 'HADOOP_OPTS=-Djava.net.preferIPv4Stack=true '
    + HADOOP_OPTS='-Djava.net.preferIPv4Stack=true '
    + acquire_kerberos_tgt mapred.keytab
    + '[' -z mapred.keytab ']'
    + '[' -n '' ']'
    + '[' tasktracker = jobtracker ']'
    + '[' tasktracker = tasktracker ']'
    + chmod 0644 /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/mapred-site.xml /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/hdfs-site.xml /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/core-site.xml /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/log4j.properties
    ++ tr -d '\n'
    ++ find /usr/share/cmf/lib/plugins -name 'tt-instrumentation-*.jar'
    + ADD_TO_CP=/usr/share/cmf/lib/plugins/tt-instrumentation-4.0.4.jar
    + '[' -z '/usr/share/cmf/lib/plugins/event-publish-4.0.4-shaded.jar:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/*:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/*::' ']'
    + export 'HADOOP_CLASSPATH=/usr/share/cmf/lib/plugins/tt-instrumentation-4.0.4.jar:/usr/share/cmf/lib/plugins/event-publish-4.0.4-shaded.jar:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/*:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/*::'
    + HADOOP_CLASSPATH='/usr/share/cmf/lib/plugins/tt-instrumentation-4.0.4.jar:/usr/share/cmf/lib/plugins/event-publish-4.0.4-shaded.jar:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/*:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/*::'
    + exec /usr/lib/hadoop-0.20-mapreduce/bin/hadoop --config /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER tasktracker
    Exploring the log files lead to this part:


    2012-10-19 09:31:40,034 INFO org.apache.hadoop.util.HostsFileReader:
    Setting the includes file to
    /var/run/cloudera-scm-agent/process/24-mapreduce-JOBTRACKER/mapred_hosts_allow.txt
    2012-10-19 09:31:40,034 INFO org.apache.hadoop.util.HostsFileReader:
    Setting the excludes file to
    /var/run/cloudera-scm-agent/process/24-mapreduce-JOBTRACKER/mapred_hosts_exclude.txt
    2012-10-19 09:31:40,034 INFO org.apache.hadoop.util.HostsFileReader:
    Refreshing hosts (include/exclude) list
    2012-10-19 09:31:40,035 INFO org.apache.hadoop.util.HostsFileReader:
    Adding localhost.localdomain to the list of hosts from
    /var/run/cloudera-scm-agent/process/24-mapreduce-JOBTRACKER/mapred_hosts_allow.txt
    2012-10-19 09:31:40,036 INFO org.apache.hadoop.util.HostsFileReader:
    Adding 127.0.0.1 to the list of hosts from
    /var/run/cloudera-scm-agent/process/24-mapreduce-JOBTRACKER/mapred_hosts_allow.txt
    2012-10-19 09:31:40,036 INFO org.apache.hadoop.mapred.JobTracker:
    Decommissioning 0 nodes
    2012-10-19 09:31:40,048 INFO org.apache.hadoop.ipc.Server: IPC Server
    Responder: starting
    2012-10-19 09:31:40,049 INFO org.apache.hadoop.ipc.Server: IPC Server
    listener on 8021: starting
    2012-10-19 09:31:40,066 INFO org.apache.hadoop.mapred.JobTracker:
    Starting RUNNING
    2012-10-19 09:31:40,403 ERROR
    org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
    as:mapred (auth:SIMPLE)
    cause:org.apache.hadoop.mapred.DisallowedTaskTrackerException: Tasktracker
    denied communication with jobtracker: tracker_localhost:localhost/
    127.0.0.1:32833
    2012-10-19 09:31:40,404 INFO org.apache.hadoop.ipc.Server: IPC Server
    handler 3 on 8021, call
    heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@6b248979, true,
    true, true, -1), rpc version=2, client version=32,
    methodsFingerPrint=-159967141 from 127.0.0.1:54999: error:
    org.apache.hadoop.mapred.DisallowedTaskTrackerException: Tasktracker denied
    communication with jobtracker: tracker_localhost:localhost/
    127.0.0.1:32833
    org.apache.hadoop.mapred.DisallowedTaskTrackerException: Tasktracker
    denied communication with jobtracker: tracker_localhost:localhost/
    127.0.0.1:32833
    at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:2857)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
    org.apache.hadoop.ipc.WritableRpcEngine$Server$WritableRpcInvoker.call(WritableRpcEngine.java:474)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:898)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1693)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1689)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1687)
    Any ideas why the services won't start or why the communication between
    the job and the task tracker would be denied? I am running CentOS 6.3 and
    selinux is disabled.

    Thanks,
    - Thomas
  • Thomas Emge at Oct 20, 2012 at 6:13 am
    Thanks Chris,

    It did work indeed.

    On Friday, October 19, 2012 3:27:56 PM UTC-7, BludGeonT wrote:

    Go to your main MapReduce page and go over to the "Actions" tab on the top
    right, click it, go down to "Refresh Job Tracker" and click that. After it
    finishes, try again.

    I had the same problem on multiple occasions, and this fixed it.

    I hope this helps.

    -Chris



    On Fri, Oct 19, 2012 at 1:48 PM, Thomas Emge <te...@esri.com <javascript:>
    wrote:
    Hi there,

    I am running Cloudera Manager Free Edition and would like to create a
    prototype Hadoop environment where everything is running on a single
    computer.

    On the first run when all the CDH services are configured and started I
    am running into an error message stating that the TaskTracker cannot be
    started.


    Supervisor returned FATAL: + set -x
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER#g' /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/core-site.xml /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/hdfs-site.xml /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/mapred-site.xml
    + '[' -e /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/topology.py ']'
    + export 'HADOOP_OPTS=-Djava.net.preferIPv4Stack=true '
    + HADOOP_OPTS='-Djava.net.preferIPv4Stack=true '
    + acquire_kerberos_tgt mapred.keytab
    + '[' -z mapred.keytab ']'
    + '[' -n '' ']'
    + '[' tasktracker = jobtracker ']'
    + '[' tasktracker = tasktracker ']'
    + chmod 0644 /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/mapred-site.xml /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/hdfs-site.xml /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/core-site.xml /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/log4j.properties
    ++ tr -d '\n'
    ++ find /usr/share/cmf/lib/plugins -name 'tt-instrumentation-*.jar'
    + ADD_TO_CP=/usr/share/cmf/lib/plugins/tt-instrumentation-4.0.4.jar
    + '[' -z '/usr/share/cmf/lib/plugins/event-publish-4.0.4-shaded.jar:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/*:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/*::' ']'
    + export 'HADOOP_CLASSPATH=/usr/share/cmf/lib/plugins/tt-instrumentation-4.0.4.jar:/usr/share/cmf/lib/plugins/event-publish-4.0.4-shaded.jar:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/*:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/*::'
    + HADOOP_CLASSPATH='/usr/share/cmf/lib/plugins/tt-instrumentation-4.0.4.jar:/usr/share/cmf/lib/plugins/event-publish-4.0.4-shaded.jar:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/*:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/*::'
    + exec /usr/lib/hadoop-0.20-mapreduce/bin/hadoop --config /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER tasktracker
    Exploring the log files lead to this part:


    2012-10-19 09:31:40,034 INFO org.apache.hadoop.util.HostsFileReader:
    Setting the includes file to
    /var/run/cloudera-scm-agent/process/24-mapreduce-JOBTRACKER/mapred_hosts_allow.txt
    2012-10-19 09:31:40,034 INFO org.apache.hadoop.util.HostsFileReader:
    Setting the excludes file to
    /var/run/cloudera-scm-agent/process/24-mapreduce-JOBTRACKER/mapred_hosts_exclude.txt
    2012-10-19 09:31:40,034 INFO org.apache.hadoop.util.HostsFileReader:
    Refreshing hosts (include/exclude) list
    2012-10-19 09:31:40,035 INFO org.apache.hadoop.util.HostsFileReader:
    Adding localhost.localdomain to the list of hosts from
    /var/run/cloudera-scm-agent/process/24-mapreduce-JOBTRACKER/mapred_hosts_allow.txt
    2012-10-19 09:31:40,036 INFO org.apache.hadoop.util.HostsFileReader:
    Adding 127.0.0.1 to the list of hosts from
    /var/run/cloudera-scm-agent/process/24-mapreduce-JOBTRACKER/mapred_hosts_allow.txt
    2012-10-19 09:31:40,036 INFO org.apache.hadoop.mapred.JobTracker:
    Decommissioning 0 nodes
    2012-10-19 09:31:40,048 INFO org.apache.hadoop.ipc.Server: IPC Server
    Responder: starting
    2012-10-19 09:31:40,049 INFO org.apache.hadoop.ipc.Server: IPC Server
    listener on 8021: starting
    2012-10-19 09:31:40,066 INFO org.apache.hadoop.mapred.JobTracker:
    Starting RUNNING
    2012-10-19 09:31:40,403 ERROR
    org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
    as:mapred (auth:SIMPLE)
    cause:org.apache.hadoop.mapred.DisallowedTaskTrackerException: Tasktracker
    denied communication with jobtracker: tracker_localhost:localhost/
    127.0.0.1:32833
    2012-10-19 09:31:40,404 INFO org.apache.hadoop.ipc.Server: IPC Server
    handler 3 on 8021, call
    heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@6b248979, true, true,
    true, -1), rpc version=2, client version=32, methodsFingerPrint=-159967141
    from 127.0.0.1:54999: error:
    org.apache.hadoop.mapred.DisallowedTaskTrackerException: Tasktracker denied
    communication with jobtracker: tracker_localhost:localhost/
    127.0.0.1:32833
    org.apache.hadoop.mapred.DisallowedTaskTrackerException: Tasktracker
    denied communication with jobtracker: tracker_localhost:localhost/
    127.0.0.1:32833
    at
    org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:2857)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
    org.apache.hadoop.ipc.WritableRpcEngine$Server$WritableRpcInvoker.call(WritableRpcEngine.java:474)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:898)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1693)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1689)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1687)
    Any ideas why the services won't start or why the communication between
    the job and the task tracker would be denied? I am running CentOS 6.3 and
    selinux is disabled.

    Thanks,
    - Thomas
  • Thomas Emge at Oct 23, 2012 at 12:24 am
    After a machine reboot I am back square one and the

    TaskTracker denied communication with jobtacker error message. Then I
    refresh the JobTracker and then restart the mapreduce1 service and after a
    couple of minutes the TaskTracker is stopped again.

    Any suggestion is greatly appreciated.

    - Thomas
    On Friday, October 19, 2012 11:13:50 PM UTC-7, Thomas Emge wrote:

    Thanks Chris,

    It did work indeed.

    On Friday, October 19, 2012 3:27:56 PM UTC-7, BludGeonT wrote:

    Go to your main MapReduce page and go over to the "Actions" tab on the
    top right, click it, go down to "Refresh Job Tracker" and click that.
    After it finishes, try again.

    I had the same problem on multiple occasions, and this fixed it.

    I hope this helps.

    -Chris


    On Fri, Oct 19, 2012 at 1:48 PM, Thomas Emge wrote:

    Hi there,

    I am running Cloudera Manager Free Edition and would like to create a
    prototype Hadoop environment where everything is running on a single
    computer.

    On the first run when all the CDH services are configured and started I
    am running into an error message stating that the TaskTracker cannot be
    started.


    Supervisor returned FATAL: + set -x
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER#g' /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/core-site.xml /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/hdfs-site.xml /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/mapred-site.xml
    + '[' -e /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/topology.py ']'
    + export 'HADOOP_OPTS=-Djava.net.preferIPv4Stack=true '
    + HADOOP_OPTS='-Djava.net.preferIPv4Stack=true '
    + acquire_kerberos_tgt mapred.keytab
    + '[' -z mapred.keytab ']'
    + '[' -n '' ']'
    + '[' tasktracker = jobtracker ']'
    + '[' tasktracker = tasktracker ']'
    + chmod 0644 /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/mapred-site.xml /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/hdfs-site.xml /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/core-site.xml /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER/log4j.properties
    ++ tr -d '\n'
    ++ find /usr/share/cmf/lib/plugins -name 'tt-instrumentation-*.jar'
    + ADD_TO_CP=/usr/share/cmf/lib/plugins/tt-instrumentation-4.0.4.jar
    + '[' -z '/usr/share/cmf/lib/plugins/event-publish-4.0.4-shaded.jar:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/*:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/*::' ']'
    + export 'HADOOP_CLASSPATH=/usr/share/cmf/lib/plugins/tt-instrumentation-4.0.4.jar:/usr/share/cmf/lib/plugins/event-publish-4.0.4-shaded.jar:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/*:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/*::'
    + HADOOP_CLASSPATH='/usr/share/cmf/lib/plugins/tt-instrumentation-4.0.4.jar:/usr/share/cmf/lib/plugins/event-publish-4.0.4-shaded.jar:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/*:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/*::'
    + exec /usr/lib/hadoop-0.20-mapreduce/bin/hadoop --config /var/run/cloudera-scm-agent/process/21-mapreduce-TASKTRACKER tasktracker
    Exploring the log files lead to this part:


    2012-10-19 09:31:40,034 INFO org.apache.hadoop.util.HostsFileReader:
    Setting the includes file to
    /var/run/cloudera-scm-agent/process/24-mapreduce-JOBTRACKER/mapred_hosts_allow.txt
    2012-10-19 09:31:40,034 INFO org.apache.hadoop.util.HostsFileReader:
    Setting the excludes file to
    /var/run/cloudera-scm-agent/process/24-mapreduce-JOBTRACKER/mapred_hosts_exclude.txt
    2012-10-19 09:31:40,034 INFO org.apache.hadoop.util.HostsFileReader:
    Refreshing hosts (include/exclude) list
    2012-10-19 09:31:40,035 INFO org.apache.hadoop.util.HostsFileReader:
    Adding localhost.localdomain to the list of hosts from
    /var/run/cloudera-scm-agent/process/24-mapreduce-JOBTRACKER/mapred_hosts_allow.txt
    2012-10-19 09:31:40,036 INFO org.apache.hadoop.util.HostsFileReader:
    Adding 127.0.0.1 to the list of hosts from
    /var/run/cloudera-scm-agent/process/24-mapreduce-JOBTRACKER/mapred_hosts_allow.txt
    2012-10-19 09:31:40,036 INFO org.apache.hadoop.mapred.JobTracker:
    Decommissioning 0 nodes
    2012-10-19 09:31:40,048 INFO org.apache.hadoop.ipc.Server: IPC Server
    Responder: starting
    2012-10-19 09:31:40,049 INFO org.apache.hadoop.ipc.Server: IPC Server
    listener on 8021: starting
    2012-10-19 09:31:40,066 INFO org.apache.hadoop.mapred.JobTracker:
    Starting RUNNING
    2012-10-19 09:31:40,403 ERROR
    org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
    as:mapred (auth:SIMPLE)
    cause:org.apache.hadoop.mapred.DisallowedTaskTrackerException: Tasktracker
    denied communication with jobtracker: tracker_localhost:localhost/
    127.0.0.1:32833
    2012-10-19 09:31:40,404 INFO org.apache.hadoop.ipc.Server: IPC Server
    handler 3 on 8021, call
    heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@6b248979, true, true,
    true, -1), rpc version=2, client version=32, methodsFingerPrint=-159967141
    from 127.0.0.1:54999: error:
    org.apache.hadoop.mapred.DisallowedTaskTrackerException: Tasktracker denied
    communication with jobtracker: tracker_localhost:localhost/
    127.0.0.1:32833
    org.apache.hadoop.mapred.DisallowedTaskTrackerException: Tasktracker
    denied communication with jobtracker: tracker_localhost:localhost/
    127.0.0.1:32833
    at
    org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:2857)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
    org.apache.hadoop.ipc.WritableRpcEngine$Server$WritableRpcInvoker.call(WritableRpcEngine.java:474)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:898)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1693)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1689)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1687)
    Any ideas why the services won't start or why the communication between
    the job and the task tracker would be denied? I am running CentOS 6.3 and
    selinux is disabled.

    Thanks,
    - Thomas
  • Philip Zeyliger at Oct 23, 2012 at 6:50 pm
    Can you look at the JobTracker's logs? Does it say why it denied
    communication? It may be because of a version mismatch, or it may be
    because of a hosts.allow/hosts.deny setting.

    Cheers,

    -- Philip
    On Mon, Oct 22, 2012 at 8:00 PM, Thomas Emge wrote:

    After a machine reboot I am back square one and the

    TaskTracker denied communication with jobtacker error message. Then I
    refresh the JobTracker and then restart the mapreduce1 service and after a
    couple of minutes the TaskTracker is stopped again.

    Any suggestion is greatly appreciated.

    - Thomas
    On Friday, October 19, 2012 11:13:50 PM UTC-7, Thomas Emge wrote:

    Thanks Chris,

    It did work indeed.

    On Friday, October 19, 2012 3:27:56 PM UTC-7, BludGeonT wrote:

    Go to your main MapReduce page and go over to the "Actions" tab on the
    top right, click it, go down to "Refresh Job Tracker" and click that.
    After it finishes, try again.

    I had the same problem on multiple occasions, and this fixed it.

    I hope this helps.

    -Chris


    On Fri, Oct 19, 2012 at 1:48 PM, Thomas Emge wrote:

    Hi there,

    I am running Cloudera Manager Free Edition and would like to create a
    prototype Hadoop environment where everything is running on a single
    computer.

    On the first run when all the CDH services are configured and started I
    am running into an error message stating that the TaskTracker cannot be
    started.


    Supervisor returned FATAL: + set -x
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/**cloudera-scm-agent/process/21-**mapreduce-TASKTRACKER#g' /var/run/cloudera-scm-agent/**process/21-mapreduce-**TASKTRACKER/core-site.xml /var/run/cloudera-scm-agent/**process/21-mapreduce-**TASKTRACKER/hdfs-site.xml /var/run/cloudera-scm-agent/**process/21-mapreduce-**TASKTRACKER/mapred-site.xml
    + '[' -e /var/run/cloudera-scm-agent/**process/21-mapreduce-**TASKTRACKER/topology.py ']'
    + export 'HADOOP_OPTS=-Djava.net.**preferIPv4Stack=true '
    + HADOOP_OPTS='-Djava.net.**preferIPv4Stack=true '
    + acquire_kerberos_tgt mapred.keytab
    + '[' -z mapred.keytab ']'
    + '[' -n '' ']'
    + '[' tasktracker = jobtracker ']'
    + '[' tasktracker = tasktracker ']'
    + chmod 0644 /var/run/cloudera-scm-agent/**process/21-mapreduce-**TASKTRACKER/mapred-site.xml /var/run/cloudera-scm-agent/**process/21-mapreduce-**TASKTRACKER/hdfs-site.xml /var/run/cloudera-scm-agent/**process/21-mapreduce-**TASKTRACKER/core-site.xml /var/run/cloudera-scm-agent/**process/21-mapreduce-**TASKTRACKER/log4j.properties
    ++ tr -d '\n'
    ++ find /usr/share/cmf/lib/plugins -name 'tt-instrumentation-*.jar'
    + ADD_TO_CP=/usr/share/cmf/lib/**plugins/tt-instrumentation-4.**0.4.jar
    + '[' -z '/usr/share/cmf/lib/plugins/**event-publish-4.0.4-shaded.**jar:/usr/lib/hadoop-hdfs/lib/***:/usr/lib/hadoop-hdfs/*:/usr/**lib/hadoop/lib/*:/usr/lib/**hadoop/*::' ']'
    + export 'HADOOP_CLASSPATH=/usr/share/**cmf/lib/plugins/tt-**instrumentation-4.0.4.jar:/**usr/share/cmf/lib/plugins/**event-publish-4.0.4-shaded.**jar:/usr/lib/hadoop-hdfs/lib/***:/usr/lib/hadoop-hdfs/*:/usr/**lib/hadoop/lib/*:/usr/lib/**hadoop/*::'
    + HADOOP_CLASSPATH='/usr/share/**cmf/lib/plugins/tt-**instrumentation-4.0.4.jar:/**usr/share/cmf/lib/plugins/**event-publish-4.0.4-shaded.**jar:/usr/lib/hadoop-hdfs/lib/***:/usr/lib/hadoop-hdfs/*:/usr/**lib/hadoop/lib/*:/usr/lib/**hadoop/*::'
    + exec /usr/lib/hadoop-0.20-**mapreduce/bin/hadoop --config /var/run/cloudera-scm-agent/**process/21-mapreduce-**TASKTRACKER tasktracker
    Exploring the log files lead to this part:


    2012-10-19 09:31:40,034 INFO org.apache.hadoop.util.**HostsFileReader:
    Setting the includes file to /var/run/cloudera-scm-agent/**
    process/24-mapreduce-**JOBTRACKER/mapred_hosts_allow.**txt
    2012-10-19 09:31:40,034 INFO org.apache.hadoop.util.**HostsFileReader:
    Setting the excludes file to /var/run/cloudera-scm-agent/**
    process/24-mapreduce-**JOBTRACKER/mapred_hosts_**exclude.txt
    2012-10-19 09:31:40,034 INFO org.apache.hadoop.util.**HostsFileReader:
    Refreshing hosts (include/exclude) list
    2012-10-19 09:31:40,035 INFO org.apache.hadoop.util.**HostsFileReader:
    Adding localhost.localdomain to the list of hosts from
    /var/run/cloudera-scm-agent/**process/24-mapreduce-**
    JOBTRACKER/mapred_hosts_allow.**txt
    2012-10-19 09:31:40,036 INFO org.apache.hadoop.util.**HostsFileReader:
    Adding 127.0.0.1 to the list of hosts from /var/run/cloudera-scm-agent/
    **process/24-mapreduce-**JOBTRACKER/mapred_hosts_allow.**txt
    2012-10-19 09:31:40,036 INFO org.apache.hadoop.mapred.**JobTracker:
    Decommissioning 0 nodes
    2012-10-19 09:31:40,048 INFO org.apache.hadoop.ipc.Server: IPC Server
    Responder: starting
    2012-10-19 09:31:40,049 INFO org.apache.hadoop.ipc.Server: IPC Server
    listener on 8021: starting
    2012-10-19 09:31:40,066 INFO org.apache.hadoop.mapred.**JobTracker:
    Starting RUNNING
    2012-10-19 09:31:40,403 ERROR org.apache.hadoop.security.**UserGroupInformation:
    PriviledgedActionException as:mapred (auth:SIMPLE) cause:org.apache.hadoop.
    **mapred.**DisallowedTaskTrackerException**: Tasktracker denied
    communication with jobtracker: tracker_localhost:localhost/12**
    7.0.0.1:32833 <http://127.0.0.1:32833>
    2012-10-19 09:31:40,404 INFO org.apache.hadoop.ipc.Server: IPC Server
    handler 3 on 8021, call heartbeat(org.apache.hadoop.**
    mapred.TaskTrackerStatus@**6b248979, true, true, true, -1), rpc
    version=2, client version=32, methodsFingerPrint=-159967141 from
    127.0.0.1:54999: error: org.apache.hadoop.mapred.**
    DisallowedTaskTrackerException**: Tasktracker denied communication
    with jobtracker: tracker_localhost:localhost/12**7.0.0.1:32833<http://127.0.0.1:32833>
    org.apache.hadoop.mapred.**DisallowedTaskTrackerException**:
    Tasktracker denied communication with jobtracker:
    tracker_localhost:localhost/12**7.0.0.1:32833 <http://127.0.0.1:32833>
    at org.apache.hadoop.mapred.**JobTracker.heartbeat(**
    JobTracker.java:2857)
    at sun.reflect.**NativeMethodAccessorImpl.**invoke0(Native Method)
    at sun.reflect.**NativeMethodAccessorImpl.**invoke(**
    NativeMethodAccessorImpl.java:**39)
    at sun.reflect.**DelegatingMethodAccessorImpl.**invoke(**
    DelegatingMethodAccessorImpl.**java:25)
    at java.lang.reflect.Method.**invoke(Method.java:597)
    at org.apache.hadoop.ipc.**WritableRpcEngine$Server$**
    WritableRpcInvoker.call(**WritableRpcEngine.java:474)
    at org.apache.hadoop.ipc.RPC$**Server.call(RPC.java:898)
    at org.apache.hadoop.ipc.Server$**Handler$1.run(Server.java:**
    1693)
    at org.apache.hadoop.ipc.Server$**Handler$1.run(Server.java:**
    1689)
    at java.security.**AccessController.doPrivileged(**Native Method)
    at javax.security.auth.Subject.**doAs(Subject.java:396)
    at org.apache.hadoop.security.**UserGroupInformation.doAs(**
    UserGroupInformation.java:**1332)
    at org.apache.hadoop.ipc.Server$**Handler.run(Server.java:1687)
    Any ideas why the services won't start or why the communication between
    the job and the task tracker would be denied? I am running CentOS 6.3 and
    selinux is disabled.

    Thanks,
    - Thomas
  • Thomas Emge at Oct 23, 2012 at 8:37 pm
    Based on log section below 127.0.0.1 and localhost.localdomain are added to
    the hosts_allow list and should work. I am not really seeing a reason for
    the communication to be denied but I might not be looking in the right
    location and/or log file.

    Extract from *hadoop-cmf-mapreduce1-JOBTRACKER-localhost.localdomain.log.out
    *
    2012-10-23 13:18:08,168 INFO org.apache.hadoop.util.HostsFileReader: Adding
    localhost.localdomain to the list of hosts from
    /var/run/cloudera-scm-agent/process/46-mapreduce-JOBTRACKER/mapred_hosts_allow.txt
    2012-10-23 13:18:08,168 INFO org.apache.hadoop.util.HostsFileReader: Adding
    127.0.0.1 to the list of hosts from
    /var/run/cloudera-scm-agent/process/46-mapreduce-JOBTRACKER/mapred_hosts_allow.txt
    2012-10-23 13:18:08,168 INFO org.apache.hadoop.mapred.JobTracker:
    Decommissioning 0 nodes
    2012-10-23 13:18:08,177 INFO org.apache.hadoop.ipc.Server: IPC Server
    Responder: starting
    2012-10-23 13:18:08,182 INFO org.apache.hadoop.ipc.Server: IPC Server
    listener on 8021: starting
    2012-10-23 13:18:08,214 INFO org.apache.hadoop.mapred.JobTracker: Starting
    RUNNING
    2012-10-23 13:18:08,607 ERROR
    org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
    as:mapred (auth:SIMPLE)
    cause:org.apache.hadoop.mapred.DisallowedTaskTrackerException: Tasktracker
    denied communication with jobtracker:
    tracker_localhost:localhost/127.0.0.1:57311
    2012-10-23 13:18:08,607 INFO org.apache.hadoop.ipc.Server: IPC Server
    handler 3 on 8021, call
    heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@63a6b16f, true, true,
    true, -1), rpc version=2, client version=32, methodsFingerPrint=-159967141
    from 127.0.0.1:58228: error:
    org.apache.hadoop.mapred.DisallowedTaskTrackerException: Tasktracker denied
    communication with jobtracker: tracker_localhost:localhost/127.0.0.1:57311
    org.apache.hadoop.mapred.DisallowedTaskTrackerException: Tasktracker denied
    communication with jobtracker: tracker_localhost:localhost/127.0.0.1:57311
    at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:2857)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
    org.apache.hadoop.ipc.WritableRpcEngine$Server$WritableRpcInvoker.call(WritableRpcEngine.java:474)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:898)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1693)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1689)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1687

    My installed components are:


    Component Version Component Version Cloudera Manager Agent Not
    applicable 4.0.4 Cloudera Manager Management Daemons Not applicable 4.0.4
    Hadoop CDH4 2.0.0+545 HBase CDH4 0.92.1+156 HDFS (CDH4 only) CDH4
    2.0.0+545 Hive CDH4 0.9.0+150 HttpFS (CDH4 only) CDH4 2.0.0+545 Hue CDH4
    2.1.0+210 Hue Plugins CDH4 2.1.0+210 MapReduce 1 (CDH4 only) CDH4
    0.20.2+1261 MapReduce 2 (CDH4 only) CDH4 2.0.0+545 Oozie CDH4 3.2.0+123 Yarn
    (CDH4 only) CDH4 2.0.0+545 Zookeeper CDH4 3.4.3+27
    On Tuesday, October 23, 2012 11:50:51 AM UTC-7, Philip Zeyliger wrote:

    Can you look at the JobTracker's logs? Does it say why it denied
    communication? It may be because of a version mismatch, or it may be
    because of a hosts.allow/hosts.deny setting.

    Cheers,

    -- Philip

  • Chris at Oct 23, 2012 at 11:21 pm
    Glad to help out, Thomas.

    Best of luck,

    Chris
    On Oct 19, 2012 11:13 PM, "Thomas Emge" wrote:

    Thanks Chris,

    It did work indeed.

    On Friday, October 19, 2012 3:27:56 PM UTC-7, BludGeonT wrote:

    Go to your main MapReduce page and go over to the "Actions" tab on the
    top right, click it, go down to "Refresh Job Tracker" and click that.
    After it finishes, try again.

    I had the same problem on multiple occasions, and this fixed it.

    I hope this helps.

    -Chris


    On Fri, Oct 19, 2012 at 1:48 PM, Thomas Emge wrote:

    Hi there,

    I am running Cloudera Manager Free Edition and would like to create a
    prototype Hadoop environment where everything is running on a single
    computer.

    On the first run when all the CDH services are configured and started I
    am running into an error message stating that the TaskTracker cannot be
    started.


    Supervisor returned FATAL: + set -x
    + perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/**cloudera-scm-agent/process/21-**mapreduce-TASKTRACKER#g' /var/run/cloudera-scm-agent/**process/21-mapreduce-**TASKTRACKER/core-site.xml /var/run/cloudera-scm-agent/**process/21-mapreduce-**TASKTRACKER/hdfs-site.xml /var/run/cloudera-scm-agent/**process/21-mapreduce-**TASKTRACKER/mapred-site.xml
    + '[' -e /var/run/cloudera-scm-agent/**process/21-mapreduce-**TASKTRACKER/topology.py ']'
    + export 'HADOOP_OPTS=-Djava.net.**preferIPv4Stack=true '
    + HADOOP_OPTS='-Djava.net.**preferIPv4Stack=true '
    + acquire_kerberos_tgt mapred.keytab
    + '[' -z mapred.keytab ']'
    + '[' -n '' ']'
    + '[' tasktracker = jobtracker ']'
    + '[' tasktracker = tasktracker ']'
    + chmod 0644 /var/run/cloudera-scm-agent/**process/21-mapreduce-**TASKTRACKER/mapred-site.xml /var/run/cloudera-scm-agent/**process/21-mapreduce-**TASKTRACKER/hdfs-site.xml /var/run/cloudera-scm-agent/**process/21-mapreduce-**TASKTRACKER/core-site.xml /var/run/cloudera-scm-agent/**process/21-mapreduce-**TASKTRACKER/log4j.properties
    ++ tr -d '\n'
    ++ find /usr/share/cmf/lib/plugins -name 'tt-instrumentation-*.jar'
    + ADD_TO_CP=/usr/share/cmf/lib/**plugins/tt-instrumentation-4.**0.4.jar
    + '[' -z '/usr/share/cmf/lib/plugins/**event-publish-4.0.4-shaded.**jar:/usr/lib/hadoop-hdfs/lib/***:/usr/lib/hadoop-hdfs/*:/usr/**lib/hadoop/lib/*:/usr/lib/**hadoop/*::' ']'
    + export 'HADOOP_CLASSPATH=/usr/share/**cmf/lib/plugins/tt-**instrumentation-4.0.4.jar:/**usr/share/cmf/lib/plugins/**event-publish-4.0.4-shaded.**jar:/usr/lib/hadoop-hdfs/lib/***:/usr/lib/hadoop-hdfs/*:/usr/**lib/hadoop/lib/*:/usr/lib/**hadoop/*::'
    + HADOOP_CLASSPATH='/usr/share/**cmf/lib/plugins/tt-**instrumentation-4.0.4.jar:/**usr/share/cmf/lib/plugins/**event-publish-4.0.4-shaded.**jar:/usr/lib/hadoop-hdfs/lib/***:/usr/lib/hadoop-hdfs/*:/usr/**lib/hadoop/lib/*:/usr/lib/**hadoop/*::'
    + exec /usr/lib/hadoop-0.20-**mapreduce/bin/hadoop --config /var/run/cloudera-scm-agent/**process/21-mapreduce-**TASKTRACKER tasktracker
    Exploring the log files lead to this part:


    2012-10-19 09:31:40,034 INFO org.apache.hadoop.util.**HostsFileReader:
    Setting the includes file to /var/run/cloudera-scm-agent/**
    process/24-mapreduce-**JOBTRACKER/mapred_hosts_allow.**txt
    2012-10-19 09:31:40,034 INFO org.apache.hadoop.util.**HostsFileReader:
    Setting the excludes file to /var/run/cloudera-scm-agent/**
    process/24-mapreduce-**JOBTRACKER/mapred_hosts_**exclude.txt
    2012-10-19 09:31:40,034 INFO org.apache.hadoop.util.**HostsFileReader:
    Refreshing hosts (include/exclude) list
    2012-10-19 09:31:40,035 INFO org.apache.hadoop.util.**HostsFileReader:
    Adding localhost.localdomain to the list of hosts from
    /var/run/cloudera-scm-agent/**process/24-mapreduce-**
    JOBTRACKER/mapred_hosts_allow.**txt
    2012-10-19 09:31:40,036 INFO org.apache.hadoop.util.**HostsFileReader:
    Adding 127.0.0.1 to the list of hosts from /var/run/cloudera-scm-agent/
    **process/24-mapreduce-**JOBTRACKER/mapred_hosts_allow.**txt
    2012-10-19 09:31:40,036 INFO org.apache.hadoop.mapred.**JobTracker:
    Decommissioning 0 nodes
    2012-10-19 09:31:40,048 INFO org.apache.hadoop.ipc.Server: IPC Server
    Responder: starting
    2012-10-19 09:31:40,049 INFO org.apache.hadoop.ipc.Server: IPC Server
    listener on 8021: starting
    2012-10-19 09:31:40,066 INFO org.apache.hadoop.mapred.**JobTracker:
    Starting RUNNING
    2012-10-19 09:31:40,403 ERROR org.apache.hadoop.security.**UserGroupInformation:
    PriviledgedActionException as:mapred (auth:SIMPLE) cause:org.apache.hadoop.
    **mapred.**DisallowedTaskTrackerException**: Tasktracker denied
    communication with jobtracker: tracker_localhost:localhost/12**
    7.0.0.1:32833 <http://127.0.0.1:32833>
    2012-10-19 09:31:40,404 INFO org.apache.hadoop.ipc.Server: IPC Server
    handler 3 on 8021, call heartbeat(org.apache.hadoop.**
    mapred.TaskTrackerStatus@**6b248979, true, true, true, -1), rpc
    version=2, client version=32, methodsFingerPrint=-159967141 from
    127.0.0.1:54999: error: org.apache.hadoop.mapred.**
    DisallowedTaskTrackerException**: Tasktracker denied communication
    with jobtracker: tracker_localhost:localhost/12**7.0.0.1:32833<http://127.0.0.1:32833>
    org.apache.hadoop.mapred.**DisallowedTaskTrackerException**:
    Tasktracker denied communication with jobtracker:
    tracker_localhost:localhost/12**7.0.0.1:32833 <http://127.0.0.1:32833>
    at org.apache.hadoop.mapred.**JobTracker.heartbeat(**
    JobTracker.java:2857)
    at sun.reflect.**NativeMethodAccessorImpl.**invoke0(Native Method)
    at sun.reflect.**NativeMethodAccessorImpl.**invoke(**
    NativeMethodAccessorImpl.java:**39)
    at sun.reflect.**DelegatingMethodAccessorImpl.**invoke(**
    DelegatingMethodAccessorImpl.**java:25)
    at java.lang.reflect.Method.**invoke(Method.java:597)
    at org.apache.hadoop.ipc.**WritableRpcEngine$Server$**
    WritableRpcInvoker.call(**WritableRpcEngine.java:474)
    at org.apache.hadoop.ipc.RPC$**Server.call(RPC.java:898)
    at org.apache.hadoop.ipc.Server$**Handler$1.run(Server.java:**1693)
    at org.apache.hadoop.ipc.Server$**Handler$1.run(Server.java:**1689)
    at java.security.**AccessController.doPrivileged(**Native Method)
    at javax.security.auth.Subject.**doAs(Subject.java:396)
    at org.apache.hadoop.security.**UserGroupInformation.doAs(**
    UserGroupInformation.java:**1332)
    at org.apache.hadoop.ipc.Server$**Handler.run(Server.java:1687)
    Any ideas why the services won't start or why the communication between
    the job and the task tracker would be denied? I am running CentOS 6.3 and
    selinux is disabled.

    Thanks,
    - Thomas
  • Thomas Emge at Oct 29, 2012 at 1:39 am
    Ok here is what I believe is the final 'solution'.
    I created a new virtual machine this time starting CentOS 6.3 from the
    LiveCD as opposed to the Development Workstation configuration from the
    DVD.

    Using the LiveCD installation and then running the Cloudera Manager with
    the defaults settings all the services were installed fine and started as
    expected. Now what was the real difference in OS configuration between the
    LiveCD vs. DVD installation I am not sure but this how I got everything
    working.

    Thanks for all the suggestions.

    - Thomas

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedOct 19, '12 at 8:54p
activeOct 29, '12 at 1:39a
posts8
users3
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase