FAQ
Hi,

I am trying to setup a hadoop cluster with 7nodes with the master node also
functioning as a slave node (i.e. runs a datanode and a tasktracker along
with the namenode and jobtracker deamons). I am able to get HDFS working.
However when I try starting the tasktrackers (bin/start-mapred.sh), I get
the errors similar to the following logged in the jobtrackers log:

org.apache.hadoop.mapred.DisallowedTaskTrackerException: Tasktracker denied
communication with jobtracker: tracker_pc405.emulab.net:localhost/
127.0.0.1:46511
at
org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:2498)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)


I am not able to figure out why this happening. I think it is not a network
issue since HDFS is able to run just fine. This website
http://www.docjar.com/docs/api/org/apache/hadoop/mapred/DisallowedTaskTrackerException.htmlsays
that this is because "tasktracker tries to register or communicate
with
the jobtracker when it does not appear on the list of included nodes, or has
been specifically excluded". However, I have included all the tasktracker
nodes in the file which is read using the "mapred.hosts" parameter. (These
are the same nodes as the HDFS nodes and hence shouldn't be a problem)
Can anyone help me out with this?

Thanks a lot,
Virajith

Search Discussions

  • Devaraj K at Jun 22, 2011 at 5:33 am
    Hi Virajith,

    This exception will be thrown when the host name present in the
    file which is the value for the property "mapred.hosts.exclude". If you
    don't mention any thing for "mapred.hosts" and "mapred.hosts.exclude"
    properties or the mentioned files don't contain any hosts, job tracker
    allows to connect any task tracker. If you include the host name in
    "mapred.hosts" and also in "mapred.hosts.exclude", it will not allow to
    connect the task tracker and it throws the below exception.

    If you want to allow all the task trackers , don't mention any thing for the
    both properties. If you want exclude specific hosts and allow the remaining
    hosts to connect then add those host names exclude
    file("mapred.hosts.exclude" file) and don't add anything for the include
    list. If you want to include specific hosts, the add those host names in
    "mapred.hosts" property file and don't add anything for the exclude list.

    "mapred.hosts" and "mapred.hosts.exclude" are for hadoop 0.20.x versions.
    For the later versions need to update these
    "mapreduce.jobtracker.hosts.exclude.filename" and
    "mapreduce.jobtracker.hosts.filename".



    Devaraj K



    _____

    From: Virajith Jalaparti
    Sent: Tuesday, June 21, 2011 7:49 PM
    To: mapreduce-user@hadoop.apache.org
    Subject: Tasktracker denied communication with jobtracker



    Hi,

    I am trying to setup a hadoop cluster with 7nodes with the master node also
    functioning as a slave node (i.e. runs a datanode and a tasktracker along
    with the namenode and jobtracker deamons). I am able to get HDFS working.
    However when I try starting the tasktrackers (bin/start-mapred.sh), I get
    the errors similar to the following logged in the jobtrackers log:

    org.apache.hadoop.mapred.DisallowedTaskTrackerException: Tasktracker denied
    communication with jobtracker:
    tracker_pc405.emulab.net:localhost/127.0.0.1:46511
    at
    org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:2498)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39
    )
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl
    .java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)


    I am not able to figure out why this happening. I think it is not a network
    issue since HDFS is able to run just fine. This website
    http://www.docjar.com/docs/api/org/apache/hadoop/mapred/DisallowedTaskTracke
    rException.html says that this is because "tasktracker tries to register or
    communicate with the jobtracker when it does not appear on the list of
    included nodes, or has been specifically excluded". However, I have included
    all the tasktracker nodes in the file which is read using the "mapred.hosts"
    parameter. (These are the same nodes as the HDFS nodes and hence shouldn't
    be a problem)
    Can anyone help me out with this?

    Thanks a lot,
    Virajith

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupmapreduce-user @
categorieshadoop
postedJun 21, '11 at 2:19p
activeJun 22, '11 at 5:33a
posts2
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Virajith Jalaparti: 1 post Devaraj K: 1 post

People

Translate

site design / logo © 2022 Grokbase