FAQ
I'm trying to run the sample RandomWriter task on 6 nodes. DFS runs
across the nodes, and I'm able to run the task when I have
mapred.job.tracker set to
"local." However, when I try to actually use a jobtracker, it doesn't
work.
I've set mapred.job.tracker to "issdm-2:50080" in hadoop-site.xml. I
start up the jobtracker and tasktrackers from issdm-2, apparently
with no errors:

[eestolan@issdm-2 ~/hadoop-trunk]$ bin/start-mapred.sh
starting jobtracker, logging to /projects/csl/projects/obsd/eestolan/
hadoop_logs/hadoop-eestolan-jobtracker-issdm-2.cse.ucsc.edu.out
issdm-2: starting tasktracker, logging to /projects/csl/projects/obsd/
eestolan/hadoop_logs/hadoop-eestolan-tasktracker-
issdm-2.cse.ucsc.edu.out
issdm-1: starting tasktracker, logging to /projects/csl/projects/obsd/
eestolan/hadoop_logs/hadoop-eestolan-tasktracker-
issdm-1.cse.ucsc.edu.out
issdm-6: starting tasktracker, logging to /projects/csl/projects/obsd/
eestolan/hadoop_logs/hadoop-eestolan-tasktracker-
issdm-6.cse.ucsc.edu.out
issdm-5: starting tasktracker, logging to /projects/csl/projects/obsd/
eestolan/hadoop_logs/hadoop-eestolan-tasktracker-
issdm-5.cse.ucsc.edu.out
issdm-3: starting tasktracker, logging to /projects/csl/projects/obsd/
eestolan/hadoop_logs/hadoop-eestolan-tasktracker-
issdm-3.cse.ucsc.edu.out
issdm-4: starting tasktracker, logging to /projects/csl/projects/obsd/
eestolan/hadoop_logs/hadoop-eestolan-tasktracker-
issdm-4.cse.ucsc.edu.out

Then, when I try to start a RandomWriter task (the exact same one
that succeeded with "local" as the jobtracker) it can't find the
jobtracker.

[eestolan@issdm-2 ~/hadoop-trunk]$ bin/hadoop jar build/hadoop-0.10.2-
dev-examples.jar randomwriter debug_conf/tinyrandomwrite_5mapx10MB.xml
07/05/15 22:16:00 INFO ipc.Client: Retrying connect to server:
issdm-2/128.114.57.137:50080. Already tried 1 time(s).
07/05/15 22:16:01 INFO ipc.Client: Retrying connect to server:
issdm-2/128.114.57.137:50080. Already tried 2 time(s).
07/05/15 22:16:02 INFO ipc.Client: Retrying connect to server:
issdm-2/128.114.57.137:50080. Already tried 3 time(s).
07/05/15 22:16:03 INFO ipc.Client: Retrying connect to server:
issdm-2/128.114.57.137:50080. Already tried 4 time(s).
07/05/15 22:16:04 INFO ipc.Client: Retrying connect to server:
issdm-2/128.114.57.137:50080. Already tried 5 time(s).
07/05/15 22:16:05 INFO ipc.Client: Retrying connect to server:
issdm-2/128.114.57.137:50080. Already tried 6 time(s).
07/05/15 22:16:06 INFO ipc.Client: Retrying connect to server:
issdm-2/128.114.57.137:50080. Already tried 7 time(s).
07/05/15 22:16:07 INFO ipc.Client: Retrying connect to server:
issdm-2/128.114.57.137:50080. Already tried 8 time(s).
07/05/15 22:16:08 INFO ipc.Client: Retrying connect to server:
issdm-2/128.114.57.137:50080. Already tried 9 time(s).
07/05/15 22:16:09 INFO ipc.Client: Retrying connect to server:
issdm-2/128.114.57.137:50080. Already tried 10 time(s).
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress
(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:519)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams
(Client.java:145)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:525)
at org.apache.hadoop.ipc.Client.call(Client.java:452)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:164)
at $Proxy0.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:248)
at org.apache.hadoop.mapred.JobClient.init(JobClient.java:200)
at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:
192)
at org.apache.hadoop.examples.RandomWriter.main
(RandomWriter.java:178)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke
(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ProgramDriver
$ProgramDescription.invoke(ProgramDriver.java:71)
at org.apache.hadoop.util.ProgramDriver.driver
(ProgramDriver.java:143)
at org.apache.hadoop.examples.ExampleDriver.main
(ExampleDriver.java:40)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke
(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:155)


Running "netstat -nap" on issdm-2 shows that java is listening on
ports 50060 and 50070, which are the ports set in hadoop-default.xml
for tasktracker.http.port and dfs.info.port, respectively; but
nothing is listening on port 50080, nor is java listening on any
other ports. Am I just making a stupid configuration mistake/omission
somewhere?

Thanks,
Esteban Molina-Estolano

Search Discussions

  • Zhao Boran at May 16, 2007 at 9:50 am
    IPV6?
    On 5/16/07, Esteban Molina-Estolano wrote:

    I'm trying to run the sample RandomWriter task on 6 nodes. DFS runs
    across the nodes, and I'm able to run the task when I have
    mapred.job.tracker set to
    "local." However, when I try to actually use a jobtracker, it doesn't
    work.
    I've set mapred.job.tracker to "issdm-2:50080" in hadoop-site.xml. I
    start up the jobtracker and tasktrackers from issdm-2, apparently
    with no errors:

    [eestolan@issdm-2 ~/hadoop-trunk]$ bin/start-mapred.sh
    starting jobtracker, logging to /projects/csl/projects/obsd/eestolan/
    hadoop_logs/hadoop-eestolan-jobtracker-issdm-2.cse.ucsc.edu.out
    issdm-2: starting tasktracker, logging to /projects/csl/projects/obsd/
    eestolan/hadoop_logs/hadoop-eestolan-tasktracker-
    issdm-2.cse.ucsc.edu.out
    issdm-1: starting tasktracker, logging to /projects/csl/projects/obsd/
    eestolan/hadoop_logs/hadoop-eestolan-tasktracker-
    issdm-1.cse.ucsc.edu.out
    issdm-6: starting tasktracker, logging to /projects/csl/projects/obsd/
    eestolan/hadoop_logs/hadoop-eestolan-tasktracker-
    issdm-6.cse.ucsc.edu.out
    issdm-5: starting tasktracker, logging to /projects/csl/projects/obsd/
    eestolan/hadoop_logs/hadoop-eestolan-tasktracker-
    issdm-5.cse.ucsc.edu.out
    issdm-3: starting tasktracker, logging to /projects/csl/projects/obsd/
    eestolan/hadoop_logs/hadoop-eestolan-tasktracker-
    issdm-3.cse.ucsc.edu.out
    issdm-4: starting tasktracker, logging to /projects/csl/projects/obsd/
    eestolan/hadoop_logs/hadoop-eestolan-tasktracker-
    issdm-4.cse.ucsc.edu.out

    Then, when I try to start a RandomWriter task (the exact same one
    that succeeded with "local" as the jobtracker) it can't find the
    jobtracker.

    [eestolan@issdm-2 ~/hadoop-trunk]$ bin/hadoop jar build/hadoop-0.10.2-
    dev-examples.jar randomwriter debug_conf/tinyrandomwrite_5mapx10MB.xml
    07/05/15 22:16:00 INFO ipc.Client: Retrying connect to server:
    issdm-2/128.114.57.137:50080. Already tried 1 time(s).
    07/05/15 22:16:01 INFO ipc.Client: Retrying connect to server:
    issdm-2/128.114.57.137:50080. Already tried 2 time(s).
    07/05/15 22:16:02 INFO ipc.Client: Retrying connect to server:
    issdm-2/128.114.57.137:50080. Already tried 3 time(s).
    07/05/15 22:16:03 INFO ipc.Client: Retrying connect to server:
    issdm-2/128.114.57.137:50080. Already tried 4 time(s).
    07/05/15 22:16:04 INFO ipc.Client: Retrying connect to server:
    issdm-2/128.114.57.137:50080. Already tried 5 time(s).
    07/05/15 22:16:05 INFO ipc.Client: Retrying connect to server:
    issdm-2/128.114.57.137:50080. Already tried 6 time(s).
    07/05/15 22:16:06 INFO ipc.Client: Retrying connect to server:
    issdm-2/128.114.57.137:50080. Already tried 7 time(s).
    07/05/15 22:16:07 INFO ipc.Client: Retrying connect to server:
    issdm-2/128.114.57.137:50080. Already tried 8 time(s).
    07/05/15 22:16:08 INFO ipc.Client: Retrying connect to server:
    issdm-2/128.114.57.137:50080. Already tried 9 time(s).
    07/05/15 22:16:09 INFO ipc.Client: Retrying connect to server:
    issdm-2/128.114.57.137:50080. Already tried 10 time(s).
    java.net.ConnectException: Connection refused
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
    at java.net.PlainSocketImpl.connectToAddress
    (PlainSocketImpl.java:195)
    at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
    at java.net.Socket.connect(Socket.java:519)
    at org.apache.hadoop.ipc.Client$Connection.setupIOstreams
    (Client.java:145)
    at org.apache.hadoop.ipc.Client.getConnection(Client.java:525)
    at org.apache.hadoop.ipc.Client.call(Client.java:452)
    at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:164)
    at $Proxy0.getProtocolVersion(Unknown Source)
    at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:248)
    at org.apache.hadoop.mapred.JobClient.init(JobClient.java:200)
    at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:
    192)
    at org.apache.hadoop.examples.RandomWriter.main
    (RandomWriter.java:178)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke
    (NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke
    (DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.ProgramDriver
    $ProgramDescription.invoke(ProgramDriver.java:71)
    at org.apache.hadoop.util.ProgramDriver.driver
    (ProgramDriver.java:143)
    at org.apache.hadoop.examples.ExampleDriver.main
    (ExampleDriver.java:40)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke
    (NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke
    (DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:155)


    Running "netstat -nap" on issdm-2 shows that java is listening on
    ports 50060 and 50070, which are the ports set in hadoop-default.xml
    for tasktracker.http.port and dfs.info.port, respectively; but
    nothing is listening on port 50080, nor is java listening on any
    other ports. Am I just making a stupid configuration mistake/omission
    somewhere?

    Thanks,
    Esteban Molina-Estolano
  • Esteban Molina-Estolano at May 16, 2007 at 7:35 pm
    I don't understand what you're asking about IPv6. I'm not
    deliberately using IPv6 anywhere.

    Java isn't listening on port 50080 at all where I've started the
    jobtracker. That's why I think I've made some configuration error. Is
    there anything I need to do to configure the jobtracker properly in
    addition to setting mapred.job.tracker (I've set it to
    "issdm-2:50080") and adding nodes to the slave file?
    Thanks,
    ~ Esteban
    On May 16, 2007, at 2:50 AM, Zhao Boran wrote:

    IPV6?
    On 5/16/07, Esteban Molina-Estolano wrote:

    I'm trying to run the sample RandomWriter task on 6 nodes. DFS runs
    across the nodes, and I'm able to run the task when I have
    mapred.job.tracker set to
    "local." However, when I try to actually use a jobtracker, it doesn't
    work.
    I've set mapred.job.tracker to "issdm-2:50080" in hadoop-site.xml. I
    start up the jobtracker and tasktrackers from issdm-2, apparently
    with no errors:

    [eestolan@issdm-2 ~/hadoop-trunk]$ bin/start-mapred.sh
    starting jobtracker, logging to /projects/csl/projects/obsd/eestolan/
    hadoop_logs/hadoop-eestolan-jobtracker-issdm-2.cse.ucsc.edu.out
    issdm-2: starting tasktracker, logging to /projects/csl/projects/
    obsd/
    eestolan/hadoop_logs/hadoop-eestolan-tasktracker-
    issdm-2.cse.ucsc.edu.out
    issdm-1: starting tasktracker, logging to /projects/csl/projects/
    obsd/
    eestolan/hadoop_logs/hadoop-eestolan-tasktracker-
    issdm-1.cse.ucsc.edu.out
    issdm-6: starting tasktracker, logging to /projects/csl/projects/
    obsd/
    eestolan/hadoop_logs/hadoop-eestolan-tasktracker-
    issdm-6.cse.ucsc.edu.out
    issdm-5: starting tasktracker, logging to /projects/csl/projects/
    obsd/
    eestolan/hadoop_logs/hadoop-eestolan-tasktracker-
    issdm-5.cse.ucsc.edu.out
    issdm-3: starting tasktracker, logging to /projects/csl/projects/
    obsd/
    eestolan/hadoop_logs/hadoop-eestolan-tasktracker-
    issdm-3.cse.ucsc.edu.out
    issdm-4: starting tasktracker, logging to /projects/csl/projects/
    obsd/
    eestolan/hadoop_logs/hadoop-eestolan-tasktracker-
    issdm-4.cse.ucsc.edu.out

    Then, when I try to start a RandomWriter task (the exact same one
    that succeeded with "local" as the jobtracker) it can't find the
    jobtracker.

    [eestolan@issdm-2 ~/hadoop-trunk]$ bin/hadoop jar build/
    hadoop-0.10.2-
    dev-examples.jar randomwriter debug_conf/
    tinyrandomwrite_5mapx10MB.xml
    07/05/15 22:16:00 INFO ipc.Client: Retrying connect to server:
    issdm-2/128.114.57.137:50080. Already tried 1 time(s).
    07/05/15 22:16:01 INFO ipc.Client: Retrying connect to server:
    issdm-2/128.114.57.137:50080. Already tried 2 time(s).
    07/05/15 22:16:02 INFO ipc.Client: Retrying connect to server:
    issdm-2/128.114.57.137:50080. Already tried 3 time(s).
    07/05/15 22:16:03 INFO ipc.Client: Retrying connect to server:
    issdm-2/128.114.57.137:50080. Already tried 4 time(s).
    07/05/15 22:16:04 INFO ipc.Client: Retrying connect to server:
    issdm-2/128.114.57.137:50080. Already tried 5 time(s).
    07/05/15 22:16:05 INFO ipc.Client: Retrying connect to server:
    issdm-2/128.114.57.137:50080. Already tried 6 time(s).
    07/05/15 22:16:06 INFO ipc.Client: Retrying connect to server:
    issdm-2/128.114.57.137:50080. Already tried 7 time(s).
    07/05/15 22:16:07 INFO ipc.Client: Retrying connect to server:
    issdm-2/128.114.57.137:50080. Already tried 8 time(s).
    07/05/15 22:16:08 INFO ipc.Client: Retrying connect to server:
    issdm-2/128.114.57.137:50080. Already tried 9 time(s).
    07/05/15 22:16:09 INFO ipc.Client: Retrying connect to server:
    issdm-2/128.114.57.137:50080. Already tried 10 time(s).
    java.net.ConnectException: Connection refused
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.PlainSocketImpl.doConnect
    (PlainSocketImpl.java:333)
    at java.net.PlainSocketImpl.connectToAddress
    (PlainSocketImpl.java:195)
    at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:
    182)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:
    366)
    at java.net.Socket.connect(Socket.java:519)
    at org.apache.hadoop.ipc.Client$Connection.setupIOstreams
    (Client.java:145)
    at org.apache.hadoop.ipc.Client.getConnection(Client.java:
    525)
    at org.apache.hadoop.ipc.Client.call(Client.java:452)
    at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:164)
    at $Proxy0.getProtocolVersion(Unknown Source)
    at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:248)
    at org.apache.hadoop.mapred.JobClient.init(JobClient.java:
    200)
    at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:
    192)
    at org.apache.hadoop.examples.RandomWriter.main
    (RandomWriter.java:178)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
    Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke
    (NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke
    (DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.ProgramDriver
    $ProgramDescription.invoke(ProgramDriver.java:71)
    at org.apache.hadoop.util.ProgramDriver.driver
    (ProgramDriver.java:143)
    at org.apache.hadoop.examples.ExampleDriver.main
    (ExampleDriver.java:40)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
    Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke
    (NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke
    (DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:155)


    Running "netstat -nap" on issdm-2 shows that java is listening on
    ports 50060 and 50070, which are the ports set in hadoop-default.xml
    for tasktracker.http.port and dfs.info.port, respectively; but
    nothing is listening on port 50080, nor is java listening on any
    other ports. Am I just making a stupid configuration mistake/omission
    somewhere?

    Thanks,
    Esteban Molina-Estolano

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedMay 16, '07 at 5:47a
activeMay 16, '07 at 7:35p
posts3
users2
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase