I'm trying to run the sample RandomWriter task on 6 nodes. DFS runs
across the nodes, and I'm able to run the task when I have
mapred.job.tracker set to
"local." However, when I try to actually use a jobtracker, it doesn't
work.
I've set mapred.job.tracker to "issdm-2:50080" in hadoop-site.xml. I
start up the jobtracker and tasktrackers from issdm-2, apparently
with no errors:
[[email protected] ~/hadoop-trunk]$ bin/start-mapred.sh
starting jobtracker, logging to /projects/csl/projects/obsd/eestolan/
hadoop_logs/hadoop-eestolan-jobtracker-issdm-2.cse.ucsc.edu.out
issdm-2: starting tasktracker, logging to /projects/csl/projects/obsd/
eestolan/hadoop_logs/hadoop-eestolan-tasktracker-
issdm-2.cse.ucsc.edu.out
issdm-1: starting tasktracker, logging to /projects/csl/projects/obsd/
eestolan/hadoop_logs/hadoop-eestolan-tasktracker-
issdm-1.cse.ucsc.edu.out
issdm-6: starting tasktracker, logging to /projects/csl/projects/obsd/
eestolan/hadoop_logs/hadoop-eestolan-tasktracker-
issdm-6.cse.ucsc.edu.out
issdm-5: starting tasktracker, logging to /projects/csl/projects/obsd/
eestolan/hadoop_logs/hadoop-eestolan-tasktracker-
issdm-5.cse.ucsc.edu.out
issdm-3: starting tasktracker, logging to /projects/csl/projects/obsd/
eestolan/hadoop_logs/hadoop-eestolan-tasktracker-
issdm-3.cse.ucsc.edu.out
issdm-4: starting tasktracker, logging to /projects/csl/projects/obsd/
eestolan/hadoop_logs/hadoop-eestolan-tasktracker-
issdm-4.cse.ucsc.edu.out
Then, when I try to start a RandomWriter task (the exact same one
that succeeded with "local" as the jobtracker) it can't find the
jobtracker.
[[email protected] ~/hadoop-trunk]$ bin/hadoop jar build/hadoop-0.10.2-
dev-examples.jar randomwriter debug_conf/tinyrandomwrite_5mapx10MB.xml
07/05/15 22:16:00 INFO ipc.Client: Retrying connect to server:
issdm-2/128.114.57.137:50080. Already tried 1 time(s).
07/05/15 22:16:01 INFO ipc.Client: Retrying connect to server:
issdm-2/128.114.57.137:50080. Already tried 2 time(s).
07/05/15 22:16:02 INFO ipc.Client: Retrying connect to server:
issdm-2/128.114.57.137:50080. Already tried 3 time(s).
07/05/15 22:16:03 INFO ipc.Client: Retrying connect to server:
issdm-2/128.114.57.137:50080. Already tried 4 time(s).
07/05/15 22:16:04 INFO ipc.Client: Retrying connect to server:
issdm-2/128.114.57.137:50080. Already tried 5 time(s).
07/05/15 22:16:05 INFO ipc.Client: Retrying connect to server:
issdm-2/128.114.57.137:50080. Already tried 6 time(s).
07/05/15 22:16:06 INFO ipc.Client: Retrying connect to server:
issdm-2/128.114.57.137:50080. Already tried 7 time(s).
07/05/15 22:16:07 INFO ipc.Client: Retrying connect to server:
issdm-2/128.114.57.137:50080. Already tried 8 time(s).
07/05/15 22:16:08 INFO ipc.Client: Retrying connect to server:
issdm-2/128.114.57.137:50080. Already tried 9 time(s).
07/05/15 22:16:09 INFO ipc.Client: Retrying connect to server:
issdm-2/128.114.57.137:50080. Already tried 10 time(s).
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress
(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:519)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams
(Client.java:145)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:525)
at org.apache.hadoop.ipc.Client.call(Client.java:452)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:164)
at $Proxy0.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:248)
at org.apache.hadoop.mapred.JobClient.init(JobClient.java:200)
at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:
192)
at org.apache.hadoop.examples.RandomWriter.main
(RandomWriter.java:178)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke
(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ProgramDriver
$ProgramDescription.invoke(ProgramDriver.java:71)
at org.apache.hadoop.util.ProgramDriver.driver
(ProgramDriver.java:143)
at org.apache.hadoop.examples.ExampleDriver.main
(ExampleDriver.java:40)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke
(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
Running "netstat -nap" on issdm-2 shows that java is listening on
ports 50060 and 50070, which are the ports set in hadoop-default.xml
for tasktracker.http.port and dfs.info.port, respectively; but
nothing is listening on port 50080, nor is java listening on any
other ports. Am I just making a stupid configuration mistake/omission
somewhere?
Thanks,
Esteban Molina-Estolano
