FAQ
Hi there,

Installed Hadoop on VMWare Workstation 7.1.4 (1 master, 1 slave)
both of the machines use 2 cores (host has 6, win7x64) with 2048 mb
ram, ubuntu 10.4. LTS (64 bit) with hadoop 0.20.2

I used this tutorial:
http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/

hadoop-hadoop-tasktracker-s-master.log:
...
2011-05-10 08:41:27,388 INFO org.mortbay.log: jetty-6.1.14
2011-05-10 08:44:31,410 INFO org.mortbay.log: Started SelectChannelConnector@0.0.0.0:50060
...
It took 3 to 9 minutes to take the next step after jetty!
problem occures also with datanode (master/client), jobtracker (took
9! minutes), secondarynamenode (took 9! minutes) and tasktracker
(master/client)

but, thats not all. Facing another problem when running a simple m/r
example. I believe these problems belong together:

trying the example from the link, the following happens (reproducible)

master.console:
hadoop@s-master:/usr/local/hadoop$ bin/hadoop dfs -ls gutenberg
Found 3 items
-rw-r--r-- 2 hadoop supergroup 674566 2011-05-10 09:01 /user/hadoop/gutenberg/20417.txt
-rw-r--r-- 2 hadoop supergroup 1573112 2011-05-10 09:01 /user/hadoop/gutenberg/4300.txt
-rw-r--r-- 2 hadoop supergroup 1423801 2011-05-10 09:01 /user/hadoop/gutenberg/5000.txt

hadoop@s-master:/usr/local/hadoop$ bin/hadoop jar hadoop-*-examples.jar wordcount gutenberg gutenberg-output
11/05/10 09:01:56 INFO input.FileInputFormat: Total input paths to process : 3
11/05/10 09:01:56 INFO mapred.JobClient: Running job: job_201105100841_0002
11/05/10 09:01:57 INFO mapred.JobClient: map 0% reduce 0%
11/05/10 09:02:08 INFO mapred.JobClient: map 33% reduce 0%
11/05/10 09:02:11 INFO mapred.JobClient: map 100% reduce 0%
11/05/10 09:02:17 INFO mapred.JobClient: map 100% reduce 11%
here it stucks...

hadoop-hadoop-tasktracker-s-slave.log
2011-05-10 09:02:00,719 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201105100841_0002_m_000002_0 task's state:UNASSIGNED
2011-05-10 09:02:00,721 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201105100841_0002_m_000002_0
2011-05-10 09:02:00,721 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch attempt_201105100841_0002_m_000002_0
2011-05-10 09:02:01,253 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201105100841_0002_m_-1532294282
2011-05-10 09:02:01,254 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201105100841_0002_m_-1532294282 spawned.
2011-05-10 09:02:01,846 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201105100841_0002_m_-1532294282 given task: attempt_201105100841_0002_m_000002_0
2011-05-10 09:02:04,400 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105100841_0002_m_000002_0 1.0%
2011-05-10 09:02:04,401 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201105100841_0002_m_000002_0 is done.
2011-05-10 09:02:04,401 INFO org.apache.hadoop.mapred.TaskTracker: reported output size for attempt_201105100841_0002_m_000002_0 was 0
2011-05-10 09:02:04,416 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 2
2011-05-10 09:02:04,526 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201105100841_0002_m_-1532294282 exited. Number of tasks it ran: 1
2011-05-10 09:02:06,764 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201105100841_0002_r_000000_0 task's state:UNASSIGNED
2011-05-10 09:02:06,765 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201105100841_0002_r_000000_0
2011-05-10 09:02:06,765 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch attempt_201105100841_0002_r_000000_0
2011-05-10 09:02:06,817 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201105100841_0002_r_-1532294282
2011-05-10 09:02:06,817 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201105100841_0002_r_-1532294282 spawned.
2011-05-10 09:02:07,306 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201105100841_0002_r_-1532294282 given task: attempt_201105100841_0002_r_000000_0
2011-05-10 09:02:13,018 INFO org.apache.hadoop.mapred.TaskTracker: Sent out 267013 bytes for reduce: 0 from map: attempt_201105100841_0002_m_000002_0 given 267013/267009
2011-05-10 09:02:13,018 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.1.1:50060, dest: 127.0.1.1:45821, bytes: 267013, op: MAPRED_SHUFFLE, cliID: attempt_201105100841_0002_m_000002_0
2011-05-10 09:02:13,855 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105100841_0002_r_000000_0 0.11111112% reduce > copy (1 of 3 at 0.04 MB/s) >
2011-05-10 09:02:16,859 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105100841_0002_r_000000_0 0.11111112% reduce > copy (1 of 3 at 0.04 MB/s) >
2011-05-10 09:02:22,868 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105100841_0002_r_000000_0 0.11111112% reduce > copy (1 of 3 at 0.04 MB/s) >
2011-05-10 09:02:28,877 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105100841_0002_r_000000_0 0.11111112% reduce > copy (1 of 3 at 0.04 MB/s) >
2011-05-10 09:02:31,883 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105100841_0002_r_000000_0 0.11111112% reduce > copy (1 of 3 at 0.04 MB/s) >
2011-05-10 09:02:37,903 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105100841_0002_r_000000_0 0.11111112% reduce > copy (1 of 3 at 0.04 MB/s) >
2011-05-10 09:02:43,912 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105100841_0002_r_000000_0 0.11111112% reduce > copy (1 of 3 at 0.04 MB/s) >
2011-05-10 09:02:46,918 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201105100841_0002_r_000000_0 0.11111112% reduce > copy (1 of 3 at 0.04 MB/s) >
and so on...

sometimes, a package gets to the master, but therefor I have to wait
10 minutes.

I can't figure out what the problem is:
- tested with VirtualBox
- tested with 32 bit Ubuntu 10.4
- tested with 64 bit Debian 6
- tested another machine (windows server 2003 x86)

please help me... :)

thx!
Stefan

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedMay 10, '11 at 4:13p
activeMay 10, '11 at 4:13p
posts1
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Stefan Wienert: 1 post

People

Translate

site design / logo © 2022 Grokbase