FAQ
Please help me, I can't figure out how to fix this problem.
I have a cluster of virtual machines under VMWare (windows XP is original
OS):

Ubuntu 8.10
Intel Pentium DUAL CPU E2180 @ 2 GHZ
Memory 1024 MB

I have a namenode and 8 more datanodes.
I want to start teragen and terasort programs and do a benchmark analysis of
a cluster running 1, 3 and all 8 datanodes.
Datanodes have only 20GB configured HDFS capacity each, so it is a total of
cca 150GB total.
I have no problem generating the input data with 2 or 8 maps but problem
comes out with terasort. When it comes to reduce phase, it generates a
following error:

10/07/13 10:59:40 INFO mapred.JobClient: Task Id :
attempt_201007131052_0002_r_000000_0, Status : FAILED
Shuffle Error: Exceeded MAX_FAILED_UNIQUE_FETCHES; bailing-out.


As I understand I have to setup these parameters in mapred-site.xml to
override default values:

<property>
<name>mapred.map.tasks</name>
<value>?</value>
</property>
<property>
<name>mapred.reduce.tasks</name>
<value>?</value>
</property>

Does anyone know how to setup number of reducers so that it works :).

Thank you...

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedJul 13, '10 at 10:05a
activeJul 13, '10 at 10:05a
posts1
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Tonci Buljan: 1 post

People

Translate

site design / logo © 2022 Grokbase