Please help me, I can't figure out how to fix this problem.
I have a cluster of virtual machines under VMWare (windows XP is original
Intel Pentium DUAL CPU E2180 @ 2 GHZ
Memory 1024 MB
I have a namenode and 8 more datanodes.
I want to start teragen and terasort programs and do a benchmark analysis of
a cluster running 1, 3 and all 8 datanodes.
Datanodes have only 20GB configured HDFS capacity each, so it is a total of
cca 150GB total.
I have no problem generating the input data with 2 or 8 maps but problem
comes out with terasort. When it comes to reduce phase, it generates a
10/07/13 10:59:40 INFO mapred.JobClient: Task Id :
attempt_201007131052_0002_r_000000_0, Status : FAILED
Shuffle Error: Exceeded MAX_FAILED_UNIQUE_FETCHES; bailing-out.
As I understand I have to setup these parameters in mapred-site.xml to
override default values:
Does anyone know how to setup number of reducers so that it works :).