Hadoop > mapreduce-user > February 2010

Hi all, As far as I know, a barrier exists between map and reduce function in one round of MR. There is another barrier for the reducer to end the job for that round. However if we want to run in ...

Hi all, I to run a map reduce task repeatedly in order to achieve the desired result. Is it possible that at the beginning of each iteration, the data set is not distributed (divided into chunks and ...

Hi, For a particular project I created a writable for holding a long and a double called LongDoublePair. My mapper outputs LongDoublePair values and the reducer receives an Iterable<LongDoublePair . ...

Hello, We're trying to do a reduce-side join by applying two different mappers (TransformationSessionMapper and TransformationActionMapper) to two different input files and joining them using ...

Hi there, I am using Hadoop 0.20.1 and I am trying to submit jobs to the cluster as jars. Here's what I'm trying to do: -inFormat org.apache.hadoop.mapred.KeyValueTextInputFormat -outKey ...

Hello, I just joined the list and got a newbie question. Operating on a 10-node Linux cluster running Hadoop 0.20.1, I've been trying out the WordCount program. I have three files: WordCount.java, ...

Hi, I've some question about the MapRed ports and how a reduce knows where the map output is to fetch. I know that MapRed uses jetty has a webserver. - The JobTracker send tasks to the TaskTracker ...

Hi, I'm new to hadoop and am confused about some serialization issues in the spill. My reference for this post is the following snippet from the O'Reilly Hadoop book: =================== Each map ...

We have a hadoop job running and have used custom counters to track few counters ( like no of successfully processed documents matching certain conditions) Since we need to get this counters even ...

Hello, I have set up a cluster with one NameNode/JobTracker and three DataNode/TaskTrackers, and having some issues with the reduce step being unable to start. Masters and slaves can ping and ssh ...

Hello i have to find Counter and i am using inetsocketAddress and connecting into the jobtracker jsp with help of it am able to find the counter when it is running on single pc but as i am trying to ...

As we have seen multiple times in the mailing list, users want to have the capability of getting all files out of a multi-level directory structure. 4/1/2008: ...

... compile-java-sources: [echo] contrib: mumak [javac] Compiling 2 source files to /data/users/zshao/hadoop_mapreduce/trunk/build/contrib/mumak/classes [javac] ...

Hi, I am new with hadoop and specially the c++ api. I am running the Wordcount example but it crashes during the Map and prints this log. Regards, Trad Riadh ./hadoop pipes -input input -output ...

Hi, In Hadoop MapRed, when I define the number of reduce tasks to run, <property <name mapred.reduce.tasks</name <value 3</value </property I've noticed that during the execution of an MapRed ...

Hi all, I am using hadoop 0.20.1 version. I am trying to append some content from a file on HDFS to another file on HDFS. I am using FileSystem.append() method to get the stream and then passed it to ...

Hello all, Is the following architecture possible? A distributed key-value store is used (HBase). So along with values, there would be a timestamp associated with the values. Map & Reduce tasks are ...

The Hadoop PMC has voted to make Vinod Kumar V. a committer on Hadoop Map-Reduce. Congratulations, Vinod. Thanks again for all your valuable contributions, we look forward to more! Arun