[MapReduce-user] Are the input values (associated with a specific key) of reduce() method sorted?
James Y. Li
Dec 6, 2011 at 12:55 am
I just begin to use Hadoop framework, and write my reduce() method. Wonder
if the Iterable input values (associated with the input key) are already
p.s. I am using the version 0.20.2-cdh3u1
: Hi James, By default, there is no guarantees on value order. Using some of the more advanced API features, you can perform a secondary sort of values. You can read a good example of it here: http://sonerbalkir.blogspot.com/2010/01/simulating-secondary-sort-on-values.html -Joey -- Joseph Echeverria Cloudera, Inc. 443.305.9434
: James, No, values do not come in any order for a given key. But it is achievable. If you need them to be ordered, you need to perform a secondary sort operation, by using multiple key components. See this 3-part series on how to do that, for example: http://www.cloudera.com/blog/2011/04/simple-moving-average-secondary-sort-and-mapreduce-part-3/
Algorithm for cross product
I keep getting multiple values for unique reduce keys
Can I use MapWritable as a key?
Are hadoop fs commands serial or parallel
sort the values in reduce side
limit of values in reduce phase?
Should mapreduce.ReduceContext reuse same object in nextKeyValue?
Avoiding value buffering in reduce
Multiple Mappers and One Reducer
output from one map reduce job as the input to another map reduce job?
1 of 3
Dec 6, '11 at 12:55a
Dec 6, '11 at 2:37a
3 users in discussion
Joey Echeverria (1)
Harsh J (1)
James Y. Li (1)
Groups & Organizations
site design / logo © 2022 Grokbase