FAQ
Hi there,

Unfortunately you can't control which mapper gets what data. The InputSplit
-> map task assignment is random. You could, however, do the join in the
reduce, by using an intermediate key as your join key.

Does that make sense?

Alex
On Sat, Jul 3, 2010 at 9:28 AM, Denim Live wrote:

Hello everyone,

I have written my custom partitioner for partitioning datasets. I want to
partition two datasets using the same partitioner and then in the next
mapreduce job, I want each mapper to handle the same partition from the two
sources and perform some function such as joining etc. How I can I ensure
that one mapper gets the split that corresponds to same partition from both
the sources?

Any help would be highly appreciated.
Alex

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 2 of 3 | next ›
Discussion Overview
groupmapreduce-user @
categorieshadoop
postedJul 3, '10 at 4:30p
activeJul 6, '10 at 9:38a
posts3
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Denim Live: 2 posts Alex Loddengaard: 1 post

People

Translate

site design / logo © 2022 Grokbase