FAQ
Hadoop does not prevent you from writing key value pair multiple times in
the same map iteration if that is what is your roadblock.

You can call collector.collect() multiple times with same or distinct key
value pairs within a single map iteration.

-Rahul
On Thu, Jul 29, 2010 at 8:10 AM, ailinykh wrote:


Hello, everybody!
I have a bunch of records. Each record has key, and two fields A,B - R(k,
A,B)
I want to build two inverted indexes, one per each field. As output I
expect
two files
IndexA =(A1- [k1,k2,k3..]),(A2 -[k1,k2,k4...]) ...
IndexB =(B1- [k1,k2,k3..]),(B2-[k1,k2,k4...]) ...

Hadoop mapper allows to write out only one key value pair. Of cause, I can
run it twice, but I don't want to read the same record twice. What is the
best way to handle this situation?

Thank you,
Andrey
--
View this message in context:
http://old.nabble.com/How-to-build-multiple-inverted-indexes--tp29297476p29297476.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.

Search Discussions

Discussion Posts

Previous

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 2 of 2 | next ›
Discussion Overview
groupcommon-user @
categorieshadoop
postedJul 29, '10 at 3:11p
activeJul 29, '10 at 5:54p
posts2
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Rahul Jain: 1 post Ailinykh: 1 post

People

Translate

site design / logo © 2022 Grokbase