[Hadoop-common-user] How can I assign the same mapper class with different data?

Is it a strict partitioning of the input that you need? If yes, why?
Why not just feed the data into the job and let it split
automatically. You can process differently based what it inputs if you
need that.

I am a beginner at hadoop. I want to ask a question , how can I configurate
a job with two map task with the same mapper class and different dataset?
For example, I want to sort the num from 1 to 100, then use one task to deal
with 1 to 50, and the other with 51 to 100, I want to control the dataset I
send to mapper. How can I make it? Can anyone help me ?

The real problem is I want to use different mapper to deal with different
hbase data. For example the data is storing in different HTable, So I should
use different mapper to connect to different Htable and get the data.How can
I made it?

2009/10/5 Huang Qian <[email protected]>

Hi Huang,

Haven't worked with Hbase but in general,
If you want to have control over what data split to go as a whole to mapper, easiest way is to compress that split in single file; making as many split files as needed. If you need to know what file is currently being processed, you can use map.input.file ( corresponds to HBase table?? )from configuration, and do file specific operations as needed.
Hope this helps

Amogh

-----Original Message-----
From: Huang Qian
Sent: Tuesday, October 06, 2009 7:15 AM
To: [email protected]
Subject: Re: How can I assign the same mapper class with different data?

The real problem is I want to use different mapper to deal with different
hbase data. For example the data is storing in different HTable, So I should
use different mapper to connect to different Htable and get the data.How can
I made it?

2009/10/5 Huang Qian <[email protected]>