FAQ
Hello,

I'm writing one MR task where I need to pass a common set of data to all Map
task.
The data required by all the MR is present in relational database.

Is it possible to get data from database before launching job and then pass
the object to all Maps?
I know we can use DistributedCache to distribute files. But is there any
facility to distribute objects?

Abhay.

Search Discussions

  • Harsh J at Apr 23, 2011 at 8:53 am
    Hello Abhay,

    On Sat, Apr 23, 2011 at 11:32 AM, abhay ratnaparkhi
    wrote:
    Is it possible to get data from database before launching job and then pass
    the object to all Maps?
    I know we can use DistributedCache to distribute files. But is there any
    facility to distribute objects?
    There's no direct way of doing that in Hadoop without manual
    ser-deser. Serialize into a file -> Add to DC -> Deserialize in Tasks,
    should do it.

    --
    Harsh J

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedApr 23, '11 at 6:02a
activeApr 23, '11 at 8:53a
posts2
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Abhay ratnaparkhi: 1 post Harsh J: 1 post

People

Translate

site design / logo © 2022 Grokbase