Hi All,

I have a simple question. I have a arraylist which i am populating through
db in main class. now i want to use the same list in my map and reduce class
so the question is how to access/send this array list in map and reduce
class. one way is to again fetch the data in mapper and reducer classes
which means 3 times db access and seems not suitable and other way either
could be define arraylist as static in main class and then access it or
might be using distributed cache.

so kindly let me know the best way of doing it

Thanks
--
Regards
Shuja-ur-Rehman Baig
<http://pk.linkedin.com/in/shujamughal>

Search Discussions

  • Chase Bradford at Feb 21, 2011 at 7:01 pm
    I did a write up on this sort of thing, but it was based on HashSets.
    You might find it useful:

    http://chasebradford.wordpress.com/2011/02/05/distributed-cache-static-objects-and-fast-setup/
    On Mon, Feb 21, 2011 at 10:57 AM, Shuja Rehman wrote:
    Hi All,

    I have a simple question. I have a arraylist which i am populating through
    db in main class. now i want to use the same list in my map and reduce class
    so the question is how to access/send this array list in map and reduce
    class. one way is to again fetch the data in mapper and reducer classes
    which means 3 times db access and seems not suitable and other way either
    could be define arraylist as static in main class and then access it or
    might be using distributed cache.

    so kindly let me know the best way of doing it

    Thanks
    --
    Regards
    Shuja-ur-Rehman Baig



    --
    Chase Bradford


    “If in physics there's something you don't understand, you can always
    hide behind the uncharted depths of nature. But if your program
    doesn't work, there is no obstinate nature. If it doesn't work, you've
    messed up.”

    - Edsger Dijkstra

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupmapreduce-user @
categorieshadoop
postedFeb 21, '11 at 6:57p
activeFeb 21, '11 at 7:01p
posts2
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Shuja Rehman: 1 post Chase Bradford: 1 post

People

Translate

site design / logo © 2022 Grokbase