FAQ
Hi,
I want to define a matrix or list that all mappers share.
So that all mappers can do operations on it.
Can I make it?
Thanks!



--


Regards!

Jun Tan

Search Discussions

  • Sudharsan Sampath at Aug 15, 2011 at 10:25 am
    Hi,

    To my knowledge, its not possible with plain map-reduce. But you can try
    using a distributed cache on top of it. To quote a few try, hazelcast (if ur
    prog lang is java) or gigaspace.

    Just a note, why would you want to share date across mappers. It defeats the
    basic assumption of map-reduce that mappers are independent work chunks.

    -Sudhan S

    2011/8/15 谭军 <tanjun_2525@163.com>
    Hi,
    I want to define a matrix or list that all mappers share.
    So that all mappers can do operations on it.
    Can I make it?
    Thanks!


    --

    Regards!

    Jun Tan

  • 谭军 at Aug 15, 2011 at 11:29 am
    Hi,
    I want to construct a cross-linklist to store a graph
    I want to get (node, neighbor-list) from (node, node) records in database file.
    Each Mapper processes one node.
    And each Mapper gets (node, node) records and passes (node, neighbor-list) records to Reducer.
    How can I make it?

    --


    Regards!

    Jun Tan


    At 2011-08-15 18:25:06,"Sudharsan Sampath" wrote:
    Hi,


    To my knowledge, its not possible with plain map-reduce. But you can try using a distributed cache on top of it. To quote a few try, hazelcast (if ur prog lang is java) or gigaspace.


    Just a note, why would you want to share date across mappers. It defeats the basic assumption of map-reduce that mappers are independent work chunks.


    -Sudhan S


    2011/8/15 谭军<tanjun_2525@163.com>

    Hi,
    I want to define a matrix or list that all mappers share.
    So that all mappers can do operations on it.
    Can I make it?
    Thanks!



    --


    Regards!

    Jun Tan

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupmapreduce-user @
categorieshadoop
postedAug 15, '11 at 9:15a
activeAug 15, '11 at 11:29a
posts3
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

谭军: 2 posts Sudharsan Sampath: 1 post

People

Translate

site design / logo © 2021 Grokbase