FAQ
Hi,

I want to send some data and messages to all nodes after I run a MR job.
then begin another job.
Is there any straight way to broadcast data under hadoop framework?

Search Discussions

  • Ioan Eugen Stan at Jan 30, 2012 at 12:17 pm

    Pe 30.01.2012 14:10, Hamid Oliaei a scris:
    Hi,

    I want to send some data and messages to all nodes after I run a MR job.
    then begin another job.
    Is there any straight way to broadcast data under hadoop framework?
    See if DistributedCahche [1] can do what you need. It makes application
    specific files easy to distribute.

    http://hadoop.apache.org/common/docs/r0.20.2/mapred_tutorial.html#DistributedCache
  • Hamid Oliaei at Jan 30, 2012 at 12:24 pm
    Thanks a lot. I am looking at it.

    Regards,

    Oliaei.
    On Mon, Jan 30, 2012 at 3:47 PM, Ioan Eugen Stan wrote:

    Pe 30.01.2012 14:10, Hamid Oliaei a scris:

    Hi,
    I want to send some data and messages to all nodes after I run a MR job.
    then begin another job.
    Is there any straight way to broadcast data under hadoop framework?
    See if DistributedCahche [1] can do what you need. It makes application
    specific files easy to distribute.

    http://hadoop.apache.org/**common/docs/r0.20.2/mapred_**
    tutorial.html#DistributedCache<http://hadoop.apache.org/common/docs/r0.20.2/mapred_tutorial.html#DistributedCache>

    --
    Ioan Eugen Stan
    http://ieugen.blogspot.com
  • Hamid Oliaei at Aug 22, 2012 at 11:46 am
    Hi,

    I tested DistributedCache module for copying some data to all nodes.
    Unfortunately, it was time-consuming and runtime is important for my work.
    I want to write a map-reduce job so that a copy of input data are generated
    in output of all reducers.
    Is that possible? How?
    I mean i want to have copies of some data to the number of reducers.

    Thanks,
    Hamid

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupmapreduce-user @
categorieshadoop
postedJan 30, '12 at 12:11p
activeAug 22, '12 at 11:46a
posts4
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Hamid Oliaei: 3 posts Ioan Eugen Stan: 1 post

People

Translate

site design / logo © 2021 Grokbase