[MapReduce-user] Broadcasting some data to all nodes
Jan 30, 2012 at 12:11 pm
I want to send some data and messages to all nodes after I run a MR job.
then begin another job.
Is there any straight way to broadcast data under hadoop framework?
Ioan Eugen Stan
: See if DistributedCahche  can do what you need. It makes application specific files easy to distribute. http://hadoop.apache.org/common/docs/r0.20.2/mapred_tutorial.html#DistributedCache -- Ioan Eugen Stan http://ieugen.blogspot.com
: Thanks a lot. I am looking at it. Regards, Oliaei.
: Hi, I tested DistributedCache module for copying some data to all nodes. Unfortunately, it was time-consuming and runtime is important for my work. I want to write a map-reduce job so that a copy of input data are generated in output of all reducers. Is that possible? How? I mean i want to have copies of some data to the number of reducers. Thanks, Hamid
Sending data to all reducers
Extension points available for data locality
Namenode won't start, has NPE
Best practices configuring libraries on the backend.
Problem setting up 0.23 on a Cluster
How is data of each job assigned to nodes in Mumak ?
How can I know how many mappers created?
Lack of data locality in Hadoop-0.20.2
FW: How to run multiple data nodes and multiple task trackers on single server.
Dead data nodes during job excution and failed tasks.
1 of 4
Jan 30, '12 at 12:11p
Aug 22, '12 at 11:46a
2 users in discussion
Hamid Oliaei (3)
Ioan Eugen Stan (1)
Groups & Organizations
site design / logo © 2021 Grokbase