FAQ
Hi Guys,

I am new to Hadoop environment. I am unable to find a solution to below
problem.

My requirement is that, each mapper should access the same values. Is there
any such location where I can put these value and each mapper will acscess
it?

Thanks,
Santosh

--
View this message in context: http://www.nabble.com/Initial-Values-for-Mapper-tp21008923p21008923.html
Sent from the Hadoop core-dev mailing list archive at Nabble.com.

Search Discussions

  • Sagar Naik at Dec 15, 2008 at 7:06 am
    DistributedCache
    http://hadoop.apache.org/core/docs/r0.18.2/mapred_tutorial.html#DistributedCache
    Bahir wrote:
    Hi Guys,

    I am new to Hadoop environment. I am unable to find a solution to below
    problem.

    My requirement is that, each mapper should access the same values. Is there
    any such location where I can put these value and each mapper will acscess
    it?

    Thanks,
    Santosh
  • Owen O'Malley at Dec 15, 2008 at 7:10 am

    On Dec 14, 2008, at 10:34 PM, Bahir wrote:

    My requirement is that, each mapper should access the same values.
    Is there
    any such location where I can put these value and each mapper will
    acscess
    it?
    Depending on the size, you can put it into the JobConf, via set

    jobConf.set("my.values", "1,7,9,1000")

    Then the mappers can all look at the configuration for the relevant
    attribute. If it is bigger, then the distributed file cache is
    appropriate. Write the file once into hdfs and add its url to the
    distributed file cache. Each slave will download it once to the local
    file system.

    -- Owen

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedDec 15, '08 at 6:34a
activeDec 15, '08 at 7:10a
posts3
users3
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase