FAQ
Hi everybody,

please explain me the steps to pass user parameters for the mapper class.
thanks.
--
View this message in context: http://www.nabble.com/how-to-pass-user-parameter-for-the-mapper-tp14496141p14496141.html
Sent from the Hadoop Users mailing list archive at Nabble.com.

Search Discussions

  • Jason Venner at Dec 25, 2007 at 5:10 pm
    We set the values in the JobConf object, in the driver, using the set
    methods defined on the JobConf class.
    In the mapper class we access the conf parameters, via the get methods
    defined on the JobConf class.

    The JobConf, configuration object is made available to the mapper class
    in the method configure.

    We define the JobConf object as a member variable in the mapper class
    and set it in the configure method as below

    /** Store away the job configuration information and start a timer. */
    @Override
    public void configure(JobConf job) {
    super.configure(job);
    this.job = job;

    Happy Holidays

    helena21 wrote:
    Hi everybody,

    please explain me the steps to pass user parameters for the mapper class.
    thanks.
  • Ted Dziuba at Dec 25, 2007 at 5:35 pm
    You can get and set variables in the JobConf. The map task's
    configure() method takes a JobConf as a parameter, and you can keep the
    reference as an instance variable.

    Ted

    helena21 wrote:
    Hi everybody,

    please explain me the steps to pass user parameters for the mapper class.
    thanks.
  • Norbert Burger at Dec 26, 2007 at 4:10 am
    As I understand, the JobConf getters/setters are best for data static
    to the entire job.

    What's the recommended way to pass a variable to the mappers/reducers
    that might be different for each InputSplit?

    For example, let's say I'm using Hadoop's grep example to extract
    information from a collection of logfiles, but each logfile has some
    header info (which may or may not be part of the current InputSplit)
    that every mapper/reducer needs access to.

    How should I approach this? Is overriding InputFileFormat so that the
    header data is tacked onto each InputSplit the best way to approach
    this?

    Thanks,
    Norbert
    On Dec 25, 2007 12:34 PM, Ted Dziuba wrote:
    You can get and set variables in the JobConf. The map task's
    configure() method takes a JobConf as a parameter, and you can keep the
    reference as an instance variable.

    Ted


    helena21 wrote:
    Hi everybody,

    please explain me the steps to pass user parameters for the mapper class.
    thanks.
  • Ted Dunning at Dec 26, 2007 at 2:23 pm
    That would be a fine way to solve the problem.

    You can also pass data in to the maps via the key since the key has little
    use for most maps.

    On 12/25/07 8:09 PM, "Norbert Burger" wrote:

    How should I approach this? Is overriding InputFileFormat so that the
    header data is tacked onto each InputSplit the best way to approach
    this?

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedDec 25, '07 at 4:44p
activeDec 26, '07 at 2:23p
posts5
users5
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase