Hi experts
I have a query with the job.xml file in map reduce.I set some
value in mapred-site.xml and *marked as final*, say
mapred.num.reduce.tasks=15. When I submit my job I explicitly specified the
number of reducers as -D mapred.num.reduce.tasks=4. Now as expected my my
job should run with 15 reducers as I marked this value as final in my
config file. Now what would be the value that would be available for
mapred.num.reduce.tasks *in job.xml*, 15 or 4 ?

Thank you

Regards
Bejoy.K.S

Search Discussions

  • Bejoy Ks at Dec 9, 2011 at 6:48 am
    Correction not mapred.num.reduce.tasks but mapred.reduce.tasks . :)
    On Fri, Dec 9, 2011 at 12:14 PM, Bejoy Ks wrote:

    Hi experts
    I have a query with the job.xml file in map reduce.I set some
    value in mapred-site.xml and *marked as final*, say
    mapred.num.reduce.tasks=15. When I submit my job I explicitly specified the
    number of reducers as -D mapred.num.reduce.tasks=4. Now as expected my my
    job should run with 15 reducers as I marked this value as final in my
    config file. Now what would be the value that would be available for
    mapred.num.reduce.tasks *in job.xml*, 15 or 4 ?

    Thank you

    Regards
    Bejoy.K.S
  • Arun C Murthy at Dec 9, 2011 at 11:14 pm
    'final' is meant for admins to ensure certain values aren't overridable.

    However, in the example you gave, you'll see 15 (since it's 'final').

    Arun
    On Dec 8, 2011, at 10:44 PM, Bejoy Ks wrote:

    Hi experts
    I have a query with the job.xml file in map reduce.I set some value in mapred-site.xml and marked as final, say mapred.num.reduce.tasks=15. When I submit my job I explicitly specified the number of reducers as -D mapred.num.reduce.tasks=4. Now as expected my my job should run with 15 reducers as I marked this value as final in my config file. Now what would be the value that would be available for mapred.num.reduce.tasks in job.xml, 15 or 4 ?

    Thank you

    Regards
    Bejoy.K.S
  • Bejoy Ks at Dec 10, 2011 at 5:43 pm
    Thanks Arun.
    To make my understanding a little more crisp. All the
    values for config parameters available in job.xml are the actual values
    used by a job for its execution. There won't be any further changes in
    these values at any point further down the lane of execution. (ie no
    changes in confing param values after the job.xml is generated). Is my
    understanding correct?
    I waned to make sure of this as in some clusters I'm
    working on I don't have acess to the config files *-site.xml . So I just
    want to make sure that job.xml is fully dependable and the right reference
    for the config values used by my job.

    Thanks a lot!....

    Regards
    Bejoy.KS
    On Sat, Dec 10, 2011 at 4:44 AM, Arun C Murthy wrote:

    'final' is meant for admins to ensure certain values aren't overridable.

    However, in the example you gave, you'll see 15 (since it's 'final').

    Arun

    On Dec 8, 2011, at 10:44 PM, Bejoy Ks wrote:

    Hi experts
    I have a query with the job.xml file in map reduce.I set some
    value in mapred-site.xml and *marked as final*, say
    mapred.num.reduce.tasks=15. When I submit my job I explicitly specified the
    number of reducers as -D mapred.num.reduce.tasks=4. Now as expected my my
    job should run with 15 reducers as I marked this value as final in my
    config file. Now what would be the value that would be available for
    mapred.num.reduce.tasks *in job.xml*, 15 or 4 ?

    Thank you

    Regards
    Bejoy.K.S

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupmapreduce-user @
categorieshadoop
postedDec 9, '11 at 6:44a
activeDec 10, '11 at 5:43p
posts4
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Bejoy Ks: 3 posts Arun C Murthy: 1 post

People

Translate

site design / logo © 2022 Grokbase