FAQ
Hi,
I want to change the following hadoop parameters before launching TeraSort
program:
io.sort.factor
io.sort.mb
mapreduce.reduce.tasks
io.file.buffer.size
io.sort.record.percent

I tried to change these properties by putting them as property values in
core-site.xml file under conf directory.
However, I am unable to observe the effect of my changes to these
parameters.

Could someone elaborate as to which config files, these parameters need to
be configured like core-site.xml or mapred-site.xml or hdfs-site.xml.

Thanks,
Bikash

Search Discussions

  • Harsh Chouraria at Apr 4, 2011 at 7:34 pm
    Hello,
    On Tue, Apr 5, 2011 at 12:52 AM, bikash sharma wrote:
    Hi,
    I want to change the following hadoop parameters before launching TeraSort
    program:
    io.sort.factor
    io.sort.mb
    mapreduce.reduce.tasks
    io.file.buffer.size
    io.sort.record.percent
    Making these changes in the mapred-site.xml should be fine.

    As a lazy fix, set all those property changes as <final>true</final>.
    This would enforce it over the defaults that are being used instead.
    [1]'s probably why it doesn't get picked up.

    Or pass generic options (with -Dprop=val) while launching the program,
    if it accepts that.

    [1] - http://wiki.apache.org/hadoop/FAQ#How_do_I_get_my_MapReduce_Java_Program_to_read_the_Cluster.27s_set_configuration_and_not_just_defaults.3F

    --
    Harsh J
    Support Engineer, Cloudera
  • Bikash sharma at Apr 4, 2011 at 8:03 pm
    Thanks Harsh.
    On Mon, Apr 4, 2011 at 3:33 PM, Harsh Chouraria wrote:

    Hello,
    On Tue, Apr 5, 2011 at 12:52 AM, bikash sharma wrote:
    Hi,
    I want to change the following hadoop parameters before launching TeraSort
    program:
    io.sort.factor
    io.sort.mb
    mapreduce.reduce.tasks
    io.file.buffer.size
    io.sort.record.percent
    Making these changes in the mapred-site.xml should be fine.

    As a lazy fix, set all those property changes as <final>true</final>.
    This would enforce it over the defaults that are being used instead.
    [1]'s probably why it doesn't get picked up.

    Or pass generic options (with -Dprop=val) while launching the program,
    if it accepts that.

    [1] -
    http://wiki.apache.org/hadoop/FAQ#How_do_I_get_my_MapReduce_Java_Program_to_read_the_Cluster.27s_set_configuration_and_not_just_defaults.3F

    --
    Harsh J
    Support Engineer, Cloudera

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedApr 4, '11 at 7:23p
activeApr 4, '11 at 8:03p
posts3
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Bikash sharma: 2 posts Harsh Chouraria: 1 post

People

Translate

site design / logo © 2022 Grokbase