Grokbase › Groups › Hadoop › common-user › July 2008
FAQ
Hi,

I'm trying to configure RandomWriter to generate less data than does
the default configuration. I created a job configuration file job.xml
and added in variables given at http://wiki.apache.org/hadoop/
RandomWriter. Tried a couple of ways of running the program below,
but configurations in job.xml were not taken by RandomWriter.

1) bin/hadoop jar hadoop-*-examples.jar randomwriter rand job.xml
2) bin/hadoop jar hadoop-*-examples.jar randomwriter rand --conf job.xml
3) bin/hadoop jar --conf job.xml hadoop-*-examples.jar randomwriter rand

Passing property values via the -D option didn't seem to work either.
Can anybody advise on how to use the job configuration file properly?

Thanks,
Heshan

Search Discussions

  • Amar Kamat at Jul 1, 2008 at 5:45 am ⇧

    Heshan Lin wrote:
    Hi,

    I'm trying to configure RandomWriter to generate less data than does
    the default configuration.
    bin/hadoop jar hadoop-*-examples.jar randomwriter
    -Dtest.randomwrite.bytes_per_map=<value>
    -Dtest.randomwrite.total_bytes=<value>
    -Dtest.randomwriter.maps_per_host=<value> <input-filename>
    The number of maps that will be spawned in this case will be
    total_bytes/bytes_per_map.
    Other parameters are test.randomwrite.min_key (size in bytes),
    test.randomwrite.max_key (size in bytes), test.randomwrite.min_value
    (size in bytes) and test.randomwrite.max_value (size in bytes).
    Amar
    I created a job configuration file job.xml and added in variables
    given at http://wiki.apache.org/hadoop/RandomWriter. Tried a couple of
    ways of running the program below, but configurations in job.xml were
    not taken by RandomWriter.

    1) bin/hadoop jar hadoop-*-examples.jar randomwriter rand job.xml
    2) bin/hadoop jar hadoop-*-examples.jar randomwriter rand --conf job.xml
    3) bin/hadoop jar --conf job.xml hadoop-*-examples.jar randomwriter rand

    Passing property values via the -D option didn't seem to work either.
    Can anybody advise on how to use the job configuration file properly?

    Thanks,
    Heshan
  • Heshan Lin at Jul 1, 2008 at 2:28 pm ⇧
    Thanks a bunch, Amar! It works fine.

    /Heshan

    On Jul 1, 2008, at 1:44 AM, Amar Kamat wrote:

    Heshan Lin wrote:
    Hi,

    I'm trying to configure RandomWriter to generate less data than
    does the default configuration.
    bin/hadoop jar hadoop-*-examples.jar randomwriter -
    Dtest.randomwrite.bytes_per_map=<value> -
    Dtest.randomwrite.total_bytes=<value> -
    Dtest.randomwriter.maps_per_host=<value> <input-filename>
    The number of maps that will be spawned in this case will be
    total_bytes/bytes_per_map.
    Other parameters are test.randomwrite.min_key (size in bytes),
    test.randomwrite.max_key (size in bytes),
    test.randomwrite.min_value (size in bytes) and
    test.randomwrite.max_value (size in bytes).
    Amar
    I created a job configuration file job.xml and added in variables
    given at http://wiki.apache.org/hadoop/RandomWriter. Tried a
    couple of ways of running the program below, but configurations in
    job.xml were not taken by RandomWriter.

    1) bin/hadoop jar hadoop-*-examples.jar randomwriter rand job.xml
    2) bin/hadoop jar hadoop-*-examples.jar randomwriter rand --conf
    job.xml
    3) bin/hadoop jar --conf job.xml hadoop-*-examples.jar
    randomwriter rand

    Passing property values via the -D option didn't seem to work
    either. Can anybody advise on how to use the job configuration
    file properly?

    Thanks,
    Heshan

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedJul 1, '08 at 5:33a
activeJul 1, '08 at 2:28p
posts3
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Heshan Lin: 2 posts Amar Kamat: 1 post

Content

People

Support

Translate

site design / logo © 2022 Grokbase