FAQ
Hi,
I'm trying to increase the job-tracker heap by adding Xmx definition to
hadoop-env.sh (adding it to HADOOP_TASKTRACKER_OPTS).
I keep getting this error while starting hadoop:
Invalid maximum heap size: -Xmx=2048m
Could not create the Java virtual machine.

Any idea what am I doing wrong ?

Search Discussions

  • Harsh J at Mar 17, 2011 at 4:45 pm
    Hello,

    Remove the "=" from "-Xmx=2048m".

    P.s. If you're looking to change the JobTracker opts, it is
    HADOOP_JOBTRACKER_OPTS, not TASKTRACKER :)
    On Thu, Mar 17, 2011 at 10:08 PM, Lior Schachter wrote:
    Hi,
    I'm trying to increase the job-tracker heap by adding Xmx definition to
    hadoop-env.sh (adding it to HADOOP_TASKTRACKER_OPTS).
    I keep getting this error while starting hadoop:
    Invalid maximum heap size: -Xmx=2048m
    Could not create the Java virtual machine.

    Any idea what am I doing wrong ?


    --
    Harsh J
    http://harshj.com
  • Lior Schachter at Mar 17, 2011 at 5:08 pm
    sure. my mistake. thanks.

    I'm also trying to increase the reducer heap size (keeping the map with the
    default 200m). I found 2 related properties:
    1. mapreduce.reduce.java.opts
    2. mapred.child.java.opts

    Which should be configured and whats the purpose of the second property.

    thanks,
    Lior
    On Thu, Mar 17, 2011 at 6:41 PM, Harsh J wrote:

    Hello,

    Remove the "=" from "-Xmx=2048m".

    P.s. If you're looking to change the JobTracker opts, it is
    HADOOP_JOBTRACKER_OPTS, not TASKTRACKER :)
    On Thu, Mar 17, 2011 at 10:08 PM, Lior Schachter wrote:
    Hi,
    I'm trying to increase the job-tracker heap by adding Xmx definition to
    hadoop-env.sh (adding it to HADOOP_TASKTRACKER_OPTS).
    I keep getting this error while starting hadoop:
    Invalid maximum heap size: -Xmx=2048m
    Could not create the Java virtual machine.

    Any idea what am I doing wrong ?


    --
    Harsh J
    http://harshj.com
  • Harsh J at Mar 17, 2011 at 5:23 pm
    What version of Hadoop are you using?

    1 is a new property in 0.21, and is the right one if you're using that.
    2 is the property to use in 0.20, and applies to both Maps and
    Reduces. In 0.20 there is no distinction available for setting JVM
    opts to types of tasks.
    On Thu, Mar 17, 2011 at 10:37 PM, Lior Schachter wrote:
    sure. my mistake. thanks.

    I'm also trying to increase the reducer heap size (keeping the map with the
    default 200m). I found 2 related properties:
    1. mapreduce.reduce.java.opts
    2. mapred.child.java.opts

    Which should be configured and whats the purpose of the second property.

    thanks,
    Lior
    On Thu, Mar 17, 2011 at 6:41 PM, Harsh J wrote:

    Hello,

    Remove the "=" from "-Xmx=2048m".

    P.s. If you're looking to change the JobTracker opts, it is
    HADOOP_JOBTRACKER_OPTS, not TASKTRACKER :)

    On Thu, Mar 17, 2011 at 10:08 PM, Lior Schachter <liors@infolinks.com>
    wrote:
    Hi,
    I'm trying to increase the job-tracker heap by adding Xmx definition to
    hadoop-env.sh (adding it to HADOOP_TASKTRACKER_OPTS).
    I keep getting this error while starting hadoop:
    Invalid maximum heap size: -Xmx=2048m
    Could not create the Java virtual machine.

    Any idea what am I doing wrong ?


    --
    Harsh J
    http://harshj.com


    --
    Harsh J
    http://harshj.com
  • Lior Schachter at Mar 17, 2011 at 5:30 pm
    we are using hbase hadoop version 0.20.3

    maybe I can do it using the jobconf (separate jvm opts for map/reduce) ?
    On Thu, Mar 17, 2011 at 7:23 PM, Harsh J wrote:

    What version of Hadoop are you using?

    1 is a new property in 0.21, and is the right one if you're using that.
    2 is the property to use in 0.20, and applies to both Maps and
    Reduces. In 0.20 there is no distinction available for setting JVM
    opts to types of tasks.
    On Thu, Mar 17, 2011 at 10:37 PM, Lior Schachter wrote:
    sure. my mistake. thanks.

    I'm also trying to increase the reducer heap size (keeping the map with the
    default 200m). I found 2 related properties:
    1. mapreduce.reduce.java.opts
    2. mapred.child.java.opts

    Which should be configured and whats the purpose of the second property.

    thanks,
    Lior
    On Thu, Mar 17, 2011 at 6:41 PM, Harsh J wrote:

    Hello,

    Remove the "=" from "-Xmx=2048m".

    P.s. If you're looking to change the JobTracker opts, it is
    HADOOP_JOBTRACKER_OPTS, not TASKTRACKER :)

    On Thu, Mar 17, 2011 at 10:08 PM, Lior Schachter <liors@infolinks.com>
    wrote:
    Hi,
    I'm trying to increase the job-tracker heap by adding Xmx definition
    to
    hadoop-env.sh (adding it to HADOOP_TASKTRACKER_OPTS).
    I keep getting this error while starting hadoop:
    Invalid maximum heap size: -Xmx=2048m
    Could not create the Java virtual machine.

    Any idea what am I doing wrong ?


    --
    Harsh J
    http://harshj.com


    --
    Harsh J
    http://harshj.com
  • Harsh J at Mar 17, 2011 at 6:19 pm
    Nope, AFAIK 0.20.x does not have properties that can apply JVM options
    separately to map and reduce tasks. "mapred.child.java.opts", which
    applies to both, is the only go [mapred-site.xml with a final, or set
    it specifically in jobs].
    On Thu, Mar 17, 2011 at 10:59 PM, Lior Schachter wrote:
    we are using hbase hadoop version 0.20.3

    maybe I can do it using the jobconf (separate jvm opts for map/reduce) ?
    On Thu, Mar 17, 2011 at 7:23 PM, Harsh J wrote:

    What version of Hadoop are you using?

    1 is a new property in 0.21, and is the right one if you're using that.
    2 is the property to use in 0.20, and applies to both Maps and
    Reduces. In 0.20 there is no distinction available for setting JVM
    opts to types of tasks.

    On Thu, Mar 17, 2011 at 10:37 PM, Lior Schachter <liors@infolinks.com>
    wrote:
    sure. my mistake. thanks.

    I'm also trying to increase the reducer heap size (keeping the map with
    the
    default 200m). I found 2 related properties:
    1. mapreduce.reduce.java.opts
    2. mapred.child.java.opts

    Which should be configured and whats the purpose of the second property.

    thanks,
    Lior
    On Thu, Mar 17, 2011 at 6:41 PM, Harsh J wrote:

    Hello,

    Remove the "=" from "-Xmx=2048m".

    P.s. If you're looking to change the JobTracker opts, it is
    HADOOP_JOBTRACKER_OPTS, not TASKTRACKER :)

    On Thu, Mar 17, 2011 at 10:08 PM, Lior Schachter <liors@infolinks.com>
    wrote:
    Hi,
    I'm trying to increase the job-tracker heap by adding Xmx definition
    to
    hadoop-env.sh (adding it to HADOOP_TASKTRACKER_OPTS).
    I keep getting this error while starting hadoop:
    Invalid maximum heap size: -Xmx=2048m
    Could not create the Java virtual machine.

    Any idea what am I doing wrong ?


    --
    Harsh J
    http://harshj.com


    --
    Harsh J
    http://harshj.com


    --
    Harsh J
    http://harshj.com

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupmapreduce-user @
categorieshadoop
postedMar 17, '11 at 4:38p
activeMar 17, '11 at 6:19p
posts6
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Lior Schachter: 3 posts Harsh J: 3 posts

People

Translate

site design / logo © 2022 Grokbase