FAQ
Hello.
I am running 0.20.1.
There is an options HADOOP_*_OPTS in file hadoop-env.sh.
When i don't put "-Xmx" in them at all, java can't initialize any VM at
all. But this happens only on 64bit VPS with 3.5G RAM. Starting hadoop on
my 32-bit laptop with 2GB ram and same hadoop-env.sh works fine.

I am starting to play with "-Xmx" putting them to HADOOP_*_OPTS in file
hadoop-env.sh.
For example, the JobTracker's log files have records about error=12 (can't
allocate mem for running "chmod"). Then i putting "-Xmx900" in
HADOOP_JOBTRACKER_OPTS and errors=12 dont appear in JobTracker's logfiles
anymore. Maybe a little :) But i don't understand, how -Xmx900 effects on
java processes, because "900 bytes" can't be a big deal) If i try to write
"-Xmx900M" then VM can't initialize again. Then i trying to write
"-Xmx128". Now JobTracker again says "error=12, memory, chmod, etc...". If
i increasing the number of megabytes, error=12 don't disappear but the
moment "cant initialize VM" comes again)
Maybe i am touching wrong option (HADOOP_JOBTRACKER_OPTS)?
Thank you.

Search Discussions

  • Sean Owen at Dec 11, 2009 at 11:53 am
    "-Xmx900" means "give the entire JVM only 900 bytes of heap space"
    which can't possibly work.

    You do not say what problem you are trying to solve here. What is the
    problem when you do not mess with these options?

    On Fri, Dec 11, 2009 at 3:05 PM, pavel kolodin
    wrote:
    Hello.
    I am running 0.20.1.
    There is an options HADOOP_*_OPTS in file hadoop-env.sh.
    When i don't put "-Xmx" in them at all, java can't initialize any VM at all.
    But this happens only on 64bit VPS with 3.5G RAM. Starting hadoop on my
    32-bit laptop with 2GB ram and same hadoop-env.sh works fine.

    I am starting to play with "-Xmx" putting them to HADOOP_*_OPTS in file
    hadoop-env.sh.
    For example, the JobTracker's log files have records about error=12 (can't
    allocate mem for running "chmod"). Then i putting "-Xmx900" in
    HADOOP_JOBTRACKER_OPTS and errors=12 dont appear in JobTracker's logfiles
    anymore. Maybe a little :) But i don't understand, how -Xmx900 effects on
    java processes, because "900 bytes" can't be a big deal) If i try to write
    "-Xmx900M" then VM can't initialize again. Then i trying to write "-Xmx128".
    Now JobTracker again says "error=12, memory, chmod, etc...". If i increasing
    the number of megabytes, error=12 don't disappear but the moment "cant
    initialize VM" comes again)
    Maybe i am touching wrong option (HADOOP_JOBTRACKER_OPTS)?
    Thank you.
  • Pavel kolodin at Dec 11, 2009 at 12:00 pm

    On Fri, 11 Dec 2009 11:52:57 -0000, Sean Owen wrote:

    Then i putting "-Xmx900" in
    HADOOP_JOBTRACKER_OPTS and errors=12 dont appear in JobTracker's
    logfiles
    anymore.


    --
    Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
  • Pavel kolodin at Dec 11, 2009 at 12:01 pm

    On Fri, 11 Dec 2009 11:52:57 -0000, Sean Owen wrote:

    "-Xmx900" means "give the entire JVM only 900 bytes of heap space"
    which can't possibly work.
    You do not say what problem you are trying to solve here. What is the
    problem when you do not mess with these options?
    As i said:
    "Then i putting "-Xmx900" in HADOOP_JOBTRACKER_OPTS and errors=12 dont
    appear in JobTracker's logfiles anymore."
    With means -Xmx900 works for some reason.

    The problem is:

    java.io.IOException: Cannot run program "chmod": java.io.IOException:
    error=12, Cannot allocate memory
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:474)
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:149)
    at org.apache.hadoop.util.Shell.run(Shell.java:134)
    at
    org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:286)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:354)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:337)
    at
    org.apache.hadoop.fs.RawLocalFileSystem.execCommand(RawLocalFileSystem.java:481)
    at
    org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:473)
    at
    org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:280)
    at
    org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:372)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:479)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:460)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:367)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:359)
    at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:186)
  • Eli Collins at Dec 15, 2009 at 9:31 am
    See HADOOP-5059.

    On Fri, Dec 11, 2009 at 7:57 AM, pavel kolodin wrote:

    On Fri, 11 Dec 2009 11:52:57 -0000, Sean Owen wrote:

    "-Xmx900" means "give the entire JVM only 900 bytes of heap space"
    which can't possibly work.
    You do not say what problem you are trying to solve here. What is the
    problem when you do not mess with these options?
    As i said:
    "Then i putting "-Xmx900" in HADOOP_JOBTRACKER_OPTS and errors=12 dont
    appear in JobTracker's logfiles anymore."
    With means -Xmx900 works for some reason.

    The problem is:


    java.io.IOException: Cannot run program "chmod": java.io.IOException:
    error=12, Cannot allocate memory
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:474)
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:149)
    at org.apache.hadoop.util.Shell.run(Shell.java:134)
    at
    org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:286)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:354)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:337)
    at
    org.apache.hadoop.fs.RawLocalFileSystem.execCommand(RawLocalFileSystem.java:481)
    at
    org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:473)
    at
    org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:280)
    at
    org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:372)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:479)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:460)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:367)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:359)
    at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:186)

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedDec 11, '09 at 11:09a
activeDec 15, '09 at 9:31a
posts5
users3
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase