FAQ
Hey Harsh,

The NodeManager's yarn.nodemanager.resource.memory-mb setting is 8192. But
on that same row under 8192, it says "Overridden by 1 instance(s) 1627
(NODEMANAGER cdh4-cm)"

What does that mean, that it's overridden? Overridden by who?

That's a great recommendation from you to try lowering the amount of RAM
for the YARN processes. I just tried the command you sent me, but
unfortunately it still doesn't seem to have worked.

It's throwing this weird memory message:
12/08/28 07:15:57 INFO mapreduce.Job: Job job_1346098903206_0002 failed
with state FAILED due to: Application application_1346098903206_0002 failed
1 times due to AM Container for appattempt_1346098903206_0002_000001 exited
with exitCode: 143 due to: Container
[pid=2585,containerID=container_1346098903206_0002_01_000001] is running
beyond virtual memory limits. Current usage: 55.0mb of 384.0mb physical
memory used; 1.9gb of 806.4mb virtual memory used. Killing container.


The command I ran:

[root@cdh4-cm ~]# *hadoop jar
/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar grep
-Dyarn.app.mapreduce.am.resource.mb=300 -Dmapreduce.map.memory.mb=300
-Dmapreduce.reduce.memory.mb=300 -Dmapred.child.java.opts=-Xmx200m
yarn_input yarn_output 'dfs[a-z.]+'*

12/08/28 07:15:53 WARN mapreduce.JobSubmitter: No job jar file set. User
classes may not be found. See Job or Job#setJar(String).
12/08/28 07:15:53 INFO input.FileInputFormat: Total input paths to process
: 3
12/08/28 07:15:53 INFO util.NativeCodeLoader: Loaded the native-hadoop
library
12/08/28 07:15:53 WARN snappy.LoadSnappy: Snappy native library is available
12/08/28 07:15:53 INFO snappy.LoadSnappy: Snappy native library loaded
12/08/28 07:15:53 INFO mapreduce.JobSubmitter: number of splits:3
12/08/28 07:15:53 WARN conf.Configuration: mapred.output.value.class is
deprecated. Instead, use mapreduce.job.output.value.class
12/08/28 07:15:53 WARN conf.Configuration: mapreduce.combine.class is
deprecated. Instead, use mapreduce.job.combine.class
12/08/28 07:15:53 WARN conf.Configuration: mapreduce.map.class is
deprecated. Instead, use mapreduce.job.map.class
12/08/28 07:15:53 WARN conf.Configuration: mapred.job.name is deprecated.
Instead, use mapreduce.job.name
12/08/28 07:15:53 WARN conf.Configuration: mapreduce.reduce.class is
deprecated. Instead, use mapreduce.job.reduce.class
12/08/28 07:15:53 WARN conf.Configuration: mapred.input.dir is deprecated.
Instead, use mapreduce.input.fileinputformat.inputdir
12/08/28 07:15:53 WARN conf.Configuration: mapred.output.dir is deprecated.
Instead, use mapreduce.output.fileoutputformat.outputdir
12/08/28 07:15:53 WARN conf.Configuration: mapreduce.outputformat.class is
deprecated. Instead, use mapreduce.job.outputformat.class
12/08/28 07:15:53 WARN conf.Configuration: mapred.map.tasks is deprecated.
Instead, use mapreduce.job.maps
12/08/28 07:15:53 WARN conf.Configuration: mapred.output.key.class is
deprecated. Instead, use mapreduce.job.output.key.class
12/08/28 07:15:53 WARN conf.Configuration: mapred.working.dir is
deprecated. Instead, use mapreduce.job.working.dir
12/08/28 07:15:54 INFO mapred.YARNRunner: Job jar is not present. Not
adding any jar to the list of resources.
12/08/28 07:15:54 INFO mapred.ResourceMgrDelegate: Submitted application
application_1346098903206_0002 to ResourceManager at
cdh4-cm-vm0/108.166.x.x:8032
12/08/28 07:15:54 INFO mapreduce.Job: The url to track the job:
http://cdh4-cm-vm0:8088/proxy/application_1346098903206_0002/
12/08/28 07:15:54 INFO mapreduce.Job: Running job: job_1346098903206_0002
12/08/28 07:15:57 INFO mapreduce.Job: Job job_1346098903206_0002 running in
uber mode : false
12/08/28 07:15:57 INFO mapreduce.Job: map 0% reduce 0%
12/08/28 07:15:57 INFO mapreduce.Job: Job job_1346098903206_0002 failed
with state FAILED due to: Application application_1346098903206_0002 failed
1 times due to AM Container for appattempt_1346098903206_0002_000001 exited
with exitCode: 143 due to: Container
[pid=2585,containerID=container_1346098903206_0002_01_000001] is running
beyond virtual memory limits. Current usage: 55.0mb of 384.0mb physical
memory used; 1.9gb of 806.4mb virtual memory used. Killing container.
Dump of the process-tree for container_1346098903206_0002_01_000001 :
- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS)
SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
- 2585 3922 2585 2585 (bash) 0 2 110821376 354 /bin/bash -c
/usr/java/jdk1.6.0_31/bin/java
-Dlog4j.configuration=container-log4j.properties
-Dyarn.app.mapreduce.container.log.dir=/var/log/hadoop-yarn/container/application_1346098903206_0002/container_1346098903206_0002_01_000001
-Dyarn.app.mapreduce.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA
  -Xmx1073741824 org.apache.hadoop.mapreduce.v2.app.MRAppMaster
1>/var/log/hadoop-yarn/container/application_1346098903206_0002/container_1346098903206_0002_01_000001/stdout
2>/var/log/hadoop-yarn/container/application_1346098903206_0002/container_1346098903206_0002_01_000001/stderr
- 2595 2585 2585 2585 (java) 171 20 1914224640 13734
/usr/java/jdk1.6.0_31/bin/java
-Dlog4j.configuration=container-log4j.properties
-Dyarn.app.mapreduce.container.log.dir=/var/log/hadoop-yarn/container/application_1346098903206_0002/container_1346098903206_0002_01_000001
-Dyarn.app.mapreduce.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA
-Xmx1073741824 org.apache.hadoop.mapreduce.v2.app.MRAppMaster


.Failing this attempt.. Failing the application.
12/08/28 07:15:57 INFO mapreduce.Job: Counters: 0
12/08/28 07:15:57 WARN mapreduce.JobSubmitter: No job jar file set. User
classes may not be found. See Job or Job#setJar(String).
12/08/28 07:15:57 INFO mapreduce.JobSubmitter: Cleaning up the staging area
/user/root/.staging/job_1346098903206_0003
12/08/28 07:15:57 ERROR security.UserGroupInformation:
PriviledgedActionException as:root (auth:SIMPLE)
cause:org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input
path does not exist: hdfs://cdh4-cm-vm0:8020/user/root/grep-temp-833820771
org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path
does not exist: hdfs://cdh4-cm-vm0:8020/user/root/grep-temp-833820771
         at
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:243)
         at
org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat.listStatus(SequenceFileInputFormat.java:59)
         at
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
         at
org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:449)
         at
org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:466)
         at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:359)
         at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1226)
         at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1223)
         at java.security.AccessController.doPrivileged(Native Method)
         at javax.security.auth.Subject.doAs(Subject.java:396)
         at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
         at org.apache.hadoop.mapreduce.Job.submit(Job.java:1223)
         at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1244)
         at org.apache.hadoop.examples.Grep.run(Grep.java:92)
         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
         at org.apache.hadoop.examples.Grep.main(Grep.java:101)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
         at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:597)
         at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
         at
org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
         at
org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
         at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:597)
         at org.apache.hadoop.util.RunJar.main(RunJar.java:208)

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 7 of 9 | next ›
Discussion Overview
groupscm-users @
categorieshadoop
postedAug 27, '12 at 7:03p
activeAug 28, '12 at 6:57p
posts9
users3
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase