My apologies for replying to the 2-year-old post.
I'm using CM 4.6, and have similar issue with Gaurav.
Harsh's hint about "Client Override" does help me understand what's it all
Now I want to set the default heap size of tasks to 1GB, and let clients
but Im not able to find Configuration -> Client on CM4.6 to set the default
Instead, I found Configuration -> Gateway having "Java Opts" related
Changing these values changes what's in the mapred-site.xml on each server,
but the jobs run from Cloudera Hue wouldn't take these settings.
So, what would be the best way to set the default heap size of tasks to
1GB, and let both HUE and CLI clients override it?
Thank you for all of your supports.
On Saturday, September 8, 2012 10:04:43 PM UTC+9, Harsh J wrote:
There is a feature in CM that lets admins hard-clamp some client-end
properties such that they are never overridable. It seems like you
have activated these. These options are marked in their names with
"(Client Override)" and are under the service settings rather than the
client ones. Please see such fields' descriptions to get a more
clearer idea for what they are for.
I'd first make sure that none of these "(Client Override)" settings
pertaining to any "Java Opts" configurations are set. After removing
all of those and setting only the configs for java opts under the
mapreduce1 -> Configuration -> Client (left sidebar classifying
option) -> Performance config set, you should not see this issue of
overriding being prevented anymore.
(You'll need to restart MR after reverting these changes)
On Thu, Sep 6, 2012 at 10:07 PM, Gaurav Dasgupta wrote:
I have CDH3 11 node cluster setup by Cloudera Manager.
In order to tune my cluster in my way, I changed the
mapred.child.java.opts, mapred.map.child.java.opts and
mapred.reduce.child.java.opts properties values to -Xmx2048m, -Xmx1024m and
-Xmx1024m respectively from CM MapReduce configuration page under the
default as well as client settings section.
Then I deployed client configuration to the cluster and restarted my CDH3
Now, when I am running any job and from JobTracker Web UI I go to the Map
Task Attempts, I can see the following warning messages:
2012-09-06 11:03:25,004 WARN org.apache.hadoop.conf.Configuration:
attempt to override final parameter: mapred.child.java.opts; Ignoring.
2012-09-06 11:03:25,006 WARN org.apache.hadoop.conf.Configuration:
attempt to override final parameter: mapred.map.child.java.opts; Ignoring.
2012-09-06 11:03:25,007 WARN org.apache.hadoop.conf.Configuration:
attempt to override final parameter: mapred.reduce.child.java.opts;
Although, in the Job.xml, I can see the values reflecting to what I set.
Is it not accepting the values I have specified? In that case, how and where
can I set these values inside CM so that it reflects to my job?
To unsubscribe from this group and stop receiving emails from it, send an email to email@example.com.