FAQ
The variables in the configuration files are not replaced with values, when the job is submitted by Job client
--------------------------------------------------------------------------------------------------------------

Key: HADOOP-7239
URL: https://issues.apache.org/jira/browse/HADOOP-7239
Project: Hadoop Common
Issue Type: Bug
Components: conf
Affects Versions: 0.21.0, 0.20.2
Environment: Linux + Sun JDK 1.6
Reporter: Venu Gopala Rao


We have a case where we wanted to create the Job names dynamically at run time.Since JobConf is an extension for the Configuration object, we thought we can make use of the variable substitution concept in configuration like below

Job job = new Job(conf, "${mapred.user.name}" + "-job" + new Random().nextInt());
job.setJarByClass(WordCount.class);
job.setMapperClass(TokenizerMapper.class);
job.setCombinerClass(IntSumReducer.class);
job.setReducerClass(IntSumReducer.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(IntWritable.class);
FileInputFormat.addInputPath(job, new Path(otherArgs[0]));
FileOutputFormat.setOutputPath(job, new Path(otherArgs[1]));
job.submit();


We set the required run time variables(in this case mapred.user.name) before calling the job submit. But on the Job tracker side the variables are not replaced correctly.


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 1 of 1 | next ›
Discussion Overview
groupcommon-dev @
categorieshadoop
postedApr 22, '11 at 6:01p
activeApr 22, '11 at 6:01p
posts1
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Venu Gopala Rao (JIRA): 1 post

People

Translate

site design / logo © 2022 Grokbase