FAQ
Hi, all

Why set mapred.reduce.tasks does not work in hive-0.70-rc1 edition?

The source code follows:

private void setNumberOfReducers() throws IOException {
// this is a temporary hack to fix things that are not fixed in the compiler
Integer numReducersFromWork = work.getNumReduceTasks();

if (work.getReducer() == null) {
console
.printInfo("Number of reduce tasks is set to 0 since there's
no reduce operator");
work.setNumReduceTasks(Integer.valueOf(0));
} else {
if (numReducersFromWork >= 0) {
console.printInfo("Number of reduce tasks determined at compile time: "
+ work.getNumReduceTasks());
} else if (job.getNumReduceTasks() > 0) {
int reducers = job.getNumReduceTasks();
work.setNumReduceTasks(reducers);
console
.printInfo("Number of reduce tasks not specified.
Defaulting to jobconf value of: "
+ reducers);
} else {
int reducers = estimateNumberOfReducers();
work.setNumReduceTasks(reducers);
console
.printInfo("Number of reduce tasks not specified.
Estimated from input data size: "
+ reducers);

}
}

I don't know the reasons. Thanks in advance.
--
Thanks,
Jander

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshive, hadoop
postedOct 10, '11 at 10:17a
activeOct 10, '11 at 10:17a
posts1
users1
websitehive.apache.org

1 user in discussion

Jander g: 1 post

People

Translate

site design / logo © 2022 Grokbase