FAQ
I would like to use streaming for the mapper and reducer but in such a way that I still get to do the usual Hadoop setup in Java, the usual Java arrangement, like this:

public class Coadd extends Configured implements Tool {
public int run(String[] args) throws Exception {
JobConf conf = new JobConf(getConf(), getClass());

...A BUNCH OF STUFF...

JobClient.runJob(conf);
}

public static void main(String[] args) throws Exception {
ToolRunner.run(new Coadd(), args);
}
}

How do I do this? If I specify the streaming jar as the entry point then I don't see how to maintain the ability to define my own driver.

Thanks.

________________________________________________________________________________
Keith Wiley kwiley@keithwiley.com www.keithwiley.com

"I do not feel obliged to believe that the same God who has endowed us with
sense, reason, and intellect has intended us to forgo their use."
-- Galileo Galilei
________________________________________________________________________________

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedJan 29, '11 at 9:14p
activeJan 29, '11 at 9:14p
posts1
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Keith Wiley: 1 post

People

Translate

site design / logo © 2022 Grokbase