I am not sure how you can avoid the filesystem, however, I did it as follows:
// For Job 1
FileInputFormat.addInputPath(job1, new Path(args));
FileOutputFormat.setOutputPath(job1, new Path(args));
// For job 2
FileInputFormat.addInputPath(job2, new Path(args));
FileOutputFormat.setOutputPath(job2, new Path(args));
args --> Input to first mapper
args --> Output of first reducer / Input to second mapper
args --> Out of second reducer
Hope this helps!
On Tue, Sep 27, 2011 at 2:09 PM, Kevin Burton wrote:
Is it possible to connect the output of one map reduce job so that it is the
input to another map reduce job.
Basically… then reduce() outputs a key, that will be passed to another map()
function without having to store intermediate data to the filesystem.
Location: San Francisco, CA
Skype-in: (415) 871-0687