[Hadoop-common-user] Merging many output files from reducer

The usual approach is to leave them as separate files. Often the need
to merge them into a single file is removed by using a total sort
order. Basically, that ensures that all of the keys in reduce-0 are
less than the keys in reduce-1, etc. There is a library that helps
doing that named org.apache.hadoop.mapred.lib.TotalOrderPartitioner.

-- Owen

Also, the need to merge often arises from a need to import the data into an
external database. That doesn't sound like your need because you already
know and have rejected dfs -cat.

It may help to think of the containing directory as the actual file and the
files inside that directory as no more interesting than the inodes and
blocks that make up a normal unix file.

In the example code from Pro Hadoop, is a sample map reduce job that uses
mapside join to merge the files into a single output.
It is part of the chapter 9 examples.

Thanks a lot Jason.My copy of that book is on the way..So soon I will be
able to use that.

Pankil