[Hadoop-common-user] Best way to Merge small XML files

On Wed, Feb 2, 2011 at 4:04 PM, Shuja Rehman wrote:

Hi Folks,

I am having hundreds of small xml files coming each hour. The size varies
from 5 Mb to 15 Mb. As Hadoop did not work well with small files so i want
to merge these small files. So what is the best option to merge these xml
files?



--
Regards
Shuja-ur-Rehman Baig
<http://pk.linkedin.com/in/shujamughal>

Hi
You can write an InputFormat which create input splits from multiple files .
It will solve your problem.