Does someone has some data regarding maximum possible number of files over HDFS ?
my second question is, I created small files with small block size up to one lac and read the files from HDFS, reading performance remains almost unaffected with increasing number of files.
The possible reasons I could think are:
1 . One lac isn't a big number to disturb HDFS performance (I used 1 namenode and 4 data nodes)
2. As reading is done directly from datanode with first time interaction with namenode, so reading from different nodes doesn't affect the performance.
If someone could add or negate some information it will be highly appreciated.