Here are some Hadoop performance numbers.
Cluster size is ~400 nodes.
Using TestDFSIO (see HADOOP-95) I run N=1,2,4,...,512 maps.
Each map either write 320 Mb into a file or reads the same amount.
The reducer collects statistics.
Finally the following info is appended to a local file
* read or write test
* date and time the test finished
* number of files
* total number of bytes processed
* throughput in mb/sec (total number of bytes / sum of processing times)
* average IO rate in mb/sec per file
* standard IO rate deviation
The resulting file is attached.
--Konstantin
