FAQ
Here are some Hadoop performance numbers.
Cluster size is ~400 nodes.
Using TestDFSIO (see HADOOP-95) I run N=1,2,4,...,512 maps.
Each map either write 320 Mb into a file or reads the same amount.
The reducer collects statistics.
Finally the following info is appended to a local file

* read or write test
* date and time the test finished
* number of files
* total number of bytes processed
* throughput in mb/sec (total number of bytes / sum of processing times)
* average IO rate in mb/sec per file
* standard IO rate deviation

The resulting file is attached.

--Konstantin

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedApr 14, '06 at 2:19a
activeApr 14, '06 at 2:19a
posts1
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Konstantin Shvachko: 1 post

People

Translate

site design / logo © 2022 Grokbase