Hi Everyone-
I'm not sure if this is the best place for this but figured I'd try.
I'm looking for TestDFSIO read and write test results to compare to my
cluster's output.
Cluster setup 4 datanodes, 8cores each, 16GB ram each.
My Tests
Write 50 files 128 mb each
12/12/05 12:54:12 INFO fs.TestDFSIO: ----- TestDFSIO ----- : write
12/12/05 12:54:12 INFO fs.TestDFSIO: Date & time: Wed Dec 05
12:54:12 EST 2012
12/12/05 12:54:12 INFO fs.TestDFSIO: Number of files: 50
12/12/05 12:54:12 INFO fs.TestDFSIO: Total MBytes processed: 6400.0
12/12/05 12:54:12 INFO fs.TestDFSIO: Throughput mb/sec:
0.8708652440810759
12/12/05 12:54:12 INFO fs.TestDFSIO: Average IO rate mb/sec:
0.9578074812889099
12/12/05 12:54:12 INFO fs.TestDFSIO: IO rate std deviation:
0.3024037240254335
12/12/05 12:54:12 INFO fs.TestDFSIO: Test exec time sec: 305.269
Read 50 files 128 mb each
12/12/05 17:36:18 INFO fs.TestDFSIO: ----- TestDFSIO ----- : read
12/12/05 17:36:18 INFO fs.TestDFSIO: Date & time: Wed Dec 05
17:36:18 EST 2012
12/12/05 17:36:18 INFO fs.TestDFSIO: Number of files: 50
12/12/05 17:36:18 INFO fs.TestDFSIO: Total MBytes processed: 6400.0
12/12/05 17:36:18 INFO fs.TestDFSIO: Throughput mb/sec:
20.35442136196522
12/12/05 17:36:18 INFO fs.TestDFSIO: Average IO rate mb/sec:
197.49246215820312
12/12/05 17:36:18 INFO fs.TestDFSIO: IO rate std deviation:
111.96049826586476
12/12/05 17:36:18 INFO fs.TestDFSIO: Test exec time sec: 54.172
12/12/05 17:38:38 INFO fs.TestDFSIO: ----- TestDFSIO ----- : read
12/12/05 17:38:38 INFO fs.TestDFSIO: Date & time: Wed Dec 05
17:38:38 EST 2012
12/12/05 17:38:38 INFO fs.TestDFSIO: Number of files: 50
12/12/05 17:38:38 INFO fs.TestDFSIO: Total MBytes processed: 6400.0
12/12/05 17:38:38 INFO fs.TestDFSIO: Throughput mb/sec:
12.805890709726475
12/12/05 17:38:38 INFO fs.TestDFSIO: Average IO rate mb/sec:
180.38516235351562
12/12/05 17:38:38 INFO fs.TestDFSIO: IO rate std deviation:
129.25890715032295
12/12/05 17:38:38 INFO fs.TestDFSIO: Test exec time sec: 64.741
12/12/05 17:40:30 INFO fs.TestDFSIO: ----- TestDFSIO ----- : read
12/12/05 17:40:30 INFO fs.TestDFSIO: Date & time: Wed Dec 05
17:40:30 EST 2012
12/12/05 17:40:30 INFO fs.TestDFSIO: Number of files: 50
12/12/05 17:40:30 INFO fs.TestDFSIO: Total MBytes processed: 6400.0
12/12/05 17:40:30 INFO fs.TestDFSIO: Throughput mb/sec:
15.852374760047061
12/12/05 17:40:30 INFO fs.TestDFSIO: Average IO rate mb/sec:
181.2574005126953
12/12/05 17:40:30 INFO fs.TestDFSIO: IO rate std deviation:
104.20816041404993
12/12/05 17:40:30 INFO fs.TestDFSIO: Test exec time sec: 61.045
One thing I noticed that when running the read test over and over I got
fairly different results.
If you could please post your results and cluster size to allow me to
compare it would be greatly appreciated.
--
