[Hadoop-common-dev] Fw: [VOTE] Should we release 0.20.204.0rc2?

+1 (non-binding)

I downloaded the release candidate 2 onto a 12-nod cluster performed the following tests:

##################################################################
Create files using S-Live:
hadoop --config $HADOOP_CONF_DIR org.apache.hadoop.fs.slive.SliveTest -appendSize 1,67108864 -append 0,uniform -baseDir /user/$USER/S-Live -blockSize 67108864,67108864 -create 100,uniform -delete 0,uniform -dirSize 16 -duration 300 -files 1024 -ls 0,uniform -maps 20 -mkdir 0,uniform -ops 10000 -packetSize 65536 -readSize 1,4294967295 -read 0,uniform -reduces 5 -rename 0,uniform -replication 1,3 -resFile $outFile -seed 12345678 -sleep 100,1000 -writeSize 1,67108864
Output:
Basic report for operation type CreateOp
-------------
Measurement "bytes_written" = 32046140254
Measurement "milliseconds_taken" = 233234
Measurement "op_count" = 7929
Rate for measurement "bytes_written" = 131.031 MB/sec
Rate for measurement "op_count" = 33.996 operations/sec

##########################################################

Do random deletes, reads, mkdirs, lists, and renames using S-Live:
hadoop --config $HADOOP_CONF_DIR org.apache.hadoop.fs.slive.SliveTest -appendSize 1,67108864 -append 0,uniform -baseDir /user/$USER/S-Live -blockSize 67108864,67108864 -create 0,uniform -delete 20,uniform -dirSize 16 -duration 300 -files 1024 -ls 20,uniform -maps 20 -mkdir 20,uniform -ops 10000 -packetSize 65536 -readSize 1,4294967295 -read 20,uniform -reduces 5 -rename 20,uniform -replication 1,3 -resFile $outFile -seed 12345678 -sleep 100,1000 -writeSize 1,67108864
Output:
Basic report for operation type DeleteOp
-------------
Measurement "milliseconds_taken" = 6008007
Measurement "op_count" = 27148
Rate for measurement "op_count" = 4.519 operations/sec
-------------

#########################################################################

I also ran randomwriter.

Thanks,
-Eric Payne


------ Forwarded Message
From: Owen O'Malley <[email protected]>
Reply-To: "common-[email protected]" <[email protected]>
Date: Thu, 18 Aug 2011 00:28:20 -0700
To: Owen O'Malley <[email protected]>
Cc: "common-[email protected]" <[email protected]>
Subject: Re: [VOTE] Should we release 0.20.204.0rc2?

thread. Unfortunately, it had the version as 0.20.204 and didn't update the
release notes. I've updated it, run the regression tests and I think we should
release it. I've put the tarball up at:
This vote is still running with no votes other than mine.

I've tested with and without security on a 60 node cluster and I'm seeing
some failures, but not that many. On a terasort with 15,000 maps and 200
reduces, I ran the following cases:

security + linux task controller : 2 failures (both mr-2651)

no security + default task controller : 6-7 failures (seems to be a race
condition in clean up)

Even in the no security case, it is only losing 0.05% of the time.

It isn't perfect, but this is the code that Yahoo is currently running. I
think we should release it.

-- Owen

------ End of Forwarded Message