FAQ
+1 (non-binding)

I downloaded the release candidate 2 onto a 12-nod cluster performed the following tests:

##################################################################
Create files using S-Live:
hadoop --config $HADOOP_CONF_DIR org.apache.hadoop.fs.slive.SliveTest -appendSize 1,67108864 -append 0,uniform -baseDir /user/$USER/S-Live -blockSize 67108864,67108864 -create 100,uniform -delete 0,uniform -dirSize 16 -duration 300 -files 1024 -ls 0,uniform -maps 20 -mkdir 0,uniform -ops 10000 -packetSize 65536 -readSize 1,4294967295 -read 0,uniform -reduces 5 -rename 0,uniform -replication 1,3 -resFile $outFile -seed 12345678 -sleep 100,1000 -writeSize 1,67108864
Output:
Basic report for operation type CreateOp
-------------
Measurement "bytes_written" = 32046140254
Measurement "milliseconds_taken" = 233234
Measurement "op_count" = 7929
Rate for measurement "bytes_written" = 131.031 MB/sec
Rate for measurement "op_count" = 33.996 operations/sec

##########################################################

Do random deletes, reads, mkdirs, lists, and renames using S-Live:
hadoop --config $HADOOP_CONF_DIR org.apache.hadoop.fs.slive.SliveTest -appendSize 1,67108864 -append 0,uniform -baseDir /user/$USER/S-Live -blockSize 67108864,67108864 -create 0,uniform -delete 20,uniform -dirSize 16 -duration 300 -files 1024 -ls 20,uniform -maps 20 -mkdir 20,uniform -ops 10000 -packetSize 65536 -readSize 1,4294967295 -read 20,uniform -reduces 5 -rename 20,uniform -replication 1,3 -resFile $outFile -seed 12345678 -sleep 100,1000 -writeSize 1,67108864
Output:
Basic report for operation type DeleteOp
-------------
Measurement "milliseconds_taken" = 6008007
Measurement "op_count" = 27148
Rate for measurement "op_count" = 4.519 operations/sec
-------------

#########################################################################

I also ran randomwriter.

Thanks,
-Eric Payne


------ Forwarded Message
From: Owen O'Malley <owen@hortonworks.com>
Reply-To: "common-dev@hadoop.apache.org" <common-dev@hadoop.apache.org>
Date: Thu, 18 Aug 2011 00:28:20 -0700
To: Owen O'Malley <owen@hortonworks.com>
Cc: "common-dev@hadoop.apache.org" <common-dev@hadoop.apache.org>
Subject: Re: [VOTE] Should we release 0.20.204.0rc2?

On Aug 9, 2011, at 8:55 AM, Owen O'Malley wrote:

All,
Matt rolled a 0.20.204.0rc1, but I think it got lost in the previous vote
thread. Unfortunately, it had the version as 0.20.204 and didn't update the
release notes. I've updated it, run the regression tests and I think we should
release it. I've put the tarball up at:
This vote is still running with no votes other than mine.

I've tested with and without security on a 60 node cluster and I'm seeing
some failures, but not that many. On a terasort with 15,000 maps and 200
reduces, I ran the following cases:

security + linux task controller : 2 failures (both mr-2651)

no security + default task controller : 6-7 failures (seems to be a race
condition in clean up)

Even in the no security case, it is only losing 0.05% of the time.

It isn't perfect, but this is the code that Yahoo is currently running. I
think we should release it.

-- Owen

------ End of Forwarded Message

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedAug 22, '11 at 5:57p
activeAug 22, '11 at 5:57p
posts1
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Eric Payne: 1 post

People

Translate

site design / logo © 2022 Grokbase