FAQ
Performance regression bug caused by locking code
-------------------------------------------------

Key: HADOOP-7255
URL: https://issues.apache.org/jira/browse/HADOOP-7255
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 0.20.203.0
Reporter: T Jake Luciani


While testing the 0.20.203 branch we have discovered quite a severe performance regression caused by the following checkin:

http://svn.apache.org/viewvc?view=revision&revision=1079260

This locking seems to cause a pause of a few seconds between tasks, example with DFSIOTest:

*CURRENT CODE*
{noformat}
11/05/02 21:16:28 INFO fs.TestDFSIO: ----- TestDFSIO ----- : read
11/05/02 21:16:28 INFO fs.TestDFSIO: Date & time: Mon May 02 21:16:28 EDT 2011
11/05/02 21:16:28 INFO fs.TestDFSIO: Number of files: 5
11/05/02 21:16:28 INFO fs.TestDFSIO: Total MBytes processed: 500
11/05/02 21:16:28 INFO fs.TestDFSIO: Throughput mb/sec: 303.3980582524272
11/05/02 21:16:28 INFO fs.TestDFSIO: Average IO rate mb/sec: 327.69122314453125
11/05/02 21:16:28 INFO fs.TestDFSIO: IO rate std deviation: 78.95767116969083
11/05/02 21:16:28 INFO fs.TestDFSIO: Test exec time sec: 19.476
11/05/02 21:16:28 INFO fs.TestDFSIO:
{noformat}


*WITH PATCH REMOVED*
{noformat}
11/05/02 21:19:03 INFO fs.TestDFSIO: ----- TestDFSIO ----- : read
11/05/02 21:19:03 INFO fs.TestDFSIO: Date & time: Mon May 02 21:19:03 EDT 2011
11/05/02 21:19:03 INFO fs.TestDFSIO: Number of files: 5
11/05/02 21:19:03 INFO fs.TestDFSIO: Total MBytes processed: 500
11/05/02 21:19:03 INFO fs.TestDFSIO: Throughput mb/sec: 366.03221083455344
11/05/02 21:19:03 INFO fs.TestDFSIO: Average IO rate mb/sec: 366.35528564453125
11/05/02 21:19:03 INFO fs.TestDFSIO: IO rate std deviation: 11.020931463080379
11/05/02 21:19:03 INFO fs.TestDFSIO: Test exec time sec: 3.543
11/05/02 21:19:03 INFO fs.TestDFSIO:
{noformat}


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedMay 3, '11 at 1:50a
activeMay 3, '11 at 1:50a
posts1
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

T Jake Luciani (JIRA): 1 post

People

Translate

site design / logo © 2022 Grokbase