FAQ
findbugs : fix "Inconsistent Synchronization" warnings in hdfs
---------------------------------------------------------------

Key: HADOOP-5854
URL: https://issues.apache.org/jira/browse/HADOOP-5854
Project: Hadoop Core
Issue Type: Improvement
Components: dfs
Reporter: Raghu Angadi
Assignee: Raghu Angadi


This jira fixes the following findbugs warnings :

* Inconsistent synchronization of org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.closed; locked 75% of time
* Inconsistent synchronization of org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.persistBlocks; locked 66% of time
* Inconsistent synchronization of org.apache.hadoop.hdfs.server.common.UpgradeManager.currentUpgrades; locked 61% of time
* Inconsistent synchronization of org.apache.hadoop.hdfs.server.common.UpgradeManager.upgradeState; locked 75% of time
* Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSDirectory.ready; locked 50% of time
* Inconsistent synchronization of org.apache.hadoop.hdfs.server.datanode.DataBlockScanner.verificationLog; locked 45% of time
* Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSNamesystem.safeMode; locked 48% of time
* Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSNamesystem$SafeModeInfo.extension; locked 80% of time
* Inconsistent synchronization of org.apache.hadoop.io.SequenceFile$Reader.sync; locked 85% of time

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Raghu Angadi (JIRA) at May 15, 2009 at 10:32 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Raghu Angadi updated HADOOP-5854:
    ---------------------------------

    Attachment: HADOOP-5854.patch

    patch attached. Fixes are of two types :

    * to synchronize properly (or explicitly so that findbugs recognizes)
    * to declare a variable volatile : these warnings are known and are justified in various other jiras. Usually either the current usage is fine or fixing it is not worth the effort.
    findbugs : fix "Inconsistent Synchronization" warnings in hdfs
    ---------------------------------------------------------------

    Key: HADOOP-5854
    URL: https://issues.apache.org/jira/browse/HADOOP-5854
    Project: Hadoop Core
    Issue Type: Improvement
    Components: dfs
    Reporter: Raghu Angadi
    Assignee: Raghu Angadi
    Attachments: HADOOP-5854.patch


    This jira fixes the following findbugs warnings :
    * Inconsistent synchronization of org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.closed; locked 75% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.persistBlocks; locked 66% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.common.UpgradeManager.currentUpgrades; locked 61% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.common.UpgradeManager.upgradeState; locked 75% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSDirectory.ready; locked 50% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.datanode.DataBlockScanner.verificationLog; locked 45% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSNamesystem.safeMode; locked 48% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSNamesystem$SafeModeInfo.extension; locked 80% of time
    * Inconsistent synchronization of org.apache.hadoop.io.SequenceFile$Reader.sync; locked 85% of time
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Raghu Angadi (JIRA) at May 15, 2009 at 10:51 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Raghu Angadi updated HADOOP-5854:
    ---------------------------------

    Fix Version/s: 0.21.0
    findbugs : fix "Inconsistent Synchronization" warnings in hdfs
    ---------------------------------------------------------------

    Key: HADOOP-5854
    URL: https://issues.apache.org/jira/browse/HADOOP-5854
    Project: Hadoop Core
    Issue Type: Improvement
    Components: dfs
    Reporter: Raghu Angadi
    Assignee: Raghu Angadi
    Fix For: 0.21.0

    Attachments: HADOOP-5854.patch


    This jira fixes the following findbugs warnings :
    * Inconsistent synchronization of org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.closed; locked 75% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.persistBlocks; locked 66% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.common.UpgradeManager.currentUpgrades; locked 61% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.common.UpgradeManager.upgradeState; locked 75% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSDirectory.ready; locked 50% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.datanode.DataBlockScanner.verificationLog; locked 45% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSNamesystem.safeMode; locked 48% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSNamesystem$SafeModeInfo.extension; locked 80% of time
    * Inconsistent synchronization of org.apache.hadoop.io.SequenceFile$Reader.sync; locked 85% of time
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Suresh Srinivas (JIRA) at May 15, 2009 at 11:11 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710011#action_12710011 ]

    Suresh Srinivas commented on HADOOP-5854:
    -----------------------------------------

    This is a good change. Comments:
    # {{isInSafeMode()}} and {{safemode}} access with synchronize(this) surrounding it, can be moved out of the synchronized block
    # {{FSDirectory.waitForReady()}} need not have {{synchronized(this)}}
    # Should {{DFSClient.createBlockOutputStream()}} where {{persistBlocks}} is set to true be synchornized? Not sure how findbugs can ignore this case by just setting the variable to volatile!

    findbugs : fix "Inconsistent Synchronization" warnings in hdfs
    ---------------------------------------------------------------

    Key: HADOOP-5854
    URL: https://issues.apache.org/jira/browse/HADOOP-5854
    Project: Hadoop Core
    Issue Type: Improvement
    Components: dfs
    Reporter: Raghu Angadi
    Assignee: Raghu Angadi
    Fix For: 0.21.0

    Attachments: HADOOP-5854.patch


    This jira fixes the following findbugs warnings :
    * Inconsistent synchronization of org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.closed; locked 75% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.persistBlocks; locked 66% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.common.UpgradeManager.currentUpgrades; locked 61% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.common.UpgradeManager.upgradeState; locked 75% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSDirectory.ready; locked 50% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.datanode.DataBlockScanner.verificationLog; locked 45% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSNamesystem.safeMode; locked 48% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSNamesystem$SafeModeInfo.extension; locked 80% of time
    * Inconsistent synchronization of org.apache.hadoop.io.SequenceFile$Reader.sync; locked 85% of time
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Raghu Angadi (JIRA) at May 15, 2009 at 11:47 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710032#action_12710032 ]

    Raghu Angadi commented on HADOOP-5854:
    --------------------------------------

    1. not sure
    2. It needs to synchronize since it is going wait on the object. This member is written so that common case (ready is true) does not need any synchronization.
    3. It is under dataQueue lock. findbugs does not detect locks in the callers. But the problem is that other access in fsync() is under a different lock. It can't be fixed easily.
    findbugs : fix "Inconsistent Synchronization" warnings in hdfs
    ---------------------------------------------------------------

    Key: HADOOP-5854
    URL: https://issues.apache.org/jira/browse/HADOOP-5854
    Project: Hadoop Core
    Issue Type: Improvement
    Components: dfs
    Reporter: Raghu Angadi
    Assignee: Raghu Angadi
    Fix For: 0.21.0

    Attachments: HADOOP-5854.patch


    This jira fixes the following findbugs warnings :
    * Inconsistent synchronization of org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.closed; locked 75% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.persistBlocks; locked 66% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.common.UpgradeManager.currentUpgrades; locked 61% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.common.UpgradeManager.upgradeState; locked 75% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSDirectory.ready; locked 50% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.datanode.DataBlockScanner.verificationLog; locked 45% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSNamesystem.safeMode; locked 48% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSNamesystem$SafeModeInfo.extension; locked 80% of time
    * Inconsistent synchronization of org.apache.hadoop.io.SequenceFile$Reader.sync; locked 85% of time
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Suresh Srinivas (JIRA) at May 16, 2009 at 12:24 am
    [ https://issues.apache.org/jira/browse/HADOOP-5854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710037#action_12710037 ]

    Suresh Srinivas commented on HADOOP-5854:
    -----------------------------------------

    After speaking to Raghu, the code changes makes sense. In case of (3), current change will solve findbugs warnings and will not make other changes needed for correct synchronization. +1.
    findbugs : fix "Inconsistent Synchronization" warnings in hdfs
    ---------------------------------------------------------------

    Key: HADOOP-5854
    URL: https://issues.apache.org/jira/browse/HADOOP-5854
    Project: Hadoop Core
    Issue Type: Improvement
    Components: dfs
    Reporter: Raghu Angadi
    Assignee: Raghu Angadi
    Fix For: 0.21.0

    Attachments: HADOOP-5854.patch


    This jira fixes the following findbugs warnings :
    * Inconsistent synchronization of org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.closed; locked 75% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.persistBlocks; locked 66% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.common.UpgradeManager.currentUpgrades; locked 61% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.common.UpgradeManager.upgradeState; locked 75% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSDirectory.ready; locked 50% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.datanode.DataBlockScanner.verificationLog; locked 45% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSNamesystem.safeMode; locked 48% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSNamesystem$SafeModeInfo.extension; locked 80% of time
    * Inconsistent synchronization of org.apache.hadoop.io.SequenceFile$Reader.sync; locked 85% of time
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Raghu Angadi (JIRA) at May 16, 2009 at 1:31 am
    [ https://issues.apache.org/jira/browse/HADOOP-5854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710048#action_12710048 ]

    Raghu Angadi commented on HADOOP-5854:
    --------------------------------------

    ant test-patch :

    {noformat}
    [exec] +1 @author. The patch does not contain any @author tags.
    [exec]
    [exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
    [exec] Please justify why no tests are needed for this patch.
    [exec]
    [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
    [exec]
    [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
    [exec]
    [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
    [exec]
    [exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
    [exec]
    [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
    [exec]
    {noformat}
    findbugs : fix "Inconsistent Synchronization" warnings in hdfs
    ---------------------------------------------------------------

    Key: HADOOP-5854
    URL: https://issues.apache.org/jira/browse/HADOOP-5854
    Project: Hadoop Core
    Issue Type: Improvement
    Components: dfs
    Reporter: Raghu Angadi
    Assignee: Raghu Angadi
    Fix For: 0.21.0

    Attachments: HADOOP-5854.patch


    This jira fixes the following findbugs warnings :
    * Inconsistent synchronization of org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.closed; locked 75% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.persistBlocks; locked 66% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.common.UpgradeManager.currentUpgrades; locked 61% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.common.UpgradeManager.upgradeState; locked 75% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSDirectory.ready; locked 50% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.datanode.DataBlockScanner.verificationLog; locked 45% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSNamesystem.safeMode; locked 48% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSNamesystem$SafeModeInfo.extension; locked 80% of time
    * Inconsistent synchronization of org.apache.hadoop.io.SequenceFile$Reader.sync; locked 85% of time
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Raghu Angadi (JIRA) at May 16, 2009 at 1:31 am
    [ https://issues.apache.org/jira/browse/HADOOP-5854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710048#action_12710048 ]

    Raghu Angadi edited comment on HADOOP-5854 at 5/15/09 6:29 PM:
    ---------------------------------------------------------------

    Thanks Suresh.

    ant test-patch :

    {noformat}
    [exec] +1 @author. The patch does not contain any @author tags.
    [exec]
    [exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
    [exec] Please justify why no tests are needed for this patch.
    [exec]
    [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
    [exec]
    [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
    [exec]
    [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
    [exec]
    [exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
    [exec]
    [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
    [exec]
    {noformat}

    was (Author: rangadi):
    ant test-patch :

    {noformat}
    [exec] +1 @author. The patch does not contain any @author tags.
    [exec]
    [exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
    [exec] Please justify why no tests are needed for this patch.
    [exec]
    [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
    [exec]
    [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
    [exec]
    [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
    [exec]
    [exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
    [exec]
    [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
    [exec]
    {noformat}
    findbugs : fix "Inconsistent Synchronization" warnings in hdfs
    ---------------------------------------------------------------

    Key: HADOOP-5854
    URL: https://issues.apache.org/jira/browse/HADOOP-5854
    Project: Hadoop Core
    Issue Type: Improvement
    Components: dfs
    Reporter: Raghu Angadi
    Assignee: Raghu Angadi
    Fix For: 0.21.0

    Attachments: HADOOP-5854.patch


    This jira fixes the following findbugs warnings :
    * Inconsistent synchronization of org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.closed; locked 75% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.persistBlocks; locked 66% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.common.UpgradeManager.currentUpgrades; locked 61% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.common.UpgradeManager.upgradeState; locked 75% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSDirectory.ready; locked 50% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.datanode.DataBlockScanner.verificationLog; locked 45% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSNamesystem.safeMode; locked 48% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSNamesystem$SafeModeInfo.extension; locked 80% of time
    * Inconsistent synchronization of org.apache.hadoop.io.SequenceFile$Reader.sync; locked 85% of time
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Raghu Angadi (JIRA) at May 16, 2009 at 1:35 am
    [ https://issues.apache.org/jira/browse/HADOOP-5854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Raghu Angadi resolved HADOOP-5854.
    ----------------------------------

    Resolution: Fixed
    Hadoop Flags: [Reviewed]

    I just committed this.
    findbugs : fix "Inconsistent Synchronization" warnings in hdfs
    ---------------------------------------------------------------

    Key: HADOOP-5854
    URL: https://issues.apache.org/jira/browse/HADOOP-5854
    Project: Hadoop Core
    Issue Type: Improvement
    Components: dfs
    Reporter: Raghu Angadi
    Assignee: Raghu Angadi
    Fix For: 0.21.0

    Attachments: HADOOP-5854.patch


    This jira fixes the following findbugs warnings :
    * Inconsistent synchronization of org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.closed; locked 75% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.persistBlocks; locked 66% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.common.UpgradeManager.currentUpgrades; locked 61% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.common.UpgradeManager.upgradeState; locked 75% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSDirectory.ready; locked 50% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.datanode.DataBlockScanner.verificationLog; locked 45% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSNamesystem.safeMode; locked 48% of time
    * Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.FSNamesystem$SafeModeInfo.extension; locked 80% of time
    * Inconsistent synchronization of org.apache.hadoop.io.SequenceFile$Reader.sync; locked 85% of time
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedMay 15, '09 at 10:14p
activeMay 16, '09 at 1:35a
posts9
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Raghu Angadi (JIRA): 9 posts

People

Translate

site design / logo © 2022 Grokbase