FAQ
Impact in NameNode scalability because heartbeat processing acquires the global lock
------------------------------------------------------------------------------------

Key: HADOOP-4051
URL: https://issues.apache.org/jira/browse/HADOOP-4051
Project: Hadoop Core
Issue Type: Bug
Components: dfs
Affects Versions: 0.18.0, 0.17.2, 0.17.1, 0.17.0
Reporter: dhruba borthakur


The heartbeat processing code recently got rearranged via HADOOP-3254. This caused the NameNode heartbeat processing code to acquire the FSNamesystem global lock for every heartbeat processing. This could impact scalability of the namenode.

This problem is present in 0.17.x and 0.18.x release only. It is not present in 0.16.x and on trunk.


--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Koji Noguchi (JIRA) at Aug 29, 2008 at 11:32 pm
    [ https://issues.apache.org/jira/browse/HADOOP-4051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12627148#action_12627148 ]

    Koji Noguchi commented on HADOOP-4051:
    --------------------------------------

    When taking jstack of 0.17 namenode, it showed many handlers with

    {noformat}
    "IPC Server handler 38 on 8020" daemon prio=10 tid=0x00002aaf84013000 nid=0x403b waiting for monitor entry [0x0000000044b52000..0x0000000044b52b00]
    java.lang.Thread.State: BLOCKED (on object monitor)
    at org.apache.hadoop.dfs.FSNamesystem.blockReportProcessed(FSNamesystem.java:1967)
    - waiting to lock <0x00002aab7184f418> (a org.apache.hadoop.dfs.FSNamesystem)
    at org.apache.hadoop.dfs.NameNode.sendHeartbeat(NameNode.java:571)
    at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
    {noformat}

    Impact in NameNode scalability because heartbeat processing acquires the global lock
    ------------------------------------------------------------------------------------

    Key: HADOOP-4051
    URL: https://issues.apache.org/jira/browse/HADOOP-4051
    Project: Hadoop Core
    Issue Type: Bug
    Components: dfs
    Affects Versions: 0.17.0, 0.17.1, 0.17.2, 0.18.0
    Reporter: dhruba borthakur

    The heartbeat processing code recently got rearranged via HADOOP-3254. This caused the NameNode heartbeat processing code to acquire the FSNamesystem global lock for every heartbeat processing. This could impact scalability of the namenode.
    This problem is present in 0.17.x and 0.18.x release only. It is not present in 0.16.x and on trunk.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedAug 29, '08 at 11:28p
activeAug 29, '08 at 11:32p
posts2
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Koji Noguchi (JIRA): 2 posts

People

Translate

site design / logo © 2022 Grokbase