Improve namenode scalability by prioritizing datanode heartbeats over block reports

Key: HDFS-1392
URL: https://issues.apache.org/jira/browse/HDFS-1392
Project: Hadoop HDFS
Issue Type: Improvement
Components: name-node
Reporter: dhruba borthakur
Assignee: dhruba borthakur

When a namenode restarts, it gets heartbeats followed by a block reports from the datanodes. The block report processing is heavyweight and can take many many seconds, especially when the number of blocks in each datanode is large. This causes the namenode to not be able to process heartbeats in time, thus leading to "lost heartbeats from datanode". The namenode declares the datanode to be dead and the datanode has to re-send its register/block report all over again. This is a scalability problem.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouphdfs-dev @
postedSep 11, '10 at 1:24a
activeSep 11, '10 at 1:24a

1 user in discussion

dhruba borthakur (JIRA): 1 post



site design / logo © 2022 Grokbase