FAQ
Reduce NN startup time by reducing the processing time of block reports
-----------------------------------------------------------------------

Key: HDFS-1147
URL: https://issues.apache.org/jira/browse/HDFS-1147
Project: Hadoop HDFS
Issue Type: Improvement
Components: name-node
Reporter: dhruba borthakur


The NameNode restart times are impacted to a large extent by the processing time of block reports. For a cluster with 150 millions blocks, the block report processing in the NN can take upto 20 minutes or so. The NN is open for business only after it has processed the block reports from most of the datanodes. If we can reduce the processing time of a block report, that will directly reduce the restart time of the NN.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • dhruba borthakur (JIRA) at Sep 17, 2010 at 1:56 am
    [ https://issues.apache.org/jira/browse/HDFS-1147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    dhruba borthakur resolved HDFS-1147.
    ------------------------------------

    Resolution: Duplicate
    Reduce NN startup time by reducing the processing time of block reports
    -----------------------------------------------------------------------

    Key: HDFS-1147
    URL: https://issues.apache.org/jira/browse/HDFS-1147
    Project: Hadoop HDFS
    Issue Type: Improvement
    Components: name-node
    Reporter: dhruba borthakur
    Assignee: dhruba borthakur

    The NameNode restart times are impacted to a large extent by the processing time of block reports. For a cluster with 150 millions blocks, the block report processing in the NN can take upto 20 minutes or so. The NN is open for business only after it has processed the block reports from most of the datanodes. If we can reduce the processing time of a block report, that will directly reduce the restart time of the NN.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouphdfs-dev @
categorieshadoop
postedMay 12, '10 at 5:23a
activeSep 17, '10 at 1:56a
posts2
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

dhruba borthakur (JIRA): 2 posts

People

Translate

site design / logo © 2022 Grokbase