FAQ
Blocks are not replicated when...
---------------------------------

Key: HADOOP-109
URL: http://issues.apache.org/jira/browse/HADOOP-109
Project: Hadoop
Type: Bug
Components: dfs
Reporter: Konstantin Shvachko


When the block is under-replicated the namenode places it into
FSNamesystem.neededReplications list.
When a datanode D1 sends getBlockwork() request to the namenode, the namenode
selects another node D2 (which it thinks is up and running) where the new replica of the
under-replicated block will be stored.
Then namenode removes the block from the neededReplications list and places it to
the pendingReplications list, and then asks D1 to replicate the block to D2.
If D2 is in fact down, then replication will fail and will never be retried later, because
the block is not in the neededReplications list, but rather in the pendingReplications list,
which namenode never checks.


--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira

Search Discussions

  • Sameer Paranjpye (JIRA) at May 1, 2006 at 8:23 pm
    [ http://issues.apache.org/jira/browse/HADOOP-109?page=all ]

    Sameer Paranjpye updated HADOOP-109:
    ------------------------------------

    Version: 0.1.0
    Assign To: Konstantin Shvachko
    Blocks are not replicated when...
    ---------------------------------

    Key: HADOOP-109
    URL: http://issues.apache.org/jira/browse/HADOOP-109
    Project: Hadoop
    Type: Bug
    Components: dfs
    Versions: 0.1.0
    Reporter: Konstantin Shvachko
    Assignee: Konstantin Shvachko
    Fix For: 0.3
    When the block is under-replicated the namenode places it into
    FSNamesystem.neededReplications list.
    When a datanode D1 sends getBlockwork() request to the namenode, the namenode
    selects another node D2 (which it thinks is up and running) where the new replica of the
    under-replicated block will be stored.
    Then namenode removes the block from the neededReplications list and places it to
    the pendingReplications list, and then asks D1 to replicate the block to D2.
    If D2 is in fact down, then replication will fail and will never be retried later, because
    the block is not in the neededReplications list, but rather in the pendingReplications list,
    which namenode never checks.
    --
    This message is automatically generated by JIRA.
    -
    If you think it was sent incorrectly contact one of the administrators:
    http://issues.apache.org/jira/secure/Administrators.jspa
    -
    For more information on JIRA, see:
    http://www.atlassian.com/software/jira
  • Sameer Paranjpye (JIRA) at May 1, 2006 at 8:23 pm
    [ http://issues.apache.org/jira/browse/HADOOP-109?page=all ]

    Sameer Paranjpye updated HADOOP-109:
    ------------------------------------

    Fix Version: 0.3
    Blocks are not replicated when...
    ---------------------------------

    Key: HADOOP-109
    URL: http://issues.apache.org/jira/browse/HADOOP-109
    Project: Hadoop
    Type: Bug
    Components: dfs
    Versions: 0.1.0
    Reporter: Konstantin Shvachko
    Assignee: Konstantin Shvachko
    Fix For: 0.3
    When the block is under-replicated the namenode places it into
    FSNamesystem.neededReplications list.
    When a datanode D1 sends getBlockwork() request to the namenode, the namenode
    selects another node D2 (which it thinks is up and running) where the new replica of the
    under-replicated block will be stored.
    Then namenode removes the block from the neededReplications list and places it to
    the pendingReplications list, and then asks D1 to replicate the block to D2.
    If D2 is in fact down, then replication will fail and will never be retried later, because
    the block is not in the neededReplications list, but rather in the pendingReplications list,
    which namenode never checks.
    --
    This message is automatically generated by JIRA.
    -
    If you think it was sent incorrectly contact one of the administrators:
    http://issues.apache.org/jira/secure/Administrators.jspa
    -
    For more information on JIRA, see:
    http://www.atlassian.com/software/jira
  • Sameer Paranjpye (JIRA) at May 30, 2006 at 6:43 pm
    [ http://issues.apache.org/jira/browse/HADOOP-109?page=all ]

    Sameer Paranjpye updated HADOOP-109:
    ------------------------------------

    Fix Version: 0.4
    (was: 0.3)
    Blocks are not replicated when...
    ---------------------------------

    Key: HADOOP-109
    URL: http://issues.apache.org/jira/browse/HADOOP-109
    Project: Hadoop
    Type: Bug
    Components: dfs
    Versions: 0.1.0
    Reporter: Konstantin Shvachko
    Assignee: Konstantin Shvachko
    Fix For: 0.4
    When the block is under-replicated the namenode places it into
    FSNamesystem.neededReplications list.
    When a datanode D1 sends getBlockwork() request to the namenode, the namenode
    selects another node D2 (which it thinks is up and running) where the new replica of the
    under-replicated block will be stored.
    Then namenode removes the block from the neededReplications list and places it to
    the pendingReplications list, and then asks D1 to replicate the block to D2.
    If D2 is in fact down, then replication will fail and will never be retried later, because
    the block is not in the neededReplications list, but rather in the pendingReplications list,
    which namenode never checks.
    --
    This message is automatically generated by JIRA.
    -
    If you think it was sent incorrectly contact one of the administrators:
    http://issues.apache.org/jira/secure/Administrators.jspa
    -
    For more information on JIRA, see:
    http://www.atlassian.com/software/jira
  • Doug Cutting (JIRA) at Jun 28, 2006 at 9:13 pm
    [ http://issues.apache.org/jira/browse/HADOOP-109?page=all ]

    Doug Cutting updated HADOOP-109:
    --------------------------------

    Fix Version: 0.5.0
    (was: 0.4.0)
    Blocks are not replicated when...
    ---------------------------------

    Key: HADOOP-109
    URL: http://issues.apache.org/jira/browse/HADOOP-109
    Project: Hadoop
    Type: Bug
    Components: dfs
    Versions: 0.1.0
    Reporter: Konstantin Shvachko
    Assignee: Konstantin Shvachko
    Fix For: 0.5.0
    When the block is under-replicated the namenode places it into
    FSNamesystem.neededReplications list.
    When a datanode D1 sends getBlockwork() request to the namenode, the namenode
    selects another node D2 (which it thinks is up and running) where the new replica of the
    under-replicated block will be stored.
    Then namenode removes the block from the neededReplications list and places it to
    the pendingReplications list, and then asks D1 to replicate the block to D2.
    If D2 is in fact down, then replication will fail and will never be retried later, because
    the block is not in the neededReplications list, but rather in the pendingReplications list,
    which namenode never checks.
    --
    This message is automatically generated by JIRA.
    -
    If you think it was sent incorrectly contact one of the administrators:
    http://issues.apache.org/jira/secure/Administrators.jspa
    -
    For more information on JIRA, see:
    http://www.atlassian.com/software/jira
  • Doug Cutting (JIRA) at Feb 28, 2007 at 8:20 pm
    [ https://issues.apache.org/jira/browse/HADOOP-109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Doug Cutting resolved HADOOP-109.
    ---------------------------------

    Resolution: Fixed
    Fix Version/s: 0.12.0

    This was fixed as a part of HADOOP-940.
    Blocks are not replicated when...
    ---------------------------------

    Key: HADOOP-109
    URL: https://issues.apache.org/jira/browse/HADOOP-109
    Project: Hadoop
    Issue Type: Bug
    Components: dfs
    Affects Versions: 0.1.0
    Reporter: Konstantin Shvachko
    Assigned To: Konstantin Shvachko
    Fix For: 0.12.0


    When the block is under-replicated the namenode places it into
    FSNamesystem.neededReplications list.
    When a datanode D1 sends getBlockwork() request to the namenode, the namenode
    selects another node D2 (which it thinks is up and running) where the new replica of the
    under-replicated block will be stored.
    Then namenode removes the block from the neededReplications list and places it to
    the pendingReplications list, and then asks D1 to replicate the block to D2.
    If D2 is in fact down, then replication will fail and will never be retried later, because
    the block is not in the neededReplications list, but rather in the pendingReplications list,
    which namenode never checks.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedMar 29, '06 at 3:43a
activeFeb 28, '07 at 8:20p
posts6
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Doug Cutting (JIRA): 6 posts

People

Translate

site design / logo © 2022 Grokbase