FAQ
Make it possible for BlockPlacementPolicy to return null
--------------------------------------------------------

Key: HDFS-1351
URL: https://issues.apache.org/jira/browse/HDFS-1351
Project: Hadoop HDFS
Issue Type: Test
Components: name-node
Affects Versions: 0.22.0
Reporter: Dmytro Molkov


The idea is to modify FSNamesystem.chooseExcessReplicates code, so it can accept a null return from chooseReplicaToDelete which will indicate that NameNode should not be deleting extra replicas.

One possible usecase - if there are nodes being added to the cluster that might have corrupt replicas on them you do not want to delete other replicas until the block scanner finished scanning every block on the datanode.

This will require additional work on the implementation of the BlockPlacementPolicy, but with this JIRA I just wanted to create a basis for future improvements.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Dmytro Molkov (JIRA) at Aug 30, 2010 at 11:55 pm
    [ https://issues.apache.org/jira/browse/HDFS-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Dmytro Molkov resolved HDFS-1351.
    ---------------------------------

    Resolution: Invalid

    Sorry, after talking with Hairong I realized that it will not be possible to make the fix this easy. The reason return value cannot be null is that at this point NameNode knows it has to delete extra replica of the block. If we skip this deletion it will not know the block has extra replicas until the next time it does full rescan (on restart). So this jira is itself invalid.
    Make it possible for BlockPlacementPolicy to return null
    --------------------------------------------------------

    Key: HDFS-1351
    URL: https://issues.apache.org/jira/browse/HDFS-1351
    Project: Hadoop HDFS
    Issue Type: Test
    Components: name-node
    Affects Versions: 0.22.0
    Reporter: Dmytro Molkov
    Assignee: Dmytro Molkov
    Attachments: HDFS-1351.patch


    The idea is to modify FSNamesystem.chooseExcessReplicates code, so it can accept a null return from chooseReplicaToDelete which will indicate that NameNode should not be deleting extra replicas.
    One possible usecase - if there are nodes being added to the cluster that might have corrupt replicas on them you do not want to delete other replicas until the block scanner finished scanning every block on the datanode.
    This will require additional work on the implementation of the BlockPlacementPolicy, but with this JIRA I just wanted to create a basis for future improvements.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouphdfs-dev @
categorieshadoop
postedAug 24, '10 at 7:25a
activeAug 30, '10 at 11:55p
posts2
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Dmytro Molkov (JIRA): 2 posts

People

Translate

site design / logo © 2022 Grokbase