[Hadoop-common-dev] [jira] Created: (HADOOP-4598) '-setrep' command skips under-replicated blocks

[ https://issues.apache.org/jira/browse/HADOOP-4598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hairong Kuang reassigned HADOOP-4598:
-------------------------------------

Assignee: Hairong Kuang

[ https://issues.apache.org/jira/browse/HADOOP-4598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12647709#action_12647709 ]

Hairong Kuang commented on HADOOP-4598:
---------------------------------------

UnderReplicatedBlocks.update has an optimization that a under-replication block does not get added to the queue if a block was under-replicated and its previous priority is the same as the current priority. This optimization is based on the assumption that a previously under-replicated block should have already been in the queue.

For the case that Koji filed, due to for some bugs like HADOOP-4597, under-replicated blocks did not get to place in the under-replicated block queue at NameNode startup time. Later on changing a file's replication factor to be a bigger number does not trigger the move of the file's blocks into the queue if the replication factor change does not change their priority.

The fix for this problem is to remove the optimization.

[ https://issues.apache.org/jira/browse/HADOOP-4598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hairong Kuang updated HADOOP-4598:
----------------------------------

Attachment: setRep.patch

A patch for review.

[ https://issues.apache.org/jira/browse/HADOOP-4598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12648836#action_12648836 ]

Konstantin Shvachko commented on HADOOP-4598:
---------------------------------------------

+1 This should work.

[ https://issues.apache.org/jira/browse/HADOOP-4598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hairong Kuang updated HADOOP-4598:
----------------------------------

Hadoop Flags: [Reviewed]
Status: Patch Available (was: Open)

[ https://issues.apache.org/jira/browse/HADOOP-4598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12649492#action_12649492 ]

Hairong Kuang commented on HADOOP-4598:
---------------------------------------

Ant test-core passed on my local machine.
BUILD SUCCESSFUL
Total time: 124 minutes 24 seconds

Ant test-patch result:
[exec] +1 overall.

[exec] +1 @author. The patch does not contain any @author tags.

[exec] +1 tests included. The patch appears to include 4 new or modified tests.

[exec] +1 javadoc. The javadoc tool did not generate any warningmessages.

[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.

[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.

[exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

[ https://issues.apache.org/jira/browse/HADOOP-4598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hairong Kuang updated HADOOP-4598:
----------------------------------

Resolution: Fixed
Fix Version/s: 0.20.0
Status: Resolved (was: Patch Available)

I just committed this!

[ https://issues.apache.org/jira/browse/HADOOP-4598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12649697#action_12649697 ]

Hudson commented on HADOOP-4598:
--------------------------------

Integrated in Hadoop-trunk #667 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/667/])
. Setrep command skips under-replicated blocks. Contributed by Hairong Kuang.