[Hadoop-common-dev] [jira] Created: (HADOOP-4945) Add BlockTool to query file and its block info

[ https://issues.apache.org/jira/browse/HADOOP-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

zhangwei updated HADOOP-4945:
-----------------------------

Attachment: HADOOP-4945.patch

I add the tool under the under the package "org.apache.hadoop.tools", and can be used by type:
$HADOOP_HOME/bin/hadoop blocktool -f <file or directory>
to fetch the block information of the file or directory, if the dest is directory it query recursively to fecth the file's info under the dir.

also you can get the block's info by type:
$HADOOP_HOME/bin/hadoop blocktool -b <block>

++++++++++++
the output style is below:

mode1:(get the file's block info)
$ ./hadoop blocktool -f /test/wordcount2/input/english_story.txt
/test/wordcount2/input/english_story.txt:
rw-r--r-- rep=3 root root 5230(blksize:268435456) 2008-12-24 14:15:24 english_story.txt
block detail: (#BlockName #OffSet #Len #Locations)
blk_155_1158 0 5230 jx-hadoop-data04.jx.baidu.com
--------

Dest can be dir yet:
$ ./hadoop blocktool -f /test/
/test/wordcount/input/english_story.txt:
rw-r--r-- rep=3 root root 5230(blksize:268435456) 2008-12-24 14:03:39 english_story.txt
block detail: (#BlockName #OffSet #Len #Locations)
blk_151_1153 0 5230 jx-hadoop-data04.jx.baidu.com
--------

/test/wordcount2/input/english_story.txt:
rw-r--r-- rep=3 root root 5230(blksize:268435456) 2008-12-24 14:15:24 english_story.txt
block detail: (#BlockName #OffSet #Len #Locations)
blk_155_1158 0 5230 jx-hadoop-data04.jx.baidu.com
--------



mode2:(get the block's info)
$ ./hadoop blocktool -b blk_155_1158
"//test/wordcount2/input/english_story.txt":root:root:rw-r--r--
Loaction of block:
jx-hadoop-data04.jx.baidu.com

[ https://issues.apache.org/jira/browse/HADOOP-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

zhangwei updated HADOOP-4945:
-----------------------------

Fix Version/s: site
Affects Version/s: (was: 0.21.0)
site
Release Note: patch is based on hadoop core trunk ,revision 728879
Status: Patch Available (was: Open)

[ https://issues.apache.org/jira/browse/HADOOP-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12659734#action_12659734 ]

zhangwei commented on HADOOP-4945:
----------------------------------

can anyone review it ?

[ https://issues.apache.org/jira/browse/HADOOP-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12659977#action_12659977 ]

dhruba borthakur commented on HADOOP-4945:
------------------------------------------
The functionality described in "mode2" is similar to running "bin/hadoop fsck -files -blocks -locations / | grep <blockid>"

Do you agree?

[ https://issues.apache.org/jira/browse/HADOOP-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12660005#action_12660005 ]

zhangwei commented on HADOOP-4945:
----------------------------------

Hi,dhruba borthakur
Indeed the mode 1 is equal to.and the fsck can do the things above,but consider the situation that when you find the block's file in a product cluster with a huge data set.
if you run "bin/hadoop fsck -files -blocks -locations / | grep <blockid>" ,it's not a effective way.
Or maybe the fsck can be impoved a little else,i will consider it .

thanks your comment

[ https://issues.apache.org/jira/browse/HADOOP-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12660106#action_12660106 ]

dhruba borthakur commented on HADOOP-4945:
------------------------------------------

Hello zhangwei, thanks for looking into whether the "mode2" functionality be incorporated into fsck itself (without making the fsck code much more complicated).

[ https://issues.apache.org/jira/browse/HADOOP-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

zhangwei updated HADOOP-4945:
-----------------------------

Status: Open (was: Patch Available)

I decided to create a new issue which add ability on the fsck facility. And abort the patch so.

[ https://issues.apache.org/jira/browse/HADOOP-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

dhruba borthakur resolved HADOOP-4945.
--------------------------------------

Resolution: Won't Fix

I am closing this issue because HADOOP-5019 can address this issue. Please re-open if you feel otherwise. Thanks.