[Hadoop-common-dev] [jira] Created: (HADOOP-5671) DistCp.sameFile(..) should return true if src fs does not support checksum

[ https://issues.apache.org/jira/browse/HADOOP-5671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tsz Wo (Nicholas), SZE updated HADOOP-5671:
-------------------------------------------

Attachment: 5671_20090414.patch

5671_20090414.patch: If src fs does not support file checksum, it may throw a FileNotFoundException. In such case, we should return true.

[ https://issues.apache.org/jira/browse/HADOOP-5671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12698984#action_12698984 ]

Tsz Wo (Nicholas), SZE commented on HADOOP-5671:
------------------------------------------------

{noformat}
[exec] -1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
[exec] Please justify why no tests are needed for this patch.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
{noformat}
Tested manually, no new tests added.

[ https://issues.apache.org/jira/browse/HADOOP-5671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tsz Wo (Nicholas), SZE updated HADOOP-5671:
-------------------------------------------

Status: Patch Available (was: Open)

[ https://issues.apache.org/jira/browse/HADOOP-5671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699389#action_12699389 ]

Suresh Srinivas commented on HADOOP-5671:
-----------------------------------------

Based on the documentation of the method {{FileSystem.getFileChecksum()}}, not supporting checksum is indicated by {{null}} return and *not* by throwing exception. So should we ensure the classes extending {{FileSystem}} honor this?

[ https://issues.apache.org/jira/browse/HADOOP-5671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tsz Wo (Nicholas), SZE updated HADOOP-5671:
-------------------------------------------

Attachment: 5671_20090415.patch
More details of the problem:
Some previous versions HftpFileSystem do not support file checksum and throw FileNotFoundException since the request url is not found. In such case, DistCp.sameFile(..) should return true.

5671_20090415.patch: rewritten the comments, no code changes.

[ https://issues.apache.org/jira/browse/HADOOP-5671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699449#action_12699449 ]

Chris Douglas commented on HADOOP-5671:
---------------------------------------

+1

[ https://issues.apache.org/jira/browse/HADOOP-5671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Douglas updated HADOOP-5671:
----------------------------------

Resolution: Fixed
Fix Version/s: 0.21.0
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)

I committed this. Thanks, Nicholas

[ https://issues.apache.org/jira/browse/HADOOP-5671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699455#action_12699455 ]

Koji Noguchi commented on HADOOP-5671:
--------------------------------------

Thanks Nicholas, Chris, Suresh!

Since this is sort of a regression for 0.19, can we also commit it to 0.19.* and 0.20.1 ?
(-update option won't work without this patch when pulling from 0.18 cluster or earlier versions)

[ https://issues.apache.org/jira/browse/HADOOP-5671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12700029#action_12700029 ]

dhruba borthakur commented on HADOOP-5671:
------------------------------------------

Really appreciate it if we can commit it to 0.19 and earlier releases.

[ https://issues.apache.org/jira/browse/HADOOP-5671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12700721#action_12700721 ]

Hudson commented on HADOOP-5671:
--------------------------------

Integrated in Hadoop-trunk #811 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/811/])

[ https://issues.apache.org/jira/browse/HADOOP-5671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Douglas updated HADOOP-5671:
----------------------------------

Fix Version/s: (was: 0.21.0)
0.19.2

Committed to 0.19.2 and 0.20.1 as well

[ https://issues.apache.org/jira/browse/HADOOP-5671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12702077#action_12702077 ]

Hudson commented on HADOOP-5671:
--------------------------------

Integrated in Hadoop-trunk #815 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/815/])
Move from 0.21 to 0.19.2