FAQ
"hadoop dfs -cp" does not copy crc files
----------------------------------------

Key: HADOOP-126
URL: http://issues.apache.org/jira/browse/HADOOP-126
Project: Hadoop
Type: Bug

Components: dfs
Reporter: Konstantin Shvachko


DFSShell.copy() uses FileUtil.copyContents() which works with "Raw" files, and does not copy crc files.
Try
hadoop dfs -cp a b
Then either
hadoop dfs -cat b
hadoop dfs -copyToLocal b c
The last two complain about crc files
In fact DFSShell.copy() should use the same FileSystem methods as copyToLocal and copyFromLocal do.


--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira

Search Discussions

  • Konstantin Shvachko (JIRA) at Apr 8, 2006 at 2:50 am
    [ http://issues.apache.org/jira/browse/HADOOP-126?page=all ]

    Konstantin Shvachko updated HADOOP-126:
    ---------------------------------------

    Attachment: copy.patch

    This fixes the problem.
    "hadoop dfs -cp" does not copy crc files
    ----------------------------------------

    Key: HADOOP-126
    URL: http://issues.apache.org/jira/browse/HADOOP-126
    Project: Hadoop
    Type: Bug
    Components: dfs
    Reporter: Konstantin Shvachko
    Attachments: copy.patch

    DFSShell.copy() uses FileUtil.copyContents() which works with "Raw" files, and does not copy crc files.
    Try
    hadoop dfs -cp a b
    Then either
    hadoop dfs -cat b
    hadoop dfs -copyToLocal b c
    The last two complain about crc files
    In fact DFSShell.copy() should use the same FileSystem methods as copyToLocal and copyFromLocal do.
    --
    This message is automatically generated by JIRA.
    -
    If you think it was sent incorrectly contact one of the administrators:
    http://issues.apache.org/jira/secure/Administrators.jspa
    -
    For more information on JIRA, see:
    http://www.atlassian.com/software/jira
  • Doug Cutting (JIRA) at Apr 10, 2006 at 7:16 pm
    [ http://issues.apache.org/jira/browse/HADOOP-126?page=all ]

    Doug Cutting resolved HADOOP-126:
    ---------------------------------

    Fix Version: 0.2
    Resolution: Fixed
    Assign To: Konstantin Shvachko

    I just committed this patch. Thanks, Konstantin!
    "hadoop dfs -cp" does not copy crc files
    ----------------------------------------

    Key: HADOOP-126
    URL: http://issues.apache.org/jira/browse/HADOOP-126
    Project: Hadoop
    Type: Bug
    Components: dfs
    Reporter: Konstantin Shvachko
    Assignee: Konstantin Shvachko
    Fix For: 0.2
    Attachments: copy.patch

    DFSShell.copy() uses FileUtil.copyContents() which works with "Raw" files, and does not copy crc files.
    Try
    hadoop dfs -cp a b
    Then either
    hadoop dfs -cat b
    hadoop dfs -copyToLocal b c
    The last two complain about crc files
    In fact DFSShell.copy() should use the same FileSystem methods as copyToLocal and copyFromLocal do.
    --
    This message is automatically generated by JIRA.
    -
    If you think it was sent incorrectly contact one of the administrators:
    http://issues.apache.org/jira/secure/Administrators.jspa
    -
    For more information on JIRA, see:
    http://www.atlassian.com/software/jira

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedApr 8, '06 at 2:48a
activeApr 10, '06 at 7:16p
posts3
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Doug Cutting (JIRA): 3 posts

People

Translate

site design / logo © 2022 Grokbase