FAQ
Copying a file to local with Crc throws an exception
----------------------------------------------------

Key: HADOOP-5085
URL: https://issues.apache.org/jira/browse/HADOOP-5085
Project: Hadoop Core
Issue Type: Bug
Components: fs
Reporter: Hairong Kuang
Fix For: 0.21.0


$ hadoop dfs -get -crc /user/aa/test.txt test.txt
get: org.apache.hadoop.dfs.DistributedFileSystem cannot be cast to
org.apache.hadoop.fs.ChecksumFileSystem

The problem seems to be caused by the line 251 in FsShell#copyToLocal:
{noformat}
250; if (copyCrc) {
251: ChecksumFileSystem csfs = (ChecksumFileSystem) srcFS;
...
}
{noformat}

Copying crc files to local should not require the source file system to be ChecksumFileSystem.


--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Raghu Angadi (JIRA) at Jan 20, 2009 at 8:29 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12665546#action_12665546 ]

    Raghu Angadi commented on HADOOP-5085:
    --------------------------------------
    Copying crc files to local should not require the source file system to be ChecksumFileSystem.
    "crc file" applies only to a ChecksumFileSystem. HDFS does not have a checksum file..
    Copying a file to local with Crc throws an exception
    ----------------------------------------------------

    Key: HADOOP-5085
    URL: https://issues.apache.org/jira/browse/HADOOP-5085
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Hairong Kuang
    Fix For: 0.21.0


    $ hadoop dfs -get -crc /user/aa/test.txt test.txt
    get: org.apache.hadoop.dfs.DistributedFileSystem cannot be cast to
    org.apache.hadoop.fs.ChecksumFileSystem
    The problem seems to be caused by the line 251 in FsShell#copyToLocal:
    {noformat}
    250; if (copyCrc) {
    251: ChecksumFileSystem csfs = (ChecksumFileSystem) srcFS;
    ...
    }
    {noformat}
    Copying crc files to local should not require the source file system to be ChecksumFileSystem.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hairong Kuang (JIRA) at Jan 29, 2009 at 11:14 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Hairong Kuang reassigned HADOOP-5085:
    -------------------------------------

    Assignee: Hairong Kuang
    Copying a file to local with Crc throws an exception
    ----------------------------------------------------

    Key: HADOOP-5085
    URL: https://issues.apache.org/jira/browse/HADOOP-5085
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Hairong Kuang
    Assignee: Hairong Kuang
    Fix For: 0.21.0

    Attachments: crcChecksum.patch


    $ hadoop dfs -get -crc /user/aa/test.txt test.txt
    get: org.apache.hadoop.dfs.DistributedFileSystem cannot be cast to
    org.apache.hadoop.fs.ChecksumFileSystem
    The problem seems to be caused by the line 251 in FsShell#copyToLocal:
    {noformat}
    250; if (copyCrc) {
    251: ChecksumFileSystem csfs = (ChecksumFileSystem) srcFS;
    ...
    }
    {noformat}
    Copying crc files to local should not require the source file system to be ChecksumFileSystem.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hairong Kuang (JIRA) at Jan 29, 2009 at 11:14 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Hairong Kuang updated HADOOP-5085:
    ----------------------------------

    Attachment: crcChecksum.patch

    This patch gives a warning if a user specifies -crc option when source file system does not have .crc file.
    Copying a file to local with Crc throws an exception
    ----------------------------------------------------

    Key: HADOOP-5085
    URL: https://issues.apache.org/jira/browse/HADOOP-5085
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Hairong Kuang
    Fix For: 0.21.0

    Attachments: crcChecksum.patch


    $ hadoop dfs -get -crc /user/aa/test.txt test.txt
    get: org.apache.hadoop.dfs.DistributedFileSystem cannot be cast to
    org.apache.hadoop.fs.ChecksumFileSystem
    The problem seems to be caused by the line 251 in FsShell#copyToLocal:
    {noformat}
    250; if (copyCrc) {
    251: ChecksumFileSystem csfs = (ChecksumFileSystem) srcFS;
    ...
    }
    {noformat}
    Copying crc files to local should not require the source file system to be ChecksumFileSystem.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Jan 30, 2009 at 12:58 am
    [ https://issues.apache.org/jira/browse/HADOOP-5085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12668685#action_12668685 ]

    Tsz Wo (Nicholas), SZE commented on HADOOP-5085:
    ------------------------------------------------

    When -crc is specified in the command and the fs does not support crc, the patch prints a warning, turns off -crc automatically and then continues. I think we should throw an exception in such case. No?
    Copying a file to local with Crc throws an exception
    ----------------------------------------------------

    Key: HADOOP-5085
    URL: https://issues.apache.org/jira/browse/HADOOP-5085
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Hairong Kuang
    Assignee: Hairong Kuang
    Fix For: 0.21.0

    Attachments: crcChecksum.patch


    $ hadoop dfs -get -crc /user/aa/test.txt test.txt
    get: org.apache.hadoop.dfs.DistributedFileSystem cannot be cast to
    org.apache.hadoop.fs.ChecksumFileSystem
    The problem seems to be caused by the line 251 in FsShell#copyToLocal:
    {noformat}
    250; if (copyCrc) {
    251: ChecksumFileSystem csfs = (ChecksumFileSystem) srcFS;
    ...
    }
    {noformat}
    Copying crc files to local should not require the source file system to be ChecksumFileSystem.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hairong Kuang (JIRA) at Jan 30, 2009 at 1:18 am
    [ https://issues.apache.org/jira/browse/HADOOP-5085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12668691#action_12668691 ]

    Hairong Kuang commented on HADOOP-5085:
    ---------------------------------------

    My solution is consistent with how FsShell handles -crc option now. If the option is specified and stdout is the destination, the code also prints a warning and continue. I think it is no harm to turn off -crc option automatically.
    Copying a file to local with Crc throws an exception
    ----------------------------------------------------

    Key: HADOOP-5085
    URL: https://issues.apache.org/jira/browse/HADOOP-5085
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Hairong Kuang
    Assignee: Hairong Kuang
    Fix For: 0.21.0

    Attachments: crcChecksum.patch


    $ hadoop dfs -get -crc /user/aa/test.txt test.txt
    get: org.apache.hadoop.dfs.DistributedFileSystem cannot be cast to
    org.apache.hadoop.fs.ChecksumFileSystem
    The problem seems to be caused by the line 251 in FsShell#copyToLocal:
    {noformat}
    250; if (copyCrc) {
    251: ChecksumFileSystem csfs = (ChecksumFileSystem) srcFS;
    ...
    }
    {noformat}
    Copying crc files to local should not require the source file system to be ChecksumFileSystem.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Jan 30, 2009 at 6:59 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Tsz Wo (Nicholas), SZE updated HADOOP-5085:
    -------------------------------------------

    Hadoop Flags: [Reviewed]

    Ok, let's make it consistent with the stdout case.

    +1
    Copying a file to local with Crc throws an exception
    ----------------------------------------------------

    Key: HADOOP-5085
    URL: https://issues.apache.org/jira/browse/HADOOP-5085
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Hairong Kuang
    Assignee: Hairong Kuang
    Fix For: 0.21.0

    Attachments: crcChecksum.patch


    $ hadoop dfs -get -crc /user/aa/test.txt test.txt
    get: org.apache.hadoop.dfs.DistributedFileSystem cannot be cast to
    org.apache.hadoop.fs.ChecksumFileSystem
    The problem seems to be caused by the line 251 in FsShell#copyToLocal:
    {noformat}
    250; if (copyCrc) {
    251: ChecksumFileSystem csfs = (ChecksumFileSystem) srcFS;
    ...
    }
    {noformat}
    Copying crc files to local should not require the source file system to be ChecksumFileSystem.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hairong Kuang (JIRA) at Feb 2, 2009 at 6:52 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Hairong Kuang updated HADOOP-5085:
    ----------------------------------

    Attachment: crcChecksum1.patch

    Thanks Nicholas for the review comments. This patch explicitly states that crc option is turned off automatically. Just to make the user to be better informed.
    Copying a file to local with Crc throws an exception
    ----------------------------------------------------

    Key: HADOOP-5085
    URL: https://issues.apache.org/jira/browse/HADOOP-5085
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Hairong Kuang
    Assignee: Hairong Kuang
    Fix For: 0.20.0

    Attachments: crcChecksum.patch, crcChecksum1.patch


    $ hadoop dfs -get -crc /user/aa/test.txt test.txt
    get: org.apache.hadoop.dfs.DistributedFileSystem cannot be cast to
    org.apache.hadoop.fs.ChecksumFileSystem
    The problem seems to be caused by the line 251 in FsShell#copyToLocal:
    {noformat}
    250; if (copyCrc) {
    251: ChecksumFileSystem csfs = (ChecksumFileSystem) srcFS;
    ...
    }
    {noformat}
    Copying crc files to local should not require the source file system to be ChecksumFileSystem.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hairong Kuang (JIRA) at Feb 2, 2009 at 6:52 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Hairong Kuang updated HADOOP-5085:
    ----------------------------------

    Fix Version/s: (was: 0.21.0)
    0.20.0
    Status: Patch Available (was: Open)
    Copying a file to local with Crc throws an exception
    ----------------------------------------------------

    Key: HADOOP-5085
    URL: https://issues.apache.org/jira/browse/HADOOP-5085
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Hairong Kuang
    Assignee: Hairong Kuang
    Fix For: 0.20.0

    Attachments: crcChecksum.patch, crcChecksum1.patch


    $ hadoop dfs -get -crc /user/aa/test.txt test.txt
    get: org.apache.hadoop.dfs.DistributedFileSystem cannot be cast to
    org.apache.hadoop.fs.ChecksumFileSystem
    The problem seems to be caused by the line 251 in FsShell#copyToLocal:
    {noformat}
    250; if (copyCrc) {
    251: ChecksumFileSystem csfs = (ChecksumFileSystem) srcFS;
    ...
    }
    {noformat}
    Copying crc files to local should not require the source file system to be ChecksumFileSystem.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Feb 2, 2009 at 11:00 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12669784#action_12669784 ]

    Hadoop QA commented on HADOOP-5085:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12399284/crcChecksum1.patch
    against trunk revision 740064.

    +1 @author. The patch does not contain any @author tags.

    -1 tests included. The patch doesn't appear to include any new or modified tests.
    Please justify why no tests are needed for this patch.

    +1 javadoc. The javadoc tool did not generate any warning messages.

    +1 javac. The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs. The patch does not introduce any new Findbugs warnings.

    +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

    +1 release audit. The applied patch does not increase the total number of release audit warnings.

    +1 core tests. The patch passed core unit tests.

    -1 contrib tests. The patch failed contrib unit tests.

    Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3789/testReport/
    Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3789/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3789/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3789/console

    This message is automatically generated.
    Copying a file to local with Crc throws an exception
    ----------------------------------------------------

    Key: HADOOP-5085
    URL: https://issues.apache.org/jira/browse/HADOOP-5085
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Hairong Kuang
    Assignee: Hairong Kuang
    Fix For: 0.20.0

    Attachments: crcChecksum.patch, crcChecksum1.patch


    $ hadoop dfs -get -crc /user/aa/test.txt test.txt
    get: org.apache.hadoop.dfs.DistributedFileSystem cannot be cast to
    org.apache.hadoop.fs.ChecksumFileSystem
    The problem seems to be caused by the line 251 in FsShell#copyToLocal:
    {noformat}
    250; if (copyCrc) {
    251: ChecksumFileSystem csfs = (ChecksumFileSystem) srcFS;
    ...
    }
    {noformat}
    Copying crc files to local should not require the source file system to be ChecksumFileSystem.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hairong Kuang (JIRA) at Feb 2, 2009 at 11:52 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Hairong Kuang updated HADOOP-5085:
    ----------------------------------

    Resolution: Fixed
    Status: Resolved (was: Patch Available)

    I've just committed this.
    Copying a file to local with Crc throws an exception
    ----------------------------------------------------

    Key: HADOOP-5085
    URL: https://issues.apache.org/jira/browse/HADOOP-5085
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Hairong Kuang
    Assignee: Hairong Kuang
    Fix For: 0.20.0

    Attachments: crcChecksum.patch, crcChecksum1.patch


    $ hadoop dfs -get -crc /user/aa/test.txt test.txt
    get: org.apache.hadoop.dfs.DistributedFileSystem cannot be cast to
    org.apache.hadoop.fs.ChecksumFileSystem
    The problem seems to be caused by the line 251 in FsShell#copyToLocal:
    {noformat}
    250; if (copyCrc) {
    251: ChecksumFileSystem csfs = (ChecksumFileSystem) srcFS;
    ...
    }
    {noformat}
    Copying crc files to local should not require the source file system to be ChecksumFileSystem.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hudson (JIRA) at Feb 16, 2009 at 5:06 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12673997#action_12673997 ]

    Hudson commented on HADOOP-5085:
    --------------------------------

    Integrated in Hadoop-trunk #756 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/756/])

    Copying a file to local with Crc throws an exception
    ----------------------------------------------------

    Key: HADOOP-5085
    URL: https://issues.apache.org/jira/browse/HADOOP-5085
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Hairong Kuang
    Assignee: Hairong Kuang
    Fix For: 0.20.0

    Attachments: crcChecksum.patch, crcChecksum1.patch


    $ hadoop dfs -get -crc /user/aa/test.txt test.txt
    get: org.apache.hadoop.dfs.DistributedFileSystem cannot be cast to
    org.apache.hadoop.fs.ChecksumFileSystem
    The problem seems to be caused by the line 251 in FsShell#copyToLocal:
    {noformat}
    250; if (copyCrc) {
    251: ChecksumFileSystem csfs = (ChecksumFileSystem) srcFS;
    ...
    }
    {noformat}
    Copying crc files to local should not require the source file system to be ChecksumFileSystem.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedJan 20, '09 at 6:49p
activeFeb 16, '09 at 5:06p
posts12
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Hudson (JIRA): 12 posts

People

Translate

site design / logo © 2022 Grokbase