FAQ
HFTP interface compatibility with older releases broken
-------------------------------------------------------

Key: HADOOP-4099
URL: https://issues.apache.org/jira/browse/HADOOP-4099
Project: Hadoop Core
Issue Type: Bug
Components: tools/distcp
Affects Versions: 0.19.0
Reporter: Kan Zhang


Using current trunk (Revision 692556) build and trying to distcp from a 0.18.0 cluster via HFTP, got the following NullPointerException. Seems like on line 165 in HftpFileSystem.java, a potential null return value isn't checked before using.

coursemud-lm:bin kan$ ./hadoop distcp hftp://ucdev18.inktomisearch.com:50070/output/part-00000 /copied/part-00001
08/09/05 19:00:10 INFO tools.DistCp: srcPaths=[hftp://ucdev18.inktomisearch.com:50070/output/part-00000]
08/09/05 19:00:10 INFO tools.DistCp: destPath=/copied/part-00001
With failures, global counters are inaccurate; consider running with -i
Copy failed: java.lang.NullPointerException
at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1215)
at java.text.DateFormat.parse(DateFormat.java:335)
at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.startElement(HftpFileSystem.java:165)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:501)
at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:179)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:377)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2740)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:647)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:508)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.fetchList(HftpFileSystem.java:194)
at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.getFileStatus(HftpFileSystem.java:205)
at org.apache.hadoop.hdfs.HftpFileSystem.getFileStatus(HftpFileSystem.java:234)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:649)
at org.apache.hadoop.tools.DistCp.checkSrcPath(DistCp.java:614)
at org.apache.hadoop.tools.DistCp.copy(DistCp.java:631)
at org.apache.hadoop.tools.DistCp.run(DistCp.java:838)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.tools.DistCp.main(DistCp.java:865)



--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Raghu Angadi (JIRA) at Sep 8, 2008 at 4:36 pm
    [ https://issues.apache.org/jira/browse/HADOOP-4099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Raghu Angadi reassigned HADOOP-4099:
    ------------------------------------

    Assignee: dhruba borthakur

    Dhruba, could you take a quick look? Looks like hsftp should handle missing accesstime field in xml.
    HFTP interface compatibility with older releases broken
    -------------------------------------------------------

    Key: HADOOP-4099
    URL: https://issues.apache.org/jira/browse/HADOOP-4099
    Project: Hadoop Core
    Issue Type: Bug
    Components: tools/distcp
    Affects Versions: 0.19.0
    Reporter: Kan Zhang
    Assignee: dhruba borthakur

    Using current trunk (Revision 692556) build and trying to distcp from a 0.18.0 cluster via HFTP, got the following NullPointerException. Seems like on line 165 in HftpFileSystem.java, a potential null return value isn't checked before using.
    coursemud-lm:bin kan$ ./hadoop distcp hftp://ucdev18.inktomisearch.com:50070/output/part-00000 /copied/part-00001
    08/09/05 19:00:10 INFO tools.DistCp: srcPaths=[hftp://ucdev18.inktomisearch.com:50070/output/part-00000]
    08/09/05 19:00:10 INFO tools.DistCp: destPath=/copied/part-00001
    With failures, global counters are inaccurate; consider running with -i
    Copy failed: java.lang.NullPointerException
    at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1215)
    at java.text.DateFormat.parse(DateFormat.java:335)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.startElement(HftpFileSystem.java:165)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:501)
    at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:179)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:377)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2740)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:647)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:508)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
    at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.fetchList(HftpFileSystem.java:194)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.getFileStatus(HftpFileSystem.java:205)
    at org.apache.hadoop.hdfs.HftpFileSystem.getFileStatus(HftpFileSystem.java:234)
    at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:649)
    at org.apache.hadoop.tools.DistCp.checkSrcPath(DistCp.java:614)
    at org.apache.hadoop.tools.DistCp.copy(DistCp.java:631)
    at org.apache.hadoop.tools.DistCp.run(DistCp.java:838)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.tools.DistCp.main(DistCp.java:865)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • dhruba borthakur (JIRA) at Sep 8, 2008 at 5:38 pm
    [ https://issues.apache.org/jira/browse/HADOOP-4099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12629224#action_12629224 ]

    dhruba borthakur commented on HADOOP-4099:
    ------------------------------------------

    Ok, looking at it...

    HFTP interface compatibility with older releases broken
    -------------------------------------------------------

    Key: HADOOP-4099
    URL: https://issues.apache.org/jira/browse/HADOOP-4099
    Project: Hadoop Core
    Issue Type: Bug
    Components: tools/distcp
    Affects Versions: 0.19.0
    Reporter: Kan Zhang
    Assignee: dhruba borthakur

    Using current trunk (Revision 692556) build and trying to distcp from a 0.18.0 cluster via HFTP, got the following NullPointerException. Seems like on line 165 in HftpFileSystem.java, a potential null return value isn't checked before using.
    coursemud-lm:bin kan$ ./hadoop distcp hftp://ucdev18.inktomisearch.com:50070/output/part-00000 /copied/part-00001
    08/09/05 19:00:10 INFO tools.DistCp: srcPaths=[hftp://ucdev18.inktomisearch.com:50070/output/part-00000]
    08/09/05 19:00:10 INFO tools.DistCp: destPath=/copied/part-00001
    With failures, global counters are inaccurate; consider running with -i
    Copy failed: java.lang.NullPointerException
    at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1215)
    at java.text.DateFormat.parse(DateFormat.java:335)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.startElement(HftpFileSystem.java:165)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:501)
    at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:179)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:377)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2740)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:647)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:508)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
    at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.fetchList(HftpFileSystem.java:194)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.getFileStatus(HftpFileSystem.java:205)
    at org.apache.hadoop.hdfs.HftpFileSystem.getFileStatus(HftpFileSystem.java:234)
    at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:649)
    at org.apache.hadoop.tools.DistCp.checkSrcPath(DistCp.java:614)
    at org.apache.hadoop.tools.DistCp.copy(DistCp.java:631)
    at org.apache.hadoop.tools.DistCp.run(DistCp.java:838)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.tools.DistCp.main(DistCp.java:865)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Sep 8, 2008 at 5:42 pm
    [ https://issues.apache.org/jira/browse/HADOOP-4099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Tsz Wo (Nicholas), SZE updated HADOOP-4099:
    -------------------------------------------

    Priority: Blocker (was: Major)

    Setting this to a 0.19 blocker.
    HFTP interface compatibility with older releases broken
    -------------------------------------------------------

    Key: HADOOP-4099
    URL: https://issues.apache.org/jira/browse/HADOOP-4099
    Project: Hadoop Core
    Issue Type: Bug
    Components: tools/distcp
    Affects Versions: 0.19.0
    Reporter: Kan Zhang
    Assignee: dhruba borthakur
    Priority: Blocker

    Using current trunk (Revision 692556) build and trying to distcp from a 0.18.0 cluster via HFTP, got the following NullPointerException. Seems like on line 165 in HftpFileSystem.java, a potential null return value isn't checked before using.
    coursemud-lm:bin kan$ ./hadoop distcp hftp://ucdev18.inktomisearch.com:50070/output/part-00000 /copied/part-00001
    08/09/05 19:00:10 INFO tools.DistCp: srcPaths=[hftp://ucdev18.inktomisearch.com:50070/output/part-00000]
    08/09/05 19:00:10 INFO tools.DistCp: destPath=/copied/part-00001
    With failures, global counters are inaccurate; consider running with -i
    Copy failed: java.lang.NullPointerException
    at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1215)
    at java.text.DateFormat.parse(DateFormat.java:335)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.startElement(HftpFileSystem.java:165)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:501)
    at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:179)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:377)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2740)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:647)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:508)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
    at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.fetchList(HftpFileSystem.java:194)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.getFileStatus(HftpFileSystem.java:205)
    at org.apache.hadoop.hdfs.HftpFileSystem.getFileStatus(HftpFileSystem.java:234)
    at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:649)
    at org.apache.hadoop.tools.DistCp.checkSrcPath(DistCp.java:614)
    at org.apache.hadoop.tools.DistCp.copy(DistCp.java:631)
    at org.apache.hadoop.tools.DistCp.run(DistCp.java:838)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.tools.DistCp.main(DistCp.java:865)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • dhruba borthakur (JIRA) at Sep 8, 2008 at 6:16 pm
    [ https://issues.apache.org/jira/browse/HADOOP-4099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    dhruba borthakur updated HADOOP-4099:
    -------------------------------------

    Status: Patch Available (was: Open)
    HFTP interface compatibility with older releases broken
    -------------------------------------------------------

    Key: HADOOP-4099
    URL: https://issues.apache.org/jira/browse/HADOOP-4099
    Project: Hadoop Core
    Issue Type: Bug
    Components: tools/distcp
    Affects Versions: 0.19.0
    Reporter: Kan Zhang
    Assignee: dhruba borthakur
    Priority: Blocker
    Attachments: hftpAccessTime.patch


    Using current trunk (Revision 692556) build and trying to distcp from a 0.18.0 cluster via HFTP, got the following NullPointerException. Seems like on line 165 in HftpFileSystem.java, a potential null return value isn't checked before using.
    coursemud-lm:bin kan$ ./hadoop distcp hftp://ucdev18.inktomisearch.com:50070/output/part-00000 /copied/part-00001
    08/09/05 19:00:10 INFO tools.DistCp: srcPaths=[hftp://ucdev18.inktomisearch.com:50070/output/part-00000]
    08/09/05 19:00:10 INFO tools.DistCp: destPath=/copied/part-00001
    With failures, global counters are inaccurate; consider running with -i
    Copy failed: java.lang.NullPointerException
    at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1215)
    at java.text.DateFormat.parse(DateFormat.java:335)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.startElement(HftpFileSystem.java:165)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:501)
    at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:179)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:377)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2740)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:647)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:508)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
    at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.fetchList(HftpFileSystem.java:194)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.getFileStatus(HftpFileSystem.java:205)
    at org.apache.hadoop.hdfs.HftpFileSystem.getFileStatus(HftpFileSystem.java:234)
    at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:649)
    at org.apache.hadoop.tools.DistCp.checkSrcPath(DistCp.java:614)
    at org.apache.hadoop.tools.DistCp.copy(DistCp.java:631)
    at org.apache.hadoop.tools.DistCp.run(DistCp.java:838)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.tools.DistCp.main(DistCp.java:865)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • dhruba borthakur (JIRA) at Sep 8, 2008 at 6:16 pm
    [ https://issues.apache.org/jira/browse/HADOOP-4099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    dhruba borthakur updated HADOOP-4099:
    -------------------------------------

    Attachment: hftpAccessTime.patch

    Use the access time only if present.
    HFTP interface compatibility with older releases broken
    -------------------------------------------------------

    Key: HADOOP-4099
    URL: https://issues.apache.org/jira/browse/HADOOP-4099
    Project: Hadoop Core
    Issue Type: Bug
    Components: tools/distcp
    Affects Versions: 0.19.0
    Reporter: Kan Zhang
    Assignee: dhruba borthakur
    Priority: Blocker
    Attachments: hftpAccessTime.patch


    Using current trunk (Revision 692556) build and trying to distcp from a 0.18.0 cluster via HFTP, got the following NullPointerException. Seems like on line 165 in HftpFileSystem.java, a potential null return value isn't checked before using.
    coursemud-lm:bin kan$ ./hadoop distcp hftp://ucdev18.inktomisearch.com:50070/output/part-00000 /copied/part-00001
    08/09/05 19:00:10 INFO tools.DistCp: srcPaths=[hftp://ucdev18.inktomisearch.com:50070/output/part-00000]
    08/09/05 19:00:10 INFO tools.DistCp: destPath=/copied/part-00001
    With failures, global counters are inaccurate; consider running with -i
    Copy failed: java.lang.NullPointerException
    at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1215)
    at java.text.DateFormat.parse(DateFormat.java:335)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.startElement(HftpFileSystem.java:165)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:501)
    at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:179)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:377)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2740)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:647)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:508)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
    at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.fetchList(HftpFileSystem.java:194)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.getFileStatus(HftpFileSystem.java:205)
    at org.apache.hadoop.hdfs.HftpFileSystem.getFileStatus(HftpFileSystem.java:234)
    at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:649)
    at org.apache.hadoop.tools.DistCp.checkSrcPath(DistCp.java:614)
    at org.apache.hadoop.tools.DistCp.copy(DistCp.java:631)
    at org.apache.hadoop.tools.DistCp.run(DistCp.java:838)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.tools.DistCp.main(DistCp.java:865)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Sep 8, 2008 at 10:14 pm
    [ https://issues.apache.org/jira/browse/HADOOP-4099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12629312#action_12629312 ]

    Hadoop QA commented on HADOOP-4099:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12389693/hftpAccessTime.patch
    against trunk revision 693048.

    +1 @author. The patch does not contain any @author tags.

    -1 tests included. The patch doesn't appear to include any new or modified tests.
    Please justify why no tests are needed for this patch.

    +1 javadoc. The javadoc tool did not generate any warning messages.

    +1 javac. The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs. The patch does not introduce any new Findbugs warnings.

    -1 core tests. The patch failed core unit tests.

    +1 contrib tests. The patch passed contrib unit tests.

    Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3210/testReport/
    Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3210/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3210/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3210/console

    This message is automatically generated.
    HFTP interface compatibility with older releases broken
    -------------------------------------------------------

    Key: HADOOP-4099
    URL: https://issues.apache.org/jira/browse/HADOOP-4099
    Project: Hadoop Core
    Issue Type: Bug
    Components: tools/distcp
    Affects Versions: 0.19.0
    Reporter: Kan Zhang
    Assignee: dhruba borthakur
    Priority: Blocker
    Attachments: hftpAccessTime.patch


    Using current trunk (Revision 692556) build and trying to distcp from a 0.18.0 cluster via HFTP, got the following NullPointerException. Seems like on line 165 in HftpFileSystem.java, a potential null return value isn't checked before using.
    coursemud-lm:bin kan$ ./hadoop distcp hftp://ucdev18.inktomisearch.com:50070/output/part-00000 /copied/part-00001
    08/09/05 19:00:10 INFO tools.DistCp: srcPaths=[hftp://ucdev18.inktomisearch.com:50070/output/part-00000]
    08/09/05 19:00:10 INFO tools.DistCp: destPath=/copied/part-00001
    With failures, global counters are inaccurate; consider running with -i
    Copy failed: java.lang.NullPointerException
    at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1215)
    at java.text.DateFormat.parse(DateFormat.java:335)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.startElement(HftpFileSystem.java:165)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:501)
    at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:179)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:377)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2740)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:647)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:508)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
    at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.fetchList(HftpFileSystem.java:194)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.getFileStatus(HftpFileSystem.java:205)
    at org.apache.hadoop.hdfs.HftpFileSystem.getFileStatus(HftpFileSystem.java:234)
    at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:649)
    at org.apache.hadoop.tools.DistCp.checkSrcPath(DistCp.java:614)
    at org.apache.hadoop.tools.DistCp.copy(DistCp.java:631)
    at org.apache.hadoop.tools.DistCp.run(DistCp.java:838)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.tools.DistCp.main(DistCp.java:865)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • dhruba borthakur (JIRA) at Sep 9, 2008 at 6:02 pm
    [ https://issues.apache.org/jira/browse/HADOOP-4099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12629554#action_12629554 ]

    dhruba borthakur commented on HADOOP-4099:
    ------------------------------------------

    @Nicholas: can you pl review this one-line patch?

    HFTP interface compatibility with older releases broken
    -------------------------------------------------------

    Key: HADOOP-4099
    URL: https://issues.apache.org/jira/browse/HADOOP-4099
    Project: Hadoop Core
    Issue Type: Bug
    Components: tools/distcp
    Affects Versions: 0.19.0
    Reporter: Kan Zhang
    Assignee: dhruba borthakur
    Priority: Blocker
    Attachments: hftpAccessTime.patch


    Using current trunk (Revision 692556) build and trying to distcp from a 0.18.0 cluster via HFTP, got the following NullPointerException. Seems like on line 165 in HftpFileSystem.java, a potential null return value isn't checked before using.
    coursemud-lm:bin kan$ ./hadoop distcp hftp://ucdev18.inktomisearch.com:50070/output/part-00000 /copied/part-00001
    08/09/05 19:00:10 INFO tools.DistCp: srcPaths=[hftp://ucdev18.inktomisearch.com:50070/output/part-00000]
    08/09/05 19:00:10 INFO tools.DistCp: destPath=/copied/part-00001
    With failures, global counters are inaccurate; consider running with -i
    Copy failed: java.lang.NullPointerException
    at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1215)
    at java.text.DateFormat.parse(DateFormat.java:335)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.startElement(HftpFileSystem.java:165)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:501)
    at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:179)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:377)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2740)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:647)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:508)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
    at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.fetchList(HftpFileSystem.java:194)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.getFileStatus(HftpFileSystem.java:205)
    at org.apache.hadoop.hdfs.HftpFileSystem.getFileStatus(HftpFileSystem.java:234)
    at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:649)
    at org.apache.hadoop.tools.DistCp.checkSrcPath(DistCp.java:614)
    at org.apache.hadoop.tools.DistCp.copy(DistCp.java:631)
    at org.apache.hadoop.tools.DistCp.run(DistCp.java:838)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.tools.DistCp.main(DistCp.java:865)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Sep 9, 2008 at 6:16 pm
    [ https://issues.apache.org/jira/browse/HADOOP-4099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Tsz Wo (Nicholas), SZE updated HADOOP-4099:
    -------------------------------------------

    Component/s: (was: tools/distcp)
    fs
    Fix Version/s: 0.19.0
    Hadoop Flags: [Reviewed]

    +1 patch looks good.
    HFTP interface compatibility with older releases broken
    -------------------------------------------------------

    Key: HADOOP-4099
    URL: https://issues.apache.org/jira/browse/HADOOP-4099
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.19.0
    Reporter: Kan Zhang
    Assignee: dhruba borthakur
    Priority: Blocker
    Fix For: 0.19.0

    Attachments: hftpAccessTime.patch


    Using current trunk (Revision 692556) build and trying to distcp from a 0.18.0 cluster via HFTP, got the following NullPointerException. Seems like on line 165 in HftpFileSystem.java, a potential null return value isn't checked before using.
    coursemud-lm:bin kan$ ./hadoop distcp hftp://ucdev18.inktomisearch.com:50070/output/part-00000 /copied/part-00001
    08/09/05 19:00:10 INFO tools.DistCp: srcPaths=[hftp://ucdev18.inktomisearch.com:50070/output/part-00000]
    08/09/05 19:00:10 INFO tools.DistCp: destPath=/copied/part-00001
    With failures, global counters are inaccurate; consider running with -i
    Copy failed: java.lang.NullPointerException
    at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1215)
    at java.text.DateFormat.parse(DateFormat.java:335)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.startElement(HftpFileSystem.java:165)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:501)
    at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:179)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:377)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2740)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:647)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:508)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
    at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.fetchList(HftpFileSystem.java:194)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.getFileStatus(HftpFileSystem.java:205)
    at org.apache.hadoop.hdfs.HftpFileSystem.getFileStatus(HftpFileSystem.java:234)
    at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:649)
    at org.apache.hadoop.tools.DistCp.checkSrcPath(DistCp.java:614)
    at org.apache.hadoop.tools.DistCp.copy(DistCp.java:631)
    at org.apache.hadoop.tools.DistCp.run(DistCp.java:838)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.tools.DistCp.main(DistCp.java:865)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • dhruba borthakur (JIRA) at Sep 9, 2008 at 6:46 pm
    [ https://issues.apache.org/jira/browse/HADOOP-4099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    dhruba borthakur updated HADOOP-4099:
    -------------------------------------

    Status: Open (was: Patch Available)
    HFTP interface compatibility with older releases broken
    -------------------------------------------------------

    Key: HADOOP-4099
    URL: https://issues.apache.org/jira/browse/HADOOP-4099
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.19.0
    Reporter: Kan Zhang
    Assignee: dhruba borthakur
    Priority: Blocker
    Fix For: 0.19.0

    Attachments: hftpAccessTime.patch


    Using current trunk (Revision 692556) build and trying to distcp from a 0.18.0 cluster via HFTP, got the following NullPointerException. Seems like on line 165 in HftpFileSystem.java, a potential null return value isn't checked before using.
    coursemud-lm:bin kan$ ./hadoop distcp hftp://ucdev18.inktomisearch.com:50070/output/part-00000 /copied/part-00001
    08/09/05 19:00:10 INFO tools.DistCp: srcPaths=[hftp://ucdev18.inktomisearch.com:50070/output/part-00000]
    08/09/05 19:00:10 INFO tools.DistCp: destPath=/copied/part-00001
    With failures, global counters are inaccurate; consider running with -i
    Copy failed: java.lang.NullPointerException
    at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1215)
    at java.text.DateFormat.parse(DateFormat.java:335)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.startElement(HftpFileSystem.java:165)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:501)
    at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:179)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:377)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2740)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:647)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:508)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
    at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.fetchList(HftpFileSystem.java:194)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.getFileStatus(HftpFileSystem.java:205)
    at org.apache.hadoop.hdfs.HftpFileSystem.getFileStatus(HftpFileSystem.java:234)
    at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:649)
    at org.apache.hadoop.tools.DistCp.checkSrcPath(DistCp.java:614)
    at org.apache.hadoop.tools.DistCp.copy(DistCp.java:631)
    at org.apache.hadoop.tools.DistCp.run(DistCp.java:838)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.tools.DistCp.main(DistCp.java:865)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • dhruba borthakur (JIRA) at Sep 9, 2008 at 6:46 pm
    [ https://issues.apache.org/jira/browse/HADOOP-4099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    dhruba borthakur updated HADOOP-4099:
    -------------------------------------

    Status: Patch Available (was: Open)

    The test failures are not related to this patch. Resubmitting patch to trigger another round of HadoopQA tests
    HFTP interface compatibility with older releases broken
    -------------------------------------------------------

    Key: HADOOP-4099
    URL: https://issues.apache.org/jira/browse/HADOOP-4099
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.19.0
    Reporter: Kan Zhang
    Assignee: dhruba borthakur
    Priority: Blocker
    Fix For: 0.19.0

    Attachments: hftpAccessTime.patch


    Using current trunk (Revision 692556) build and trying to distcp from a 0.18.0 cluster via HFTP, got the following NullPointerException. Seems like on line 165 in HftpFileSystem.java, a potential null return value isn't checked before using.
    coursemud-lm:bin kan$ ./hadoop distcp hftp://ucdev18.inktomisearch.com:50070/output/part-00000 /copied/part-00001
    08/09/05 19:00:10 INFO tools.DistCp: srcPaths=[hftp://ucdev18.inktomisearch.com:50070/output/part-00000]
    08/09/05 19:00:10 INFO tools.DistCp: destPath=/copied/part-00001
    With failures, global counters are inaccurate; consider running with -i
    Copy failed: java.lang.NullPointerException
    at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1215)
    at java.text.DateFormat.parse(DateFormat.java:335)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.startElement(HftpFileSystem.java:165)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:501)
    at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:179)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:377)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2740)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:647)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:508)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
    at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.fetchList(HftpFileSystem.java:194)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.getFileStatus(HftpFileSystem.java:205)
    at org.apache.hadoop.hdfs.HftpFileSystem.getFileStatus(HftpFileSystem.java:234)
    at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:649)
    at org.apache.hadoop.tools.DistCp.checkSrcPath(DistCp.java:614)
    at org.apache.hadoop.tools.DistCp.copy(DistCp.java:631)
    at org.apache.hadoop.tools.DistCp.run(DistCp.java:838)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.tools.DistCp.main(DistCp.java:865)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Sep 9, 2008 at 10:02 pm
    [ https://issues.apache.org/jira/browse/HADOOP-4099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12629630#action_12629630 ]

    Hadoop QA commented on HADOOP-4099:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12389693/hftpAccessTime.patch
    against trunk revision 693545.

    +1 @author. The patch does not contain any @author tags.

    -1 tests included. The patch doesn't appear to include any new or modified tests.
    Please justify why no tests are needed for this patch.

    +1 javadoc. The javadoc tool did not generate any warning messages.

    +1 javac. The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs. The patch does not introduce any new Findbugs warnings.

    -1 core tests. The patch failed core unit tests.

    +1 contrib tests. The patch passed contrib unit tests.

    Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3221/testReport/
    Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3221/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3221/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3221/console

    This message is automatically generated.
    HFTP interface compatibility with older releases broken
    -------------------------------------------------------

    Key: HADOOP-4099
    URL: https://issues.apache.org/jira/browse/HADOOP-4099
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.19.0
    Reporter: Kan Zhang
    Assignee: dhruba borthakur
    Priority: Blocker
    Fix For: 0.19.0

    Attachments: hftpAccessTime.patch


    Using current trunk (Revision 692556) build and trying to distcp from a 0.18.0 cluster via HFTP, got the following NullPointerException. Seems like on line 165 in HftpFileSystem.java, a potential null return value isn't checked before using.
    coursemud-lm:bin kan$ ./hadoop distcp hftp://ucdev18.inktomisearch.com:50070/output/part-00000 /copied/part-00001
    08/09/05 19:00:10 INFO tools.DistCp: srcPaths=[hftp://ucdev18.inktomisearch.com:50070/output/part-00000]
    08/09/05 19:00:10 INFO tools.DistCp: destPath=/copied/part-00001
    With failures, global counters are inaccurate; consider running with -i
    Copy failed: java.lang.NullPointerException
    at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1215)
    at java.text.DateFormat.parse(DateFormat.java:335)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.startElement(HftpFileSystem.java:165)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:501)
    at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:179)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:377)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2740)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:647)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:508)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
    at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.fetchList(HftpFileSystem.java:194)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.getFileStatus(HftpFileSystem.java:205)
    at org.apache.hadoop.hdfs.HftpFileSystem.getFileStatus(HftpFileSystem.java:234)
    at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:649)
    at org.apache.hadoop.tools.DistCp.checkSrcPath(DistCp.java:614)
    at org.apache.hadoop.tools.DistCp.copy(DistCp.java:631)
    at org.apache.hadoop.tools.DistCp.run(DistCp.java:838)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.tools.DistCp.main(DistCp.java:865)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Owen O'Malley (JIRA) at Sep 13, 2008 at 8:57 am
    [ https://issues.apache.org/jira/browse/HADOOP-4099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Owen O'Malley updated HADOOP-4099:
    ----------------------------------

    Resolution: Fixed
    Status: Resolved (was: Patch Available)

    I just committed this. Thanks, Dhruba!
    HFTP interface compatibility with older releases broken
    -------------------------------------------------------

    Key: HADOOP-4099
    URL: https://issues.apache.org/jira/browse/HADOOP-4099
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.19.0
    Reporter: Kan Zhang
    Assignee: dhruba borthakur
    Priority: Blocker
    Fix For: 0.19.0

    Attachments: hftpAccessTime.patch


    Using current trunk (Revision 692556) build and trying to distcp from a 0.18.0 cluster via HFTP, got the following NullPointerException. Seems like on line 165 in HftpFileSystem.java, a potential null return value isn't checked before using.
    coursemud-lm:bin kan$ ./hadoop distcp hftp://ucdev18.inktomisearch.com:50070/output/part-00000 /copied/part-00001
    08/09/05 19:00:10 INFO tools.DistCp: srcPaths=[hftp://ucdev18.inktomisearch.com:50070/output/part-00000]
    08/09/05 19:00:10 INFO tools.DistCp: destPath=/copied/part-00001
    With failures, global counters are inaccurate; consider running with -i
    Copy failed: java.lang.NullPointerException
    at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1215)
    at java.text.DateFormat.parse(DateFormat.java:335)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.startElement(HftpFileSystem.java:165)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:501)
    at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:179)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:377)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2740)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:647)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:508)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
    at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.fetchList(HftpFileSystem.java:194)
    at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.getFileStatus(HftpFileSystem.java:205)
    at org.apache.hadoop.hdfs.HftpFileSystem.getFileStatus(HftpFileSystem.java:234)
    at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:649)
    at org.apache.hadoop.tools.DistCp.checkSrcPath(DistCp.java:614)
    at org.apache.hadoop.tools.DistCp.copy(DistCp.java:631)
    at org.apache.hadoop.tools.DistCp.run(DistCp.java:838)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.tools.DistCp.main(DistCp.java:865)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedSep 6, '08 at 2:07a
activeSep 13, '08 at 8:57a
posts13
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Owen O'Malley (JIRA): 13 posts

People

Translate

site design / logo © 2022 Grokbase