FAQ
Distcp should log to specified location
---------------------------------------

Key: HADOOP-1533
URL: https://issues.apache.org/jira/browse/HADOOP-1533
Project: Hadoop
Issue Type: Improvement
Components: util
Reporter: Senthil Subramanian
Fix For: 0.14.0


Distcp errors are now not logged anywhere (other that reporter.setStatus which gets overwritten). Add support to mention a log URI where the log files should go.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Senthil Subramanian (JIRA) at Jun 27, 2007 at 11:35 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Senthil Subramanian updated HADOOP-1533:
    ----------------------------------------

    Status: Patch Available (was: Open)
    Distcp should log to specified location
    ---------------------------------------

    Key: HADOOP-1533
    URL: https://issues.apache.org/jira/browse/HADOOP-1533
    Project: Hadoop
    Issue Type: Improvement
    Components: util
    Reporter: Senthil Subramanian
    Fix For: 0.14.0

    Attachments: HADOOP-1533_1.patch


    Distcp errors are now not logged anywhere (other that reporter.setStatus which gets overwritten). Add support to mention a log URI where the log files should go.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Senthil Subramanian (JIRA) at Jun 27, 2007 at 11:35 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Senthil Subramanian updated HADOOP-1533:
    ----------------------------------------

    Attachment: HADOOP-1533_1.patch

    distcp now logs errors to specified logDir as outputs from map and Reduce=None
    Distcp should log to specified location
    ---------------------------------------

    Key: HADOOP-1533
    URL: https://issues.apache.org/jira/browse/HADOOP-1533
    Project: Hadoop
    Issue Type: Improvement
    Components: util
    Reporter: Senthil Subramanian
    Fix For: 0.14.0

    Attachments: HADOOP-1533_1.patch


    Distcp errors are now not logged anywhere (other that reporter.setStatus which gets overwritten). Add support to mention a log URI where the log files should go.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Jun 28, 2007 at 12:06 am
    [ https://issues.apache.org/jira/browse/HADOOP-1533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12508688 ]

    Hadoop QA commented on HADOOP-1533:
    -----------------------------------

    +0, new Findbugs warnings

    http://issues.apache.org/jira/secure/attachment/12360696/HADOOP-1533_1.patch
    applied and successfully tested against trunk revision r551361,
    but there appear to be new Findbugs warnings introduced by this patch.

    New Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/341/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/341/testReport/
    Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/341/console
    Distcp should log to specified location
    ---------------------------------------

    Key: HADOOP-1533
    URL: https://issues.apache.org/jira/browse/HADOOP-1533
    Project: Hadoop
    Issue Type: Improvement
    Components: util
    Reporter: Senthil Subramanian
    Fix For: 0.14.0

    Attachments: HADOOP-1533_1.patch


    Distcp errors are now not logged anywhere (other that reporter.setStatus which gets overwritten). Add support to mention a log URI where the log files should go.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Senthil Subramanian (JIRA) at Jun 28, 2007 at 12:16 am
    [ https://issues.apache.org/jira/browse/HADOOP-1533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Senthil Subramanian updated HADOOP-1533:
    ----------------------------------------

    Attachment: HADOOP-1533_1.patch

    Fixed findbug warning
    Distcp should log to specified location
    ---------------------------------------

    Key: HADOOP-1533
    URL: https://issues.apache.org/jira/browse/HADOOP-1533
    Project: Hadoop
    Issue Type: Improvement
    Components: util
    Reporter: Senthil Subramanian
    Fix For: 0.14.0

    Attachments: HADOOP-1533_1.patch


    Distcp errors are now not logged anywhere (other that reporter.setStatus which gets overwritten). Add support to mention a log URI where the log files should go.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Senthil Subramanian (JIRA) at Jun 28, 2007 at 12:16 am
    [ https://issues.apache.org/jira/browse/HADOOP-1533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Senthil Subramanian updated HADOOP-1533:
    ----------------------------------------

    Attachment: (was: HADOOP-1533_1.patch)
    Distcp should log to specified location
    ---------------------------------------

    Key: HADOOP-1533
    URL: https://issues.apache.org/jira/browse/HADOOP-1533
    Project: Hadoop
    Issue Type: Improvement
    Components: util
    Reporter: Senthil Subramanian
    Fix For: 0.14.0

    Attachments: HADOOP-1533_1.patch


    Distcp errors are now not logged anywhere (other that reporter.setStatus which gets overwritten). Add support to mention a log URI where the log files should go.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Senthil Subramanian (JIRA) at Jun 28, 2007 at 12:16 am
    [ https://issues.apache.org/jira/browse/HADOOP-1533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Senthil Subramanian updated HADOOP-1533:
    ----------------------------------------

    Status: Open (was: Patch Available)
    Distcp should log to specified location
    ---------------------------------------

    Key: HADOOP-1533
    URL: https://issues.apache.org/jira/browse/HADOOP-1533
    Project: Hadoop
    Issue Type: Improvement
    Components: util
    Reporter: Senthil Subramanian
    Fix For: 0.14.0

    Attachments: HADOOP-1533_1.patch


    Distcp errors are now not logged anywhere (other that reporter.setStatus which gets overwritten). Add support to mention a log URI where the log files should go.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Senthil Subramanian (JIRA) at Jun 28, 2007 at 12:18 am
    [ https://issues.apache.org/jira/browse/HADOOP-1533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Senthil Subramanian updated HADOOP-1533:
    ----------------------------------------

    Status: Patch Available (was: Open)
    Distcp should log to specified location
    ---------------------------------------

    Key: HADOOP-1533
    URL: https://issues.apache.org/jira/browse/HADOOP-1533
    Project: Hadoop
    Issue Type: Improvement
    Components: util
    Reporter: Senthil Subramanian
    Fix For: 0.14.0

    Attachments: HADOOP-1533_1.patch


    Distcp errors are now not logged anywhere (other that reporter.setStatus which gets overwritten). Add support to mention a log URI where the log files should go.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Jun 28, 2007 at 1:07 am
    [ https://issues.apache.org/jira/browse/HADOOP-1533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12508699 ]

    Hadoop QA commented on HADOOP-1533:
    -----------------------------------

    +1

    http://issues.apache.org/jira/secure/attachment/12360699/HADOOP-1533_1.patch applied and successfully tested against trunk revision r551361.

    Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/342/testReport/
    Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/342/console
    Distcp should log to specified location
    ---------------------------------------

    Key: HADOOP-1533
    URL: https://issues.apache.org/jira/browse/HADOOP-1533
    Project: Hadoop
    Issue Type: Improvement
    Components: util
    Reporter: Senthil Subramanian
    Fix For: 0.14.0

    Attachments: HADOOP-1533_1.patch


    Distcp errors are now not logged anywhere (other that reporter.setStatus which gets overwritten). Add support to mention a log URI where the log files should go.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Owen O'Malley (JIRA) at Jul 1, 2007 at 6:08 am
    [ https://issues.apache.org/jira/browse/HADOOP-1533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Owen O'Malley updated HADOOP-1533:
    ----------------------------------

    Status: Open (was: Patch Available)

    This patch seems confused whether the "log path" is a Path, URI, or a String. Since you are implementing it as the output of the Map (with reduces = 0), it pretty clearly must be a Path. It should be coded as such.

    Instead of requiring a parameter for the log path, I think it should be an option like:
    ... - log hdfs://server:1234/my/log
    and I'd propose that the default be: "$output_dir/_distcp_logs"

    That will work well in most cases and keep the output/logs together.

    The chunk of code that looks like:

    + String logURL = "hdfs://" + fsConfig.get("fs.default.name", "local") +
    + archiveDirectory + "distcpLogDir." + System.currentTimeMillis();

    is problematic, since it won't construct useful urls in a wide variety of values for fs.default.name.
    Distcp should log to specified location
    ---------------------------------------

    Key: HADOOP-1533
    URL: https://issues.apache.org/jira/browse/HADOOP-1533
    Project: Hadoop
    Issue Type: Improvement
    Components: util
    Reporter: Senthil Subramanian
    Fix For: 0.14.0

    Attachments: HADOOP-1533_1.patch


    Distcp errors are now not logged anywhere (other that reporter.setStatus which gets overwritten). Add support to mention a log URI where the log files should go.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Senthil Subramanian (JIRA) at Jul 3, 2007 at 1:00 am
    [ https://issues.apache.org/jira/browse/HADOOP-1533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Senthil Subramanian updated HADOOP-1533:
    ----------------------------------------

    Attachment: (was: HADOOP-1533_1.patch)
    Distcp should log to specified location
    ---------------------------------------

    Key: HADOOP-1533
    URL: https://issues.apache.org/jira/browse/HADOOP-1533
    Project: Hadoop
    Issue Type: Improvement
    Components: util
    Reporter: Senthil Subramanian
    Fix For: 0.14.0


    Distcp errors are now not logged anywhere (other that reporter.setStatus which gets overwritten). Add support to mention a log URI where the log files should go.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Senthil Subramanian (JIRA) at Jul 3, 2007 at 1:08 am
    [ https://issues.apache.org/jira/browse/HADOOP-1533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Senthil Subramanian updated HADOOP-1533:
    ----------------------------------------

    Attachment: HADOOP-1533_2.patch

    Based on Owen's comments, I have made the following changes:
    - logPath is now clearly a Path
    - logPath is now optional and can be specified with a -log switch. Default is "$output_dir/_distcp_logs
    _System.currentTimeMillis()"
    Distcp should log to specified location
    ---------------------------------------

    Key: HADOOP-1533
    URL: https://issues.apache.org/jira/browse/HADOOP-1533
    Project: Hadoop
    Issue Type: Improvement
    Components: util
    Reporter: Senthil Subramanian
    Fix For: 0.14.0

    Attachments: HADOOP-1533_2.patch


    Distcp errors are now not logged anywhere (other that reporter.setStatus which gets overwritten). Add support to mention a log URI where the log files should go.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Senthil Subramanian (JIRA) at Jul 3, 2007 at 1:10 am
    [ https://issues.apache.org/jira/browse/HADOOP-1533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Senthil Subramanian updated HADOOP-1533:
    ----------------------------------------

    Status: Patch Available (was: Open)
    Distcp should log to specified location
    ---------------------------------------

    Key: HADOOP-1533
    URL: https://issues.apache.org/jira/browse/HADOOP-1533
    Project: Hadoop
    Issue Type: Improvement
    Components: util
    Reporter: Senthil Subramanian
    Fix For: 0.14.0

    Attachments: HADOOP-1533_2.patch


    Distcp errors are now not logged anywhere (other that reporter.setStatus which gets overwritten). Add support to mention a log URI where the log files should go.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Jul 3, 2007 at 3:03 am
    [ https://issues.apache.org/jira/browse/HADOOP-1533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12509739 ]

    Hadoop QA commented on HADOOP-1533:
    -----------------------------------

    +1

    http://issues.apache.org/jira/secure/attachment/12360981/HADOOP-1533_2.patch applied and successfully tested against trunk revision r552548.

    Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/356/testReport/
    Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/356/console
    Distcp should log to specified location
    ---------------------------------------

    Key: HADOOP-1533
    URL: https://issues.apache.org/jira/browse/HADOOP-1533
    Project: Hadoop
    Issue Type: Improvement
    Components: util
    Reporter: Senthil Subramanian
    Fix For: 0.14.0

    Attachments: HADOOP-1533_2.patch


    Distcp errors are now not logged anywhere (other that reporter.setStatus which gets overwritten). Add support to mention a log URI where the log files should go.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Doug Cutting (JIRA) at Jul 3, 2007 at 7:51 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Doug Cutting reassigned HADOOP-1533:
    ------------------------------------

    Assignee: Senthil Subramanian
    Distcp should log to specified location
    ---------------------------------------

    Key: HADOOP-1533
    URL: https://issues.apache.org/jira/browse/HADOOP-1533
    Project: Hadoop
    Issue Type: Improvement
    Components: util
    Reporter: Senthil Subramanian
    Assignee: Senthil Subramanian
    Fix For: 0.14.0

    Attachments: HADOOP-1533_2.patch


    Distcp errors are now not logged anywhere (other that reporter.setStatus which gets overwritten). Add support to mention a log URI where the log files should go.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Owen O'Malley (JIRA) at Jul 11, 2007 at 6:39 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Owen O'Malley updated HADOOP-1533:
    ----------------------------------

    Resolution: Fixed
    Status: Resolved (was: Patch Available)

    This has been committed. Thanks, Senthil.
    Distcp should log to specified location
    ---------------------------------------

    Key: HADOOP-1533
    URL: https://issues.apache.org/jira/browse/HADOOP-1533
    Project: Hadoop
    Issue Type: Improvement
    Components: util
    Reporter: Senthil Subramanian
    Assignee: Senthil Subramanian
    Fix For: 0.14.0

    Attachments: HADOOP-1533_2.patch


    Distcp errors are now not logged anywhere (other that reporter.setStatus which gets overwritten). Add support to mention a log URI where the log files should go.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Konstantin Shvachko (JIRA) at Jul 11, 2007 at 10:23 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Konstantin Shvachko reopened HADOOP-1533:
    -----------------------------------------


    This breaks Windows build. checkPath() fails.

    Copy failed: java.lang.IllegalArgumentException: Wrong FS: file://C:/Work/nutch-0.7.1/Hadoop/build/test/data/srcdat, expected: file:///
    at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:204)
    at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:50)
    at org.apache.hadoop.fs.RawLocalFileSystem.exists(RawLocalFileSystem.java:219)
    at org.apache.hadoop.fs.FilterFileSystem.exists(FilterFileSystem.java:156)
    at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:832)
    at org.apache.hadoop.util.ToolBase.doMain(ToolBase.java:187)
    at org.apache.hadoop.fs.TestCopyFiles.testCopyFromLocalToLocal(TestCopyFiles.java:166)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:585)
    at junit.framework.TestCase.runTest(TestCase.java:154)
    at junit.framework.TestCase.runBare(TestCase.java:127)
    at junit.framework.TestResult$1.protect(TestResult.java:106)
    at junit.framework.TestResult.runProtected(TestResult.java:124)
    at junit.framework.TestResult.run(TestResult.java:109)
    at junit.framework.TestCase.run(TestCase.java:118)
    at junit.framework.TestSuite.runTest(TestSuite.java:208)
    at junit.framework.TestSuite.run(TestSuite.java:203)
    at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:297)
    at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:672)
    at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:567)
    Distcp should log to specified location
    ---------------------------------------

    Key: HADOOP-1533
    URL: https://issues.apache.org/jira/browse/HADOOP-1533
    Project: Hadoop
    Issue Type: Improvement
    Components: util
    Reporter: Senthil Subramanian
    Assignee: Senthil Subramanian
    Fix For: 0.14.0

    Attachments: HADOOP-1533_2.patch


    Distcp errors are now not logged anywhere (other that reporter.setStatus which gets overwritten). Add support to mention a log URI where the log files should go.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hudson (JIRA) at Jul 12, 2007 at 11:42 am
    [ https://issues.apache.org/jira/browse/HADOOP-1533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12512051 ]

    Hudson commented on HADOOP-1533:
    --------------------------------

    Integrated in Hadoop-Nightly #152 (See [http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/152/])
    Distcp should log to specified location
    ---------------------------------------

    Key: HADOOP-1533
    URL: https://issues.apache.org/jira/browse/HADOOP-1533
    Project: Hadoop
    Issue Type: Improvement
    Components: util
    Reporter: Senthil Subramanian
    Assignee: Senthil Subramanian
    Fix For: 0.14.0

    Attachments: HADOOP-1533_2.patch


    Distcp errors are now not logged anywhere (other that reporter.setStatus which gets overwritten). Add support to mention a log URI where the log files should go.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Senthil Subramanian (JIRA) at Sep 4, 2007 at 7:25 pm
    [ https://issues.apache.org/jira/browse/HADOOP-1533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Senthil Subramanian resolved HADOOP-1533.
    -----------------------------------------

    Resolution: Fixed
    Distcp should log to specified location
    ---------------------------------------

    Key: HADOOP-1533
    URL: https://issues.apache.org/jira/browse/HADOOP-1533
    Project: Hadoop
    Issue Type: Improvement
    Components: util
    Reporter: Senthil Subramanian
    Assignee: Senthil Subramanian
    Attachments: HADOOP-1533_2.patch


    Distcp errors are now not logged anywhere (other that reporter.setStatus which gets overwritten). Add support to mention a log URI where the log files should go.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedJun 26, '07 at 9:18p
activeSep 4, '07 at 7:25p
posts19
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Senthil Subramanian (JIRA): 19 posts

People

Translate

site design / logo © 2022 Grokbase