FAQ
discp can preserve modification times of files
----------------------------------------------

Key: HADOOP-5620
URL: https://issues.apache.org/jira/browse/HADOOP-5620
Project: Hadoop Core
Issue Type: Improvement
Components: tools/distcp
Reporter: dhruba borthakur


It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • dhruba borthakur (JIRA) at Apr 3, 2009 at 6:58 am
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    dhruba borthakur reassigned HADOOP-5620:
    ----------------------------------------

    Assignee: Rodrigo Schmidt
    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt

    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Rodrigo Schmidt (JIRA) at May 21, 2009 at 6:13 am
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711484#action_12711484 ]

    Rodrigo Schmidt commented on HADOOP-5620:
    -----------------------------------------

    distcp itself updates the access times of the files it copies. How should this option work in such case?

    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt

    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • dhruba borthakur (JIRA) at May 21, 2009 at 1:53 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711624#action_12711624 ]

    dhruba borthakur commented on HADOOP-5620:
    ------------------------------------------

    The method FileSystem.setTime() allows setting the access time of the file.
    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt

    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • dhruba borthakur (JIRA) at May 21, 2009 at 1:53 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711623#action_12711623 ]

    dhruba borthakur commented on HADOOP-5620:
    ------------------------------------------

    After distcp copies a file from src to dest, the dest file should have the same access time as the one in the src file.
    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt

    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Rodrigo Schmidt (JIRA) at May 21, 2009 at 2:34 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711636#action_12711636 ]

    Rodrigo Schmidt commented on HADOOP-5620:
    -----------------------------------------

    let's assume we have a file /src/src.file, whose access time is Mar 30, 2005, and then we run

    $ bin/hadoop dstcp /src/src.file /dst

    By the end of the execution, since distcp reads the source, the access time of /src/src.file will be close to now().

    My question is more fundamental: Does it make sense to preserve the access time, if the copying itself will update the access time of the source? What are we trying to preserve with it?

    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt

    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Raghu Angadi (JIRA) at May 21, 2009 at 7:12 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711761#action_12711761 ]

    Raghu Angadi commented on HADOOP-5620:
    --------------------------------------

    Of course, modification time is the more often used stat. But setting access time does not cost more and seems like the right thing to do. Even though access times for both source and dest files (usually) changes during copy, it would not be same unless set explicitly.

    Alternately we could just do what ever Linux 'cp -a' command does. If it preserves access time, we should.

    Also some file systems might not configured to update access times all the time.

    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt

    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Rodrigo Schmidt (JIRA) at May 21, 2009 at 7:58 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711777#action_12711777 ]

    Rodrigo Schmidt commented on HADOOP-5620:
    -----------------------------------------

    The main reason we needed this option was to preserve the properties of files that are being migrated from one cluster to another one, without losing last access and modification times (and without interfering with it). In that sense, I think the best option is to set the destination with last modification and access times of the source file immediately before it is copied.
    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt
    Fix For: 0.21.0

    Attachments: HADOOP-5620.patch


    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Rodrigo Schmidt (JIRA) at May 21, 2009 at 7:58 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Rodrigo Schmidt updated HADOOP-5620:
    ------------------------------------

    Fix Version/s: 0.21.0
    Status: Patch Available (was: Open)
    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt
    Fix For: 0.21.0

    Attachments: HADOOP-5620.patch


    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Rodrigo Schmidt (JIRA) at May 21, 2009 at 7:58 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Rodrigo Schmidt updated HADOOP-5620:
    ------------------------------------

    Attachment: HADOOP-5620.patch
    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt
    Fix For: 0.21.0

    Attachments: HADOOP-5620.patch


    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at May 24, 2009 at 8:35 am
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712525#action_12712525 ]

    Hadoop QA commented on HADOOP-5620:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12408734/HADOOP-5620.patch
    against trunk revision 777761.

    +1 @author. The patch does not contain any @author tags.

    +1 tests included. The patch appears to include 4 new or modified tests.

    +1 javadoc. The javadoc tool did not generate any warning messages.

    +1 javac. The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs. The patch does not introduce any new Findbugs warnings.

    +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

    +1 release audit. The applied patch does not increase the total number of release audit warnings.

    +1 core tests. The patch passed core unit tests.

    -1 contrib tests. The patch failed contrib unit tests.

    Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/390/testReport/
    Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/390/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/390/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/390/console

    This message is automatically generated.
    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt
    Fix For: 0.21.0

    Attachments: HADOOP-5620.patch


    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at May 26, 2009 at 6:01 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Tsz Wo (Nicholas), SZE updated HADOOP-5620:
    -------------------------------------------

    Hadoop Flags: [Incompatible change]

    - Should discp also preserve modification times of directories? Otherwise, the new -t option seems confusing.

    - It might be better to get mtime and atime inside the if-statement
    {code}
    if ( preserve_status && preseved.contains(FileAttribute.TIMES)) {
    ...
    }
    {code}
    since srcstat remains unchanged in the method.

    - Marking this as an incompatible change because it change the meaning of -p.
    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt
    Fix For: 0.21.0

    Attachments: HADOOP-5620.patch


    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Rodrigo Schmidt (JIRA) at May 26, 2009 at 8:26 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12713232#action_12713232 ]

    Rodrigo Schmidt commented on HADOOP-5620:
    -----------------------------------------

    - Dhruba told me modification times of directories are not persistent, that is, on namenode restart they are set to the latest modification time amongst the files they contain.

    - If we get atime inside the if, it will be the copy time (last access after copying the file) instead of the latest access time before copying, which is what we need for migration.
    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt
    Fix For: 0.21.0

    Attachments: HADOOP-5620.patch


    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at May 26, 2009 at 9:15 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12713260#action_12713260 ]

    Tsz Wo (Nicholas), SZE commented on HADOOP-5620:
    ------------------------------------------------
    Dhruba told me modification times of directories are not persistent, that is, on namenode restart they are set to the latest modification time amongst the files they contain.
    I just have checked the codes. It seems not true.

    Also, DistCp works on general FileSystem. It should not depend on a particular implementation.
    If we get atime inside the if, it will be the copy time (last access after copying the file) instead of the latest access time before copying, which is what we need for migration.
    FileStatus is a local object. Once it has been obtained from a FileSystem it remains unchanged even the actual status of the file is changed. So the atime inside the if-statement will be the latest access time before copying since getFileStatus is called before copying.


    BTW, there is a white space change in the patch, could you remove it?
    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt
    Fix For: 0.21.0

    Attachments: HADOOP-5620.patch


    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Rodrigo Schmidt (JIRA) at May 26, 2009 at 10:44 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Rodrigo Schmidt updated HADOOP-5620:
    ------------------------------------

    Attachment: HADOOP-5620.2.patch

    Thanks for the comments! The new patch has a much more elegant solution that works for both files and directories.
    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt
    Fix For: 0.21.0

    Attachments: HADOOP-5620.2.patch, HADOOP-5620.patch


    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at May 26, 2009 at 10:53 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12713312#action_12713312 ]

    Tsz Wo (Nicholas), SZE commented on HADOOP-5620:
    ------------------------------------------------

    Yes, the new patch is much more elegant.

    Could you rename the method updatePermissions(..) to something like updateStatus(..)? Otherwise, patch looks good.
    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt
    Fix For: 0.21.0

    Attachments: HADOOP-5620.2.patch, HADOOP-5620.patch


    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Rodrigo Schmidt (JIRA) at May 26, 2009 at 11:56 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Rodrigo Schmidt updated HADOOP-5620:
    ------------------------------------

    Attachment: HADOOP-5620.3.patch

    Changed it to updateDestStatus since method updateStatus already exists.
    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt
    Fix For: 0.21.0

    Attachments: HADOOP-5620.2.patch, HADOOP-5620.3.patch, HADOOP-5620.patch


    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at May 27, 2009 at 12:37 am
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Tsz Wo (Nicholas), SZE updated HADOOP-5620:
    -------------------------------------------

    Status: Open (was: Patch Available)
    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt
    Fix For: 0.21.0

    Attachments: HADOOP-5620.2.patch, HADOOP-5620.3.patch, HADOOP-5620.patch


    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at May 27, 2009 at 12:39 am
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Tsz Wo (Nicholas), SZE updated HADOOP-5620:
    -------------------------------------------

    Hadoop Flags: [Incompatible change, Reviewed] (was: [Incompatible change])
    Status: Patch Available (was: Open)

    +1 new patch looks good.
    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt
    Fix For: 0.21.0

    Attachments: HADOOP-5620.2.patch, HADOOP-5620.3.patch, HADOOP-5620.patch


    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Rodrigo Schmidt (JIRA) at May 27, 2009 at 8:12 am
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Rodrigo Schmidt updated HADOOP-5620:
    ------------------------------------

    Status: In Progress (was: Patch Available)
    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt
    Fix For: 0.21.0

    Attachments: HADOOP-5620.2.patch, HADOOP-5620.3.patch, HADOOP-5620.patch


    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Rodrigo Schmidt (JIRA) at May 27, 2009 at 8:12 am
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Rodrigo Schmidt updated HADOOP-5620:
    ------------------------------------

    Status: Patch Available (was: In Progress)
    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt
    Fix For: 0.21.0

    Attachments: HADOOP-5620.2.patch, HADOOP-5620.3.patch, HADOOP-5620.patch


    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Rodrigo Schmidt (JIRA) at May 27, 2009 at 7:57 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12713708#action_12713708 ]

    Rodrigo Schmidt commented on HADOOP-5620:
    -----------------------------------------

    $ ant -Dpatch.file=/.../HADOOP-5620.3.patch -Dforrest.home=/.../apache-forrest-0.8/ -Dfindbugs.home=/.../findbugs-1.3.8/ -Djava5.home=/.../jdk1.5.0_07/ test-patch

    [exec]
    [exec] +1 overall.
    [exec]
    [exec] +1 @author. The patch does not contain any @author tags.
    [exec]
    [exec] +1 tests included. The patch appears to include 4 new or modified tests.
    [exec]
    [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
    [exec]
    [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
    [exec]
    [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
    [exec]
    [exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
    [exec]
    [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
    [exec]

    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt
    Fix For: 0.21.0

    Attachments: HADOOP-5620.2.patch, HADOOP-5620.3.patch, HADOOP-5620.patch


    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at May 27, 2009 at 10:55 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12713773#action_12713773 ]

    Tsz Wo (Nicholas), SZE commented on HADOOP-5620:
    ------------------------------------------------

    Hi Rodrigo, have you also run the unit tests in your local machine?

    BTW, since this is an incompatible change (which changes the distcp command options), please add release note in this jira. Could you also update the forrest doc (in this or a separated jira)? .
    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt
    Fix For: 0.21.0

    Attachments: HADOOP-5620.2.patch, HADOOP-5620.3.patch, HADOOP-5620.patch


    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Rodrigo Schmidt (JIRA) at May 27, 2009 at 10:56 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Rodrigo Schmidt updated HADOOP-5620:
    ------------------------------------

    Release Note: DistCp can now preserve last modification and last access times of copied files with option -pt
    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt
    Fix For: 0.21.0

    Attachments: HADOOP-5620.2.patch, HADOOP-5620.3.patch, HADOOP-5620.patch


    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Rodrigo Schmidt (JIRA) at May 27, 2009 at 10:57 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12713786#action_12713786 ]

    Rodrigo Schmidt commented on HADOOP-5620:
    -----------------------------------------

    I ran TestCopyFiles manually. I was expecting "ant ... test-patch" to run all the unit tests, but I was surprised to see it didn't. What's the option to run the exact same tests performed by Hadoop QA?

    I'm changing the forrest doc...
    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt
    Fix For: 0.21.0

    Attachments: HADOOP-5620.2.patch, HADOOP-5620.3.patch, HADOOP-5620.patch


    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at May 27, 2009 at 11:00 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12713790#action_12713790 ]

    Tsz Wo (Nicholas), SZE commented on HADOOP-5620:
    ------------------------------------------------
    What's the option to run the exact same tests performed by Hadoop QA?
    You may do "ant test" for running all unit tests.
    I'm changing the forrest doc... Thanks.
    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt
    Fix For: 0.21.0

    Attachments: HADOOP-5620.2.patch, HADOOP-5620.3.patch, HADOOP-5620.patch


    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Rodrigo Schmidt (JIRA) at May 27, 2009 at 11:03 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12713793#action_12713793 ]

    Rodrigo Schmidt commented on HADOOP-5620:
    -----------------------------------------

    I realize it makes more sense to change the forrest doc only once, with all the new features we are introducing. I created HADOOP-5927 for that.
    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt
    Fix For: 0.21.0

    Attachments: HADOOP-5620.2.patch, HADOOP-5620.3.patch, HADOOP-5620.patch


    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Rodrigo Schmidt (JIRA) at May 28, 2009 at 10:19 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714196#action_12714196 ]

    Rodrigo Schmidt commented on HADOOP-5620:
    -----------------------------------------

    I ran the unit tests and the three following errors appeared. They all seem to be unrelated to this patch:

    --

    {code}
    Testsuite: org.apache.hadoop.mapred.TestJobTrackerRestartWithLostTracker
    Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec

    Testcase: testRestartWithLostTracker took 0.003 sec
    Caused an ERROR
    Timeout occurred. Please note the time in the report does not reflect the time until the timeout.
    junit.framework.AssertionFailedError: Timeout occurred. Please note the time in the report does not reflect the time until the timeout.
    {code}

    --

    {code}
    Testsuite: org.apache.hadoop.mapred.TestReduceFetch
    Tests run: 3, Failures: 1, Errors: 0, Time elapsed: 282.598 sec

    Testcase: testReduceFromPartialMem took 27.634 sec
    FAILED
    Expected at least 1MB fewer bytes read from local (21159650) than written to HDFS (21036680)
    junit.framework.AssertionFailedError: Expected at least 1MB fewer bytes read from local (21159650) than written to HDFS (21036680)
    at org.apache.hadoop.mapred.TestReduceFetch.testReduceFromPartialMem(TestReduceFetch.java:276)
    at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
    at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
    at junit.extensions.TestSetup.run(TestSetup.java:27)
    {code}

    --

    {code}
    Testsuite: org.apache.hadoop.mapred.TestTaskFail
    Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 92.65 sec

    Testcase: testWithDFS took 92.629 sec
    FAILED
    expected:<FAILED> but was:<KILLED>
    junit.framework.AssertionFailedError: expected:<FAILED> but was:(TestTaskFail.java:139)
    at org.apache.hadoop.mapred.TestTaskFail.testWithDFS(TestTaskFail.java:170)
    {code}
    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt
    Fix For: 0.21.0

    Attachments: HADOOP-5620.2.patch, HADOOP-5620.3.patch, HADOOP-5620.patch


    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at May 28, 2009 at 11:32 pm
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Tsz Wo (Nicholas), SZE updated HADOOP-5620:
    -------------------------------------------

    Resolution: Fixed
    Status: Resolved (was: Patch Available)

    I have committed this. Thanks, Rodrigo!
    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt
    Fix For: 0.21.0

    Attachments: HADOOP-5620.2.patch, HADOOP-5620.3.patch, HADOOP-5620.patch


    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at May 29, 2009 at 2:14 am
    [ https://issues.apache.org/jira/browse/HADOOP-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714273#action_12714273 ]

    Hadoop QA commented on HADOOP-5620:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12409098/HADOOP-5620.3.patch
    against trunk revision 779656.

    +1 @author. The patch does not contain any @author tags.

    +1 tests included. The patch appears to include 4 new or modified tests.

    +1 javadoc. The javadoc tool did not generate any warning messages.

    +1 javac. The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs. The patch does not introduce any new Findbugs warnings.

    +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

    +1 release audit. The applied patch does not increase the total number of release audit warnings.

    -1 core tests. The patch failed core unit tests.

    -1 contrib tests. The patch failed contrib unit tests.

    Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/421/testReport/
    Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/421/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/421/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/421/console

    This message is automatically generated.
    discp can preserve modification times of files
    ----------------------------------------------

    Key: HADOOP-5620
    URL: https://issues.apache.org/jira/browse/HADOOP-5620
    Project: Hadoop Core
    Issue Type: Improvement
    Components: tools/distcp
    Reporter: dhruba borthakur
    Assignee: Rodrigo Schmidt
    Fix For: 0.21.0

    Attachments: HADOOP-5620.2.patch, HADOOP-5620.3.patch, HADOOP-5620.patch


    It will be helpful if distcp can preserve the modification time and access time of files. This helps to archive/unarchive hdfs files.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedApr 3, '09 at 6:54a
activeMay 29, '09 at 2:14a
posts30
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Hadoop QA (JIRA): 30 posts

People

Translate

site design / logo © 2022 Grokbase