FAQ
Handling of Trash with quota
-----------------------------

Key: HADOOP-6080
URL: https://issues.apache.org/jira/browse/HADOOP-6080
Project: Hadoop Core
Issue Type: New Feature
Components: fs
Reporter: Koji Noguchi


Currently with quota turned on, user cannot call '-rmr' on large directory that causes over quota.

{noformat}
[knoguchi src]$ hadoop dfs -rmr /tmp/net2
rmr: Failed to move to trash: hdfs://abc.def.com/tmp/net2
[knoguchi src]$ hadoop dfs -mv /tmp/net2 /user/knoguchi/.Trash/Current
mv: org.apache.hadoop.hdfs.protocol.QuotaExceededException: The quota of /user/knoguchi is exceeded: namespace
quota=37500 file count=37757, diskspace quota=-1 diskspace=1991250043353
{noformat}

Besides from error message being unfriendly, how should this be handled?


--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Koji Noguchi (JIRA) at Jun 18, 2009 at 7:04 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12721420#action_12721420 ]

    Koji Noguchi commented on HADOOP-6080:
    --------------------------------------

    1) Tell users to use -Dfs.trash.interval=0 when deleting large directory
    2) Exclude /user/<username>/.Trash from the quota
    3) Move .Trash out of /user directory. Maybe /Trash/<username> and set different quota.
    4) When -rm/rmr fail with quota, automatically delete them.
    5) Introduce a separate command that does (1). Something like -rmr -skipTrash for force delete.

    Handling of Trash with quota
    -----------------------------

    Key: HADOOP-6080
    URL: https://issues.apache.org/jira/browse/HADOOP-6080
    Project: Hadoop Core
    Issue Type: New Feature
    Components: fs
    Reporter: Koji Noguchi

    Currently with quota turned on, user cannot call '-rmr' on large directory that causes over quota.
    {noformat}
    [knoguchi src]$ hadoop dfs -rmr /tmp/net2
    rmr: Failed to move to trash: hdfs://abc.def.com/tmp/net2
    [knoguchi src]$ hadoop dfs -mv /tmp/net2 /user/knoguchi/.Trash/Current
    mv: org.apache.hadoop.hdfs.protocol.QuotaExceededException: The quota of /user/knoguchi is exceeded: namespace
    quota=37500 file count=37757, diskspace quota=-1 diskspace=1991250043353
    {noformat}
    Besides from error message being unfriendly, how should this be handled?
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jakob Homan (JIRA) at Jun 18, 2009 at 7:32 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jakob Homan reassigned HADOOP-6080:
    -----------------------------------

    Assignee: Jakob Homan
    Handling of Trash with quota
    -----------------------------

    Key: HADOOP-6080
    URL: https://issues.apache.org/jira/browse/HADOOP-6080
    Project: Hadoop Core
    Issue Type: New Feature
    Components: fs
    Reporter: Koji Noguchi
    Assignee: Jakob Homan

    Currently with quota turned on, user cannot call '-rmr' on large directory that causes over quota.
    {noformat}
    [knoguchi src]$ hadoop dfs -rmr /tmp/net2
    rmr: Failed to move to trash: hdfs://abc.def.com/tmp/net2
    [knoguchi src]$ hadoop dfs -mv /tmp/net2 /user/knoguchi/.Trash/Current
    mv: org.apache.hadoop.hdfs.protocol.QuotaExceededException: The quota of /user/knoguchi is exceeded: namespace
    quota=37500 file count=37757, diskspace quota=-1 diskspace=1991250043353
    {noformat}
    Besides from error message being unfriendly, how should this be handled?
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Raghu Angadi (JIRA) at Jun 18, 2009 at 7:46 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12721430#action_12721430 ]

    Raghu Angadi commented on HADOOP-6080:
    --------------------------------------

    +1 for (5).
    Handling of Trash with quota
    -----------------------------

    Key: HADOOP-6080
    URL: https://issues.apache.org/jira/browse/HADOOP-6080
    Project: Hadoop Core
    Issue Type: New Feature
    Components: fs
    Reporter: Koji Noguchi
    Assignee: Jakob Homan

    Currently with quota turned on, user cannot call '-rmr' on large directory that causes over quota.
    {noformat}
    [knoguchi src]$ hadoop dfs -rmr /tmp/net2
    rmr: Failed to move to trash: hdfs://abc.def.com/tmp/net2
    [knoguchi src]$ hadoop dfs -mv /tmp/net2 /user/knoguchi/.Trash/Current
    mv: org.apache.hadoop.hdfs.protocol.QuotaExceededException: The quota of /user/knoguchi is exceeded: namespace
    quota=37500 file count=37757, diskspace quota=-1 diskspace=1991250043353
    {noformat}
    Besides from error message being unfriendly, how should this be handled?
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Jun 18, 2009 at 10:12 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12721525#action_12721525 ]

    Tsz Wo (Nicholas), SZE commented on HADOOP-6080:
    ------------------------------------------------

    If we do (5), we still have to do (1), i.e. add a message telling the user to use -skipTrash .
    Handling of Trash with quota
    -----------------------------

    Key: HADOOP-6080
    URL: https://issues.apache.org/jira/browse/HADOOP-6080
    Project: Hadoop Core
    Issue Type: New Feature
    Components: fs
    Reporter: Koji Noguchi
    Assignee: Jakob Homan

    Currently with quota turned on, user cannot call '-rmr' on large directory that causes over quota.
    {noformat}
    [knoguchi src]$ hadoop dfs -rmr /tmp/net2
    rmr: Failed to move to trash: hdfs://abc.def.com/tmp/net2
    [knoguchi src]$ hadoop dfs -mv /tmp/net2 /user/knoguchi/.Trash/Current
    mv: org.apache.hadoop.hdfs.protocol.QuotaExceededException: The quota of /user/knoguchi is exceeded: namespace
    quota=37500 file count=37757, diskspace quota=-1 diskspace=1991250043353
    {noformat}
    Besides from error message being unfriendly, how should this be handled?
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jakob Homan (JIRA) at Jun 26, 2009 at 12:12 am
    [ https://issues.apache.org/jira/browse/HADOOP-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12724349#action_12724349 ]

    Jakob Homan commented on HADOOP-6080:
    -------------------------------------

    I'm going to go ahead and implement 5.
    Handling of Trash with quota
    -----------------------------

    Key: HADOOP-6080
    URL: https://issues.apache.org/jira/browse/HADOOP-6080
    Project: Hadoop Common
    Issue Type: New Feature
    Components: fs
    Reporter: Koji Noguchi
    Assignee: Jakob Homan

    Currently with quota turned on, user cannot call '-rmr' on large directory that causes over quota.
    {noformat}
    [knoguchi src]$ hadoop dfs -rmr /tmp/net2
    rmr: Failed to move to trash: hdfs://abc.def.com/tmp/net2
    [knoguchi src]$ hadoop dfs -mv /tmp/net2 /user/knoguchi/.Trash/Current
    mv: org.apache.hadoop.hdfs.protocol.QuotaExceededException: The quota of /user/knoguchi is exceeded: namespace
    quota=37500 file count=37757, diskspace quota=-1 diskspace=1991250043353
    {noformat}
    Besides from error message being unfriendly, how should this be handled?
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jakob Homan (JIRA) at Jun 30, 2009 at 1:15 am
    [ https://issues.apache.org/jira/browse/HADOOP-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jakob Homan updated HADOOP-6080:
    --------------------------------

    Attachment: HADOOP-6080.patch

    Patch adds a new option to the fsshell rm and rmr commands: -skipTrash, which performs as expected. Adds to trash unit test to verify correct execution. Changes documentation to reflect new option. Docs suggest this option as being a solution when a directory is over quota.

    Passes all commons unit tests. Running test patch now. Will post those results when done.
    Handling of Trash with quota
    -----------------------------

    Key: HADOOP-6080
    URL: https://issues.apache.org/jira/browse/HADOOP-6080
    Project: Hadoop Common
    Issue Type: New Feature
    Components: fs
    Reporter: Koji Noguchi
    Assignee: Jakob Homan
    Attachments: HADOOP-6080.patch


    Currently with quota turned on, user cannot call '-rmr' on large directory that causes over quota.
    {noformat}
    [knoguchi src]$ hadoop dfs -rmr /tmp/net2
    rmr: Failed to move to trash: hdfs://abc.def.com/tmp/net2
    [knoguchi src]$ hadoop dfs -mv /tmp/net2 /user/knoguchi/.Trash/Current
    mv: org.apache.hadoop.hdfs.protocol.QuotaExceededException: The quota of /user/knoguchi is exceeded: namespace
    quota=37500 file count=37757, diskspace quota=-1 diskspace=1991250043353
    {noformat}
    Besides from error message being unfriendly, how should this be handled?
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jakob Homan (JIRA) at Jun 30, 2009 at 1:17 am
    [ https://issues.apache.org/jira/browse/HADOOP-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jakob Homan updated HADOOP-6080:
    --------------------------------

    Status: Patch Available (was: Open)

    submitting patch.
    Handling of Trash with quota
    -----------------------------

    Key: HADOOP-6080
    URL: https://issues.apache.org/jira/browse/HADOOP-6080
    Project: Hadoop Common
    Issue Type: New Feature
    Components: fs
    Reporter: Koji Noguchi
    Assignee: Jakob Homan
    Attachments: HADOOP-6080.patch


    Currently with quota turned on, user cannot call '-rmr' on large directory that causes over quota.
    {noformat}
    [knoguchi src]$ hadoop dfs -rmr /tmp/net2
    rmr: Failed to move to trash: hdfs://abc.def.com/tmp/net2
    [knoguchi src]$ hadoop dfs -mv /tmp/net2 /user/knoguchi/.Trash/Current
    mv: org.apache.hadoop.hdfs.protocol.QuotaExceededException: The quota of /user/knoguchi is exceeded: namespace
    quota=37500 file count=37757, diskspace quota=-1 diskspace=1991250043353
    {noformat}
    Besides from error message being unfriendly, how should this be handled?
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Jun 30, 2009 at 1:21 am
    [ https://issues.apache.org/jira/browse/HADOOP-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Tsz Wo (Nicholas), SZE updated HADOOP-6080:
    -------------------------------------------

    Hadoop Flags: [Reviewed]

    +1 patch looks good
    Handling of Trash with quota
    -----------------------------

    Key: HADOOP-6080
    URL: https://issues.apache.org/jira/browse/HADOOP-6080
    Project: Hadoop Common
    Issue Type: New Feature
    Components: fs
    Reporter: Koji Noguchi
    Assignee: Jakob Homan
    Attachments: HADOOP-6080.patch


    Currently with quota turned on, user cannot call '-rmr' on large directory that causes over quota.
    {noformat}
    [knoguchi src]$ hadoop dfs -rmr /tmp/net2
    rmr: Failed to move to trash: hdfs://abc.def.com/tmp/net2
    [knoguchi src]$ hadoop dfs -mv /tmp/net2 /user/knoguchi/.Trash/Current
    mv: org.apache.hadoop.hdfs.protocol.QuotaExceededException: The quota of /user/knoguchi is exceeded: namespace
    quota=37500 file count=37757, diskspace quota=-1 diskspace=1991250043353
    {noformat}
    Besides from error message being unfriendly, how should this be handled?
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jakob Homan (JIRA) at Jun 30, 2009 at 1:33 am
    [ https://issues.apache.org/jira/browse/HADOOP-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jakob Homan updated HADOOP-6080:
    --------------------------------

    Attachment: javac_warnings_diff.txt

    Test-patch:
    {noformat}[exec] -1 overall.
    [exec]
    [exec] +1 @author. The patch does not contain any @author tags.
    [exec]
    [exec] +1 tests included. The patch appears to include 3 new or modified tests.
    [exec]
    [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
    [exec]
    [exec] -1 javac. The applied patch generated 64 javac compiler warnings (more than the trunk's current 124 warnings).
    [exec]
    [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
    [exec]
    [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
    {noformat}
    This is weird. I've attached the javac warnings it says are new and they have nothing to do with this patch. test-patch must be broken in this regard. I believe the patch is ready to go.
    Handling of Trash with quota
    -----------------------------

    Key: HADOOP-6080
    URL: https://issues.apache.org/jira/browse/HADOOP-6080
    Project: Hadoop Common
    Issue Type: New Feature
    Components: fs
    Reporter: Koji Noguchi
    Assignee: Jakob Homan
    Attachments: HADOOP-6080.patch, javac_warnings_diff.txt


    Currently with quota turned on, user cannot call '-rmr' on large directory that causes over quota.
    {noformat}
    [knoguchi src]$ hadoop dfs -rmr /tmp/net2
    rmr: Failed to move to trash: hdfs://abc.def.com/tmp/net2
    [knoguchi src]$ hadoop dfs -mv /tmp/net2 /user/knoguchi/.Trash/Current
    mv: org.apache.hadoop.hdfs.protocol.QuotaExceededException: The quota of /user/knoguchi is exceeded: namespace
    quota=37500 file count=37757, diskspace quota=-1 diskspace=1991250043353
    {noformat}
    Besides from error message being unfriendly, how should this be handled?
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jakob Homan (JIRA) at Jun 30, 2009 at 3:26 am
    [ https://issues.apache.org/jira/browse/HADOOP-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jakob Homan updated HADOOP-6080:
    --------------------------------

    Attachment: HADOOP-6080-v20.patch
    HADOOP-6080.patch

    Attaching two new files:
    * Updated patch. Previous patch missed updating the help text for rmr to include -skipTrash option. No change to actual code.
    * Patch for Hadoop 20 off of the Hadoop-20 branch from svn. Nothing had to be changed for patch, just file locations were different. Code is still the same. Passes unit tests.

    Handling of Trash with quota
    -----------------------------

    Key: HADOOP-6080
    URL: https://issues.apache.org/jira/browse/HADOOP-6080
    Project: Hadoop Common
    Issue Type: New Feature
    Components: fs
    Reporter: Koji Noguchi
    Assignee: Jakob Homan
    Attachments: HADOOP-6080-v20.patch, HADOOP-6080.patch, HADOOP-6080.patch, javac_warnings_diff.txt


    Currently with quota turned on, user cannot call '-rmr' on large directory that causes over quota.
    {noformat}
    [knoguchi src]$ hadoop dfs -rmr /tmp/net2
    rmr: Failed to move to trash: hdfs://abc.def.com/tmp/net2
    [knoguchi src]$ hadoop dfs -mv /tmp/net2 /user/knoguchi/.Trash/Current
    mv: org.apache.hadoop.hdfs.protocol.QuotaExceededException: The quota of /user/knoguchi is exceeded: namespace
    quota=37500 file count=37757, diskspace quota=-1 diskspace=1991250043353
    {noformat}
    Besides from error message being unfriendly, how should this be handled?
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jakob Homan (JIRA) at Jun 30, 2009 at 5:38 am
    [ https://issues.apache.org/jira/browse/HADOOP-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jakob Homan updated HADOOP-6080:
    --------------------------------

    Status: Open (was: Patch Available)

    Canceling patch to double check something.
    Handling of Trash with quota
    -----------------------------

    Key: HADOOP-6080
    URL: https://issues.apache.org/jira/browse/HADOOP-6080
    Project: Hadoop Common
    Issue Type: New Feature
    Components: fs
    Reporter: Koji Noguchi
    Assignee: Jakob Homan
    Attachments: HADOOP-6080-v20.patch, HADOOP-6080.patch, HADOOP-6080.patch, javac_warnings_diff.txt


    Currently with quota turned on, user cannot call '-rmr' on large directory that causes over quota.
    {noformat}
    [knoguchi src]$ hadoop dfs -rmr /tmp/net2
    rmr: Failed to move to trash: hdfs://abc.def.com/tmp/net2
    [knoguchi src]$ hadoop dfs -mv /tmp/net2 /user/knoguchi/.Trash/Current
    mv: org.apache.hadoop.hdfs.protocol.QuotaExceededException: The quota of /user/knoguchi is exceeded: namespace
    quota=37500 file count=37757, diskspace quota=-1 diskspace=1991250043353
    {noformat}
    Besides from error message being unfriendly, how should this be handled?
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jakob Homan (JIRA) at Jun 30, 2009 at 7:07 am
    [ https://issues.apache.org/jira/browse/HADOOP-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jakob Homan updated HADOOP-6080:
    --------------------------------

    Attachment: (was: HADOOP-6080.patch)
    Handling of Trash with quota
    -----------------------------

    Key: HADOOP-6080
    URL: https://issues.apache.org/jira/browse/HADOOP-6080
    Project: Hadoop Common
    Issue Type: New Feature
    Components: fs
    Reporter: Koji Noguchi
    Assignee: Jakob Homan
    Attachments: javac_warnings_diff.txt


    Currently with quota turned on, user cannot call '-rmr' on large directory that causes over quota.
    {noformat}
    [knoguchi src]$ hadoop dfs -rmr /tmp/net2
    rmr: Failed to move to trash: hdfs://abc.def.com/tmp/net2
    [knoguchi src]$ hadoop dfs -mv /tmp/net2 /user/knoguchi/.Trash/Current
    mv: org.apache.hadoop.hdfs.protocol.QuotaExceededException: The quota of /user/knoguchi is exceeded: namespace
    quota=37500 file count=37757, diskspace quota=-1 diskspace=1991250043353
    {noformat}
    Besides from error message being unfriendly, how should this be handled?
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jakob Homan (JIRA) at Jun 30, 2009 at 7:07 am
    [ https://issues.apache.org/jira/browse/HADOOP-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jakob Homan updated HADOOP-6080:
    --------------------------------

    Attachment: (was: HADOOP-6080.patch)
    Handling of Trash with quota
    -----------------------------

    Key: HADOOP-6080
    URL: https://issues.apache.org/jira/browse/HADOOP-6080
    Project: Hadoop Common
    Issue Type: New Feature
    Components: fs
    Reporter: Koji Noguchi
    Assignee: Jakob Homan
    Attachments: javac_warnings_diff.txt


    Currently with quota turned on, user cannot call '-rmr' on large directory that causes over quota.
    {noformat}
    [knoguchi src]$ hadoop dfs -rmr /tmp/net2
    rmr: Failed to move to trash: hdfs://abc.def.com/tmp/net2
    [knoguchi src]$ hadoop dfs -mv /tmp/net2 /user/knoguchi/.Trash/Current
    mv: org.apache.hadoop.hdfs.protocol.QuotaExceededException: The quota of /user/knoguchi is exceeded: namespace
    quota=37500 file count=37757, diskspace quota=-1 diskspace=1991250043353
    {noformat}
    Besides from error message being unfriendly, how should this be handled?
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jakob Homan (JIRA) at Jun 30, 2009 at 7:07 am
    [ https://issues.apache.org/jira/browse/HADOOP-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jakob Homan updated HADOOP-6080:
    --------------------------------

    Attachment: (was: HADOOP-6080-v20.patch)
    Handling of Trash with quota
    -----------------------------

    Key: HADOOP-6080
    URL: https://issues.apache.org/jira/browse/HADOOP-6080
    Project: Hadoop Common
    Issue Type: New Feature
    Components: fs
    Reporter: Koji Noguchi
    Assignee: Jakob Homan
    Attachments: javac_warnings_diff.txt


    Currently with quota turned on, user cannot call '-rmr' on large directory that causes over quota.
    {noformat}
    [knoguchi src]$ hadoop dfs -rmr /tmp/net2
    rmr: Failed to move to trash: hdfs://abc.def.com/tmp/net2
    [knoguchi src]$ hadoop dfs -mv /tmp/net2 /user/knoguchi/.Trash/Current
    mv: org.apache.hadoop.hdfs.protocol.QuotaExceededException: The quota of /user/knoguchi is exceeded: namespace
    quota=37500 file count=37757, diskspace quota=-1 diskspace=1991250043353
    {noformat}
    Besides from error message being unfriendly, how should this be handled?
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jakob Homan (JIRA) at Jun 30, 2009 at 7:13 am
    [ https://issues.apache.org/jira/browse/HADOOP-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jakob Homan updated HADOOP-6080:
    --------------------------------

    Attachment: HADOOP-6080.patch
    HADOOP-6080-v20.patch

    Ran into a problem that I didn't notice with the HDFS version of TestTrash. I think there's an issue with the FileSystem.listStatus methods between LocalFileSystem and DistributedFileSystem, which I'll look into. In the meantime, modified test so that it doesn't rely on that method and works on both local and distributed file systems.
    Will run full test suite tonight, report tomorrow morning. Also, deleted old patches to avoid confusion. New patches for both trunk and v20 should be good to go.
    Handling of Trash with quota
    -----------------------------

    Key: HADOOP-6080
    URL: https://issues.apache.org/jira/browse/HADOOP-6080
    Project: Hadoop Common
    Issue Type: New Feature
    Components: fs
    Reporter: Koji Noguchi
    Assignee: Jakob Homan
    Attachments: HADOOP-6080-v20.patch, HADOOP-6080.patch, javac_warnings_diff.txt


    Currently with quota turned on, user cannot call '-rmr' on large directory that causes over quota.
    {noformat}
    [knoguchi src]$ hadoop dfs -rmr /tmp/net2
    rmr: Failed to move to trash: hdfs://abc.def.com/tmp/net2
    [knoguchi src]$ hadoop dfs -mv /tmp/net2 /user/knoguchi/.Trash/Current
    mv: org.apache.hadoop.hdfs.protocol.QuotaExceededException: The quota of /user/knoguchi is exceeded: namespace
    quota=37500 file count=37757, diskspace quota=-1 diskspace=1991250043353
    {noformat}
    Besides from error message being unfriendly, how should this be handled?
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jakob Homan (JIRA) at Jun 30, 2009 at 3:06 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12725659#action_12725659 ]

    Jakob Homan commented on HADOOP-6080:
    -------------------------------------

    Updated patches are good to go on all commons unit tests for trunk and all tests for v20. Test-patch is fine except the incorrect javac warnings, which are not related.
    {noformat} [exec] -1 overall.
    [exec]
    [exec] +1 @author. The patch does not contain any @author tags.
    [exec]
    [exec] +1 tests included. The patch appears to include 3 new or modified tests.
    [exec]
    [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
    [exec]
    [exec] -1 javac. The applied patch generated 64 javac compiler warnings (more than the trunk's current 124 warnings).
    [exec]
    [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
    [exec]
    [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.{noformat}
    Handling of Trash with quota
    -----------------------------

    Key: HADOOP-6080
    URL: https://issues.apache.org/jira/browse/HADOOP-6080
    Project: Hadoop Common
    Issue Type: New Feature
    Components: fs
    Reporter: Koji Noguchi
    Assignee: Jakob Homan
    Attachments: HADOOP-6080-v20.patch, HADOOP-6080.patch, javac_warnings_diff.txt


    Currently with quota turned on, user cannot call '-rmr' on large directory that causes over quota.
    {noformat}
    [knoguchi src]$ hadoop dfs -rmr /tmp/net2
    rmr: Failed to move to trash: hdfs://abc.def.com/tmp/net2
    [knoguchi src]$ hadoop dfs -mv /tmp/net2 /user/knoguchi/.Trash/Current
    mv: org.apache.hadoop.hdfs.protocol.QuotaExceededException: The quota of /user/knoguchi is exceeded: namespace
    quota=37500 file count=37757, diskspace quota=-1 diskspace=1991250043353
    {noformat}
    Besides from error message being unfriendly, how should this be handled?
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Sanjay Radia (JIRA) at Jun 30, 2009 at 4:27 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12725686#action_12725686 ]

    Sanjay Radia commented on HADOOP-6080:
    --------------------------------------

    Should we consider excluding trash for files deleted from /tmp (ie make -skipTrash implicit when deleting from /tmp.)?
    Handling of Trash with quota
    -----------------------------

    Key: HADOOP-6080
    URL: https://issues.apache.org/jira/browse/HADOOP-6080
    Project: Hadoop Common
    Issue Type: New Feature
    Components: fs
    Reporter: Koji Noguchi
    Assignee: Jakob Homan
    Attachments: HADOOP-6080-v20.patch, HADOOP-6080.patch, javac_warnings_diff.txt


    Currently with quota turned on, user cannot call '-rmr' on large directory that causes over quota.
    {noformat}
    [knoguchi src]$ hadoop dfs -rmr /tmp/net2
    rmr: Failed to move to trash: hdfs://abc.def.com/tmp/net2
    [knoguchi src]$ hadoop dfs -mv /tmp/net2 /user/knoguchi/.Trash/Current
    mv: org.apache.hadoop.hdfs.protocol.QuotaExceededException: The quota of /user/knoguchi is exceeded: namespace
    quota=37500 file count=37757, diskspace quota=-1 diskspace=1991250043353
    {noformat}
    Besides from error message being unfriendly, how should this be handled?
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jakob Homan (JIRA) at Jun 30, 2009 at 4:57 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12725696#action_12725696 ]

    Jakob Homan commented on HADOOP-6080:
    -------------------------------------

    bq. Should we consider excluding trash for files deleted from /tmp (ie make -skipTrash implicit when deleting from /tmp.)?
    I'm not a fan of special cases for certain directories, even for /tmp, and particularly when we're already straying away from the posix world with the trash feature. Minimizing surprise seems a good goal, and I'd be very surprised if I were accustomed to explicitly skipping the trash when I want and discovering something I had expected to be trashed had been helpfully nuked by the system.
    Handling of Trash with quota
    -----------------------------

    Key: HADOOP-6080
    URL: https://issues.apache.org/jira/browse/HADOOP-6080
    Project: Hadoop Common
    Issue Type: New Feature
    Components: fs
    Reporter: Koji Noguchi
    Assignee: Jakob Homan
    Attachments: HADOOP-6080-v20.patch, HADOOP-6080.patch, javac_warnings_diff.txt


    Currently with quota turned on, user cannot call '-rmr' on large directory that causes over quota.
    {noformat}
    [knoguchi src]$ hadoop dfs -rmr /tmp/net2
    rmr: Failed to move to trash: hdfs://abc.def.com/tmp/net2
    [knoguchi src]$ hadoop dfs -mv /tmp/net2 /user/knoguchi/.Trash/Current
    mv: org.apache.hadoop.hdfs.protocol.QuotaExceededException: The quota of /user/knoguchi is exceeded: namespace
    quota=37500 file count=37757, diskspace quota=-1 diskspace=1991250043353
    {noformat}
    Besides from error message being unfriendly, how should this be handled?
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Konstantin Shvachko (JIRA) at Jun 30, 2009 at 7:03 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Konstantin Shvachko resolved HADOOP-6080.
    -----------------------------------------

    Resolution: Fixed
    Fix Version/s: 0.20.1

    I just committed this. Thank you Jakob.
    Handling of Trash with quota
    -----------------------------

    Key: HADOOP-6080
    URL: https://issues.apache.org/jira/browse/HADOOP-6080
    Project: Hadoop Common
    Issue Type: New Feature
    Components: fs
    Reporter: Koji Noguchi
    Assignee: Jakob Homan
    Fix For: 0.20.1

    Attachments: HADOOP-6080-v20.patch, HADOOP-6080.patch, javac_warnings_diff.txt


    Currently with quota turned on, user cannot call '-rmr' on large directory that causes over quota.
    {noformat}
    [knoguchi src]$ hadoop dfs -rmr /tmp/net2
    rmr: Failed to move to trash: hdfs://abc.def.com/tmp/net2
    [knoguchi src]$ hadoop dfs -mv /tmp/net2 /user/knoguchi/.Trash/Current
    mv: org.apache.hadoop.hdfs.protocol.QuotaExceededException: The quota of /user/knoguchi is exceeded: namespace
    quota=37500 file count=37757, diskspace quota=-1 diskspace=1991250043353
    {noformat}
    Besides from error message being unfriendly, how should this be handled?
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedJun 18, '09 at 7:02p
activeJun 30, '09 at 7:03p
posts20
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Konstantin Shvachko (JIRA): 20 posts

People

Translate

site design / logo © 2022 Grokbase