FAQ
dfs -lsr fail with "Could not get listing "
-------------------------------------------

Key: HADOOP-3121
URL: https://issues.apache.org/jira/browse/HADOOP-3121
Project: Hadoop Core
Issue Type: Bug
Components: fs
Affects Versions: 0.16.1
Reporter: Koji Noguchi
Priority: Minor


(This happened a lot when namenode was extremely slow due to some other reasons.)

% hadoop dfs -lsr /

failed with
Could not get listing for /aaa/bbb/randomfile
It's probably because file was deleted between items = fs.listStatus( and ls(items[i]

I think lsr should ignore this case and continue.


--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • dhruba borthakur (JIRA) at Mar 28, 2008 at 7:00 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12583143#action_12583143 ]

    dhruba borthakur commented on HADOOP-3121:
    ------------------------------------------

    This does not seem to be a regression and I propose that we fix it for 0.17.
    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Priority: Minor

    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • T Meyarivan (JIRA) at Oct 28, 2008 at 5:24 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12643273#action_12643273 ]

    T Meyarivan commented on HADOOP-3121:
    -------------------------------------

    Is it likely to be fixed in 0.18.x ?

    --
    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Priority: Minor

    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Nov 5, 2008 at 12:55 am
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Tsz Wo (Nicholas), SZE updated HADOOP-3121:
    -------------------------------------------

    Assignee: Tsz Wo (Nicholas), SZE

    This seems already fixed in trunk. At least, the behavior should be different since I could not find "Could not get listing " in the ls codes. I will check this in details.
    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Assignee: Tsz Wo (Nicholas), SZE
    Priority: Minor

    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Nov 5, 2008 at 10:23 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12645346#action_12645346 ]

    Tsz Wo (Nicholas), SZE commented on HADOOP-3121:
    ------------------------------------------------
    This seems already fixed in trunk...
    Unfortunately, we still have this bug but the error message have been changed.
    {noformat}
    "lsr /", then remove /a right before lsr printing the result of /a

    bash-3.2$ ./bin/hadoop fs -lsr /
    drwxr-xr-x - tsz supergroup 0 2008-11-05 14:01 /a
    lsr: hdfs://namenode9000/a: No such file or directory.
    {noformat}

    I plan to change lsr such that lsr prints an error message and then keep listing other items.
    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Assignee: Tsz Wo (Nicholas), SZE
    Priority: Minor

    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Nov 5, 2008 at 10:35 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Tsz Wo (Nicholas), SZE updated HADOOP-3121:
    -------------------------------------------

    Attachment: 3121_20081105.patch

    3121_20081105.patch: print an error message and then keep listing other items, instead of throwing an exception and quit.

    {noformat}
    Same as before: "lsr /", then remove /a right before lsr printing the result of /a

    bash-3.2$ ./bin/hadoop fs -lsr /
    drwxr-xr-x - tsz supergroup 0 2008-11-05 14:18 /a
    hdfs://namenode:9000/a: No such file or directory.
    drwxr-xr-x - tsz supergroup 0 2008-11-05 14:18 /b
    ...
    {noformat}

    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Assignee: Tsz Wo (Nicholas), SZE
    Priority: Minor
    Attachments: 3121_20081105.patch


    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Raghu Angadi (JIRA) at Nov 5, 2008 at 10:47 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12645350#action_12645350 ]

    Raghu Angadi commented on HADOOP-3121:
    --------------------------------------


    Shouldn't it be the same for other exceptions? What if dir '/a' does not have permissions for this user?

    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Assignee: Tsz Wo (Nicholas), SZE
    Priority: Minor
    Attachments: 3121_20081105.patch


    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Nov 6, 2008 at 12:59 am
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Tsz Wo (Nicholas), SZE updated HADOOP-3121:
    -------------------------------------------

    Fix Version/s: 0.18.3
    Shouldn't it be the same for other exceptions? What if dir '/a' does not have permissions for this user?
    Since this issue is target for 0.18, I don't want to introduce big changes. How about we fix other exceptions in a separated issue?
    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Assignee: Tsz Wo (Nicholas), SZE
    Priority: Minor
    Fix For: 0.18.3

    Attachments: 3121_20081105.patch


    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Nov 6, 2008 at 1:07 am
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Tsz Wo (Nicholas), SZE updated HADOOP-3121:
    -------------------------------------------

    Status: Patch Available (was: Open)
    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Assignee: Tsz Wo (Nicholas), SZE
    Priority: Minor
    Fix For: 0.18.3

    Attachments: 3121_20081105.patch


    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Raghu Angadi (JIRA) at Nov 6, 2008 at 6:33 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12645533#action_12645533 ]

    Raghu Angadi commented on HADOOP-3121:
    --------------------------------------


    Then don't think it is a proper fix for the real issue. It fixes just one symptom reported.
    How about we fix other exceptions in a separated issue?
    may be. +0 from me.
    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Assignee: Tsz Wo (Nicholas), SZE
    Priority: Minor
    Fix For: 0.18.3

    Attachments: 3121_20081105.patch


    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Raghu Angadi (JIRA) at Nov 6, 2008 at 6:55 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12645533#action_12645533 ]

    rangadi edited comment on HADOOP-3121 at 11/6/08 10:54 AM:
    ----------------------------------------------------------------

    Then I don't think it is a proper fix for the real issue. It fixes just one symptom reported.
    How about we fix other exceptions in a separated issue?
    may be. +0 from me.

    was (Author: rangadi):

    Then don't think it is a proper fix for the real issue. It fixes just one symptom reported.
    How about we fix other exceptions in a separated issue?
    may be. +0 from me.
    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Assignee: Tsz Wo (Nicholas), SZE
    Priority: Minor
    Fix For: 0.18.3

    Attachments: 3121_20081105.patch


    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Nov 6, 2008 at 10:54 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Tsz Wo (Nicholas), SZE updated HADOOP-3121:
    -------------------------------------------

    Attachment: 3121_20081106.patch

    Raghu, I see your point now. Here is a new patch.

    3121_20081106.patch: try-catch all IOException.
    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Assignee: Tsz Wo (Nicholas), SZE
    Priority: Minor
    Fix For: 0.18.3

    Attachments: 3121_20081105.patch, 3121_20081106.patch


    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Raghu Angadi (JIRA) at Nov 6, 2008 at 11:03 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12645610#action_12645610 ]

    Raghu Angadi commented on HADOOP-3121:
    --------------------------------------

    Couple of comments:

    - do we need to rename ListStatus() to FileStatus()?
    - getFileStatus() name is too close to FS.getFileStatus(), might be confusing while reading the code.

    I suggest renaming it to some thing like 'shellListStatus()'
    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Assignee: Tsz Wo (Nicholas), SZE
    Priority: Minor
    Fix For: 0.18.3

    Attachments: 3121_20081105.patch, 3121_20081106.patch


    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Nov 6, 2008 at 11:14 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Tsz Wo (Nicholas), SZE updated HADOOP-3121:
    -------------------------------------------

    Attachment: 3121_20081106b.patch

    3121_20081106b.patch: renamed getFileStatus(...) to getListStatus(...)
    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Assignee: Tsz Wo (Nicholas), SZE
    Priority: Minor
    Fix For: 0.18.3

    Attachments: 3121_20081105.patch, 3121_20081106.patch, 3121_20081106b.patch


    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Nov 6, 2008 at 11:16 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Tsz Wo (Nicholas), SZE updated HADOOP-3121:
    -------------------------------------------

    Attachment: (was: 3121_20081106b.patch)
    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Assignee: Tsz Wo (Nicholas), SZE
    Priority: Minor
    Fix For: 0.18.3

    Attachments: 3121_20081105.patch, 3121_20081106.patch, 3121_20081106b.patch


    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Nov 6, 2008 at 11:16 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Tsz Wo (Nicholas), SZE updated HADOOP-3121:
    -------------------------------------------

    Attachment: 3121_20081106b.patch
    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Assignee: Tsz Wo (Nicholas), SZE
    Priority: Minor
    Fix For: 0.18.3

    Attachments: 3121_20081105.patch, 3121_20081106.patch, 3121_20081106b.patch


    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Nov 6, 2008 at 11:16 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12645615#action_12645615 ]

    szetszwo edited comment on HADOOP-3121 at 11/6/08 3:14 PM:
    -------------------------------------------------------------------------

    3121_20081106b.patch: renamed getFileStatus(...) to shellListStatus(...)

    was (Author: szetszwo):
    3121_20081106b.patch: renamed getFileStatus(...) to getListStatus(...)
    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Assignee: Tsz Wo (Nicholas), SZE
    Priority: Minor
    Fix For: 0.18.3

    Attachments: 3121_20081105.patch, 3121_20081106.patch, 3121_20081106b.patch


    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Raghu Angadi (JIRA) at Nov 7, 2008 at 1:33 am
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12645657#action_12645657 ]

    Raghu Angadi commented on HADOOP-3121:
    --------------------------------------

    Thanks Nicholas. looks good. One nit:

    instead of ls throwing exception with "There were totally..." message, I think it is better to make ls return -1 and replace {{ls(..)}} with ' {{exitCode = ls()}} ' in doall().
    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Assignee: Tsz Wo (Nicholas), SZE
    Priority: Minor
    Fix For: 0.18.3

    Attachments: 3121_20081105.patch, 3121_20081106.patch, 3121_20081106b.patch


    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Nov 7, 2008 at 1:47 am
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12645661#action_12645661 ]

    Tsz Wo (Nicholas), SZE commented on HADOOP-3121:
    ------------------------------------------------
    instead of ls throwing exception with "There were totally..." message, I think it is better to make ls return -1 and replace ls(..) with ' exitCode = ls() ' in doall().
    I am not sure what do you mean. Could you post a patch for this?
    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Assignee: Tsz Wo (Nicholas), SZE
    Priority: Minor
    Fix For: 0.18.3

    Attachments: 3121_20081105.patch, 3121_20081106.patch, 3121_20081106b.patch


    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Raghu Angadi (JIRA) at Nov 7, 2008 at 2:09 am
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Raghu Angadi updated HADOOP-3121:
    ---------------------------------

    Attachment: HADOOP-3121.patch

    I hope attached patch clarifies what I mean. It behaves same as before and does not add extra line of output that is unecessary.

    For e.g. compare 'bin/hadoop fs -ls /other' :

    - with the patch attached (*and* trunk) : {noformat}
    ls: could not get get listing for '/other' : org.apache.hadoop.security.AccessControlException: Permission denied: [...]
    {noformat}


    - with your patch : {noformat}
    ls: could not get get listing for '/other' : org.apache.hadoop.security.AccessControlException: Permission denied: [...]
    ls: There were totaly 1 exception(s).
    {noformat}

    I don't think most command print extra lines like 'ls: There were totaly 1 exception(s)' at the end..'
    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Assignee: Tsz Wo (Nicholas), SZE
    Priority: Minor
    Fix For: 0.18.3

    Attachments: 3121_20081105.patch, 3121_20081106.patch, 3121_20081106b.patch, HADOOP-3121.patch


    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Nov 7, 2008 at 5:05 am
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12645687#action_12645687 ]

    Hadoop QA commented on HADOOP-3121:
    -----------------------------------

    +1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12393479/HADOOP-3121.patch
    against trunk revision 712033.

    +1 @author. The patch does not contain any @author tags.

    +1 tests included. The patch appears to include 3 new or modified tests.

    +1 javadoc. The javadoc tool did not generate any warning messages.

    +1 javac. The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs. The patch does not introduce any new Findbugs warnings.

    +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

    +1 core tests. The patch passed core unit tests.

    +1 contrib tests. The patch passed contrib unit tests.

    Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3545/testReport/
    Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3545/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3545/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3545/console

    This message is automatically generated.
    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Assignee: Tsz Wo (Nicholas), SZE
    Priority: Minor
    Fix For: 0.18.3

    Attachments: 3121_20081105.patch, 3121_20081106.patch, 3121_20081106b.patch, HADOOP-3121.patch


    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Nov 7, 2008 at 6:25 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Tsz Wo (Nicholas), SZE updated HADOOP-3121:
    -------------------------------------------

    Status: Open (was: Patch Available)
    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Assignee: Tsz Wo (Nicholas), SZE
    Priority: Minor
    Fix For: 0.18.3

    Attachments: 3121_20081105.patch, 3121_20081106.patch, 3121_20081106b.patch, 3121_20081107.patch, HADOOP-3121.patch


    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Nov 7, 2008 at 6:25 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Tsz Wo (Nicholas), SZE updated HADOOP-3121:
    -------------------------------------------

    Status: Patch Available (was: Open)
    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Assignee: Tsz Wo (Nicholas), SZE
    Priority: Minor
    Fix For: 0.18.3

    Attachments: 3121_20081105.patch, 3121_20081106.patch, 3121_20081106b.patch, 3121_20081107.patch, HADOOP-3121.patch


    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Nov 7, 2008 at 6:25 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Tsz Wo (Nicholas), SZE updated HADOOP-3121:
    -------------------------------------------

    Attachment: 3121_20081107.patch
    I don't think most command print extra lines like 'ls: There were totaly 1 exception(s)' at the end..'
    I agree that the extra line is odd. Thanks.

    3121_20081107.patch: We should also change FsShell.run(...) to update exitCode.
    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Assignee: Tsz Wo (Nicholas), SZE
    Priority: Minor
    Fix For: 0.18.3

    Attachments: 3121_20081105.patch, 3121_20081106.patch, 3121_20081106b.patch, 3121_20081107.patch, HADOOP-3121.patch


    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Raghu Angadi (JIRA) at Nov 7, 2008 at 6:31 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12645845#action_12645845 ]

    Raghu Angadi commented on HADOOP-3121:
    --------------------------------------

    +1 for your patch. thanks.
    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Assignee: Tsz Wo (Nicholas), SZE
    Priority: Minor
    Fix For: 0.18.3

    Attachments: 3121_20081105.patch, 3121_20081106.patch, 3121_20081106b.patch, 3121_20081107.patch, HADOOP-3121.patch


    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Nov 7, 2008 at 6:54 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12645849#action_12645849 ]

    Tsz Wo (Nicholas), SZE commented on HADOOP-3121:
    ------------------------------------------------

    ant test-patch
    {noformat}
    [exec] +1 overall.

    [exec] +1 @author. The patch does not contain any @author tags.

    [exec] +1 tests included. The patch appears to include 3 new or modified tests.

    [exec] +1 javadoc. The javadoc tool did not generate any warning messages.

    [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.

    [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.

    [exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
    {noformat}
    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Assignee: Tsz Wo (Nicholas), SZE
    Priority: Minor
    Fix For: 0.18.3

    Attachments: 3121_20081105.patch, 3121_20081106.patch, 3121_20081106b.patch, 3121_20081107.patch, HADOOP-3121.patch


    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Nov 7, 2008 at 9:46 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12645883#action_12645883 ]

    Tsz Wo (Nicholas), SZE commented on HADOOP-3121:
    ------------------------------------------------

    The patch passed all unit tests locally.
    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Assignee: Tsz Wo (Nicholas), SZE
    Priority: Minor
    Fix For: 0.18.3

    Attachments: 3121_20081105.patch, 3121_20081106.patch, 3121_20081106b.patch, 3121_20081107.patch, HADOOP-3121.patch


    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Nov 7, 2008 at 9:48 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Tsz Wo (Nicholas), SZE updated HADOOP-3121:
    -------------------------------------------

    Attachment: 3121_20081107_0.18.patch

    3121_20081107_0.18.patch: for 0.18. It also passed all unit tests locally.
    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Assignee: Tsz Wo (Nicholas), SZE
    Priority: Minor
    Fix For: 0.18.3

    Attachments: 3121_20081105.patch, 3121_20081106.patch, 3121_20081106b.patch, 3121_20081107.patch, 3121_20081107_0.18.patch, HADOOP-3121.patch


    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Nov 7, 2008 at 10:10 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Tsz Wo (Nicholas), SZE updated HADOOP-3121:
    -------------------------------------------

    Resolution: Fixed
    Hadoop Flags: [Reviewed]
    Status: Resolved (was: Patch Available)

    I just committed this.
    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Assignee: Tsz Wo (Nicholas), SZE
    Priority: Minor
    Fix For: 0.18.3

    Attachments: 3121_20081105.patch, 3121_20081106.patch, 3121_20081106b.patch, 3121_20081107.patch, 3121_20081107_0.18.patch, HADOOP-3121.patch


    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hudson (JIRA) at Nov 8, 2008 at 4:49 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12645997#action_12645997 ]

    Hudson commented on HADOOP-3121:
    --------------------------------

    Integrated in Hadoop-trunk #655 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/655/])
    . lsr should keep listing the remaining items but not terminate if there is any IOException. (szetszwo)

    dfs -lsr fail with "Could not get listing "
    -------------------------------------------

    Key: HADOOP-3121
    URL: https://issues.apache.org/jira/browse/HADOOP-3121
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Affects Versions: 0.16.1
    Reporter: Koji Noguchi
    Assignee: Tsz Wo (Nicholas), SZE
    Priority: Minor
    Fix For: 0.18.3

    Attachments: 3121_20081105.patch, 3121_20081106.patch, 3121_20081106b.patch, 3121_20081107.patch, 3121_20081107_0.18.patch, HADOOP-3121.patch


    (This happened a lot when namenode was extremely slow due to some other reasons.)
    % hadoop dfs -lsr /
    failed with
    Could not get listing for /aaa/bbb/randomfile
    It's probably because file was deleted between items = fs.listStatus( and ls(items[i]
    I think lsr should ignore this case and continue.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedMar 28, '08 at 6:59p
activeNov 8, '08 at 4:49p
posts30
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Hudson (JIRA): 30 posts

People

Translate

site design / logo © 2022 Grokbase