Grokbase › Groups › Hadoop › common-dev › March 2008
FAQ
[ https://issues.apache.org/jira/browse/HADOOP-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

lohit vijayarenu updated HADOOP-2027:
-------------------------------------

Status: Open (was: Patch Available)
FileSystem should provide byte ranges for file locations
--------------------------------------------------------

Key: HADOOP-2027
URL: https://issues.apache.org/jira/browse/HADOOP-2027
Project: Hadoop Core
Issue Type: Bug
Components: fs
Reporter: Owen O'Malley
Assignee: lohit vijayarenu
Attachments: HADOOP-2027-1.patch, HADOOP-2027-10.patch, HADOOP-2027-2.patch, HADOOP-2027-3.patch, HADOOP-2027-4.patch, HADOOP-2027-5.patch, HADOOP-2027-6.patch, HADOOP-2027-7.patch, HADOOP-2027-8.patch, HADOOP-2027-9.patch


FileSystem's getFileCacheHints should be replaced with something more useful. I'd suggest replacing getFileCacheHints with a new method:
{code}
BlockLocation[] getFileLocations(Path file, long offset, long range) throws IOException;
{code}
and adding
{code}
class BlockLocation implements Writable {
String[] getHosts();
long getOffset();
long getLength();
}
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • lohit vijayarenu (JIRA) at Mar 6, 2008 at 6:40 am ⇧
    [ https://issues.apache.org/jira/browse/HADOOP-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    lohit vijayarenu updated HADOOP-2027:
    -------------------------------------

    Attachment: HADOOP-2559-11.patch

    TestRackAwareTaskPlacement was failing because of my latest changes. While adding getNames() method, I tried to derive getHosts() from string returned by getName(), this had ipaddress:port. But we needed hostnames. So, I created 2 separate arrays within BlockLocation. Now both getHosts() and getNames() return the expected output. Attaching a new patch.
    FileSystem should provide byte ranges for file locations
    --------------------------------------------------------

    Key: HADOOP-2027
    URL: https://issues.apache.org/jira/browse/HADOOP-2027
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Owen O'Malley
    Assignee: lohit vijayarenu
    Attachments: HADOOP-2027-1.patch, HADOOP-2027-10.patch, HADOOP-2027-2.patch, HADOOP-2027-3.patch, HADOOP-2027-4.patch, HADOOP-2027-5.patch, HADOOP-2027-6.patch, HADOOP-2027-7.patch, HADOOP-2027-8.patch, HADOOP-2027-9.patch, HADOOP-2559-11.patch


    FileSystem's getFileCacheHints should be replaced with something more useful. I'd suggest replacing getFileCacheHints with a new method:
    {code}
    BlockLocation[] getFileLocations(Path file, long offset, long range) throws IOException;
    {code}
    and adding
    {code}
    class BlockLocation implements Writable {
    String[] getHosts();
    long getOffset();
    long getLength();
    }
    {code}
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • lohit vijayarenu (JIRA) at Mar 6, 2008 at 6:40 am ⇧
    [ https://issues.apache.org/jira/browse/HADOOP-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    lohit vijayarenu updated HADOOP-2027:
    -------------------------------------

    Status: Patch Available (was: Open)
    FileSystem should provide byte ranges for file locations
    --------------------------------------------------------

    Key: HADOOP-2027
    URL: https://issues.apache.org/jira/browse/HADOOP-2027
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Owen O'Malley
    Assignee: lohit vijayarenu
    Attachments: HADOOP-2027-1.patch, HADOOP-2027-10.patch, HADOOP-2027-2.patch, HADOOP-2027-3.patch, HADOOP-2027-4.patch, HADOOP-2027-5.patch, HADOOP-2027-6.patch, HADOOP-2027-7.patch, HADOOP-2027-8.patch, HADOOP-2027-9.patch, HADOOP-2559-11.patch


    FileSystem's getFileCacheHints should be replaced with something more useful. I'd suggest replacing getFileCacheHints with a new method:
    {code}
    BlockLocation[] getFileLocations(Path file, long offset, long range) throws IOException;
    {code}
    and adding
    {code}
    class BlockLocation implements Writable {
    String[] getHosts();
    long getOffset();
    long getLength();
    }
    {code}
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • lohit vijayarenu (JIRA) at Mar 6, 2008 at 7:57 am ⇧
    [ https://issues.apache.org/jira/browse/HADOOP-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    lohit vijayarenu updated HADOOP-2027:
    -------------------------------------

    Status: Open (was: Patch Available)
    FileSystem should provide byte ranges for file locations
    --------------------------------------------------------

    Key: HADOOP-2027
    URL: https://issues.apache.org/jira/browse/HADOOP-2027
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Owen O'Malley
    Assignee: lohit vijayarenu
    Attachments: HADOOP-2027-1.patch, HADOOP-2027-10.patch, HADOOP-2027-2.patch, HADOOP-2027-3.patch, HADOOP-2027-4.patch, HADOOP-2027-5.patch, HADOOP-2027-6.patch, HADOOP-2027-7.patch, HADOOP-2027-8.patch, HADOOP-2027-9.patch, HADOOP-2559-11.patch


    FileSystem's getFileCacheHints should be replaced with something more useful. I'd suggest replacing getFileCacheHints with a new method:
    {code}
    BlockLocation[] getFileLocations(Path file, long offset, long range) throws IOException;
    {code}
    and adding
    {code}
    class BlockLocation implements Writable {
    String[] getHosts();
    long getOffset();
    long getLength();
    }
    {code}
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • lohit vijayarenu (JIRA) at Mar 6, 2008 at 8:48 am ⇧
    [ https://issues.apache.org/jira/browse/HADOOP-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    lohit vijayarenu updated HADOOP-2027:
    -------------------------------------

    Attachment: HADOOP-2559-12.patch

    modified TestTextInputFormat. With the updated patch, if file is of 0 length it will not be added to the splits.
    FileSystem should provide byte ranges for file locations
    --------------------------------------------------------

    Key: HADOOP-2027
    URL: https://issues.apache.org/jira/browse/HADOOP-2027
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Owen O'Malley
    Assignee: lohit vijayarenu
    Attachments: HADOOP-2027-1.patch, HADOOP-2027-10.patch, HADOOP-2027-2.patch, HADOOP-2027-3.patch, HADOOP-2027-4.patch, HADOOP-2027-5.patch, HADOOP-2027-6.patch, HADOOP-2027-7.patch, HADOOP-2027-8.patch, HADOOP-2027-9.patch, HADOOP-2559-11.patch, HADOOP-2559-12.patch


    FileSystem's getFileCacheHints should be replaced with something more useful. I'd suggest replacing getFileCacheHints with a new method:
    {code}
    BlockLocation[] getFileLocations(Path file, long offset, long range) throws IOException;
    {code}
    and adding
    {code}
    class BlockLocation implements Writable {
    String[] getHosts();
    long getOffset();
    long getLength();
    }
    {code}
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • lohit vijayarenu (JIRA) at Mar 6, 2008 at 8:50 am ⇧
    [ https://issues.apache.org/jira/browse/HADOOP-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    lohit vijayarenu updated HADOOP-2027:
    -------------------------------------

    Status: Patch Available (was: Open)
    FileSystem should provide byte ranges for file locations
    --------------------------------------------------------

    Key: HADOOP-2027
    URL: https://issues.apache.org/jira/browse/HADOOP-2027
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Owen O'Malley
    Assignee: lohit vijayarenu
    Attachments: HADOOP-2027-1.patch, HADOOP-2027-10.patch, HADOOP-2027-2.patch, HADOOP-2027-3.patch, HADOOP-2027-4.patch, HADOOP-2027-5.patch, HADOOP-2027-6.patch, HADOOP-2027-7.patch, HADOOP-2027-8.patch, HADOOP-2027-9.patch, HADOOP-2559-11.patch, HADOOP-2559-12.patch


    FileSystem's getFileCacheHints should be replaced with something more useful. I'd suggest replacing getFileCacheHints with a new method:
    {code}
    BlockLocation[] getFileLocations(Path file, long offset, long range) throws IOException;
    {code}
    and adding
    {code}
    class BlockLocation implements Writable {
    String[] getHosts();
    long getOffset();
    long getLength();
    }
    {code}
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • lohit vijayarenu (JIRA) at Mar 6, 2008 at 6:06 pm ⇧
    [ https://issues.apache.org/jira/browse/HADOOP-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    lohit vijayarenu updated HADOOP-2027:
    -------------------------------------

    Status: Open (was: Patch Available)
    FileSystem should provide byte ranges for file locations
    --------------------------------------------------------

    Key: HADOOP-2027
    URL: https://issues.apache.org/jira/browse/HADOOP-2027
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Owen O'Malley
    Assignee: lohit vijayarenu
    Attachments: HADOOP-2027-1.patch, HADOOP-2027-10.patch, HADOOP-2027-2.patch, HADOOP-2027-3.patch, HADOOP-2027-4.patch, HADOOP-2027-5.patch, HADOOP-2027-6.patch, HADOOP-2027-7.patch, HADOOP-2027-8.patch, HADOOP-2027-9.patch, HADOOP-2559-11.patch, HADOOP-2559-12.patch


    FileSystem's getFileCacheHints should be replaced with something more useful. I'd suggest replacing getFileCacheHints with a new method:
    {code}
    BlockLocation[] getFileLocations(Path file, long offset, long range) throws IOException;
    {code}
    and adding
    {code}
    class BlockLocation implements Writable {
    String[] getHosts();
    long getOffset();
    long getLength();
    }
    {code}
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • lohit vijayarenu (JIRA) at Mar 6, 2008 at 6:08 pm ⇧
    [ https://issues.apache.org/jira/browse/HADOOP-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    lohit vijayarenu updated HADOOP-2027:
    -------------------------------------

    Attachment: HADOOP-2559-13.patch

    For now, creating empty hosts array when of input file is zero length. Owen opened HADOOP-2952 to address zero length files. Attaching another patch.
    FileSystem should provide byte ranges for file locations
    --------------------------------------------------------

    Key: HADOOP-2027
    URL: https://issues.apache.org/jira/browse/HADOOP-2027
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Owen O'Malley
    Assignee: lohit vijayarenu
    Attachments: HADOOP-2027-1.patch, HADOOP-2027-10.patch, HADOOP-2027-2.patch, HADOOP-2027-3.patch, HADOOP-2027-4.patch, HADOOP-2027-5.patch, HADOOP-2027-6.patch, HADOOP-2027-7.patch, HADOOP-2027-8.patch, HADOOP-2027-9.patch, HADOOP-2559-11.patch, HADOOP-2559-12.patch, HADOOP-2559-13.patch


    FileSystem's getFileCacheHints should be replaced with something more useful. I'd suggest replacing getFileCacheHints with a new method:
    {code}
    BlockLocation[] getFileLocations(Path file, long offset, long range) throws IOException;
    {code}
    and adding
    {code}
    class BlockLocation implements Writable {
    String[] getHosts();
    long getOffset();
    long getLength();
    }
    {code}
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • lohit vijayarenu (JIRA) at Mar 6, 2008 at 6:08 pm ⇧
    [ https://issues.apache.org/jira/browse/HADOOP-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    lohit vijayarenu updated HADOOP-2027:
    -------------------------------------

    Status: Patch Available (was: Open)
    FileSystem should provide byte ranges for file locations
    --------------------------------------------------------

    Key: HADOOP-2027
    URL: https://issues.apache.org/jira/browse/HADOOP-2027
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Owen O'Malley
    Assignee: lohit vijayarenu
    Attachments: HADOOP-2027-1.patch, HADOOP-2027-10.patch, HADOOP-2027-2.patch, HADOOP-2027-3.patch, HADOOP-2027-4.patch, HADOOP-2027-5.patch, HADOOP-2027-6.patch, HADOOP-2027-7.patch, HADOOP-2027-8.patch, HADOOP-2027-9.patch, HADOOP-2559-11.patch, HADOOP-2559-12.patch, HADOOP-2559-13.patch


    FileSystem's getFileCacheHints should be replaced with something more useful. I'd suggest replacing getFileCacheHints with a new method:
    {code}
    BlockLocation[] getFileLocations(Path file, long offset, long range) throws IOException;
    {code}
    and adding
    {code}
    class BlockLocation implements Writable {
    String[] getHosts();
    long getOffset();
    long getLength();
    }
    {code}
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • lohit vijayarenu (JIRA) at Mar 10, 2008 at 11:32 pm ⇧
    [ https://issues.apache.org/jira/browse/HADOOP-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    lohit vijayarenu updated HADOOP-2027:
    -------------------------------------

    Status: Open (was: Patch Available)
    FileSystem should provide byte ranges for file locations
    --------------------------------------------------------

    Key: HADOOP-2027
    URL: https://issues.apache.org/jira/browse/HADOOP-2027
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Owen O'Malley
    Assignee: lohit vijayarenu
    Attachments: HADOOP-2027-1.patch, HADOOP-2027-10.patch, HADOOP-2027-2.patch, HADOOP-2027-3.patch, HADOOP-2027-4.patch, HADOOP-2027-5.patch, HADOOP-2027-6.patch, HADOOP-2027-7.patch, HADOOP-2027-8.patch, HADOOP-2027-9.patch, HADOOP-2559-11.patch, HADOOP-2559-12.patch, HADOOP-2559-13.patch


    FileSystem's getFileCacheHints should be replaced with something more useful. I'd suggest replacing getFileCacheHints with a new method:
    {code}
    BlockLocation[] getFileLocations(Path file, long offset, long range) throws IOException;
    {code}
    and adding
    {code}
    class BlockLocation implements Writable {
    String[] getHosts();
    long getOffset();
    long getLength();
    }
    {code}
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • lohit vijayarenu (JIRA) at Mar 10, 2008 at 11:34 pm ⇧
    [ https://issues.apache.org/jira/browse/HADOOP-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    lohit vijayarenu updated HADOOP-2027:
    -------------------------------------

    Attachment: HADOOP-2027-14.patch

    Attaching another patch after changing getFileCacheHints in FileSystem.java and KFS.
    FileSystem should provide byte ranges for file locations
    --------------------------------------------------------

    Key: HADOOP-2027
    URL: https://issues.apache.org/jira/browse/HADOOP-2027
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Owen O'Malley
    Assignee: lohit vijayarenu
    Attachments: HADOOP-2027-1.patch, HADOOP-2027-10.patch, HADOOP-2027-14.patch, HADOOP-2027-2.patch, HADOOP-2027-3.patch, HADOOP-2027-4.patch, HADOOP-2027-5.patch, HADOOP-2027-6.patch, HADOOP-2027-7.patch, HADOOP-2027-8.patch, HADOOP-2027-9.patch, HADOOP-2559-11.patch, HADOOP-2559-12.patch, HADOOP-2559-13.patch


    FileSystem's getFileCacheHints should be replaced with something more useful. I'd suggest replacing getFileCacheHints with a new method:
    {code}
    BlockLocation[] getFileLocations(Path file, long offset, long range) throws IOException;
    {code}
    and adding
    {code}
    class BlockLocation implements Writable {
    String[] getHosts();
    long getOffset();
    long getLength();
    }
    {code}
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • lohit vijayarenu (JIRA) at Mar 10, 2008 at 11:34 pm ⇧
    [ https://issues.apache.org/jira/browse/HADOOP-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    lohit vijayarenu updated HADOOP-2027:
    -------------------------------------

    Status: Patch Available (was: Open)
    FileSystem should provide byte ranges for file locations
    --------------------------------------------------------

    Key: HADOOP-2027
    URL: https://issues.apache.org/jira/browse/HADOOP-2027
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Owen O'Malley
    Assignee: lohit vijayarenu
    Attachments: HADOOP-2027-1.patch, HADOOP-2027-10.patch, HADOOP-2027-14.patch, HADOOP-2027-2.patch, HADOOP-2027-3.patch, HADOOP-2027-4.patch, HADOOP-2027-5.patch, HADOOP-2027-6.patch, HADOOP-2027-7.patch, HADOOP-2027-8.patch, HADOOP-2027-9.patch, HADOOP-2559-11.patch, HADOOP-2559-12.patch, HADOOP-2559-13.patch


    FileSystem's getFileCacheHints should be replaced with something more useful. I'd suggest replacing getFileCacheHints with a new method:
    {code}
    BlockLocation[] getFileLocations(Path file, long offset, long range) throws IOException;
    {code}
    and adding
    {code}
    class BlockLocation implements Writable {
    String[] getHosts();
    long getOffset();
    long getLength();
    }
    {code}
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Owen O'Malley (JIRA) at Mar 19, 2008 at 11:58 pm ⇧
    [ https://issues.apache.org/jira/browse/HADOOP-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Owen O'Malley updated HADOOP-2027:
    ----------------------------------

    Resolution: Fixed
    Fix Version/s: 0.17.0
    Status: Resolved (was: Patch Available)

    I just committed this. Thanks, Lohit!
    FileSystem should provide byte ranges for file locations
    --------------------------------------------------------

    Key: HADOOP-2027
    URL: https://issues.apache.org/jira/browse/HADOOP-2027
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Owen O'Malley
    Assignee: lohit vijayarenu
    Fix For: 0.17.0

    Attachments: HADOOP-2027-1.patch, HADOOP-2027-10.patch, HADOOP-2027-14.patch, HADOOP-2027-2.patch, HADOOP-2027-3.patch, HADOOP-2027-4.patch, HADOOP-2027-5.patch, HADOOP-2027-6.patch, HADOOP-2027-7.patch, HADOOP-2027-8.patch, HADOOP-2027-9.patch, HADOOP-2559-11.patch, HADOOP-2559-12.patch, HADOOP-2559-13.patch


    FileSystem's getFileCacheHints should be replaced with something more useful. I'd suggest replacing getFileCacheHints with a new method:
    {code}
    BlockLocation[] getFileLocations(Path file, long offset, long range) throws IOException;
    {code}
    and adding
    {code}
    class BlockLocation implements Writable {
    String[] getHosts();
    long getOffset();
    long getLength();
    }
    {code}
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedMar 6, '08 at 6:36a
activeMar 19, '08 at 11:58p
posts13
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Owen O'Malley (JIRA): 13 posts

Content

People

Support

Translate

site design / logo © 2022 Grokbase