FAQ
BlockLocation deserialization is incorrect
------------------------------------------

Key: HADOOP-6004
URL: https://issues.apache.org/jira/browse/HADOOP-6004
Project: Hadoop Core
Issue Type: Bug
Components: fs
Reporter: Jakob Homan
Assignee: Jakob Homan


The {{readFields}} method of {{BlockLocation}} does not correctly allocate new arrays for topologyPaths and hosts. Patch shortly.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Jakob Homan (JIRA) at Jun 9, 2009 at 7:30 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jakob Homan updated HADOOP-6004:
    --------------------------------

    Status: Patch Available (was: Open)

    submitting patch.
    BlockLocation deserialization is incorrect
    ------------------------------------------

    Key: HADOOP-6004
    URL: https://issues.apache.org/jira/browse/HADOOP-6004
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Jakob Homan
    Assignee: Jakob Homan
    Attachments: HADOOP-6004.patch


    The {{readFields}} method of {{BlockLocation}} does not correctly allocate new arrays for topologyPaths and hosts. Patch shortly.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jakob Homan (JIRA) at Jun 9, 2009 at 7:30 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jakob Homan updated HADOOP-6004:
    --------------------------------

    Attachment: HADOOP-6004.patch

    {noformat}
    [exec] +1 overall.
    [exec]
    [exec] +1 @author. The patch does not contain any @author tags.
    [exec]
    [exec] +1 tests included. The patch appears to include 2 new or modified tests.
    [exec]
    [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
    [exec]
    [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
    [exec]
    [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
    [exec]
    [exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
    [exec]
    [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.{noformat}
    Patch fixes bug where new array is not allocated during deserialization and provides new unit test to verify.

    Is there any reason BlockLocation is in fs rather than hdfs package?
    BlockLocation deserialization is incorrect
    ------------------------------------------

    Key: HADOOP-6004
    URL: https://issues.apache.org/jira/browse/HADOOP-6004
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Jakob Homan
    Assignee: Jakob Homan
    Attachments: HADOOP-6004.patch


    The {{readFields}} method of {{BlockLocation}} does not correctly allocate new arrays for topologyPaths and hosts. Patch shortly.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Jun 11, 2009 at 3:05 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12718491#action_12718491 ]

    Hadoop QA commented on HADOOP-6004:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12410245/HADOOP-6004.patch
    against trunk revision 783672.

    +1 @author. The patch does not contain any @author tags.

    +1 tests included. The patch appears to include 2 new or modified tests.

    +1 javadoc. The javadoc tool did not generate any warning messages.

    +1 javac. The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs. The patch does not introduce any new Findbugs warnings.

    +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

    +1 release audit. The applied patch does not increase the total number of release audit warnings.

    +1 core tests. The patch passed core unit tests.

    -1 contrib tests. The patch failed contrib unit tests.

    Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/490/testReport/
    Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/490/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/490/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/490/console

    This message is automatically generated.
    BlockLocation deserialization is incorrect
    ------------------------------------------

    Key: HADOOP-6004
    URL: https://issues.apache.org/jira/browse/HADOOP-6004
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Jakob Homan
    Assignee: Jakob Homan
    Attachments: HADOOP-6004.patch


    The {{readFields}} method of {{BlockLocation}} does not correctly allocate new arrays for topologyPaths and hosts. Patch shortly.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Bryan Duxbury (JIRA) at Jun 11, 2009 at 3:51 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12718525#action_12718525 ]

    Bryan Duxbury commented on HADOOP-6004:
    ---------------------------------------

    What kind of issues would this problem cause? Is only trunk affected?
    BlockLocation deserialization is incorrect
    ------------------------------------------

    Key: HADOOP-6004
    URL: https://issues.apache.org/jira/browse/HADOOP-6004
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Jakob Homan
    Assignee: Jakob Homan
    Attachments: HADOOP-6004.patch


    The {{readFields}} method of {{BlockLocation}} does not correctly allocate new arrays for topologyPaths and hosts. Patch shortly.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jakob Homan (JIRA) at Jun 12, 2009 at 8:59 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12718973#action_12718973 ]

    Jakob Homan commented on HADOOP-6004:
    -------------------------------------

    Failing contrib test is known-bad testhdfsproxy. Patch is good to go.
    BlockLocation deserialization is incorrect
    ------------------------------------------

    Key: HADOOP-6004
    URL: https://issues.apache.org/jira/browse/HADOOP-6004
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Jakob Homan
    Assignee: Jakob Homan
    Attachments: HADOOP-6004.patch


    The {{readFields}} method of {{BlockLocation}} does not correctly allocate new arrays for topologyPaths and hosts. Patch shortly.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jakob Homan (JIRA) at Jun 12, 2009 at 9:05 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12718977#action_12718977 ]

    Jakob Homan commented on HADOOP-6004:
    -------------------------------------

    bq. What kind of issues would this problem cause?
    Since the arrays were not resized before trying to assign new values, if you were trying to deserialize a BL with multiple hosts or topologyPaths, you would get an array index out of bounds exception, which is a runtime exception and wouldn't have been caught. Since this happened, there must be no code that's trying to do. The deserialization is still broken and should be fixed though.

    bq. Is only trunk affected?
    I checked back to the 18.3 code and this problem is there. It would probably be a good idea to do a patch for that version as well.

    BlockLocation deserialization is incorrect
    ------------------------------------------

    Key: HADOOP-6004
    URL: https://issues.apache.org/jira/browse/HADOOP-6004
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Jakob Homan
    Assignee: Jakob Homan
    Attachments: HADOOP-6004.patch


    The {{readFields}} method of {{BlockLocation}} does not correctly allocate new arrays for topologyPaths and hosts. Patch shortly.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Jun 15, 2009 at 6:06 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12719678#action_12719678 ]

    Tsz Wo (Nicholas), SZE commented on HADOOP-6004:
    ------------------------------------------------

    {code}
    + topologyPaths = new String[numHosts];
    {code}
    Should it use numTops?
    BlockLocation deserialization is incorrect
    ------------------------------------------

    Key: HADOOP-6004
    URL: https://issues.apache.org/jira/browse/HADOOP-6004
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Jakob Homan
    Assignee: Jakob Homan
    Attachments: HADOOP-6004.patch


    The {{readFields}} method of {{BlockLocation}} does not correctly allocate new arrays for topologyPaths and hosts. Patch shortly.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jakob Homan (JIRA) at Jun 15, 2009 at 6:28 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jakob Homan updated HADOOP-6004:
    --------------------------------

    Attachment: HADOOP-6004.patch

    D'oh. Yes, it should be. Good catch. Uploaded new version.

    The BlockLocation class could be improved as there is an implicit requirement that the length of the hosts, names and topologies all should be the same length, which currently is not verified or enforced. A better implementation would be a list of tuples of {host, name, topology} with accompanying offset and length.
    BlockLocation deserialization is incorrect
    ------------------------------------------

    Key: HADOOP-6004
    URL: https://issues.apache.org/jira/browse/HADOOP-6004
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Jakob Homan
    Assignee: Jakob Homan
    Attachments: HADOOP-6004.patch, HADOOP-6004.patch


    The {{readFields}} method of {{BlockLocation}} does not correctly allocate new arrays for topologyPaths and hosts. Patch shortly.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Jun 15, 2009 at 6:36 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12719691#action_12719691 ]

    Tsz Wo (Nicholas), SZE commented on HADOOP-6004:
    ------------------------------------------------

    +1 patch looks good.

    A nit: It is better to replace the existing codes
    {code}
    Text t = new Text();
    t.readFields(in);
    variable = t.toString();
    {code}
    by
    {code}
    variable = Text.readString(in);
    {code}
    for preventing object creation.
    BlockLocation deserialization is incorrect
    ------------------------------------------

    Key: HADOOP-6004
    URL: https://issues.apache.org/jira/browse/HADOOP-6004
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Jakob Homan
    Assignee: Jakob Homan
    Attachments: HADOOP-6004.patch, HADOOP-6004.patch


    The {{readFields}} method of {{BlockLocation}} does not correctly allocate new arrays for topologyPaths and hosts. Patch shortly.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jakob Homan (JIRA) at Jun 15, 2009 at 6:45 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12719702#action_12719702 ]

    Jakob Homan commented on HADOOP-6004:
    -------------------------------------

    Quite right. A better refactoring would be to pull out the whole:
    {noformat}
    int numHosts = in.readInt();
    this.hosts = new String[numHosts];
    for (int i = 0; i < numHosts; i++) {
    Text host = new Text();
    host.readFields(in);
    hosts[i] = host.toString();
    }
    {noformat}
    block into a separate method, since it's repeated three times for each of the sections...
    BlockLocation deserialization is incorrect
    ------------------------------------------

    Key: HADOOP-6004
    URL: https://issues.apache.org/jira/browse/HADOOP-6004
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Jakob Homan
    Assignee: Jakob Homan
    Attachments: HADOOP-6004.patch, HADOOP-6004.patch


    The {{readFields}} method of {{BlockLocation}} does not correctly allocate new arrays for topologyPaths and hosts. Patch shortly.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Jun 16, 2009 at 12:30 am
    [ https://issues.apache.org/jira/browse/HADOOP-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12719870#action_12719870 ]

    Tsz Wo (Nicholas), SZE commented on HADOOP-6004:
    ------------------------------------------------
    ... A better refactoring would be ...
    My suggestion is not a code refactoring: it reduces the number of object creations since "new Text()" is not needed.
    BlockLocation deserialization is incorrect
    ------------------------------------------

    Key: HADOOP-6004
    URL: https://issues.apache.org/jira/browse/HADOOP-6004
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Jakob Homan
    Assignee: Jakob Homan
    Attachments: HADOOP-6004.patch, HADOOP-6004.patch


    The {{readFields}} method of {{BlockLocation}} does not correctly allocate new arrays for topologyPaths and hosts. Patch shortly.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jakob Homan (JIRA) at Jun 16, 2009 at 12:36 am
    [ https://issues.apache.org/jira/browse/HADOOP-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12719873#action_12719873 ]

    Jakob Homan commented on HADOOP-6004:
    -------------------------------------

    I know. I moved the text section so it would be consistent with the other two sections. My comment was towards a more general fix of the code.
    BlockLocation deserialization is incorrect
    ------------------------------------------

    Key: HADOOP-6004
    URL: https://issues.apache.org/jira/browse/HADOOP-6004
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Jakob Homan
    Assignee: Jakob Homan
    Attachments: HADOOP-6004.patch, HADOOP-6004.patch


    The {{readFields}} method of {{BlockLocation}} does not correctly allocate new arrays for topologyPaths and hosts. Patch shortly.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jakob Homan (JIRA) at Jun 16, 2009 at 12:38 am
    [ https://issues.apache.org/jira/browse/HADOOP-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jakob Homan updated HADOOP-6004:
    --------------------------------

    Status: Open (was: Patch Available)

    canceling patch to get submit for hudson again.
    BlockLocation deserialization is incorrect
    ------------------------------------------

    Key: HADOOP-6004
    URL: https://issues.apache.org/jira/browse/HADOOP-6004
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Jakob Homan
    Assignee: Jakob Homan
    Attachments: HADOOP-6004.patch, HADOOP-6004.patch


    The {{readFields}} method of {{BlockLocation}} does not correctly allocate new arrays for topologyPaths and hosts. Patch shortly.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jakob Homan (JIRA) at Jun 16, 2009 at 12:38 am
    [ https://issues.apache.org/jira/browse/HADOOP-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jakob Homan updated HADOOP-6004:
    --------------------------------

    Status: Patch Available (was: Open)

    re-submitting to hudson.
    BlockLocation deserialization is incorrect
    ------------------------------------------

    Key: HADOOP-6004
    URL: https://issues.apache.org/jira/browse/HADOOP-6004
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Jakob Homan
    Assignee: Jakob Homan
    Attachments: HADOOP-6004.patch, HADOOP-6004.patch


    The {{readFields}} method of {{BlockLocation}} does not correctly allocate new arrays for topologyPaths and hosts. Patch shortly.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jakob Homan (JIRA) at Jun 17, 2009 at 7:58 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12720855#action_12720855 ]

    Jakob Homan commented on HADOOP-6004:
    -------------------------------------

    Hudson is too slow. Ran unit tests locally. All pass except known-bad.
    {noformat}
    [exec] +1 overall.
    [exec]
    [exec] +1 @author. The patch does not contain any @author tags.
    [exec]
    [exec] +1 tests included. The patch appears to include 2 new or modified tests.
    [exec]
    [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
    [exec]
    [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
    [exec]
    [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
    [exec]
    [exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
    [exec]
    [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.{noformat}
    BlockLocation deserialization is incorrect
    ------------------------------------------

    Key: HADOOP-6004
    URL: https://issues.apache.org/jira/browse/HADOOP-6004
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Jakob Homan
    Assignee: Jakob Homan
    Attachments: HADOOP-6004.patch, HADOOP-6004.patch


    The {{readFields}} method of {{BlockLocation}} does not correctly allocate new arrays for topologyPaths and hosts. Patch shortly.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Jun 18, 2009 at 9:05 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Tsz Wo (Nicholas), SZE updated HADOOP-6004:
    -------------------------------------------

    Resolution: Fixed
    Fix Version/s: 0.21.0
    Hadoop Flags: [Reviewed]
    Status: Resolved (was: Patch Available)

    I have committed this. Thanks, Jakob!
    BlockLocation deserialization is incorrect
    ------------------------------------------

    Key: HADOOP-6004
    URL: https://issues.apache.org/jira/browse/HADOOP-6004
    Project: Hadoop Core
    Issue Type: Bug
    Components: fs
    Reporter: Jakob Homan
    Assignee: Jakob Homan
    Fix For: 0.21.0

    Attachments: HADOOP-6004.patch, HADOOP-6004.patch


    The {{readFields}} method of {{BlockLocation}} does not correctly allocate new arrays for topologyPaths and hosts. Patch shortly.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hudson (JIRA) at Jun 20, 2009 at 3:18 pm
    [ https://issues.apache.org/jira/browse/HADOOP-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12722207#action_12722207 ]

    Hudson commented on HADOOP-6004:
    --------------------------------

    Integrated in Hadoop-trunk #872 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/872/])

    BlockLocation deserialization is incorrect
    ------------------------------------------

    Key: HADOOP-6004
    URL: https://issues.apache.org/jira/browse/HADOOP-6004
    Project: Hadoop Common
    Issue Type: Bug
    Components: fs
    Reporter: Jakob Homan
    Assignee: Jakob Homan
    Fix For: 0.21.0

    Attachments: HADOOP-6004.patch, HADOOP-6004.patch


    The {{readFields}} method of {{BlockLocation}} does not correctly allocate new arrays for topologyPaths and hosts. Patch shortly.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedJun 9, '09 at 6:40p
activeJun 20, '09 at 3:18p
posts18
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Hudson (JIRA): 18 posts

People

Translate

site design / logo © 2022 Grokbase