FAQ
Writes from map serialization include redundant checks for accounting space
---------------------------------------------------------------------------

Key: HADOOP-3617
URL: https://issues.apache.org/jira/browse/HADOOP-3617
Project: Hadoop Core
Issue Type: Improvement
Components: mapred
Affects Versions: 0.17.0
Reporter: Chris Douglas
Assignee: Chris Douglas
Fix For: 0.19.0


In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Chris Douglas (JIRA) at Jun 22, 2008 at 8:32 am
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-3617:
    ----------------------------------

    Attachment: 3617-0.patch

    This moves the check for accounting space into collect and makes the spill thread persistent. No test cases are included, as this cannot be observed from outside the collector.
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Jun 22, 2008 at 8:34 am
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-3617:
    ----------------------------------

    Status: Patch Available (was: Open)
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Jun 22, 2008 at 10:18 am
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12607077#action_12607077 ]

    Hadoop QA commented on HADOOP-3617:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12384445/3617-0.patch
    against trunk revision 670278.

    +1 @author. The patch does not contain any @author tags.

    -1 tests included. The patch doesn't appear to include any new or modified tests.
    Please justify why no tests are needed for this patch.

    +1 javadoc. The javadoc tool did not generate any warning messages.

    +1 javac. The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs. The patch does not introduce any new Findbugs warnings.

    +1 release audit. The applied patch does not increase the total number of release audit warnings.

    -1 core tests. The patch failed core unit tests.

    +1 contrib tests. The patch passed contrib unit tests.

    Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2717/testReport/
    Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2717/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2717/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2717/console

    This message is automatically generated.
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Jun 22, 2008 at 11:02 am
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12607080#action_12607080 ]

    Chris Douglas commented on HADOOP-3617:
    ---------------------------------------

    The failing test, dfs.TestDatanodeDeath.testDatanodeDeath, is not related.
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Jun 23, 2008 at 10:04 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-3617:
    ----------------------------------

    Attachment: 3617-1.patch

    Added a comment explaining why an interrupt is used instead of a shutdown flag and removed a spurious sync/notify
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch, 3617-1.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Jun 23, 2008 at 10:06 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-3617:
    ----------------------------------

    Status: Open (was: Patch Available)
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch, 3617-1.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Jun 23, 2008 at 10:06 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-3617:
    ----------------------------------

    Status: Patch Available (was: Open)
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch, 3617-1.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Jun 23, 2008 at 11:34 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12607409#action_12607409 ]

    Hadoop QA commented on HADOOP-3617:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12384537/3617-1.patch
    against trunk revision 670709.

    +1 @author. The patch does not contain any @author tags.

    -1 tests included. The patch doesn't appear to include any new or modified tests.
    Please justify why no tests are needed for this patch.

    +1 javadoc. The javadoc tool did not generate any warning messages.

    +1 javac. The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs. The patch does not introduce any new Findbugs warnings.

    +1 release audit. The applied patch does not increase the total number of release audit warnings.

    +1 core tests. The patch passed core unit tests.

    +1 contrib tests. The patch passed contrib unit tests.

    Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2721/testReport/
    Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2721/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2721/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2721/console

    This message is automatically generated.
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch, 3617-1.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Jun 24, 2008 at 4:36 am
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-3617:
    ----------------------------------

    Status: Open (was: Patch Available)

    There remains a (practically unrealizable) possibility of deadlock in this patch. Given the spill thread:
    {noformat}
    while (true) {
    synchronized (spillLock) {
    spillLock.notify();
    while (kvstart == kvend) {
    spillLock.wait();
    }
    }
    try {
    sortAndSpill();
    } catch (Throwable e) {
    sortSpillException = e;
    } finally {
    synchronized (spillLock) {
    // adjust indices
    }
    }
    }
    {noformat}

    It's possible to acquire the lock in the finally block, release it, block at the front of the loop as the other thread acquires the lock and signals before the spill thread can reacquire the lock at the top of the loop. In practice, the only possibility of deadlock would be from the collection thread acquiring the lock in a write that cannot be satisfied without spilling- again- immediately. Still, it needs to be fixed.

    What this attempts to effect seems ill-suited to block-level synchronization and will either require some additional state or more fine-grained locking (i.e. java.util.concurrent.lock).
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch, 3617-1.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Jun 25, 2008 at 12:18 am
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-3617:
    ----------------------------------

    Attachment: 3617-2.patch

    Updated patch.
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch, 3617-1.patch, 3617-2.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Jun 25, 2008 at 12:20 am
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-3617:
    ----------------------------------

    Status: Patch Available (was: Open)
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch, 3617-1.patch, 3617-2.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Jun 25, 2008 at 1:51 am
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12607836#action_12607836 ]

    Hadoop QA commented on HADOOP-3617:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12384638/3617-2.patch
    against trunk revision 671277.

    +1 @author. The patch does not contain any @author tags.

    -1 tests included. The patch doesn't appear to include any new or modified tests.
    Please justify why no tests are needed for this patch.

    +1 javadoc. The javadoc tool did not generate any warning messages.

    +1 javac. The applied patch does not increase the total number of javac compiler warnings.

    -1 findbugs. The patch appears to introduce 2 new Findbugs warnings.

    +1 release audit. The applied patch does not increase the total number of release audit warnings.

    +1 core tests. The patch passed core unit tests.

    +1 contrib tests. The patch passed contrib unit tests.

    Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2729/testReport/
    Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2729/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2729/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2729/console

    This message is automatically generated.
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch, 3617-1.patch, 3617-2.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Jun 25, 2008 at 7:27 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-3617:
    ----------------------------------

    Attachment: 3617-3.patch

    Fixed one findbugs warning, but the second is spurious. We need to release the lock before we wait for the spill thread to die, but we don't need to reacquire it as we exit the flush since there's no contention. In a circumstance where we throw before we release the lock, it's still unlocked in the finally block.
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch, 3617-1.patch, 3617-2.patch, 3617-3.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Jun 25, 2008 at 9:41 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-3617:
    ----------------------------------

    Attachment: 3617-3.patch

    OK, findbugs. You win.
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch, 3617-1.patch, 3617-2.patch, 3617-3.patch, 3617-3.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Jun 25, 2008 at 9:43 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-3617:
    ----------------------------------

    Attachment: (was: 3617-3.patch)
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch, 3617-1.patch, 3617-2.patch, 3617-3.patch, 3617-3.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Jun 25, 2008 at 9:43 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-3617:
    ----------------------------------

    Attachment: 3617-3.patch
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch, 3617-1.patch, 3617-2.patch, 3617-3.patch, 3617-3.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Jun 25, 2008 at 9:47 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-3617:
    ----------------------------------

    Attachment: (was: 3617-3.patch)
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch, 3617-1.patch, 3617-2.patch, 3617-3.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Jun 25, 2008 at 10:25 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-3617:
    ----------------------------------

    Attachment: (was: 3617-3.patch)
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch, 3617-1.patch, 3617-2.patch, 3617-3.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Jun 25, 2008 at 10:25 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-3617:
    ----------------------------------

    Attachment: 3617-3.patch
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch, 3617-1.patch, 3617-2.patch, 3617-3.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Jun 25, 2008 at 10:25 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-3617:
    ----------------------------------

    Attachment: (was: 3617-3.patch)
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch, 3617-1.patch, 3617-2.patch, 3617-3.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Jun 25, 2008 at 10:25 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-3617:
    ----------------------------------

    Attachment: 3617-3.patch
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch, 3617-1.patch, 3617-2.patch, 3617-3.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Jun 25, 2008 at 10:27 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-3617:
    ----------------------------------

    Attachment: 3617-3.patch
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch, 3617-1.patch, 3617-2.patch, 3617-3.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Jun 25, 2008 at 10:27 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-3617:
    ----------------------------------

    Attachment: (was: 3617-3.patch)
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch, 3617-1.patch, 3617-2.patch, 3617-3.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Jun 25, 2008 at 10:27 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-3617:
    ----------------------------------

    Status: Patch Available (was: Open)
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch, 3617-1.patch, 3617-2.patch, 3617-3.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Jun 25, 2008 at 10:27 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-3617:
    ----------------------------------

    Status: Open (was: Patch Available)
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch, 3617-1.patch, 3617-2.patch, 3617-3.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Jun 26, 2008 at 1:01 am
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12608242#action_12608242 ]

    Hadoop QA commented on HADOOP-3617:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12384722/3617-3.patch
    against trunk revision 671563.

    +1 @author. The patch does not contain any @author tags.

    -1 tests included. The patch doesn't appear to include any new or modified tests.
    Please justify why no tests are needed for this patch.

    +1 javadoc. The javadoc tool did not generate any warning messages.

    +1 javac. The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs. The patch does not introduce any new Findbugs warnings.

    +1 release audit. The applied patch does not increase the total number of release audit warnings.

    +1 core tests. The patch passed core unit tests.

    -1 contrib tests. The patch failed contrib unit tests.

    Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2741/testReport/
    Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2741/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2741/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2741/console

    This message is automatically generated.
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch, 3617-1.patch, 3617-2.patch, 3617-3.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Arun C Murthy (JIRA) at Jul 16, 2008 at 11:57 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Arun C Murthy updated HADOOP-3617:
    ----------------------------------

    Resolution: Fixed
    Status: Resolved (was: Patch Available)

    I just committed this. Thanks, Chris!
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch, 3617-1.patch, 3617-2.patch, 3617-3.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hudson (JIRA) at Aug 22, 2008 at 12:40 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12624766#action_12624766 ]

    Hudson commented on HADOOP-3617:
    --------------------------------

    Integrated in Hadoop-trunk #581 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/581/])
    Writes from map serialization include redundant checks for accounting space
    ---------------------------------------------------------------------------

    Key: HADOOP-3617
    URL: https://issues.apache.org/jira/browse/HADOOP-3617
    Project: Hadoop Core
    Issue Type: Improvement
    Components: mapred
    Affects Versions: 0.17.0
    Reporter: Chris Douglas
    Assignee: Chris Douglas
    Fix For: 0.19.0

    Attachments: 3617-0.patch, 3617-1.patch, 3617-2.patch, 3617-3.patch


    In BlockingBuffer::write, the collector verifies that there is sufficient space to hold metadata when it returns to MapOutputBuffer::collect. This check is redundant for serialized records that make more than one call to write.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedJun 22, '08 at 8:28a
activeAug 22, '08 at 12:40p
posts29
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Hudson (JIRA): 29 posts

People

Translate

site design / logo © 2022 Grokbase