FAQ
Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
------------------------------------------------------------------------------------------------------------

Key: HADOOP-3422
URL: https://issues.apache.org/jira/browse/HADOOP-3422
Project: Hadoop Core
Issue Type: Bug
Components: metrics
Affects Versions: 0.16.4
Reporter: Jason
Fix For: 0.17.0


The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..

This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Jason (JIRA) at May 20, 2008 at 5:38 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jason updated HADOOP-3422:
    --------------------------

    Attachment: diff-20080520-1025.txt

    A patch against trunk to provide this.
    Modifies JobInProgress and GangliaContext
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.16.4
    Reporter: Jason
    Fix For: 0.17.0

    Attachments: diff-20080520-1025.txt

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Nigel Daley (JIRA) at May 20, 2008 at 5:42 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Nigel Daley updated HADOOP-3422:
    --------------------------------

    Fix Version/s: (was: 0.17.0)
    0.18.0

    This is not a blocker for 0.17.0 (for which a release candidate is being voted on). Moving to 0.18
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.16.4
    Reporter: Jason
    Fix For: 0.18.0

    Attachments: diff-20080520-1025.txt

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Doug Cutting (JIRA) at May 20, 2008 at 9:31 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12598482#action_12598482 ]

    Doug Cutting commented on HADOOP-3422:
    --------------------------------------

    Did these counters work with Ganglia in a prior release? If so, then this is a regression and might be a candidate for 0.17.1...
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.16.4
    Reporter: Jason
    Fix For: 0.18.0

    Attachments: diff-20080520-1025.txt

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jason (JIRA) at May 20, 2008 at 9:45 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12598492#action_12598492 ]

    Jason commented on HADOOP-3422:
    -------------------------------

    They did not work correctly in 0.15.3 or in 0.16.0. It is a very minor change and I think people would like it.

    The only other issue i currently have is that i have to set up my hadoop-metrics.properties files such that the report to node:
    jvm.servers=HOST:8649 ..., where HOST is the head node, if i use localhost, the metrics end up all in a bucket under the psuedo machine localhost.localdomain
    If the HOST is a real host, they show up under the node's statistics
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.16.4
    Reporter: Jason
    Fix For: 0.18.0

    Attachments: diff-20080520-1025.txt

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jason (JIRA) at May 21, 2008 at 2:20 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12598652#action_12598652 ]

    Jason commented on HADOOP-3422:
    -------------------------------

    The names being produced for ganglia are long and tend to overflow the bounding box in the graph.
    mapred.job.FQCN.COUNTER_NAME
    where FQCN is the fully qualified class that contains the counter ie: org.apache.hadoop.mapred.XXX.YYY

    And it would be nice to get this from the individual machines as well as the cluster agreggate so you could tell at a glance how the individual machines were doing. I suspect that will be annother patch for that new feature.
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.16.4
    Reporter: Jason
    Fix For: 0.18.0

    Attachments: diff-20080520-1025.txt

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • dhruba borthakur (JIRA) at May 29, 2008 at 5:28 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12600845#action_12600845 ]

    dhruba borthakur commented on HADOOP-3422:
    ------------------------------------------

    +1. This patch is very helpful. Are there any unit tests that tests the HadoopMetrics API?
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.16.4
    Reporter: Jason
    Fix For: 0.18.0

    Attachments: diff-20080520-1025.txt

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Mukund Madhugiri (JIRA) at Jun 7, 2008 at 1:28 am
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Mukund Madhugiri updated HADOOP-3422:
    -------------------------------------

    Fix Version/s: (was: 0.18.0)
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.16.4
    Reporter: Jason
    Attachments: diff-20080520-1025.txt

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Lohit Vijayarenu (JIRA) at Jul 24, 2008 at 3:42 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616506#action_12616506 ]

    Lohit Vijayarenu commented on HADOOP-3422:
    ------------------------------------------

    bq. Are there any unit tests that tests the HadoopMetrics API?
    Not as of now. Have opened a JIRA HADOOP-3634 for this. Should be easy if we model something along the lines of FileContext.
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.16.4
    Reporter: Jason
    Attachments: diff-20080520-1025.txt

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Lohit Vijayarenu (JIRA) at Jul 24, 2008 at 8:03 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616619#action_12616619 ]

    Lohit Vijayarenu commented on HADOOP-3422:
    ------------------------------------------

    Trunk has changed, would it be possible to regenerate this patch? We can make it PA once done.
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.16.4
    Reporter: Jason
    Attachments: diff-20080520-1025.txt

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Brian Bockelman (JIRA) at Oct 4, 2008 at 6:20 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Brian Bockelman updated HADOOP-3422:
    ------------------------------------

    Attachment: ganglia-patch-3422-and-4137.patch

    Patch against more recent trunk (rev 701680). Combined the existing patch with the one submitter for #HADOOP-4137; without both, Ganglia metrics appear to be broken.
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.18.1
    Reporter: Jason
    Attachments: diff-20080520-1025.txt, ganglia-patch-3422-and-4137.patch

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Brian Bockelman (JIRA) at Oct 4, 2008 at 6:20 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Brian Bockelman updated HADOOP-3422:
    ------------------------------------

    Affects Version/s: (was: 0.16.4)
    0.18.1
    Status: Patch Available (was: Open)
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.18.1
    Reporter: Jason
    Attachments: diff-20080520-1025.txt, ganglia-patch-3422-and-4137.patch

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Oct 4, 2008 at 11:30 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12636863#action_12636863 ]

    Hadoop QA commented on HADOOP-3422:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12391475/ganglia-patch-3422-and-4137.patch
    against trunk revision 701476.

    +1 @author. The patch does not contain any @author tags.

    -1 tests included. The patch doesn't appear to include any new or modified tests.
    Please justify why no tests are needed for this patch.

    +1 javadoc. The javadoc tool did not generate any warning messages.

    +1 javac. The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs. The patch does not introduce any new Findbugs warnings.

    +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

    -1 core tests. The patch failed core unit tests.

    +1 contrib tests. The patch passed contrib unit tests.

    Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3433/testReport/
    Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3433/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3433/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3433/console

    This message is automatically generated.
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.18.1
    Reporter: Jason
    Attachments: diff-20080520-1025.txt, ganglia-patch-3422-and-4137.patch

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Brian Bockelman (JIRA) at Oct 7, 2008 at 11:21 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Brian Bockelman updated HADOOP-3422:
    ------------------------------------

    Attachment: ganglia-patch-3422-and-4137.patch

    Updated patch: Silly me, Ganglia has no long data type - and I mapped it to int32, which overflows quickly (especially when reporting bytes transferred!).

    This now maps Long to float in Ganglia.
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.18.1
    Reporter: Jason
    Attachments: diff-20080520-1025.txt, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-and-4137.patch

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • stack (JIRA) at Oct 10, 2008 at 6:02 am
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12638491#action_12638491 ]

    stack commented on HADOOP-3422:
    -------------------------------

    I took a look at the patch. You have this in a few places '// TODO: Log this.'. Why not log the exception?

    You keep a ThreadLocal StringBuffer to save on construction cost? Could you use a StringBuilder instead of a StringBuffer?

    Otherwise, this patch is great. I tried it. Ganglia graphs work with this in place.
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.18.1
    Reporter: Jason
    Attachments: diff-20080520-1025.txt, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-and-4137.patch

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Brian Bockelman (JIRA) at Oct 11, 2008 at 3:27 am
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12638708#action_12638708 ]

    Brian Bockelman commented on HADOOP-3422:
    -----------------------------------------

    Hey stack,

    1) I didn't add the log statements because I don't know the "preferred" way to add a logger to classes for this project. I hoped someone who knew would come along and help out.
    2) I don't know the logic behind the ThreadLocal StringBuffer stuff. Adoption from the original patch.
    3) The update to the patch which maps Long to float is bust. The class doesn't have a way to serialize floats with XDR, so I'm not sure what would happen.

    For now, use the original, not the updated patch - having the buffer overflow is better than nothing at all with floats.
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.18.1
    Reporter: Jason
    Attachments: diff-20080520-1025.txt, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-and-4137.patch

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • stack (JIRA) at Oct 11, 2008 at 4:39 am
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12638717#action_12638717 ]

    stack commented on HADOOP-3422:
    -------------------------------

    Good stuff Brian.

    1). Just by way of FYI, see MetricsUtil in same package for how to add logging. Import two classes, then just inside your class opening, add something like:

    {code}
    private static final Log LOG =
    LogFactory.getLog("org.apache.hadoop.util.GangliaContext");
    {code}

    In code, do something like this: LOG.warn("Type is null"); ... and so on.

    2). Let me write Jason and see what he was up to with the ThreadLocal. I don't understand. Its not keeping any state so why bother with it.
    3). Doesn't your patch also address HADOOP-4137 whereas diff-20080520-1025.txt does not?

    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.18.1
    Reporter: Jason
    Attachments: diff-20080520-1025.txt, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-and-4137.patch

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Jason (JIRA) at Oct 20, 2008 at 4:57 am
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Jason updated HADOOP-3422:
    --------------------------

    Attachment: ganglia-patch-3422-qand-4137-1.patch

    This patch removes the thread local code and uses StringBuilder instead of StringBuffer.

    The core ganglia pieces are unchanged.

    At this point an outstanding issue is that it is not possible to distinguish the metrics from different jvms.
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.18.1
    Reporter: Jason
    Attachments: diff-20080520-1025.txt, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-qand-4137-1.patch

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Chris Douglas (JIRA) at Nov 13, 2008 at 11:45 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Chris Douglas updated HADOOP-3422:
    ----------------------------------

    Status: Open (was: Patch Available)

    What is the status of this issue and HADOOP-4137? The latter patch has been absorbed into the one posted here, but the two fixes appear independent (and the latest conflict). I'm canceling both patches for now, since it is unclear whether either is to be committed.
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.18.1
    Reporter: Jason
    Attachments: diff-20080520-1025.txt, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-qand-4137-1.patch

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Brian Bockelman (JIRA) at Nov 17, 2008 at 4:56 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Brian Bockelman updated HADOOP-3422:
    ------------------------------------

    Attachment: ganglia-patch-3422-updated.patch

    The attached patch does the following:
    1) Removes the TODO lines by adding the logger.
    2) Incorporates HADOOP-4137. I'll mark that one as a duplicate of this.
    3) Includes Jason's updated patch.

    Builds cleanly against trunk.
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.18.1
    Reporter: Jason
    Attachments: diff-20080520-1025.txt, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-qand-4137-1.patch, ganglia-patch-3422-updated.patch

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Brian Bockelman (JIRA) at Nov 17, 2008 at 5:00 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Brian Bockelman updated HADOOP-3422:
    ------------------------------------

    Fix Version/s: 0.20.0
    0.19.1
    hudson
    Affects Version/s: 0.17.2
    0.18.0
    0.18.2
    Status: Patch Available (was: Open)
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.18.2, 0.18.1, 0.18.0, 0.17.2
    Reporter: Jason
    Fix For: hudson, 0.19.1, 0.20.0

    Attachments: diff-20080520-1025.txt, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-qand-4137-1.patch, ganglia-patch-3422-updated.patch

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • stack (JIRA) at Nov 18, 2008 at 6:54 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12648700#action_12648700 ]

    stack commented on HADOOP-3422:
    -------------------------------

    Couple of nitpicks: Use StringBuilder instead of StringBuffer and the set of StringBuffer length to zero on line #47 is not needed since you just created a new instance. Also, make comment on line #45 < 80 characters. Otherwise, I tried the patch against 0.19.0. All works.

    What you think we should do about rpc and jvm 'record' names? They both use 'metrics'. Its kinda useless. Means all rpc and jvm metrics get lumped together. Other metrics records are better named: e.g. 'datanode', 'namenode', etc. Should we fix this as part of this patch or in a new issue?

    The jvm metrics record might be named for the application its hosting. And rpc the same?

    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.17.2, 0.18.0, 0.18.1, 0.18.2
    Reporter: Jason
    Fix For: hudson, 0.19.1, 0.20.0

    Attachments: diff-20080520-1025.txt, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-qand-4137-1.patch, ganglia-patch-3422-updated.patch

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Nov 20, 2008 at 7:41 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12649453#action_12649453 ]

    Hadoop QA commented on HADOOP-3422:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12394071/ganglia-patch-3422-updated.patch
    against trunk revision 719152.

    +1 @author. The patch does not contain any @author tags.

    -1 tests included. The patch doesn't appear to include any new or modified tests.
    Please justify why no tests are needed for this patch.

    +1 javadoc. The javadoc tool did not generate any warning messages.

    +1 javac. The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs. The patch does not introduce any new Findbugs warnings.

    +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

    -1 core tests. The patch failed core unit tests.

    +1 contrib tests. The patch passed contrib unit tests.

    Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3611/testReport/
    Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3611/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3611/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3611/console

    This message is automatically generated.
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.17.2, 0.18.0, 0.18.1, 0.18.2
    Reporter: Jason
    Fix For: hudson, 0.19.1, 0.20.0

    Attachments: diff-20080520-1025.txt, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-qand-4137-1.patch, ganglia-patch-3422-updated.patch

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Tsz Wo (Nicholas), SZE (JIRA) at Nov 20, 2008 at 11:29 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12649531#action_12649531 ]

    Tsz Wo (Nicholas), SZE commented on HADOOP-3422:
    ------------------------------------------------

    Seems that the failed test, TestFsck, is not robust. There was another failing in HADOOP-4647. I believe both of the failing cases are not related to the patches posted.
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.17.2, 0.18.0, 0.18.1, 0.18.2
    Reporter: Jason
    Fix For: hudson, 0.19.1, 0.20.0

    Attachments: diff-20080520-1025.txt, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-qand-4137-1.patch, ganglia-patch-3422-updated.patch

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • stack (JIRA) at Nov 20, 2008 at 11:35 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12649534#action_12649534 ]

    stack commented on HADOOP-3422:
    -------------------------------

    I agree the test failure is unrelated but would like to post another patch anyways to address comments above.
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.17.2, 0.18.0, 0.18.1, 0.18.2
    Reporter: Jason
    Fix For: hudson, 0.19.1, 0.20.0

    Attachments: diff-20080520-1025.txt, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-qand-4137-1.patch, ganglia-patch-3422-updated.patch

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • stack (JIRA) at Nov 21, 2008 at 7:08 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    stack updated HADOOP-3422:
    --------------------------

    Attachment: ganglia-patch-3422-updated-v2.patch

    Version of patch that addresses the above comments (Uses StringBuilder instead of StringBuffer, removes extraneous setLength(0), etc.). Also adds overload of JVMMetrics#init method so can pass name of record to use. Default is to use the old 'metrics' name. Overload makes it so can pass in alternate record name. This makes it so can have metrics per jvm rather than have all jvm metrics lumped together. For example, in hbase, we can now have graphs for both the master and regionserver's jvms show in ganglia. Doing similar for rpc didn't make as much sense since method names are shown (and override of RpcMetrics creation is not possible currently).
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.17.2, 0.18.0, 0.18.1, 0.18.2
    Reporter: Jason
    Fix For: hudson, 0.19.1, 0.20.0

    Attachments: diff-20080520-1025.txt, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-qand-4137-1.patch, ganglia-patch-3422-updated-v2.patch, ganglia-patch-3422-updated.patch

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • stack (JIRA) at Nov 21, 2008 at 7:08 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    stack reassigned HADOOP-3422:
    -----------------------------

    Assignee: stack
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.17.2, 0.18.0, 0.18.1, 0.18.2
    Reporter: Jason
    Assignee: stack
    Fix For: hudson, 0.19.1, 0.20.0

    Attachments: diff-20080520-1025.txt, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-qand-4137-1.patch, ganglia-patch-3422-updated-v2.patch, ganglia-patch-3422-updated.patch

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • stack (JIRA) at Nov 21, 2008 at 7:08 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    stack updated HADOOP-3422:
    --------------------------

    Status: In Progress (was: Patch Available)
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.18.2, 0.18.1, 0.18.0, 0.17.2
    Reporter: Jason
    Fix For: hudson, 0.19.1, 0.20.0

    Attachments: diff-20080520-1025.txt, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-qand-4137-1.patch, ganglia-patch-3422-updated-v2.patch, ganglia-patch-3422-updated.patch

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • stack (JIRA) at Nov 21, 2008 at 7:10 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    stack updated HADOOP-3422:
    --------------------------

    Status: Patch Available (was: In Progress)

    Try hudson w/ new version of patch.
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.18.2, 0.18.1, 0.18.0, 0.17.2
    Reporter: Jason
    Assignee: stack
    Fix For: hudson, 0.19.1, 0.20.0

    Attachments: diff-20080520-1025.txt, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-qand-4137-1.patch, ganglia-patch-3422-updated-v2.patch, ganglia-patch-3422-updated.patch

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Hadoop QA (JIRA) at Nov 23, 2008 at 1:48 am
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12649991#action_12649991 ]

    Hadoop QA commented on HADOOP-3422:
    -----------------------------------

    -1 overall. Here are the results of testing the latest attachment
    http://issues.apache.org/jira/secure/attachment/12394440/ganglia-patch-3422-updated-v2.patch
    against trunk revision 719787.

    +1 @author. The patch does not contain any @author tags.

    -1 tests included. The patch doesn't appear to include any new or modified tests.
    Please justify why no tests are needed for this patch.

    +1 javadoc. The javadoc tool did not generate any warning messages.

    +1 javac. The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs. The patch does not introduce any new Findbugs warnings.

    +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

    +1 core tests. The patch passed core unit tests.

    +1 contrib tests. The patch passed contrib unit tests.

    Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3635/testReport/
    Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3635/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
    Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3635/artifact/trunk/build/test/checkstyle-errors.html
    Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3635/console

    This message is automatically generated.
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.17.2, 0.18.0, 0.18.1, 0.18.2
    Reporter: Jason
    Assignee: stack
    Fix For: hudson, 0.19.1, 0.20.0

    Attachments: diff-20080520-1025.txt, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-qand-4137-1.patch, ganglia-patch-3422-updated-v2.patch, ganglia-patch-3422-updated.patch

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Brian Bockelman (JIRA) at Nov 24, 2008 at 2:45 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650215#action_12650215 ]

    Brian Bockelman commented on HADOOP-3422:
    -----------------------------------------

    I set HADOOP-4675 to depend on this, as we can't start supporting Ganglia 3.1 until these basic Ganglia issues are resolved.

    I like stack's refinements. Am I allowed to vote +1 on it if I helped push the patch along? I guess probably not...
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.17.2, 0.18.0, 0.18.1, 0.18.2
    Reporter: Jason
    Assignee: stack
    Fix For: hudson, 0.19.1, 0.20.0

    Attachments: diff-20080520-1025.txt, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-qand-4137-1.patch, ganglia-patch-3422-updated-v2.patch, ganglia-patch-3422-updated.patch

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Doug Cutting (JIRA) at Nov 24, 2008 at 5:36 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650255#action_12650255 ]

    Doug Cutting commented on HADOOP-3422:
    --------------------------------------
    Am I allowed to vote +1 on it if I helped push the patch along? I guess probably not...
    Yes, of course you are allowed to vote. In the case of a dispute, PMC votes trump contributors, but everyone is encouraged to vote, and their opinions are taken seriously.

    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.17.2, 0.18.0, 0.18.1, 0.18.2
    Reporter: Jason
    Assignee: stack
    Fix For: hudson, 0.19.1, 0.20.0

    Attachments: diff-20080520-1025.txt, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-qand-4137-1.patch, ganglia-patch-3422-updated-v2.patch, ganglia-patch-3422-updated.patch

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • stack (JIRA) at Nov 24, 2008 at 7:00 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650279#action_12650279 ]

    stack commented on HADOOP-3422:
    -------------------------------

    Thanks for the review Brian.

    Unless objection, I'll commit to TRUNK in the next day or so.

    Should I commit to branch-0.19 too? I'd like to but its only a 'major' bug, not critical nor a blocker.
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.17.2, 0.18.0, 0.18.1, 0.18.2
    Reporter: Jason
    Assignee: stack
    Fix For: hudson, 0.19.1, 0.20.0

    Attachments: diff-20080520-1025.txt, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-qand-4137-1.patch, ganglia-patch-3422-updated-v2.patch, ganglia-patch-3422-updated.patch

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • stack (JIRA) at Nov 25, 2008 at 10:43 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    stack updated HADOOP-3422:
    --------------------------

    Resolution: Fixed
    Fix Version/s: (was: 0.19.1)
    (was: hudson)
    Hadoop Flags: [Reviewed]
    Status: Resolved (was: Patch Available)

    Committed. Thanks for the patch Jason and Brian.
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.17.2, 0.18.0, 0.18.1, 0.18.2
    Reporter: Jason
    Assignee: stack
    Fix For: 0.20.0

    Attachments: diff-20080520-1025.txt, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-qand-4137-1.patch, ganglia-patch-3422-updated-v2.patch, ganglia-patch-3422-updated.patch

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Robert Chansler (JIRA) at Mar 3, 2009 at 7:01 pm
    [ https://issues.apache.org/jira/browse/HADOOP-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Robert Chansler updated HADOOP-3422:
    ------------------------------------

    Release Note: Changed names of ganglia metrics to avoid conflicts and to better identify source function. (was: Produce ganglia metrics with names of the form:
    jvm.metrics.*
    mapred.job.Job Counters .*
    mapred.job.Map-Reduce Framework.*
    mapred.job.Map-Reduce Framework.*
    mapred.job.Map-Reduce Framework.*
    mapred.jobtracker.*
    mapred.shuffleInput.*
    mapred.tasktracker.*

    instead of just using the last token name, or in the case of Per Job Counters, the string value, for all counters.

    The net result of this change is that the metrics names make it clear which part of hadoop is reporting, and the counter values each have their own RRD file instead of all the counters being aggregated under the name "value")
    Hadoop Flags: [Incompatible change, Reviewed] (was: [Reviewed])

    Edit release note for publication.

    jvm.metrics.*
    mapred.job.Job Counters .*
    mapred.job.Map-Reduce Framework.*
    mapred.job.Map-Reduce Framework.*
    mapred.job.Map-Reduce Framework.*
    mapred.jobtracker.*
    mapred.shuffleInput.*
    mapred.tasktracker.*

    instead of just using the last token name, or in the case of Per Job Counters, the string value, for all counters.

    The net result of this change is that the metrics names make it clear which part of hadoop is reporting, and the counter values each have their own RRD file instead of all the counters being aggregated under the name "value"
    Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen
    ------------------------------------------------------------------------------------------------------------

    Key: HADOOP-3422
    URL: https://issues.apache.org/jira/browse/HADOOP-3422
    Project: Hadoop Core
    Issue Type: Bug
    Components: metrics
    Affects Versions: 0.17.2, 0.18.0, 0.18.1, 0.18.2
    Reporter: Jason
    Assignee: stack
    Fix For: 0.20.0

    Attachments: diff-20080520-1025.txt, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-and-4137.patch, ganglia-patch-3422-qand-4137-1.patch, ganglia-patch-3422-updated-v2.patch, ganglia-patch-3422-updated.patch

    Original Estimate: 1h
    Remaining Estimate: 1h

    The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..
    This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedMay 20, '08 at 5:36p
activeMar 3, '09 at 7:01p
posts35
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Robert Chansler (JIRA): 35 posts

People

Translate

site design / logo © 2022 Grokbase