Grokbase Groups HBase dev July 2012
FAQ
Hi,
See the following comment for background:
https://issues.apache.org/jira/browse/HBASE-4050?focusedCommentId=13408679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13408679

metrics2 framework is in both hadoop 1.0 and hadoop 2.0
We would like to migrate to metrics2 framework. But maintaining two metrics
frameworks in HBase is costly and results in lower readability.

However, if you have cluster monitoring tool that depends on (old) metrics
framework, we want to give you smooth migration experience.

Please provide your opinions on whether (old) metrics framework should be
deprecated in 0.94 or in 0.96.

Thanks

Search Discussions

  • Jonathan Hsieh at Jul 9, 2012 at 4:29 pm
    I'm ok if we moved solely to metrics2 in 0.96 and if we deprecated in 0.94.

    Jon.
    On Sat, Jul 7, 2012 at 7:23 AM, Ted Yu wrote:

    Hi,
    See the following comment for background:

    https://issues.apache.org/jira/browse/HBASE-4050?focusedCommentId=13408679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13408679

    metrics2 framework is in both hadoop 1.0 and hadoop 2.0
    We would like to migrate to metrics2 framework. But maintaining two metrics
    frameworks in HBase is costly and results in lower readability.

    However, if you have cluster monitoring tool that depends on (old) metrics
    framework, we want to give you smooth migration experience.

    Please provide your opinions on whether (old) metrics framework should be
    deprecated in 0.94 or in 0.96.

    Thanks


    --
    // Jonathan Hsieh (shay)
    // Software Engineer, Cloudera
    // jon@cloudera.com
  • Ted Yu at Jul 9, 2012 at 4:36 pm
    Thanks for the response, Jon.

    One more detail: the metrics2 classes in hadoop 1.0 and hadoop 2.0 are
    different. Alex is working on a shim layer to make the difference
    transparent. If we were to keep 0.96 metric (1) compatible, we would be
    maintaining 2.5 metric systems.

    Cheers
    On Mon, Jul 9, 2012 at 9:28 AM, Jonathan Hsieh wrote:

    I'm ok if we moved solely to metrics2 in 0.96 and if we deprecated in 0.94.

    Jon.
    On Sat, Jul 7, 2012 at 7:23 AM, Ted Yu wrote:

    Hi,
    See the following comment for background:

    https://issues.apache.org/jira/browse/HBASE-4050?focusedCommentId=13408679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13408679
    metrics2 framework is in both hadoop 1.0 and hadoop 2.0
    We would like to migrate to metrics2 framework. But maintaining two metrics
    frameworks in HBase is costly and results in lower readability.

    However, if you have cluster monitoring tool that depends on (old) metrics
    framework, we want to give you smooth migration experience.

    Please provide your opinions on whether (old) metrics framework should be
    deprecated in 0.94 or in 0.96.

    Thanks


    --
    // Jonathan Hsieh (shay)
    // Software Engineer, Cloudera
    // jon@cloudera.com
  • Ted Yu at Jul 9, 2012 at 7:29 pm
    If I don't get objection by Friday the 13th @ mid night (PST), I will
    create a JIRA that marks HBase metric classes that depend on
    org.apache.hadoop.metrics.* deprecated.

    Thanks
    On Mon, Jul 9, 2012 at 9:35 AM, Ted Yu wrote:

    Thanks for the response, Jon.

    One more detail: the metrics2 classes in hadoop 1.0 and hadoop 2.0 are
    different. Alex is working on a shim layer to make the difference
    transparent. If we were to keep 0.96 metric (1) compatible, we would be
    maintaining 2.5 metric systems.

    Cheers

    On Mon, Jul 9, 2012 at 9:28 AM, Jonathan Hsieh wrote:

    I'm ok if we moved solely to metrics2 in 0.96 and if we deprecated in
    0.94.

    Jon.
    On Sat, Jul 7, 2012 at 7:23 AM, Ted Yu wrote:

    Hi,
    See the following comment for background:

    https://issues.apache.org/jira/browse/HBASE-4050?focusedCommentId=13408679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13408679
    metrics2 framework is in both hadoop 1.0 and hadoop 2.0
    We would like to migrate to metrics2 framework. But maintaining two metrics
    frameworks in HBase is costly and results in lower readability.

    However, if you have cluster monitoring tool that depends on (old) metrics
    framework, we want to give you smooth migration experience.

    Please provide your opinions on whether (old) metrics framework should be
    deprecated in 0.94 or in 0.96.

    Thanks


    --
    // Jonathan Hsieh (shay)
    // Software Engineer, Cloudera
    // jon@cloudera.com
  • Elliott Clark at Jul 9, 2012 at 9:35 pm
    HBASE-6323 has the first Metrics2 implementation that I've gotten to. We
    will have to add lots of documentation around hadoop-metrics2.properties
    and the jmx locations are not consistent, however Metrics2 is much nicer to
    use for dynamic metrics.
    On Mon, Jul 9, 2012 at 12:28 PM, Ted Yu wrote:

    If I don't get objection by Friday the 13th @ mid night (PST), I will
    create a JIRA that marks HBase metric classes that depend on
    org.apache.hadoop.metrics.* deprecated.

    Thanks
    On Mon, Jul 9, 2012 at 9:35 AM, Ted Yu wrote:

    Thanks for the response, Jon.

    One more detail: the metrics2 classes in hadoop 1.0 and hadoop 2.0 are
    different. Alex is working on a shim layer to make the difference
    transparent. If we were to keep 0.96 metric (1) compatible, we would be
    maintaining 2.5 metric systems.

    Cheers

    On Mon, Jul 9, 2012 at 9:28 AM, Jonathan Hsieh wrote:

    I'm ok if we moved solely to metrics2 in 0.96 and if we deprecated in
    0.94.

    Jon.
    On Sat, Jul 7, 2012 at 7:23 AM, Ted Yu wrote:

    Hi,
    See the following comment for background:
    https://issues.apache.org/jira/browse/HBASE-4050?focusedCommentId=13408679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13408679
    metrics2 framework is in both hadoop 1.0 and hadoop 2.0
    We would like to migrate to metrics2 framework. But maintaining two metrics
    frameworks in HBase is costly and results in lower readability.

    However, if you have cluster monitoring tool that depends on (old) metrics
    framework, we want to give you smooth migration experience.

    Please provide your opinions on whether (old) metrics framework should be
    deprecated in 0.94 or in 0.96.

    Thanks


    --
    // Jonathan Hsieh (shay)
    // Software Engineer, Cloudera
    // jon@cloudera.com
  • Lars hofhansl at Jul 9, 2012 at 11:48 pm
    Let's not deprecate until the replacement is ready (including docs).

    -0


    -- Lars


    ----- Original Message -----
    From: Elliott Clark <eclark@stumbleupon.com>
    To: dev@hbase.apache.org
    Cc:
    Sent: Monday, July 9, 2012 2:34 PM
    Subject: Re: deprecating (old) metrics in favor of metrics2 framework

    HBASE-6323 has the first Metrics2 implementation that I've gotten to.  We
    will have to add lots of documentation around hadoop-metrics2.properties
    and the jmx locations are not consistent, however Metrics2 is much nicer to
    use for dynamic metrics.
    On Mon, Jul 9, 2012 at 12:28 PM, Ted Yu wrote:

    If I don't get objection by Friday the 13th @ mid night (PST), I will
    create a JIRA that marks HBase metric classes that depend on
    org.apache.hadoop.metrics.* deprecated.

    Thanks
    On Mon, Jul 9, 2012 at 9:35 AM, Ted Yu wrote:

    Thanks for the response, Jon.

    One more detail: the metrics2 classes in hadoop 1.0 and hadoop 2.0 are
    different. Alex is working on a shim layer to make the difference
    transparent. If we were to keep 0.96 metric (1) compatible, we would be
    maintaining 2.5 metric systems.

    Cheers

    On Mon, Jul 9, 2012 at 9:28 AM, Jonathan Hsieh wrote:

    I'm ok if we moved solely to metrics2 in 0.96 and if we deprecated in
    0.94.

    Jon.
    On Sat, Jul 7, 2012 at 7:23 AM, Ted Yu wrote:

    Hi,
    See the following comment for background:
    https://issues.apache.org/jira/browse/HBASE-4050?focusedCommentId=13408679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13408679
    metrics2 framework is in both hadoop 1.0 and hadoop 2.0
    We would like to migrate to metrics2 framework. But maintaining two metrics
    frameworks in HBase is costly and results in lower readability.

    However, if you have cluster monitoring tool that depends on (old) metrics
    framework, we want to give you smooth migration experience.

    Please provide your opinions on whether (old) metrics framework should be
    deprecated in 0.94 or in 0.96.

    Thanks


    --
    // Jonathan Hsieh (shay)
    // Software Engineer, Cloudera
    // jon@cloudera.com
  • Ted Yu at Jul 9, 2012 at 11:53 pm
    Lars:
    Do you think 0.96 should only support metrics2 framework ?

    Cheers
    On Mon, Jul 9, 2012 at 4:47 PM, lars hofhansl wrote:

    Let's not deprecate until the replacement is ready (including docs).

    -0


    -- Lars


    ----- Original Message -----
    From: Elliott Clark <eclark@stumbleupon.com>
    To: dev@hbase.apache.org
    Cc:
    Sent: Monday, July 9, 2012 2:34 PM
    Subject: Re: deprecating (old) metrics in favor of metrics2 framework

    HBASE-6323 has the first Metrics2 implementation that I've gotten to. We
    will have to add lots of documentation around hadoop-metrics2.properties
    and the jmx locations are not consistent, however Metrics2 is much nicer to
    use for dynamic metrics.
    On Mon, Jul 9, 2012 at 12:28 PM, Ted Yu wrote:

    If I don't get objection by Friday the 13th @ mid night (PST), I will
    create a JIRA that marks HBase metric classes that depend on
    org.apache.hadoop.metrics.* deprecated.

    Thanks
    On Mon, Jul 9, 2012 at 9:35 AM, Ted Yu wrote:

    Thanks for the response, Jon.

    One more detail: the metrics2 classes in hadoop 1.0 and hadoop 2.0 are
    different. Alex is working on a shim layer to make the difference
    transparent. If we were to keep 0.96 metric (1) compatible, we would be
    maintaining 2.5 metric systems.

    Cheers

    On Mon, Jul 9, 2012 at 9:28 AM, Jonathan Hsieh wrote:

    I'm ok if we moved solely to metrics2 in 0.96 and if we deprecated in
    0.94.

    Jon.
    On Sat, Jul 7, 2012 at 7:23 AM, Ted Yu wrote:

    Hi,
    See the following comment for background:
    https://issues.apache.org/jira/browse/HBASE-4050?focusedCommentId=13408679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13408679
    metrics2 framework is in both hadoop 1.0 and hadoop 2.0
    We would like to migrate to metrics2 framework. But maintaining two metrics
    frameworks in HBase is costly and results in lower readability.

    However, if you have cluster monitoring tool that depends on (old) metrics
    framework, we want to give you smooth migration experience.

    Please provide your opinions on whether (old) metrics framework
    should
    be
    deprecated in 0.94 or in 0.96.

    Thanks


    --
    // Jonathan Hsieh (shay)
    // Software Engineer, Cloudera
    // jon@cloudera.com
  • Lars hofhansl at Jul 9, 2012 at 11:57 pm
    It seems that if we want to make that switch 0.96 (the singularity) should be release.
    Sorry, I should give a clear vote.
    +1 on metrics2 in 0.96 only (if we're ready by then of course)



    ________________________________
    From: Ted Yu <yuzhihong@gmail.com>
    To: dev@hbase.apache.org; lars hofhansl <lhofhansl@yahoo.com>
    Sent: Monday, July 9, 2012 4:52 PM
    Subject: Re: deprecating (old) metrics in favor of metrics2 framework


    Lars:
    Do you think 0.96 should only support metrics2 framework ?

    Cheers


    On Mon, Jul 9, 2012 at 4:47 PM, lars hofhansl wrote:

    Let's not deprecate until the replacement is ready (including docs).
    -0


    -- Lars



    ----- Original Message -----
    From: Elliott Clark <eclark@stumbleupon.com>
    To: dev@hbase.apache.org
    Cc:
    Sent: Monday, July 9, 2012 2:34 PM
    Subject: Re: deprecating (old) metrics in favor of metrics2 framework


    HBASE-6323 has the first Metrics2 implementation that I've gotten to.  We
    will have to add lots of documentation around hadoop-metrics2.properties
    and the jmx locations are not consistent, however Metrics2 is much nicer to
    use for dynamic metrics.
    On Mon, Jul 9, 2012 at 12:28 PM, Ted Yu wrote:

    If I don't get objection by Friday the 13th @ mid night (PST), I will
    create a JIRA that marks HBase metric classes that depend on
    org.apache.hadoop.metrics.* deprecated.

    Thanks
    On Mon, Jul 9, 2012 at 9:35 AM, Ted Yu wrote:

    Thanks for the response, Jon.

    One more detail: the metrics2 classes in hadoop 1.0 and hadoop 2.0 are
    different. Alex is working on a shim layer to make the difference
    transparent. If we were to keep 0.96 metric (1) compatible, we would be
    maintaining 2.5 metric systems.

    Cheers

    On Mon, Jul 9, 2012 at 9:28 AM, Jonathan Hsieh wrote:

    I'm ok if we moved solely to metrics2 in 0.96 and if we deprecated in
    0.94.

    Jon.
    On Sat, Jul 7, 2012 at 7:23 AM, Ted Yu wrote:

    Hi,
    See the following comment for background:
    https://issues.apache.org/jira/browse/HBASE-4050?focusedCommentId=13408679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13408679
    metrics2 framework is in both hadoop 1.0 and hadoop 2.0
    We would like to migrate to metrics2 framework. But maintaining two metrics
    frameworks in HBase is costly and results in lower readability.

    However, if you have cluster monitoring tool that depends on (old) metrics
    framework, we want to give you smooth migration experience.

    Please provide your opinions on whether (old) metrics framework should be
    deprecated in 0.94 or in 0.96.

    Thanks


    --
    // Jonathan Hsieh (shay)
    // Software Engineer, Cloudera
    // jon@cloudera.com
  • Otis Gospodnetic at Jul 10, 2012 at 4:05 am
    +1 for deprecating and moving to metrics2 faster.
    I'd normally make the same argument about deprecating that Lars H. made, but I think current HBase metrics really need improvement, so if I had voting power I'd vote to drop old stuff and move to metrics2 ASAP.

    Otis
    ----- Original Message -----
    From: Ted Yu <yuzhihong@gmail.com>
    To: dev@hbase.apache.org
    Cc:
    Sent: Saturday, July 7, 2012 10:23 AM
    Subject: deprecating (old) metrics in favor of metrics2 framework

    Hi,
    See the following comment for background:
    https://issues.apache.org/jira/browse/HBASE-4050?focusedCommentId=13408679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13408679

    metrics2 framework is in both hadoop 1.0 and hadoop 2.0
    We would like to migrate to metrics2 framework. But maintaining two metrics
    frameworks in HBase is costly and results in lower readability.

    However, if you have cluster monitoring tool that depends on (old) metrics
    framework, we want to give you smooth migration experience.

    Please provide your opinions on whether (old) metrics framework should be
    deprecated in 0.94 or in 0.96.

    Thanks
  • Ted Yu at Jul 10, 2012 at 8:46 am
    In another email thread 'HBase 0.94.1', Lars has kindly accepted the motion
    that deprecating current metric classes be done for 0.94.1

    Since 0.94.1 contains critical bug fixes, I want to get public opinion on
    closing this poll earlier than Friday.

    Thanks for the understanding.
    On Mon, Jul 9, 2012 at 9:05 PM, Otis Gospodnetic wrote:

    +1 for deprecating and moving to metrics2 faster.
    I'd normally make the same argument about deprecating that Lars H. made,
    but I think current HBase metrics really need improvement, so if I had
    voting power I'd vote to drop old stuff and move to metrics2 ASAP.

    Otis
    ----- Original Message -----
    From: Ted Yu <yuzhihong@gmail.com>
    To: dev@hbase.apache.org
    Cc:
    Sent: Saturday, July 7, 2012 10:23 AM
    Subject: deprecating (old) metrics in favor of metrics2 framework

    Hi,
    See the following comment for background:
    https://issues.apache.org/jira/browse/HBASE-4050?focusedCommentId=13408679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13408679
    metrics2 framework is in both hadoop 1.0 and hadoop 2.0
    We would like to migrate to metrics2 framework. But maintaining two metrics
    frameworks in HBase is costly and results in lower readability.

    However, if you have cluster monitoring tool that depends on (old) metrics
    framework, we want to give you smooth migration experience.

    Please provide your opinions on whether (old) metrics framework should be
    deprecated in 0.94 or in 0.96.

    Thanks
  • Gary Helmling at Jul 10, 2012 at 6:26 pm
    Do we have any usage of metrics2 currently in 0.94? If not, it seems
    odd to deprecate existing metrics implementations. Aside from whether
    or not we should deprecate in a point release, normally deprecation
    happens when the new implementation you're supposed to move to has
    already been added, so that you have both old and new versions in
    parallel.

    If we deprecate existing metrics in 0.94 and only have metrics2
    implementation in 0.96, that gives no chance for switching back and
    forth between the two in the same release. While most of the metrics
    classes can be seen as private, I think that MetricsContext is
    arguably a public interface. Implementing your own MetricsContext to
    plugin to your own monitoring system is not that crazy a thing to do.
    So if that is changing in the move from metrics to metrics2, I think
    it would be hurting our users to not have both implementations
    available at the same time.

    On Tue, Jul 10, 2012 at 1:45 AM, Ted Yu wrote:
    In another email thread 'HBase 0.94.1', Lars has kindly accepted the motion
    that deprecating current metric classes be done for 0.94.1

    Since 0.94.1 contains critical bug fixes, I want to get public opinion on
    closing this poll earlier than Friday.

    Thanks for the understanding.

    On Mon, Jul 9, 2012 at 9:05 PM, Otis Gospodnetic <otis_gospodnetic@yahoo.com
    wrote:
    +1 for deprecating and moving to metrics2 faster.
    I'd normally make the same argument about deprecating that Lars H. made,
    but I think current HBase metrics really need improvement, so if I had
    voting power I'd vote to drop old stuff and move to metrics2 ASAP.

    Otis
    ----- Original Message -----
    From: Ted Yu <yuzhihong@gmail.com>
    To: dev@hbase.apache.org
    Cc:
    Sent: Saturday, July 7, 2012 10:23 AM
    Subject: deprecating (old) metrics in favor of metrics2 framework

    Hi,
    See the following comment for background:
    https://issues.apache.org/jira/browse/HBASE-4050?focusedCommentId=13408679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13408679
    metrics2 framework is in both hadoop 1.0 and hadoop 2.0
    We would like to migrate to metrics2 framework. But maintaining two metrics
    frameworks in HBase is costly and results in lower readability.

    However, if you have cluster monitoring tool that depends on (old) metrics
    framework, we want to give you smooth migration experience.

    Please provide your opinions on whether (old) metrics framework should be
    deprecated in 0.94 or in 0.96.

    Thanks
  • Ted Yu at Jul 10, 2012 at 7:02 pm
    Thanks for the analysis, Gary.

    Whether we support 2 (actually more than 2) metrics frameworks in 0.96 can
    be debated in the next 2 months.

    As Todd mentioned in the thread 'HBase 0.94.1', we will try our best to
    keep JMX interface the same across 0.94 and 0.96. Does this somehow reduce
    the concern you raised ?

    As for using MetricsContext, I assume the user also uses hadoop in his /
    her deployment. Then he / she should be aware of the deprecation of
    metrics.* classes in both hadoop 1.0 and 2.0
    Meaning he / she should be prepared to endorse metrics2 framework.

    Correct me if I am wrong.
    On Tue, Jul 10, 2012 at 11:26 AM, Gary Helmling wrote:

    Do we have any usage of metrics2 currently in 0.94? If not, it seems
    odd to deprecate existing metrics implementations. Aside from whether
    or not we should deprecate in a point release, normally deprecation
    happens when the new implementation you're supposed to move to has
    already been added, so that you have both old and new versions in
    parallel.

    If we deprecate existing metrics in 0.94 and only have metrics2
    implementation in 0.96, that gives no chance for switching back and
    forth between the two in the same release. While most of the metrics
    classes can be seen as private, I think that MetricsContext is
    arguably a public interface. Implementing your own MetricsContext to
    plugin to your own monitoring system is not that crazy a thing to do.
    So if that is changing in the move from metrics to metrics2, I think
    it would be hurting our users to not have both implementations
    available at the same time.

    On Tue, Jul 10, 2012 at 1:45 AM, Ted Yu wrote:
    In another email thread 'HBase 0.94.1', Lars has kindly accepted the motion
    that deprecating current metric classes be done for 0.94.1

    Since 0.94.1 contains critical bug fixes, I want to get public opinion on
    closing this poll earlier than Friday.

    Thanks for the understanding.

    On Mon, Jul 9, 2012 at 9:05 PM, Otis Gospodnetic <
    otis_gospodnetic@yahoo.com
    wrote:
    +1 for deprecating and moving to metrics2 faster.
    I'd normally make the same argument about deprecating that Lars H. made,
    but I think current HBase metrics really need improvement, so if I had
    voting power I'd vote to drop old stuff and move to metrics2 ASAP.

    Otis
    ----- Original Message -----
    From: Ted Yu <yuzhihong@gmail.com>
    To: dev@hbase.apache.org
    Cc:
    Sent: Saturday, July 7, 2012 10:23 AM
    Subject: deprecating (old) metrics in favor of metrics2 framework

    Hi,
    See the following comment for background:
    https://issues.apache.org/jira/browse/HBASE-4050?focusedCommentId=13408679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13408679
    metrics2 framework is in both hadoop 1.0 and hadoop 2.0
    We would like to migrate to metrics2 framework. But maintaining two metrics
    frameworks in HBase is costly and results in lower readability.

    However, if you have cluster monitoring tool that depends on (old) metrics
    framework, we want to give you smooth migration experience.

    Please provide your opinions on whether (old) metrics framework
    should be
    deprecated in 0.94 or in 0.96.

    Thanks
  • Gary Helmling at Jul 10, 2012 at 8:30 pm

    Whether we support 2 (actually more than 2) metrics frameworks in 0.96 can
    be debated in the next 2 months.
    I'm not sure I agree that deprecating without having something in
    place for users to move to makes sense.
    As Todd mentioned in the thread 'HBase 0.94.1', we will try our best to
    keep JMX interface the same across 0.94 and 0.96. Does this somehow reduce
    the concern you raised ?
    I think that maintaining consistency with the existing JMX naming
    conventions (to the extent possible) is important for operational
    concerns, but it's independent of the MetricsContext question and the
    question of whether other metrics classes of our own need a proper
    deprecation cycle.
    As for using MetricsContext, I assume the user also uses hadoop in his /
    her deployment. Then he / she should be aware of the deprecation of
    metrics.* classes in both hadoop 1.0 and 2.0
    Meaning he / she should be prepared to endorse metrics2 framework.
    Hadoop deprecating metrics in favor of metrics2 is independent of us
    deprecating HBase metrics classes. TimeStampingFileContext is one
    MetricsContext implementation in HBase that would need to be
    deprecated and could be used or possibly extended by current users.

    Ultimately it's up to Lars H as RM for 0.94 to decide what he wants to
    include. It just feels to me like we're rushing to deprecate metrics
    in 0.94 so that it can be removed in 0.96, instead of what seems to me
    like the more standard path of deprecating metrics in 0.96, while also
    including new metrics2 implementations, which would give users a
    smoother path to actually switch over. I'm just not sure I understand
    the motivation for deprecating in 0.94 instead of 0.96.
  • Ted Yu at Jul 10, 2012 at 8:35 pm
    Gary:
    Your comment makes sense.

    Part of this poll originates from the fact that 0.96 is our singularity
    release. RPC, coprocessor, etc have undergone considerable changes.
    Users migrating to 0.96 would have to deal with a lot of updates in their
    codebase.

    It seems to me that doing all upgrades in one shot is almost the same as
    upgrading components other than metrics framework.

    Cheers
    On Tue, Jul 10, 2012 at 1:29 PM, Gary Helmling wrote:


    Whether we support 2 (actually more than 2) metrics frameworks in 0.96 can
    be debated in the next 2 months.
    I'm not sure I agree that deprecating without having something in
    place for users to move to makes sense.
    As Todd mentioned in the thread 'HBase 0.94.1', we will try our best to
    keep JMX interface the same across 0.94 and 0.96. Does this somehow reduce
    the concern you raised ?
    I think that maintaining consistency with the existing JMX naming
    conventions (to the extent possible) is important for operational
    concerns, but it's independent of the MetricsContext question and the
    question of whether other metrics classes of our own need a proper
    deprecation cycle.
    As for using MetricsContext, I assume the user also uses hadoop in his /
    her deployment. Then he / she should be aware of the deprecation of
    metrics.* classes in both hadoop 1.0 and 2.0
    Meaning he / she should be prepared to endorse metrics2 framework.
    Hadoop deprecating metrics in favor of metrics2 is independent of us
    deprecating HBase metrics classes. TimeStampingFileContext is one
    MetricsContext implementation in HBase that would need to be
    deprecated and could be used or possibly extended by current users.

    Ultimately it's up to Lars H as RM for 0.94 to decide what he wants to
    include. It just feels to me like we're rushing to deprecate metrics
    in 0.94 so that it can be removed in 0.96, instead of what seems to me
    like the more standard path of deprecating metrics in 0.96, while also
    including new metrics2 implementations, which would give users a
    smoother path to actually switch over. I'm just not sure I understand
    the motivation for deprecating in 0.94 instead of 0.96.
  • Gary Helmling at Jul 10, 2012 at 8:56 pm
    I agree that having a new metrics2 implementation in 0.96 would be
    great to see and seems like a natural fit. I'm 100% for that. But I
    do think that having metrics2 and (deprecated) metrics v1 in the same
    release would be very helpful to users making the transition. So to
    me it seems more natural for 0.96 to be that release with both
    implementations, since that's where it seems like the metrics2
    implementation will land.

    Otherwise it seems like we risk introducing the same disruptions that
    Hadoop did when metrics2 initially replaced the metrics v1
    implementation, instead of living along side. This did cause us as a
    project some trouble until metrics v1 was added back in. So it would
    be unfortunate to repeat the same mistake ourselves.

    If there's considerable pain or overhead in having both
    implementations live in parallel, maybe it's worth doing a straight
    switch over in 0.96. I haven't looked at the differences enough
    myself to know. But otherwise it seems like an easier migration path
    to deprecate v1 in 0.96 and remove the release after.

    On Tue, Jul 10, 2012 at 1:35 PM, Ted Yu wrote:
    Gary:
    Your comment makes sense.

    Part of this poll originates from the fact that 0.96 is our singularity
    release. RPC, coprocessor, etc have undergone considerable changes.
    Users migrating to 0.96 would have to deal with a lot of updates in their
    codebase.

    It seems to me that doing all upgrades in one shot is almost the same as
    upgrading components other than metrics framework.

    Cheers
    On Tue, Jul 10, 2012 at 1:29 PM, Gary Helmling wrote:


    Whether we support 2 (actually more than 2) metrics frameworks in 0.96 can
    be debated in the next 2 months.
    I'm not sure I agree that deprecating without having something in
    place for users to move to makes sense.
    As Todd mentioned in the thread 'HBase 0.94.1', we will try our best to
    keep JMX interface the same across 0.94 and 0.96. Does this somehow reduce
    the concern you raised ?
    I think that maintaining consistency with the existing JMX naming
    conventions (to the extent possible) is important for operational
    concerns, but it's independent of the MetricsContext question and the
    question of whether other metrics classes of our own need a proper
    deprecation cycle.
    As for using MetricsContext, I assume the user also uses hadoop in his /
    her deployment. Then he / she should be aware of the deprecation of
    metrics.* classes in both hadoop 1.0 and 2.0
    Meaning he / she should be prepared to endorse metrics2 framework.
    Hadoop deprecating metrics in favor of metrics2 is independent of us
    deprecating HBase metrics classes. TimeStampingFileContext is one
    MetricsContext implementation in HBase that would need to be
    deprecated and could be used or possibly extended by current users.

    Ultimately it's up to Lars H as RM for 0.94 to decide what he wants to
    include. It just feels to me like we're rushing to deprecate metrics
    in 0.94 so that it can be removed in 0.96, instead of what seems to me
    like the more standard path of deprecating metrics in 0.96, while also
    including new metrics2 implementations, which would give users a
    smoother path to actually switch over. I'm just not sure I understand
    the motivation for deprecating in 0.94 instead of 0.96.
  • Elliott Clark at Jul 10, 2012 at 9:01 pm
    I think Gary here, and Stack in the other thread have raised enough points
    that if I could vote I would vote in favor of holding onto the classes that
    might be used externally past 0.96.0. Right now there's no alternative to
    several of our Metrics classes and the deprecations weren't out in 0.94.0.
    With all that said I don't see a large down side with keeping those
    classes around while the actual hbase code base starts using something else
    (the metrics2 replacements). I added comment in HBASE-6365 with all of the
    classes that I could find that would be deprecated (They all derive from or
    use the old metrics code). There are a lot

    On Tue, Jul 10, 2012 at 1:55 PM, Gary Helmling wrote:

    I agree that having a new metrics2 implementation in 0.96 would be
    great to see and seems like a natural fit. I'm 100% for that. But I
    do think that having metrics2 and (deprecated) metrics v1 in the same
    release would be very helpful to users making the transition. So to
    me it seems more natural for 0.96 to be that release with both
    implementations, since that's where it seems like the metrics2
    implementation will land.

    Otherwise it seems like we risk introducing the same disruptions that
    Hadoop did when metrics2 initially replaced the metrics v1
    implementation, instead of living along side. This did cause us as a
    project some trouble until metrics v1 was added back in. So it would
    be unfortunate to repeat the same mistake ourselves.

    If there's considerable pain or overhead in having both
    implementations live in parallel, maybe it's worth doing a straight
    switch over in 0.96. I haven't looked at the differences enough
    myself to know. But otherwise it seems like an easier migration path
    to deprecate v1 in 0.96 and remove the release after.

    On Tue, Jul 10, 2012 at 1:35 PM, Ted Yu wrote:
    Gary:
    Your comment makes sense.

    Part of this poll originates from the fact that 0.96 is our singularity
    release. RPC, coprocessor, etc have undergone considerable changes.
    Users migrating to 0.96 would have to deal with a lot of updates in their
    codebase.

    It seems to me that doing all upgrades in one shot is almost the same as
    upgrading components other than metrics framework.

    Cheers
    On Tue, Jul 10, 2012 at 1:29 PM, Gary Helmling wrote:


    Whether we support 2 (actually more than 2) metrics frameworks in 0.96 can
    be debated in the next 2 months.
    I'm not sure I agree that deprecating without having something in
    place for users to move to makes sense.
    As Todd mentioned in the thread 'HBase 0.94.1', we will try our best
    to
    keep JMX interface the same across 0.94 and 0.96. Does this somehow reduce
    the concern you raised ?
    I think that maintaining consistency with the existing JMX naming
    conventions (to the extent possible) is important for operational
    concerns, but it's independent of the MetricsContext question and the
    question of whether other metrics classes of our own need a proper
    deprecation cycle.
    As for using MetricsContext, I assume the user also uses hadoop in
    his /
    her deployment. Then he / she should be aware of the deprecation of
    metrics.* classes in both hadoop 1.0 and 2.0
    Meaning he / she should be prepared to endorse metrics2 framework.
    Hadoop deprecating metrics in favor of metrics2 is independent of us
    deprecating HBase metrics classes. TimeStampingFileContext is one
    MetricsContext implementation in HBase that would need to be
    deprecated and could be used or possibly extended by current users.

    Ultimately it's up to Lars H as RM for 0.94 to decide what he wants to
    include. It just feels to me like we're rushing to deprecate metrics
    in 0.94 so that it can be removed in 0.96, instead of what seems to me
    like the more standard path of deprecating metrics in 0.96, while also
    including new metrics2 implementations, which would give users a
    smoother path to actually switch over. I'm just not sure I understand
    the motivation for deprecating in 0.94 instead of 0.96.
  • Ted Yu at Jul 10, 2012 at 9:04 pm
    Points taken.

    Thanks for the education of metrics framework history.
    On Tue, Jul 10, 2012 at 1:55 PM, Gary Helmling wrote:

    I agree that having a new metrics2 implementation in 0.96 would be
    great to see and seems like a natural fit. I'm 100% for that. But I
    do think that having metrics2 and (deprecated) metrics v1 in the same
    release would be very helpful to users making the transition. So to
    me it seems more natural for 0.96 to be that release with both
    implementations, since that's where it seems like the metrics2
    implementation will land.

    Otherwise it seems like we risk introducing the same disruptions that
    Hadoop did when metrics2 initially replaced the metrics v1
    implementation, instead of living along side. This did cause us as a
    project some trouble until metrics v1 was added back in. So it would
    be unfortunate to repeat the same mistake ourselves.

    If there's considerable pain or overhead in having both
    implementations live in parallel, maybe it's worth doing a straight
    switch over in 0.96. I haven't looked at the differences enough
    myself to know. But otherwise it seems like an easier migration path
    to deprecate v1 in 0.96 and remove the release after.

    On Tue, Jul 10, 2012 at 1:35 PM, Ted Yu wrote:
    Gary:
    Your comment makes sense.

    Part of this poll originates from the fact that 0.96 is our singularity
    release. RPC, coprocessor, etc have undergone considerable changes.
    Users migrating to 0.96 would have to deal with a lot of updates in their
    codebase.

    It seems to me that doing all upgrades in one shot is almost the same as
    upgrading components other than metrics framework.

    Cheers
    On Tue, Jul 10, 2012 at 1:29 PM, Gary Helmling wrote:


    Whether we support 2 (actually more than 2) metrics frameworks in 0.96 can
    be debated in the next 2 months.
    I'm not sure I agree that deprecating without having something in
    place for users to move to makes sense.
    As Todd mentioned in the thread 'HBase 0.94.1', we will try our best
    to
    keep JMX interface the same across 0.94 and 0.96. Does this somehow reduce
    the concern you raised ?
    I think that maintaining consistency with the existing JMX naming
    conventions (to the extent possible) is important for operational
    concerns, but it's independent of the MetricsContext question and the
    question of whether other metrics classes of our own need a proper
    deprecation cycle.
    As for using MetricsContext, I assume the user also uses hadoop in
    his /
    her deployment. Then he / she should be aware of the deprecation of
    metrics.* classes in both hadoop 1.0 and 2.0
    Meaning he / she should be prepared to endorse metrics2 framework.
    Hadoop deprecating metrics in favor of metrics2 is independent of us
    deprecating HBase metrics classes. TimeStampingFileContext is one
    MetricsContext implementation in HBase that would need to be
    deprecated and could be used or possibly extended by current users.

    Ultimately it's up to Lars H as RM for 0.94 to decide what he wants to
    include. It just feels to me like we're rushing to deprecate metrics
    in 0.94 so that it can be removed in 0.96, instead of what seems to me
    like the more standard path of deprecating metrics in 0.96, while also
    including new metrics2 implementations, which would give users a
    smoother path to actually switch over. I'm just not sure I understand
    the motivation for deprecating in 0.94 instead of 0.96.
  • Alex Baranau at Jul 10, 2012 at 10:25 pm

    If there's considerable pain or overhead in having both
    implementations live in parallel, maybe it's worth doing a straight
    switch over in 0.96.
    As far as I can tell, they can live together easily. This should not be a
    big issue. E.g. it should be much smaller issue than the fact that code
    implemented on top of metrics2 will not compile against 1.0+, 2.0+ and 3.0+
    hadoop at the same time because class names, etc. changed between 1.0 and
    2.0. But that's a separate story, will look at it tomorrow (Ted pointed me
    to smth to look at).

    Alex Baranau
    On Tue, Jul 10, 2012 at 5:03 PM, Ted Yu wrote:

    Points taken.

    Thanks for the education of metrics framework history.
    On Tue, Jul 10, 2012 at 1:55 PM, Gary Helmling wrote:

    I agree that having a new metrics2 implementation in 0.96 would be
    great to see and seems like a natural fit. I'm 100% for that. But I
    do think that having metrics2 and (deprecated) metrics v1 in the same
    release would be very helpful to users making the transition. So to
    me it seems more natural for 0.96 to be that release with both
    implementations, since that's where it seems like the metrics2
    implementation will land.

    Otherwise it seems like we risk introducing the same disruptions that
    Hadoop did when metrics2 initially replaced the metrics v1
    implementation, instead of living along side. This did cause us as a
    project some trouble until metrics v1 was added back in. So it would
    be unfortunate to repeat the same mistake ourselves.

    If there's considerable pain or overhead in having both
    implementations live in parallel, maybe it's worth doing a straight
    switch over in 0.96. I haven't looked at the differences enough
    myself to know. But otherwise it seems like an easier migration path
    to deprecate v1 in 0.96 and remove the release after.

    On Tue, Jul 10, 2012 at 1:35 PM, Ted Yu wrote:
    Gary:
    Your comment makes sense.

    Part of this poll originates from the fact that 0.96 is our singularity
    release. RPC, coprocessor, etc have undergone considerable changes.
    Users migrating to 0.96 would have to deal with a lot of updates in
    their
    codebase.

    It seems to me that doing all upgrades in one shot is almost the same
    as
    upgrading components other than metrics framework.

    Cheers

    On Tue, Jul 10, 2012 at 1:29 PM, Gary Helmling <ghelmling@gmail.com>
    wrote:
    Whether we support 2 (actually more than 2) metrics frameworks in
    0.96
    can
    be debated in the next 2 months.
    I'm not sure I agree that deprecating without having something in
    place for users to move to makes sense.
    As Todd mentioned in the thread 'HBase 0.94.1', we will try our best
    to
    keep JMX interface the same across 0.94 and 0.96. Does this somehow reduce
    the concern you raised ?
    I think that maintaining consistency with the existing JMX naming
    conventions (to the extent possible) is important for operational
    concerns, but it's independent of the MetricsContext question and the
    question of whether other metrics classes of our own need a proper
    deprecation cycle.
    As for using MetricsContext, I assume the user also uses hadoop in
    his /
    her deployment. Then he / she should be aware of the deprecation of
    metrics.* classes in both hadoop 1.0 and 2.0
    Meaning he / she should be prepared to endorse metrics2 framework.
    Hadoop deprecating metrics in favor of metrics2 is independent of us
    deprecating HBase metrics classes. TimeStampingFileContext is one
    MetricsContext implementation in HBase that would need to be
    deprecated and could be used or possibly extended by current users.

    Ultimately it's up to Lars H as RM for 0.94 to decide what he wants to
    include. It just feels to me like we're rushing to deprecate metrics
    in 0.94 so that it can be removed in 0.96, instead of what seems to me
    like the more standard path of deprecating metrics in 0.96, while also
    including new metrics2 implementations, which would give users a
    smoother path to actually switch over. I'm just not sure I understand
    the motivation for deprecating in 0.94 instead of 0.96.
  • Elliott Clark at Jul 10, 2012 at 10:33 pm
    https://issues.apache.org/jira/browse/HADOOP-7734

    As far as I can tell this basically sinks all Metrics2 usage in HBase.
    On Tue, Jul 10, 2012 at 3:24 PM, Alex Baranau wrote:

    If there's considerable pain or overhead in having both
    implementations live in parallel, maybe it's worth doing a straight
    switch over in 0.96.
    As far as I can tell, they can live together easily. This should not be a
    big issue. E.g. it should be much smaller issue than the fact that code
    implemented on top of metrics2 will not compile against 1.0+, 2.0+ and 3.0+
    hadoop at the same time because class names, etc. changed between 1.0 and
    2.0. But that's a separate story, will look at it tomorrow (Ted pointed me
    to smth to look at).

    Alex Baranau
    On Tue, Jul 10, 2012 at 5:03 PM, Ted Yu wrote:

    Points taken.

    Thanks for the education of metrics framework history.

    On Tue, Jul 10, 2012 at 1:55 PM, Gary Helmling <ghelmling@gmail.com>
    wrote:
    I agree that having a new metrics2 implementation in 0.96 would be
    great to see and seems like a natural fit. I'm 100% for that. But I
    do think that having metrics2 and (deprecated) metrics v1 in the same
    release would be very helpful to users making the transition. So to
    me it seems more natural for 0.96 to be that release with both
    implementations, since that's where it seems like the metrics2
    implementation will land.

    Otherwise it seems like we risk introducing the same disruptions that
    Hadoop did when metrics2 initially replaced the metrics v1
    implementation, instead of living along side. This did cause us as a
    project some trouble until metrics v1 was added back in. So it would
    be unfortunate to repeat the same mistake ourselves.

    If there's considerable pain or overhead in having both
    implementations live in parallel, maybe it's worth doing a straight
    switch over in 0.96. I haven't looked at the differences enough
    myself to know. But otherwise it seems like an easier migration path
    to deprecate v1 in 0.96 and remove the release after.

    On Tue, Jul 10, 2012 at 1:35 PM, Ted Yu wrote:
    Gary:
    Your comment makes sense.

    Part of this poll originates from the fact that 0.96 is our
    singularity
    release. RPC, coprocessor, etc have undergone considerable changes.
    Users migrating to 0.96 would have to deal with a lot of updates in
    their
    codebase.

    It seems to me that doing all upgrades in one shot is almost the same
    as
    upgrading components other than metrics framework.

    Cheers

    On Tue, Jul 10, 2012 at 1:29 PM, Gary Helmling <ghelmling@gmail.com>
    wrote:
    Whether we support 2 (actually more than 2) metrics frameworks in
    0.96
    can
    be debated in the next 2 months.
    I'm not sure I agree that deprecating without having something in
    place for users to move to makes sense.
    As Todd mentioned in the thread 'HBase 0.94.1', we will try our
    best
    to
    keep JMX interface the same across 0.94 and 0.96. Does this
    somehow
    reduce
    the concern you raised ?
    I think that maintaining consistency with the existing JMX naming
    conventions (to the extent possible) is important for operational
    concerns, but it's independent of the MetricsContext question and
    the
    question of whether other metrics classes of our own need a proper
    deprecation cycle.
    As for using MetricsContext, I assume the user also uses hadoop in
    his /
    her deployment. Then he / she should be aware of the deprecation
    of
    metrics.* classes in both hadoop 1.0 and 2.0
    Meaning he / she should be prepared to endorse metrics2 framework.
    Hadoop deprecating metrics in favor of metrics2 is independent of us
    deprecating HBase metrics classes. TimeStampingFileContext is one
    MetricsContext implementation in HBase that would need to be
    deprecated and could be used or possibly extended by current users.

    Ultimately it's up to Lars H as RM for 0.94 to decide what he wants
    to
    include. It just feels to me like we're rushing to deprecate
    metrics
    in 0.94 so that it can be removed in 0.96, instead of what seems to
    me
    like the more standard path of deprecating metrics in 0.96, while
    also
    including new metrics2 implementations, which would give users a
    smoother path to actually switch over. I'm just not sure I
    understand
    the motivation for deprecating in 0.94 instead of 0.96.
  • Ted Yu at Jul 10, 2012 at 11:14 pm
    Thanks for sharing this, Elliot.

    7734 is marked blocked by HADOOP-7742: Evolve metrics2 in 0.23 and trunk to
    allow coexistence with 0.20-security releases
    On Tue, Jul 10, 2012 at 3:33 PM, Elliott Clark wrote:

    https://issues.apache.org/jira/browse/HADOOP-7734

    As far as I can tell this basically sinks all Metrics2 usage in HBase.

    On Tue, Jul 10, 2012 at 3:24 PM, Alex Baranau <alex.baranov.v@gmail.com
    wrote:
    If there's considerable pain or overhead in having both
    implementations live in parallel, maybe it's worth doing a straight
    switch over in 0.96.
    As far as I can tell, they can live together easily. This should not be a
    big issue. E.g. it should be much smaller issue than the fact that code
    implemented on top of metrics2 will not compile against 1.0+, 2.0+ and 3.0+
    hadoop at the same time because class names, etc. changed between 1.0 and
    2.0. But that's a separate story, will look at it tomorrow (Ted pointed me
    to smth to look at).

    Alex Baranau
    On Tue, Jul 10, 2012 at 5:03 PM, Ted Yu wrote:

    Points taken.

    Thanks for the education of metrics framework history.

    On Tue, Jul 10, 2012 at 1:55 PM, Gary Helmling <ghelmling@gmail.com>
    wrote:
    I agree that having a new metrics2 implementation in 0.96 would be
    great to see and seems like a natural fit. I'm 100% for that. But I
    do think that having metrics2 and (deprecated) metrics v1 in the same
    release would be very helpful to users making the transition. So to
    me it seems more natural for 0.96 to be that release with both
    implementations, since that's where it seems like the metrics2
    implementation will land.

    Otherwise it seems like we risk introducing the same disruptions that
    Hadoop did when metrics2 initially replaced the metrics v1
    implementation, instead of living along side. This did cause us as a
    project some trouble until metrics v1 was added back in. So it would
    be unfortunate to repeat the same mistake ourselves.

    If there's considerable pain or overhead in having both
    implementations live in parallel, maybe it's worth doing a straight
    switch over in 0.96. I haven't looked at the differences enough
    myself to know. But otherwise it seems like an easier migration path
    to deprecate v1 in 0.96 and remove the release after.

    On Tue, Jul 10, 2012 at 1:35 PM, Ted Yu wrote:
    Gary:
    Your comment makes sense.

    Part of this poll originates from the fact that 0.96 is our
    singularity
    release. RPC, coprocessor, etc have undergone considerable changes.
    Users migrating to 0.96 would have to deal with a lot of updates in
    their
    codebase.

    It seems to me that doing all upgrades in one shot is almost the
    same
    as
    upgrading components other than metrics framework.

    Cheers

    On Tue, Jul 10, 2012 at 1:29 PM, Gary Helmling <
    ghelmling@gmail.com>
    wrote:
    Whether we support 2 (actually more than 2) metrics frameworks
    in
    0.96
    can
    be debated in the next 2 months.
    I'm not sure I agree that deprecating without having something in
    place for users to move to makes sense.
    As Todd mentioned in the thread 'HBase 0.94.1', we will try our
    best
    to
    keep JMX interface the same across 0.94 and 0.96. Does this
    somehow
    reduce
    the concern you raised ?
    I think that maintaining consistency with the existing JMX naming
    conventions (to the extent possible) is important for operational
    concerns, but it's independent of the MetricsContext question and
    the
    question of whether other metrics classes of our own need a proper
    deprecation cycle.
    As for using MetricsContext, I assume the user also uses hadoop
    in
    his /
    her deployment. Then he / she should be aware of the deprecation
    of
    metrics.* classes in both hadoop 1.0 and 2.0
    Meaning he / she should be prepared to endorse metrics2
    framework.
    Hadoop deprecating metrics in favor of metrics2 is independent of
    us
    deprecating HBase metrics classes. TimeStampingFileContext is one
    MetricsContext implementation in HBase that would need to be
    deprecated and could be used or possibly extended by current
    users.
    Ultimately it's up to Lars H as RM for 0.94 to decide what he
    wants
    to
    include. It just feels to me like we're rushing to deprecate
    metrics
    in 0.94 so that it can be removed in 0.96, instead of what seems
    to
    me
    like the more standard path of deprecating metrics in 0.96, while
    also
    including new metrics2 implementations, which would give users a
    smoother path to actually switch over. I'm just not sure I
    understand
    the motivation for deprecating in 0.94 instead of 0.96.
  • Andrew Purtell at Jul 10, 2012 at 11:54 pm

    On Tue, Jul 10, 2012 at 3:33 PM, Elliott Clark wrote:
    https://issues.apache.org/jira/browse/HADOOP-7734

    As far as I can tell this basically sinks all Metrics2 usage in HBase.
    So can we settle this then as something to do 0.96 and beyond and get
    the RC out?

    Best regards,

    - Andy

    Problems worthy of attack prove their worth by hitting back. - Piet
    Hein (via Tom White)
  • Elliott Clark at Jul 10, 2012 at 11:59 pm
    I would think so yes. Talked with Todd on irc and he suggested a good
    solution, shim jars that are loaded based on what version of hadoop is on
    the class path. However to get that working is a lot more work than just
    rewriting some classes to use the metrics2 namespace. As such it seems way
    too early to think about removing the working code.
    On Tue, Jul 10, 2012 at 4:54 PM, Andrew Purtell wrote:
    On Tue, Jul 10, 2012 at 3:33 PM, Elliott Clark wrote:
    https://issues.apache.org/jira/browse/HADOOP-7734

    As far as I can tell this basically sinks all Metrics2 usage in HBase.
    So can we settle this then as something to do 0.96 and beyond and get
    the RC out?

    Best regards,

    - Andy

    Problems worthy of attack prove their worth by hitting back. - Piet
    Hein (via Tom White)
  • Alex Baranau at Jul 11, 2012 at 4:29 pm
    Hi, I don't know the details of what Todd suggested, could you please take
    a look at last two comments at
    https://issues.apache.org/jira/browse/HBASE-4050, to see if I understand
    correctly the options here.

    Thank you in advance,
    Alex Baranau
    ------
    Sematext :: http://blog.sematext.com/ : Hadoop - HBase - ElasticSearch -
    Solr - Lucene
    On Tue, Jul 10, 2012 at 7:58 PM, Elliott Clark wrote:

    I would think so yes. Talked with Todd on irc and he suggested a good
    solution, shim jars that are loaded based on what version of hadoop is on
    the class path. However to get that working is a lot more work than just
    rewriting some classes to use the metrics2 namespace. As such it seems way
    too early to think about removing the working code.
    On Tue, Jul 10, 2012 at 4:54 PM, Andrew Purtell wrote:

    On Tue, Jul 10, 2012 at 3:33 PM, Elliott Clark <eclark@stumbleupon.com>
    wrote:
    https://issues.apache.org/jira/browse/HADOOP-7734

    As far as I can tell this basically sinks all Metrics2 usage in HBase.
    So can we settle this then as something to do 0.96 and beyond and get
    the RC out?

    Best regards,

    - Andy

    Problems worthy of attack prove their worth by hitting back. - Piet
    Hein (via Tom White)


    --
    Alex Baranau
    ------
    Sematext :: http://blog.sematext.com/ :: Hadoop - HBase - ElasticSearch -
    Solr - Lucene
  • Elliott Clark at Jul 11, 2012 at 5:51 pm
    He and I had discussed something close to #2. (This is my understanding
    and is partially based on Todd's irc thoughts so feel free to correct me if
    I get anything wrong)

    The reflection option seems like it would be a big perf hit. Reflection
    basically means that nothing ever gets inlined so all function calls into
    metrics2 code would be very expensive. Since it seems like we are adding
    more and more instrumentation this perf impact would only grow.
    In addition as more hadoop versions come out all of our reflection code
    would get much more complicated and brittle.

    The conditionally loaded jar would be nice in that the JIT would only see
    one version of the factory classes on the classpath and everything could be
    optimized just like any other jvm run jit'd code. In addition there are
    other places that we use reflection to do things conditionally and a
    conditionally loaded jar would be nice.

    (putting this on jira as well.)
    On Wed, Jul 11, 2012 at 9:29 AM, Alex Baranau wrote:

    Hi, I don't know the details of what Todd suggested, could you please take
    a look at last two comments at
    https://issues.apache.org/jira/browse/HBASE-4050, to see if I understand
    correctly the options here.

    Thank you in advance,
    Alex Baranau
    ------
    Sematext :: http://blog.sematext.com/ : Hadoop - HBase - ElasticSearch -
    Solr - Lucene

    On Tue, Jul 10, 2012 at 7:58 PM, Elliott Clark <eclark@stumbleupon.com
    wrote:
    I would think so yes. Talked with Todd on irc and he suggested a good
    solution, shim jars that are loaded based on what version of hadoop is on
    the class path. However to get that working is a lot more work than just
    rewriting some classes to use the metrics2 namespace. As such it seems way
    too early to think about removing the working code.

    On Tue, Jul 10, 2012 at 4:54 PM, Andrew Purtell <apurtell@apache.org>
    wrote:
    On Tue, Jul 10, 2012 at 3:33 PM, Elliott Clark <eclark@stumbleupon.com
    wrote:
    https://issues.apache.org/jira/browse/HADOOP-7734

    As far as I can tell this basically sinks all Metrics2 usage in
    HBase.
    So can we settle this then as something to do 0.96 and beyond and get
    the RC out?

    Best regards,

    - Andy

    Problems worthy of attack prove their worth by hitting back. - Piet
    Hein (via Tom White)


    --
    Alex Baranau
    ------
    Sematext :: http://blog.sematext.com/ :: Hadoop - HBase - ElasticSearch -
    Solr - Lucene
  • Andrew Purtell at Jul 11, 2012 at 6:57 pm
    On Wed, Jul 11, 2012 at 10:50 AM, Elliott Clark wrote:
    [...]
    The reflection option seems like it would be a big perf hit. Reflection
    basically means that nothing ever gets inlined so all function calls into
    metrics2 code would be very expensive. Since it seems like we are adding
    more and more instrumentation this perf impact would only grow.
    In addition as more hadoop versions come out all of our reflection code
    would get much more complicated and brittle.

    The conditionally loaded jar would be nice in that the JIT would only see
    one version of the factory classes on the classpath and everything could be
    optimized just like any other jvm run jit'd code. In addition there are
    other places that we use reflection to do things conditionally and a
    conditionally loaded jar would be nice.
    Coprocessors are a solution close to conditionally loaded JARs: there
    is not reflection, very frequently executed code would be JITed, the
    only thing missing is direct inlining.

    More and more (useful) metrics are going into hot paths, we want all
    three I think.

    We keep inching toward DI.

    Best regards,

    - Andy

    Problems worthy of attack prove their worth by hitting back. - Piet
    Hein (via Tom White)
  • Elliott Clark at Jul 12, 2012 at 2:07 am
    Yep. This is pretty darn close to DI. I would be willing to change the
    patch that I just put up to use Guice if that's what people wanted. I know
    some of our classes could use some DI loving to help with testability.
    On Wed, Jul 11, 2012 at 11:57 AM, Andrew Purtell wrote:

    On Wed, Jul 11, 2012 at 10:50 AM, Elliott Clark wrote:
    [...]
    The reflection option seems like it would be a big perf hit. Reflection
    basically means that nothing ever gets inlined so all function calls into
    metrics2 code would be very expensive. Since it seems like we are adding
    more and more instrumentation this perf impact would only grow.
    In addition as more hadoop versions come out all of our reflection code
    would get much more complicated and brittle.

    The conditionally loaded jar would be nice in that the JIT would only see
    one version of the factory classes on the classpath and everything could be
    optimized just like any other jvm run jit'd code. In addition there are
    other places that we use reflection to do things conditionally and a
    conditionally loaded jar would be nice.
    Coprocessors are a solution close to conditionally loaded JARs: there
    is not reflection, very frequently executed code would be JITed, the
    only thing missing is direct inlining.

    More and more (useful) metrics are going into hot paths, we want all
    three I think.

    We keep inching toward DI.

    Best regards,

    - Andy

    Problems worthy of attack prove their worth by hitting back. - Piet
    Hein (via Tom White)
  • Michael Stack at Jul 12, 2012 at 8:24 am

    On Thu, Jul 12, 2012 at 4:07 AM, Elliott Clark wrote:
    Yep. This is pretty darn close to DI. I would be willing to change the
    patch that I just put up to use Guice if that's what people wanted. I know
    some of our classes could use some DI loving to help with testability.
    So, you think we could Guice the metrics and then over time, Guice
    everything else Elliott?
    St.Ack
  • Elliott Clark at Jul 13, 2012 at 12:12 am
    From what I know about using guice it's best to only have on injector per
    application. So It's not really possible to only implement metrics with
    guice. There would need to be a medium amount of of work to pipe it all
    the way from HMasterCommandLine/HRegionserverCommandLine alll the way to
    metrics. Probably something we should look into after the current patch
    goes in so that we don't couple the two discussions too tightly.
    On Thu, Jul 12, 2012 at 1:23 AM, Stack wrote:
    On Thu, Jul 12, 2012 at 4:07 AM, Elliott Clark wrote:
    Yep. This is pretty darn close to DI. I would be willing to change the
    patch that I just put up to use Guice if that's what people wanted. I know
    some of our classes could use some DI loving to help with testability.
    So, you think we could Guice the metrics and then over time, Guice
    everything else Elliott?
    St.Ack

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categorieshbase, hadoop
postedJul 7, '12 at 2:24p
activeJul 13, '12 at 12:12a
posts28
users9
websitehbase.apache.org

People

Translate

site design / logo © 2022 Grokbase