Grokbase Groups HBase dev June 2010
FAQ
Hey all,

Quick update on the development branch 0.89.20100621.

I think the following patches still need to go in:
- HBASE-2767 (failed tests building with HDFS-1209)
- HBASE-2729 (fix bug when flush hits IOE)
- I'd also like to disable the TestAcidGuarantees test in this branch, since
that is a known bug.
- I'd like to commit a short KNOWN_BUGS file which describes a few of the
open issues that we're currently working on. Certainly doesn't have to be
exhaustive list of all open JIRAs, but just a few things that users may run
into.

Aside from that, my testing indicates the branch should be ready to release.

Some performance issues were raised during testing at StumbleUpon -- any
luck figuring those out, Ryan/Stack? It would be good to address them for
the dev release, since it sounds like the RS barely makes progress when this
bug is triggered.

Given the above, I'd like to see if we can get the above two jiras reviewed
and committed later tonight, and I'll try to roll a release candidate before
I go to sleep. I think the correct way to release in this maven land is to
simply do an svn export and vote on that, and then separately do an
assembly:assembly tar as a binary release artifact (since assembly tar
doesn't include unpacked source, etc).

[
Obligatory reminder that the purpose of this dev release is to get something
usable into users' hands, and not to claim that it's a release quality
artifact - hence I'm hoping we can vote tomorrow/Thurs and release by Friday
at the latest. If there are big problems, we'll just document them as known
bugs and do another release next week, but I think showing release momentum
coming into the Hadoop Summit will do great things for the project.
]

-Todd
--
Todd Lipcon
Software Engineer, Cloudera

Search Discussions

  • Stack at Jun 23, 2010 at 4:05 am

    On Tue, Jun 22, 2010 at 6:35 PM, Todd Lipcon wrote:
    Quick update on the development branch 0.89.20100621.
    (Todd you want to explain the version number or do you want me to?)

    I think the following patches still need to go in:
    - HBASE-2767 (failed tests building with HDFS-1209) Reviewed.
    - HBASE-2729 (fix bug when flush hits IOE)
    Please paste patch into issue so can review (review.hbase.org is down)

    - I'd also like to disable the TestAcidGuarantees test in this branch, since
    that is a known bug.

    Grand.
    - I'd like to commit a short KNOWN_BUGS file which describes a few of the
    open issues that we're currently working on. Certainly doesn't have to be
    exhaustive list of all open JIRAs, but just a few things that users may run
    into.
    Yeah. Just call out the biggies and refer use to the short list
    (ahem) of other issues we have against next major release.

    Some performance issues were raised during testing at StumbleUpon -- any
    luck figuring those out, Ryan/Stack? It would be good to address them for
    the dev release, since it sounds like the RS barely makes progress when this
    bug is triggered.

    Yeah, its a beaut. Sucks all resources for some period of time until
    it gets over first flush then its good to go for a while at least.
    Still trying to figure it.

    Given the above, I'd like to see if we can get the above two jiras reviewed
    and committed later tonight, and I'll try to roll a release candidate before
    I go to sleep. I think the correct way to release in this maven land is to
    simply do an svn export and vote on that, and then separately do an
    assembly:assembly tar as a binary release artifact (since assembly tar
    doesn't include unpacked source, etc).
    Why not just vote on the assembly? Thats what we'd run? If you do a
    site before assembly:assembly, you'll even have docs (mvn install site
    assembly;assembly). The source is there to review if wanted. Doing
    an svn export will have to build ourselves.

    St.Ack
  • Todd Lipcon at Jun 23, 2010 at 5:21 am

    On Tue, Jun 22, 2010 at 9:05 PM, Stack wrote:
    On Tue, Jun 22, 2010 at 6:35 PM, Todd Lipcon wrote:

    Quick update on the development branch 0.89.20100621.
    (Todd you want to explain the version number or do you want me to?)
    Sure, here's an explanation, geared towards a general user audience. If
    people think this looks good, I'll post it on the wiki (and maybe a blog
    post next week):

    "The last few releases of HBase have been in lockstep with Hadoop releases
    (eg hbase 0.X.* would work with Hadoop 0.X.*). However, Hadoop's release
    process has slowed down, and we have the desire to release HBase on a
    different timeline from Hadoop, with each HBase release potentially being
    compatible with multiple versions of Hadoop. To signify this departure from
    lockstep releases, and since we intend for the next release of HBase to
    continue to support Hadoop 0.20, we don't want to call the next release
    0.21.

    There's also a general sentiment that, feature-wise, HBase is nearing a 1.0
    level. The project has implemented the majority of the features described in
    the Bigtable paper plus many more. It's also starting to take a more central
    role in the production infrastructures of many companies. So, we'd really
    like to do a 1.0 release some time in the coming year (no dates, but maybe
    early 2011?) However, we're not at 1.0 yet. There are still stability bugs
    to iron out, and some features in progress that we'd like to have done for
    1.0.

    So, given that, we decided on the dev list that tne next stable release
    would be called HBase 0.90. This indicates (a) a big step up from 0.20, (b)
    we are nearing 1.0, and (c) we are not tied to Hadoop version numbering.

    We're not ready to release 0.90 yet - there are a lot of blockers still open
    affecting reliability and stability, and we haven't done extensive testing
    of trunk under production workloads. But, the development community feels
    there's a lot of stuff in trunk that's worth showing to the user community.
    To that end, we are releasing a series of development builds cut from trunk
    leading up to the 0.90 release. This development series will have version
    numbers 0.89.YYYYMMDD, the last segment of the version number indicating the
    date on which the release was branched from trunk. These releases won't have
    followup patch releases, and will only go through basic cluster testing, but
    provide usable snapshots of trunk development so that the community can
    begin to work with the new code and provide early feedback based on their
    use cases and testing. We're confident this will help make 0.90 the most
    stable release yet.

    The first release in the 0.89 series will be 0.89.20100621, due to be
    released the week of 6/21/2010.
    "

    Is that wording clear and walk the right line between "you should help test
    this release" and "this release may not work great"?

    I think the following patches still need to go in:
    - HBASE-2767 (failed tests building with HDFS-1209)
    Reviewed.
    Thanks, will commit momentarily.
    - HBASE-2729 (fix bug when flush hits IOE)
    Please paste patch into issue so can review (review.hbase.org is down)
    Patch posted... sorry about review.hbase.org. I was being cheap and using
    spot instances, but apparently spot instances are the first thing to get
    shut down when ec2 has capacity issues. I'll upgrade it to a real instance
    and send the bill to the Cloudera bosses :)

    - I'd also like to disable the TestAcidGuarantees test in this branch, since
    that is a known bug.

    Grand.
    - I'd like to commit a short KNOWN_BUGS file which describes a few of the
    open issues that we're currently working on. Certainly doesn't have to be
    exhaustive list of all open JIRAs, but just a few things that users may run
    into.
    Yeah. Just call out the biggies and refer use to the short list
    (ahem) of other issues we have against next major release.

    Some performance issues were raised during testing at StumbleUpon -- any
    luck figuring those out, Ryan/Stack? It would be good to address them for
    the dev release, since it sounds like the RS barely makes progress when this
    bug is triggered.

    Yeah, its a beaut. Sucks all resources for some period of time until
    it gets over first flush then its good to go for a while at least.
    Still trying to figure it.
    OK. I'll continue to do cluster testing and see if I can reproduce this one.
    I'd love to fix it before release, but if we have to release with the bug I
    think it's worth doing rather than delaying. Agreed? We'll call it out in
    known bugs. Alternatively if it would fix it, we could patch out the CAS
    spin loop in completeMemstoreInsert and mark a known bug that
    read-your-writes consistency is lost under rare circumstances. Whichever you
    think is better.

    Given the above, I'd like to see if we can get the above two jiras reviewed
    and committed later tonight, and I'll try to roll a release candidate before
    I go to sleep. I think the correct way to release in this maven land is to
    simply do an svn export and vote on that, and then separately do an
    assembly:assembly tar as a binary release artifact (since assembly tar
    doesn't include unpacked source, etc).
    Why not just vote on the assembly? Thats what we'd run? If you do a
    site before assembly:assembly, you'll even have docs (mvn install site
    assembly;assembly). The source is there to review if wanted. Doing
    an svn export will have to build ourselves.
    The avro project had this discussion a bit recently, and what basically came
    out of it is that Apache releases are source, not binary. It happens that in
    the past our releases have had both source and binary in one tar, but it's
    important that the actual *release* artifact contain the source. This allows
    people to rebuild from the signed release tarball if they like, etc. Since
    the mvn assembly tar isn't re-buildable, I think the release artifact has to
    be a tarred svn export, and then we have a second release artifact which is
    binary+site+docs like you say above. People can choose whether to download
    the source release or the binary.

    At least the above was my understanding - I'll ping Doug with this thread to
    see if he can clarify/confirm.

    Thanks
    -Todd


    --
    Todd Lipcon
    Software Engineer, Cloudera
  • Jonathan Gray at Jun 23, 2010 at 5:46 am
    +1

    Works for me
    -----Original Message-----
    From: Todd Lipcon
    Sent: Tuesday, June 22, 2010 10:21 PM
    To: dev@hbase.apache.org
    Subject: Re: 0.89.20100621 development branch status
    On Tue, Jun 22, 2010 at 9:05 PM, Stack wrote:
    On Tue, Jun 22, 2010 at 6:35 PM, Todd Lipcon wrote:

    Quick update on the development branch 0.89.20100621.
    (Todd you want to explain the version number or do you want me to?)
    Sure, here's an explanation, geared towards a general user audience. If
    people think this looks good, I'll post it on the wiki (and maybe a
    blog
    post next week):

    "The last few releases of HBase have been in lockstep with Hadoop
    releases
    (eg hbase 0.X.* would work with Hadoop 0.X.*). However, Hadoop's
    release
    process has slowed down, and we have the desire to release HBase on a
    different timeline from Hadoop, with each HBase release potentially
    being
    compatible with multiple versions of Hadoop. To signify this departure
    from
    lockstep releases, and since we intend for the next release of HBase to
    continue to support Hadoop 0.20, we don't want to call the next release
    0.21.

    There's also a general sentiment that, feature-wise, HBase is nearing a
    1.0
    level. The project has implemented the majority of the features
    described in
    the Bigtable paper plus many more. It's also starting to take a more
    central
    role in the production infrastructures of many companies. So, we'd
    really
    like to do a 1.0 release some time in the coming year (no dates, but
    maybe
    early 2011?) However, we're not at 1.0 yet. There are still stability
    bugs
    to iron out, and some features in progress that we'd like to have done
    for
    1.0.

    So, given that, we decided on the dev list that tne next stable release
    would be called HBase 0.90. This indicates (a) a big step up from 0.20,
    (b)
    we are nearing 1.0, and (c) we are not tied to Hadoop version
    numbering.

    We're not ready to release 0.90 yet - there are a lot of blockers still
    open
    affecting reliability and stability, and we haven't done extensive
    testing
    of trunk under production workloads. But, the development community
    feels
    there's a lot of stuff in trunk that's worth showing to the user
    community.
    To that end, we are releasing a series of development builds cut from
    trunk
    leading up to the 0.90 release. This development series will have
    version
    numbers 0.89.YYYYMMDD, the last segment of the version number
    indicating the
    date on which the release was branched from trunk. These releases won't
    have
    followup patch releases, and will only go through basic cluster
    testing, but
    provide usable snapshots of trunk development so that the community can
    begin to work with the new code and provide early feedback based on
    their
    use cases and testing. We're confident this will help make 0.90 the
    most
    stable release yet.

    The first release in the 0.89 series will be 0.89.20100621, due to be
    released the week of 6/21/2010.
    "

    Is that wording clear and walk the right line between "you should help
    test
    this release" and "this release may not work great"?

    I think the following patches still need to go in:
    - HBASE-2767 (failed tests building with HDFS-1209)
    Reviewed.
    Thanks, will commit momentarily.
    - HBASE-2729 (fix bug when flush hits IOE)
    Please paste patch into issue so can review (review.hbase.org is
    down)
    Patch posted... sorry about review.hbase.org. I was being cheap and
    using
    spot instances, but apparently spot instances are the first thing to
    get
    shut down when ec2 has capacity issues. I'll upgrade it to a real
    instance
    and send the bill to the Cloudera bosses :)

    - I'd also like to disable the TestAcidGuarantees test in this
    branch,
    since
    that is a known bug.

    Grand.
    - I'd like to commit a short KNOWN_BUGS file which describes a few
    of the
    open issues that we're currently working on. Certainly doesn't have
    to be
    exhaustive list of all open JIRAs, but just a few things that users
    may
    run
    into.
    Yeah. Just call out the biggies and refer use to the short list
    (ahem) of other issues we have against next major release.

    Some performance issues were raised during testing at StumbleUpon -
    - any
    luck figuring those out, Ryan/Stack? It would be good to address
    them for
    the dev release, since it sounds like the RS barely makes progress
    when
    this
    bug is triggered.

    Yeah, its a beaut. Sucks all resources for some period of time until
    it gets over first flush then its good to go for a while at least.
    Still trying to figure it.
    OK. I'll continue to do cluster testing and see if I can reproduce this
    one.
    I'd love to fix it before release, but if we have to release with the
    bug I
    think it's worth doing rather than delaying. Agreed? We'll call it out
    in
    known bugs. Alternatively if it would fix it, we could patch out the
    CAS
    spin loop in completeMemstoreInsert and mark a known bug that
    read-your-writes consistency is lost under rare circumstances.
    Whichever you
    think is better.

    Given the above, I'd like to see if we can get the above two jiras reviewed
    and committed later tonight, and I'll try to roll a release
    candidate
    before
    I go to sleep. I think the correct way to release in this maven
    land is
    to
    simply do an svn export and vote on that, and then separately do an
    assembly:assembly tar as a binary release artifact (since assembly
    tar
    doesn't include unpacked source, etc).
    Why not just vote on the assembly? Thats what we'd run? If you do a
    site before assembly:assembly, you'll even have docs (mvn install site
    assembly;assembly). The source is there to review if wanted. Doing
    an svn export will have to build ourselves.
    The avro project had this discussion a bit recently, and what basically
    came
    out of it is that Apache releases are source, not binary. It happens
    that in
    the past our releases have had both source and binary in one tar, but
    it's
    important that the actual *release* artifact contain the source. This
    allows
    people to rebuild from the signed release tarball if they like, etc.
    Since
    the mvn assembly tar isn't re-buildable, I think the release artifact
    has to
    be a tarred svn export, and then we have a second release artifact
    which is
    binary+site+docs like you say above. People can choose whether to
    download
    the source release or the binary.

    At least the above was my understanding - I'll ping Doug with this
    thread to
    see if he can clarify/confirm.

    Thanks
    -Todd


    --
    Todd Lipcon
    Software Engineer, Cloudera
  • Andrew Purtell at Jun 23, 2010 at 10:08 am
    +1

    Nice writeup.

    - Andy
  • Stack at Jun 23, 2010 at 7:08 am

    On Tue, Jun 22, 2010 at 10:20 PM, Todd Lipcon wrote:
    ...
    Is that wording clear and walk the right line between "you should help test
    this release" and "this release may not work great"?
    Its excellent. Ship it.

    OK. I'll continue to do cluster testing and see if I can reproduce this one.
    I'd love to fix it before release, but if we have to release with the bug I
    think it's worth doing rather than delaying. Agreed? We'll call it out in
    known bugs.

    This is fine by me. We spin crazy for some short period then the
    problem "dissolves". We can fix for the next release (my loading goes
    on to complete after this spike).
    The avro project had this discussion a bit recently, and what basically came
    out of it is that Apache releases are source, not binary. It happens that in
    the past our releases have had both source and binary in one tar, but it's
    important that the actual *release* artifact contain the source.
    assembly:assembly adds in a src jar alongside the bin jar.

    This allows
    people to rebuild from the signed release tarball if they like, etc.
    You can't do this w/ our bundle. We can sign the tgz, it'll include
    src, but you can't build it in-situ once you undo the tgz. We could
    try make it so you could build in-situ but IIRC, in that avro
    discussion was suggested that we might all want to move away from this
    style of deliverable?



    Since
    the mvn assembly tar isn't re-buildable, I think the release artifact has to
    be a tarred svn export, and then we have a second release artifact which is
    binary+site+docs like you say above. People can choose whether to download
    the source release or the binary.
    OK. I'd say do the svn export for now. For next time, we'll have mvn
    generate a -src bundle to sit beside the -bin bundle it currently
    creates.


    St.Ack
  • Paul Smith at Jun 23, 2010 at 10:45 am


    Since
    the mvn assembly tar isn't re-buildable, I think the release artifact has to
    be a tarred svn export, and then we have a second release artifact which is
    binary+site+docs like you say above. People can choose whether to download
    the source release or the binary.
    OK. I'd say do the svn export for now. For next time, we'll have mvn
    generate a -src bundle to sit beside the -bin bundle it currently
    creates.

    if you want a fully rebuildable assembly, then you may want to revisit using a secondary assembly descriptor:

    http://maven.apache.org/plugins/maven-assembly-plugin/descriptor-refs.html#project

    I had this as an extra binary artifact as part of the build early on, but it was weighty, and slowed down the build process, so I suggested removing it. Perhaps you could consider configuring a Maven profile so that only when a -D argument is defined on the maven command line that this additional assembly is created (basically prevents this assembly from being built until you need it).

    It should be pretty simple, I can help with this, but take a look at the commit SHA:

    2ea3dbac74778279f8479ebb012198cb9bc7c9d8

    from HBASE-2334, just the top-level pom.xml change, this is the reversal I did:

    @@ -308,10 +308,6 @@
    <descriptors>
    <descriptor>src/assembly/bin.xml</descriptor>
    </descriptors>
    - <descriptorRefs>
    - <descriptorRef>src</descriptorRef>
    - <descriptorRef>project</descriptorRef>
    - </descriptorRefs>
    </configuration>
    </plugin>
    </plugins>



    Paul
  • Todd Lipcon at Jun 23, 2010 at 7:27 pm

    On Tue, Jun 22, 2010 at 9:05 PM, Stack wrote:
    On Tue, Jun 22, 2010 at 6:35 PM, Todd Lipcon wrote:

    Quick update on the development branch 0.89.20100621.
    (Todd you want to explain the version number or do you want me to?)

    I think the following patches still need to go in:
    - HBASE-2767 (failed tests building with HDFS-1209)
    Reviewed.

    Committed
    - HBASE-2729 (fix bug when flush hits IOE)
    Please paste patch into issue so can review (review.hbase.org is down)
    Stack reviewed and I committed

    - I'd also like to disable the TestAcidGuarantees test in this branch, since
    that is a known bug.
    Done
    Grand.
    - I'd like to commit a short KNOWN_BUGS file which describes a few of the
    open issues that we're currently working on. Certainly doesn't have to be
    exhaustive list of all open JIRAs, but just a few things that users may run
    into.
    Yeah. Just call out the biggies and refer use to the short list
    (ahem) of other issues we have against next major release.

    Done
    Some performance issues were raised during testing at StumbleUpon -- any
    luck figuring those out, Ryan/Stack? It would be good to address them for
    the dev release, since it sounds like the RS barely makes progress when this
    bug is triggered.

    Yeah, its a beaut. Sucks all resources for some period of time until
    it gets over first flush then its good to go for a while at least.
    Still trying to figure it.
    This one is still mysterious - it happens a lot over at StumbleUpon but I
    haven't been able to repro on my cluster. I put it in the known bugs list.

    Given the above, I'd like to see if we can get the above two jiras reviewed
    and committed later tonight, and I'll try to roll a release candidate before
    I go to sleep. I think the correct way to release in this maven land is to
    simply do an svn export and vote on that, and then separately do an
    assembly:assembly tar as a binary release artifact (since assembly tar
    doesn't include unpacked source, etc).
    Why not just vote on the assembly? Thats what we'd run? If you do a
    site before assembly:assembly, you'll even have docs (mvn install site
    assembly;assembly). The source is there to review if wanted. Doing
    an svn export will have to build ourselves.
    Doug says:
    "Yes, as an open-source foundation, ASF projects fundamentally produce
    source-code releases. Binaries are optional, sources not.

    Overall this sounds like a good plan. An ASF release needn't be top-quality
    and bug-free, but it needs to be voted on. When voting on a release a
    primary concern should be that it contains properly licensed code. Voters
    may additionally wish releases to have a certain level of quality, e.g.,
    more for final releases than snapshot releases, but that's not a requirement
    of every ASF release."

    So until we figure out a way to make a mvn assembly that is
    self-hosting/recompilable, let's just vote an an svn export tarball. Of
    course right next to it will be a "bin" tarball (which includes src jar,
    bin, docs, etc)

    Given that I think we have everything in place, I'm going to kick off some
    new cluster tests here and start a vote thread.

    -Todd

    --
    Todd Lipcon
    Software Engineer, Cloudera
  • Ryan Rawson at Jun 23, 2010 at 11:17 pm
    By moving away from spins to explicit synchronization we managed to
    squish the bug we have been seeing. I also moved us to use volatiles
    in RWCC just because. We should probably ship with this patch once
    it's committed.

    On Wed, Jun 23, 2010 at 12:27 PM, Todd Lipcon wrote:
    On Tue, Jun 22, 2010 at 9:05 PM, Stack wrote:
    On Tue, Jun 22, 2010 at 6:35 PM, Todd Lipcon wrote:

    Quick update on the development branch 0.89.20100621.
    (Todd you want to explain the version number or do you want me to?)

    I think the following patches still need to go in:
    - HBASE-2767 (failed tests building with HDFS-1209)
    Reviewed.

    Committed
    - HBASE-2729 (fix bug when flush hits IOE)
    Please paste patch into issue so can review (review.hbase.org is down)
    Stack reviewed and I committed

    - I'd also like to disable the TestAcidGuarantees test in this branch, since
    that is a known bug.
    Done
    Grand.
    - I'd like to commit a short KNOWN_BUGS file which describes a few of the
    open issues that we're currently working on. Certainly doesn't have to be
    exhaustive list of all open JIRAs, but just a few things that users may run
    into.
    Yeah.  Just call out the biggies and refer use to the short list
    (ahem) of other issues we have against next major release.

    Done
    Some performance issues were raised during testing at StumbleUpon -- any
    luck figuring those out, Ryan/Stack? It would be good to address them for
    the dev release, since it sounds like the RS barely makes progress when this
    bug is triggered.

    Yeah, its a beaut.  Sucks all resources for some period of time until
    it gets over first flush then its  good to go for a while at least.
    Still trying to figure it.
    This one is still mysterious - it happens a lot over at StumbleUpon but I
    haven't been able to repro on my cluster. I put it in the known bugs list.

    Given the above, I'd like to see if we can get the above two jiras reviewed
    and committed later tonight, and I'll try to roll a release candidate before
    I go to sleep. I think the correct way to release in this maven land is to
    simply do an svn export and vote on that, and then separately do an
    assembly:assembly tar as a binary release artifact (since assembly tar
    doesn't include unpacked source, etc).
    Why not just vote on the assembly?  Thats what we'd run?   If you do a
    site before assembly:assembly, you'll even have docs (mvn install site
    assembly;assembly).  The source is there to review if wanted.  Doing
    an svn export will have to build ourselves.
    Doug says:
    "Yes, as an open-source foundation, ASF projects fundamentally produce
    source-code releases.  Binaries are optional, sources not.

    Overall this sounds like a good plan.  An ASF release needn't be top-quality
    and bug-free, but it needs to be voted on.  When voting on a release a
    primary concern should be that it contains properly licensed code.  Voters
    may additionally wish releases to have a certain level of quality, e.g.,
    more for final releases than snapshot releases, but that's not a requirement
    of every ASF release."

    So until we figure out a way to make a mvn assembly that is
    self-hosting/recompilable, let's just vote an an svn export tarball. Of
    course right next to it will be a "bin" tarball (which includes src jar,
    bin, docs, etc)

    Given that I think we have everything in place, I'm going to kick off some
    new cluster tests here and start a vote thread.

    -Todd

    --
    Todd Lipcon
    Software Engineer, Cloudera
  • Todd Lipcon at Jun 23, 2010 at 11:36 pm

    On Wed, Jun 23, 2010 at 4:16 PM, Ryan Rawson wrote:

    By moving away from spins to explicit synchronization we managed to
    squish the bug we have been seeing. I also moved us to use volatiles
    in RWCC just because. We should probably ship with this patch once
    it's committed.
    OK, I can build a new rc later tonight. Let's keep testing the previous one,
    though, and count a +1 towards either as a +1 towards both :)

    -Todd

    On Wed, Jun 23, 2010 at 12:27 PM, Todd Lipcon wrote:
    On Tue, Jun 22, 2010 at 9:05 PM, Stack wrote:
    On Tue, Jun 22, 2010 at 6:35 PM, Todd Lipcon wrote:

    Quick update on the development branch 0.89.20100621.
    (Todd you want to explain the version number or do you want me to?)

    I think the following patches still need to go in:
    - HBASE-2767 (failed tests building with HDFS-1209)
    Reviewed.

    Committed
    - HBASE-2729 (fix bug when flush hits IOE)
    Please paste patch into issue so can review (review.hbase.org is down)
    Stack reviewed and I committed

    - I'd also like to disable the TestAcidGuarantees test in this branch, since
    that is a known bug.
    Done
    Grand.
    - I'd like to commit a short KNOWN_BUGS file which describes a few of
    the
    open issues that we're currently working on. Certainly doesn't have to
    be
    exhaustive list of all open JIRAs, but just a few things that users
    may
    run
    into.
    Yeah. Just call out the biggies and refer use to the short list
    (ahem) of other issues we have against next major release.

    Done
    Some performance issues were raised during testing at StumbleUpon --
    any
    luck figuring those out, Ryan/Stack? It would be good to address them
    for
    the dev release, since it sounds like the RS barely makes progress
    when
    this
    bug is triggered.

    Yeah, its a beaut. Sucks all resources for some period of time until
    it gets over first flush then its good to go for a while at least.
    Still trying to figure it.
    This one is still mysterious - it happens a lot over at StumbleUpon but I
    haven't been able to repro on my cluster. I put it in the known bugs
    list.
    Given the above, I'd like to see if we can get the above two jiras reviewed
    and committed later tonight, and I'll try to roll a release candidate before
    I go to sleep. I think the correct way to release in this maven land
    is
    to
    simply do an svn export and vote on that, and then separately do an
    assembly:assembly tar as a binary release artifact (since assembly tar
    doesn't include unpacked source, etc).
    Why not just vote on the assembly? Thats what we'd run? If you do a
    site before assembly:assembly, you'll even have docs (mvn install site
    assembly;assembly). The source is there to review if wanted. Doing
    an svn export will have to build ourselves.
    Doug says:
    "Yes, as an open-source foundation, ASF projects fundamentally produce
    source-code releases. Binaries are optional, sources not.

    Overall this sounds like a good plan. An ASF release needn't be
    top-quality
    and bug-free, but it needs to be voted on. When voting on a release a
    primary concern should be that it contains properly licensed code. Voters
    may additionally wish releases to have a certain level of quality, e.g.,
    more for final releases than snapshot releases, but that's not a
    requirement
    of every ASF release."

    So until we figure out a way to make a mvn assembly that is
    self-hosting/recompilable, let's just vote an an svn export tarball. Of
    course right next to it will be a "bin" tarball (which includes src jar,
    bin, docs, etc)

    Given that I think we have everything in place, I'm going to kick off some
    new cluster tests here and start a vote thread.

    -Todd

    --
    Todd Lipcon
    Software Engineer, Cloudera


    --
    Todd Lipcon
    Software Engineer, Cloudera

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categorieshbase, hadoop
postedJun 23, '10 at 1:36a
activeJun 23, '10 at 11:36p
posts10
users6
websitehbase.apache.org

People

Translate

site design / logo © 2021 Grokbase