FAQ
I have created a candidate build for Hadoop 0.21.0. This fixes over
one thousand issues in Common, HDFS, and MapReduce since 0.20.2.

This release is being classified as a minor release, which means that
it is API compatible with 0.20.2.

*** This release candidate has not been tested extensively, so it
should not be considered stable.
*** Please download, test and vote before the vote closes on Thursday 8 July.

http://people.apache.org/~tomwhite/hadoop-0.21.0-candidate-0/

The hadoop-0.21.0.tar.gz file is an old-style combined release which
includes Common, HDFS, and MapReduce.

Thanks,
Tom

P.S. I have not yet pushed the JARs to Apache's Maven Staging
Repository (see HADOOP-6847), but this shouldn't prevent testing and
voting while it is sorted out.

Search Discussions

  • Tom White at Jul 8, 2010 at 12:15 pm
    A problem was found with Job.getCounters() returning null (see
    https://issues.apache.org/jira/browse/MAPREDUCE-1920). I plan to roll
    a new release candidate when this is fixed.

    Cheers,
    Tom
    On Fri, Jul 2, 2010 at 8:25 AM, Tom White wrote:
    I have created a candidate build for Hadoop 0.21.0. This fixes over
    one thousand issues in Common, HDFS, and MapReduce since 0.20.2.

    This release is being classified as a minor release, which means that
    it is API compatible with 0.20.2.

    *** This release candidate has not been tested extensively, so it
    should not be considered stable.
    *** Please download, test and vote before the vote closes on Thursday 8 July.

    http://people.apache.org/~tomwhite/hadoop-0.21.0-candidate-0/

    The hadoop-0.21.0.tar.gz file is an old-style combined release which
    includes Common, HDFS, and MapReduce.

    Thanks,
    Tom

    P.S. I have not yet pushed the JARs to Apache's Maven Staging
    Repository (see HADOOP-6847), but this shouldn't prevent testing and
    voting while it is sorted out.
  • Stephen Watt at Jul 8, 2010 at 5:05 pm
    Hi Tom,

    I'm trying to build Hadoop 0.21.0 locally, but its failing because the
    hadoop root dir is missing the build.xml in the tar.gz. Is there a new
    build process ? I'm currently using the ant clean tar test-core
    directives.

    Regards
    Steve Watt



    From:
    Tom White <tom@cloudera.com>
    To:
    common-dev <common-dev@hadoop.apache.org>
    Date:
    07/08/2010 07:16 AM
    Subject:
    Re: [VOTE] Release Hadoop 0.21.0 (candidate 0)



    A problem was found with Job.getCounters() returning null (see
    https://issues.apache.org/jira/browse/MAPREDUCE-1920). I plan to roll
    a new release candidate when this is fixed.

    Cheers,
    Tom
    On Fri, Jul 2, 2010 at 8:25 AM, Tom White wrote:
    I have created a candidate build for Hadoop 0.21.0. This fixes over
    one thousand issues in Common, HDFS, and MapReduce since 0.20.2.

    This release is being classified as a minor release, which means that
    it is API compatible with 0.20.2.

    *** This release candidate has not been tested extensively, so it
    should not be considered stable.
    *** Please download, test and vote before the vote closes on Thursday 8 July.
    http://people.apache.org/~tomwhite/hadoop-0.21.0-candidate-0/

    The hadoop-0.21.0.tar.gz file is an old-style combined release which
    includes Common, HDFS, and MapReduce.

    Thanks,
    Tom

    P.S. I have not yet pushed the JARs to Apache's Maven Staging
    Repository (see HADOOP-6847), but this shouldn't prevent testing and
    voting while it is sorted out.
  • Felix Halim at Jul 12, 2010 at 11:04 pm
    Hi Tom,

    Just want to let you know that back then when I tried to circumvent
    this problem, I used job.submit() and retrieve the counter
    asynchronously. I found out that the counter values always zero during
    the execution (so the job is still running halfway). So, in this case,
    the job is not retired yet, right? so, it might suggest another
    problem?

    Felix Halim
    On Thu, Jul 8, 2010 at 8:13 PM, Tom White wrote:
    A problem was found with Job.getCounters() returning null (see
    https://issues.apache.org/jira/browse/MAPREDUCE-1920). I plan to roll
    a new release candidate when this is fixed.

    Cheers,
    Tom
    On Fri, Jul 2, 2010 at 8:25 AM, Tom White wrote:
    I have created a candidate build for Hadoop 0.21.0. This fixes over
    one thousand issues in Common, HDFS, and MapReduce since 0.20.2.

    This release is being classified as a minor release, which means that
    it is API compatible with 0.20.2.

    *** This release candidate has not been tested extensively, so it
    should not be considered stable.
    *** Please download, test and vote before the vote closes on Thursday 8 July.

    http://people.apache.org/~tomwhite/hadoop-0.21.0-candidate-0/

    The hadoop-0.21.0.tar.gz file is an old-style combined release which
    includes Common, HDFS, and MapReduce.

    Thanks,
    Tom

    P.S. I have not yet pushed the JARs to Apache's Maven Staging
    Repository (see HADOOP-6847), but this shouldn't prevent testing and
    voting while it is sorted out.
  • Tom White at Jul 16, 2010 at 10:18 am
    Steve, Yes, this is a consequence of the project split. It's still
    possible to build Common, HDFS, MapReduce individually (from their
    respective distributions), but I agree it would be nice to be able to
    build from the combined distribution. When I create the next RC, I'll
    see if I can provide instructions on how to do this.

    Felix, I don't think this is a bug since counters are not stable until
    the job is complete, so the values you see may be zero if you retrieve
    them asynchronously.

    Cheers,
    Tom

    PS. I'm travelling/on vacation for the next couple of weeks, so I
    won't be working on the next release candidate until August. If anyone
    wants to work on the (blocker) JIRAs I listed in my previous email in
    this thread then be my guest!
    On Tue, Jul 13, 2010 at 12:04 AM, Felix Halim wrote:
    Hi Tom,

    Just want to let you know that back then when I tried to circumvent
    this problem, I used job.submit() and retrieve the counter
    asynchronously. I found out that the counter values always zero during
    the execution (so the job is still running halfway). So, in this case,
    the job is not retired yet, right? so, it might suggest another
    problem?

    Felix Halim
    On Thu, Jul 8, 2010 at 8:13 PM, Tom White wrote:
    A problem was found with Job.getCounters() returning null (see
    https://issues.apache.org/jira/browse/MAPREDUCE-1920). I plan to roll
    a new release candidate when this is fixed.

    Cheers,
    Tom
    On Fri, Jul 2, 2010 at 8:25 AM, Tom White wrote:
    I have created a candidate build for Hadoop 0.21.0. This fixes over
    one thousand issues in Common, HDFS, and MapReduce since 0.20.2.

    This release is being classified as a minor release, which means that
    it is API compatible with 0.20.2.

    *** This release candidate has not been tested extensively, so it
    should not be considered stable.
    *** Please download, test and vote before the vote closes on Thursday 8 July.

    http://people.apache.org/~tomwhite/hadoop-0.21.0-candidate-0/

    The hadoop-0.21.0.tar.gz file is an old-style combined release which
    includes Common, HDFS, and MapReduce.

    Thanks,
    Tom

    P.S. I have not yet pushed the JARs to Apache's Maven Staging
    Repository (see HADOOP-6847), but this shouldn't prevent testing and
    voting while it is sorted out.
  • Felix Halim at Jul 20, 2010 at 9:51 am

    On Fri, Jul 16, 2010 at 6:17 PM, Tom White wrote:
    Felix, I don't think this is a bug since counters are not stable until
    the job is complete, so the values you see may be zero if you retrieve
    them asynchronously.
    Just to clarify.. these counters that are retrieved asynchronously
    will be updated as soon as one mapper or reducer completes, right?
    Or is it going to be delayed arbitrarily until the job is finished?

    BTW, I have a warning when running a job:

    10/07/20 17:44:19 WARN conf.Configuration: mapred.task.id is
    deprecated. Instead, use mapreduce.task.attempt.id

    What causes this warning? I don't use mapred.task.id anywhere in the
    configurations...

    Felix Halim
  • Tom White at Jul 29, 2010 at 5:00 pm

    On Tue, Jul 20, 2010 at 10:50 AM, Felix Halim wrote:
    On Fri, Jul 16, 2010 at 6:17 PM, Tom White wrote:
    Felix, I don't think this is a bug since counters are not stable until
    the job is complete, so the values you see may be zero if you retrieve
    them asynchronously.
    Just to clarify.. these counters that are retrieved asynchronously
    will be updated as soon as one mapper or reducer completes, right?
    Or is it going to be delayed arbitrarily until the job is finished?
    To be clear, the counters objects will not be updated, you have to
    retrieve them again, and you can only be sure they are correct after
    the job completes.
    BTW, I have a warning when running a job:

    10/07/20 17:44:19 WARN conf.Configuration: mapred.task.id is
    deprecated. Instead, use mapreduce.task.attempt.id

    What causes this warning? I don't use mapred.task.id anywhere in the
    configurations...
    I think it's used internally, so there's nothing to worry about.

    Tom
    Felix Halim
  • Aaron Kimball at Jul 8, 2010 at 6:56 pm
    I downloaded the "combined" tarball of 0.21.0-rc0 and set it up as a
    pseudo-distributed Hadoop cluster.

    Everything seems to work; basic smoke tests pass. Did not run internal unit
    tests. I tested Sqoop 1.0.0 against this release. All sqoop unit tests pass.
    Sqoop can operate on the command-line as a "normal" Hadoop application
    against 0.21.0 successfully as long as MAPREDUCE-1920 is applied (or the
    appropriate configuration changes are made in mapred-site.xml).

    The other problem I found is that build-utils.xml is missing which prevents
    recompilation.

    I am a (nonbinding) +1 to this release candidate along with modifications to
    include MR-1920 and fix the missing build-utils.xml issue.

    - Aaron
    On Fri, Jul 2, 2010 at 12:25 AM, Tom White wrote:

    I have created a candidate build for Hadoop 0.21.0. This fixes over
    one thousand issues in Common, HDFS, and MapReduce since 0.20.2.

    This release is being classified as a minor release, which means that
    it is API compatible with 0.20.2.

    *** This release candidate has not been tested extensively, so it
    should not be considered stable.
    *** Please download, test and vote before the vote closes on Thursday 8
    July.

    http://people.apache.org/~tomwhite/hadoop-0.21.0-candidate-0/

    The hadoop-0.21.0.tar.gz file is an old-style combined release which
    includes Common, HDFS, and MapReduce.

    Thanks,
    Tom

    P.S. I have not yet pushed the JARs to Apache's Maven Staging
    Repository (see HADOOP-6847), but this shouldn't prevent testing and
    voting while it is sorted out.
  • Tom White at Jul 9, 2010 at 11:08 am
    A number of issues have been identified with RC0. The following JIRAs
    need fixing before the next release candidate can be built:

    HDFS-1288 - start-all.sh / stop-all.sh does not seem to work with HDFS
    MAPREDUCE-1920 - Job.getCounters() returns null when using a cluster
    MAPREDUCE-1926 - MapReduce distribution is missing build-utils.xml

    In addition, the following JIRAs need fixing before the release is finalized:

    HDFS-1292 - Allow artifacts to be published to the staging Apache
    Nexus Maven Repository
    MAPREDUCE-1929 - Allow artifacts to be published to the staging Apache
    Nexus Maven Repository

    Tom
    On Thu, Jul 8, 2010 at 7:55 PM, Aaron Kimball wrote:
    I downloaded the "combined" tarball of 0.21.0-rc0 and set it up as a
    pseudo-distributed Hadoop cluster.

    Everything seems to work; basic smoke tests pass. Did not run internal unit
    tests. I tested Sqoop 1.0.0 against this release. All sqoop unit tests pass.
    Sqoop can operate on the command-line as a "normal" Hadoop application
    against 0.21.0 successfully as long as MAPREDUCE-1920 is applied (or the
    appropriate configuration changes are made in mapred-site.xml).

    The other problem I found is that build-utils.xml is missing which prevents
    recompilation.

    I am a (nonbinding) +1 to this release candidate along with modifications to
    include MR-1920 and fix the missing build-utils.xml issue.

    - Aaron
    On Fri, Jul 2, 2010 at 12:25 AM, Tom White wrote:

    I have created a candidate build for Hadoop 0.21.0. This fixes over
    one thousand issues in Common, HDFS, and MapReduce since 0.20.2.

    This release is being classified as a minor release, which means that
    it is API compatible with 0.20.2.

    *** This release candidate has not been tested extensively, so it
    should not be considered stable.
    *** Please download, test and vote before the vote closes on Thursday 8
    July.

    http://people.apache.org/~tomwhite/hadoop-0.21.0-candidate-0/

    The hadoop-0.21.0.tar.gz file is an old-style combined release which
    includes Common, HDFS, and MapReduce.

    Thanks,
    Tom

    P.S. I have not yet pushed the JARs to Apache's Maven Staging
    Repository (see HADOOP-6847), but this shouldn't prevent testing and
    voting while it is sorted out.
  • Stephen Watt at Jul 12, 2010 at 7:13 pm
    This is likely a result of how things are now being built post
    project-split, but previously, for the hadoop-0.20.x releases there was a
    top level build.xml file which would orchestrate building the sub-projects
    which were split underneath the src directory, resulting in a final
    hadoop-20.x-core.jar. I see now each sub-project (common, hdfs and mapred)
    has its own subdirectory off of the root, as opposed to forking under the
    src/ directory and in turn their own sub-project jar files and there is
    now no aggregated hadoop-x-core.jar file.

    I also saw Tom's JIRA with the 0.21 build scripts. My questions are:

    1) Is this the way we're going to be building going forward for the
    foreseeable future?
    2) Could we consider still keeping a top level build file (i.e committing
    a variant of tom's scripts) that orchestrates the building of all the
    sub-projects?

    Kind regards
    Steve Watt

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedJul 2, '10 at 7:26a
activeJul 29, '10 at 5:00p
posts10
users4
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase