Grokbase Groups Pig user April 2013
FAQ
Hi,

[ I know this question is probably CDH specific, yet I'm hoping one of you
may be able to point me in the right direction. ]

I want to make a small change to the piggybank for pig 0.10 that is in CDH
4.2.0 and recreate an RPM from that for easier deployment.

So I downloaded
http://archive.cloudera.com/cdh4/cdh/4/pig-0.10.0-cdh4.2.0.tar.gz and tried
to simply build pig using ant.

When I do that on CentOS 6.4 x86_64 it starts downloading and compiling and
then it fails with:

*compile-sources:*
* [javac] /home/niels/tmp/pig-0.10.0-cdh4.2.0/build.xml:575: warning:
'includeantruntime' was not set, defaulting to build.sysclasspath=last; set
to false for repeatable builds*
* [javac] Compiling 788 source files to
/home/niels/tmp/pig-0.10.0-cdh4.2.0/build/classes*
* [javac]
/home/niels/tmp/pig-0.10.0-cdh4.2.0/src/org/apache/pig/data/Tuple.java:23:
cannot find symbol*
* [javac] symbol : class WritableComparable*
* [javac] location: package org.apache.hadoop.io*
* [javac] import org.apache.hadoop.io.WritableComparable;*
* [javac] ^*
* [javac]
/home/niels/tmp/pig-0.10.0-cdh4.2.0/src/org/apache/pig/data/Tuple.java:47:
cannot find symbol*
* [javac] symbol: class WritableComparable*
* [javac] public interface Tuple extends WritableComparable,
Serializable, Iterable<Object> {*
* [javac] ^*

Apparently it is unable to find the most basic Hadoop dependencies

This is strange because during the download phase I see:
*[ivy:resolve] found org.apache.hadoop#hadoop-annotations;2.0.0-cdh4.2.0
in cloudera-releases*
*[ivy:resolve] found org.apache.hadoop#hadoop-auth;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve] found org.apache.hadoop#hadoop-common;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve] found org.apache.hadoop#hadoop-hdfs;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve] found
org.apache.hadoop#hadoop-mapreduce-client-core;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve] found
org.apache.hadoop#hadoop-mapreduce-client-jobclient;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve] found
org.apache.hadoop#hadoop-yarn-server-tests;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve] found
org.apache.hadoop#hadoop-mapreduce-client-app;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve] found
org.apache.hadoop#hadoop-mapreduce-client-shuffle;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve] found
org.apache.hadoop#hadoop-mapreduce-client-common;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve] found org.apache.hadoop#hadoop-yarn-api;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve] found org.apache.hadoop#hadoop-yarn-client;2.0.0-cdh4.2.0
in cloudera-releases*
*[ivy:resolve] found org.apache.hadoop#hadoop-yarn-common;2.0.0-cdh4.2.0
in cloudera-releases*
*[ivy:resolve] found org.apache.hadoop#hadoop-yarn-server;2.0.0-cdh4.2.0
in cloudera-releases*
*[ivy:resolve] found
org.apache.hadoop#hadoop-yarn-server-web-proxy;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve] found
org.apache.hadoop#hadoop-yarn-server-common;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve] found
org.apache.hadoop#hadoop-yarn-server-nodemanager;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve] found
org.apache.hadoop#hadoop-yarn-server-resourcemanager;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve] found
org.apache.hadoop#hadoop-mapreduce-client-hs;2.0.0-cdh4.2.0 in
cloudera-releases*

I've also tried to just build the piggybank in the contrib folder and when
I do that I get similar (dependency) errors:

* [javac]
/home/niels/tmp/pig-0.10.0-cdh4.2.0/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/evaluation/ExtremalTupleByNthField.java:23:
cannot find symbol*
* [javac] symbol : class Accumulator*
* [javac] location: package org.apache.pig*
* [javac] import org.apache.pig.Accumulator;*
* [javac] ^*
* [javac]
/home/niels/tmp/pig-0.10.0-cdh4.2.0/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/evaluation/ExtremalTupleByNthField.java:24:
cannot find symbol*
* [javac] symbol : class Algebraic*
* [javac] location: package org.apache.pig*
* [javac] import org.apache.pig.Algebraic;*
* [javac] ^*
* [javac]
/home/niels/tmp/pig-0.10.0-cdh4.2.0/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/evaluation/ExtremalTupleByNthField.java:25:
cannot find symbol*
* [javac] symbol : class EvalFunc*
* [javac] location: package org.apache.pig*
* [javac] import org.apache.pig.EvalFunc;*
* [javac] ^*

What should I do to fix this?
Is there a 'better' manual on building pig/piggybank than what is described
in the RELEASE_NOTES.txt ?

--
Best regards / Met vriendelijke groeten,

Niels Basjes

Search Discussions

  • Cheolsoo Park at Apr 24, 2013 at 9:54 pm
    Hi,

    Please try:
    ant clean jar-all piggybank -Dhadoopversion=23

    Thanks,
    Cheolsoo

    On Wed, Apr 24, 2013 at 1:45 PM, Niels Basjes wrote:

    Hi,

    [ I know this question is probably CDH specific, yet I'm hoping one of you
    may be able to point me in the right direction. ]

    I want to make a small change to the piggybank for pig 0.10 that is in CDH
    4.2.0 and recreate an RPM from that for easier deployment.

    So I downloaded
    http://archive.cloudera.com/cdh4/cdh/4/pig-0.10.0-cdh4.2.0.tar.gz and
    tried
    to simply build pig using ant.

    When I do that on CentOS 6.4 x86_64 it starts downloading and compiling and
    then it fails with:

    *compile-sources:*
    * [javac] /home/niels/tmp/pig-0.10.0-cdh4.2.0/build.xml:575: warning:
    'includeantruntime' was not set, defaulting to build.sysclasspath=last; set
    to false for repeatable builds*
    * [javac] Compiling 788 source files to
    /home/niels/tmp/pig-0.10.0-cdh4.2.0/build/classes*
    * [javac]
    /home/niels/tmp/pig-0.10.0-cdh4.2.0/src/org/apache/pig/data/Tuple.java:23:
    cannot find symbol*
    * [javac] symbol : class WritableComparable*
    * [javac] location: package org.apache.hadoop.io*
    * [javac] import org.apache.hadoop.io.WritableComparable;*
    * [javac] ^*
    * [javac]
    /home/niels/tmp/pig-0.10.0-cdh4.2.0/src/org/apache/pig/data/Tuple.java:47:
    cannot find symbol*
    * [javac] symbol: class WritableComparable*
    * [javac] public interface Tuple extends WritableComparable,
    Serializable, Iterable<Object> {*
    * [javac] ^*

    Apparently it is unable to find the most basic Hadoop dependencies

    This is strange because during the download phase I see:
    *[ivy:resolve] found org.apache.hadoop#hadoop-annotations;2.0.0-cdh4.2.0
    in cloudera-releases*
    *[ivy:resolve] found org.apache.hadoop#hadoop-auth;2.0.0-cdh4.2.0 in
    cloudera-releases*
    *[ivy:resolve] found org.apache.hadoop#hadoop-common;2.0.0-cdh4.2.0 in
    cloudera-releases*
    *[ivy:resolve] found org.apache.hadoop#hadoop-hdfs;2.0.0-cdh4.2.0 in
    cloudera-releases*
    *[ivy:resolve] found
    org.apache.hadoop#hadoop-mapreduce-client-core;2.0.0-cdh4.2.0 in
    cloudera-releases*
    *[ivy:resolve] found
    org.apache.hadoop#hadoop-mapreduce-client-jobclient;2.0.0-cdh4.2.0 in
    cloudera-releases*
    *[ivy:resolve] found
    org.apache.hadoop#hadoop-yarn-server-tests;2.0.0-cdh4.2.0 in
    cloudera-releases*
    *[ivy:resolve] found
    org.apache.hadoop#hadoop-mapreduce-client-app;2.0.0-cdh4.2.0 in
    cloudera-releases*
    *[ivy:resolve] found
    org.apache.hadoop#hadoop-mapreduce-client-shuffle;2.0.0-cdh4.2.0 in
    cloudera-releases*
    *[ivy:resolve] found
    org.apache.hadoop#hadoop-mapreduce-client-common;2.0.0-cdh4.2.0 in
    cloudera-releases*
    *[ivy:resolve] found org.apache.hadoop#hadoop-yarn-api;2.0.0-cdh4.2.0 in
    cloudera-releases*
    *[ivy:resolve] found org.apache.hadoop#hadoop-yarn-client;2.0.0-cdh4.2.0
    in cloudera-releases*
    *[ivy:resolve] found org.apache.hadoop#hadoop-yarn-common;2.0.0-cdh4.2.0
    in cloudera-releases*
    *[ivy:resolve] found org.apache.hadoop#hadoop-yarn-server;2.0.0-cdh4.2.0
    in cloudera-releases*
    *[ivy:resolve] found
    org.apache.hadoop#hadoop-yarn-server-web-proxy;2.0.0-cdh4.2.0 in
    cloudera-releases*
    *[ivy:resolve] found
    org.apache.hadoop#hadoop-yarn-server-common;2.0.0-cdh4.2.0 in
    cloudera-releases*
    *[ivy:resolve] found
    org.apache.hadoop#hadoop-yarn-server-nodemanager;2.0.0-cdh4.2.0 in
    cloudera-releases*
    *[ivy:resolve] found
    org.apache.hadoop#hadoop-yarn-server-resourcemanager;2.0.0-cdh4.2.0 in
    cloudera-releases*
    *[ivy:resolve] found
    org.apache.hadoop#hadoop-mapreduce-client-hs;2.0.0-cdh4.2.0 in
    cloudera-releases*

    I've also tried to just build the piggybank in the contrib folder and when
    I do that I get similar (dependency) errors:

    * [javac]

    /home/niels/tmp/pig-0.10.0-cdh4.2.0/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/evaluation/ExtremalTupleByNthField.java:23:
    cannot find symbol*
    * [javac] symbol : class Accumulator*
    * [javac] location: package org.apache.pig*
    * [javac] import org.apache.pig.Accumulator;*
    * [javac] ^*
    * [javac]

    /home/niels/tmp/pig-0.10.0-cdh4.2.0/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/evaluation/ExtremalTupleByNthField.java:24:
    cannot find symbol*
    * [javac] symbol : class Algebraic*
    * [javac] location: package org.apache.pig*
    * [javac] import org.apache.pig.Algebraic;*
    * [javac] ^*
    * [javac]

    /home/niels/tmp/pig-0.10.0-cdh4.2.0/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/evaluation/ExtremalTupleByNthField.java:25:
    cannot find symbol*
    * [javac] symbol : class EvalFunc*
    * [javac] location: package org.apache.pig*
    * [javac] import org.apache.pig.EvalFunc;*
    * [javac] ^*

    What should I do to fix this?
    Is there a 'better' manual on building pig/piggybank than what is described
    in the RELEASE_NOTES.txt ?

    --
    Best regards / Met vriendelijke groeten,

    Niels Basjes
  • Niels Basjes at Apr 25, 2013 at 9:02 pm
    Hi Cheolsoo,
    On Wed, Apr 24, 2013 at 11:53 PM, Cheolsoo Park wrote:

    Please try:
    ant clean jar-all piggybank -Dhadoopversion=23
    Thanks, this really helped.
    Perhaps this should be part of a "Developer / Build Howto" ?


    I downloaded the current trunk of pig (via github) and ran into a
    completely different hurdle:
    With hadoopversion=23 this target is fired
         <target name="include-meta" if="isHadoop23">

    And even though I installed ant 1.9.0 (the latest) it causes an NPE in ant
    itself.

    *BUILD FAILED
    *
    */home/nbasjes/workspace/pig/build.xml:658: The following error occurred
    while executing this line:*
    */home/nbasjes/workspace/pig/build.xml:667: java.lang.NullPointerException*
    * at org.apache.tools.ant.taskdefs.Zip.zipFile(Zip.java:1827)*
    * at org.apache.tools.ant.taskdefs.Jar.zipFile(Jar.java:702)*
    * at org.apache.tools.ant.taskdefs.Zip.zipFile(Zip.java:1856)*
    *...*

    Has anyone seen this too? Is this a bug in ant or am I doing something
    wrong?

    --
    Best regards / Met vriendelijke groeten,

    Niels Basjes
  • Cheolsoo Park at Apr 25, 2013 at 11:20 pm
    Hi Niels,
    Perhaps this should be part of a "Developer / Build Howto" ?
    You probably want to ask it to CDH user group. I don't think Apache
    projects document about specific distros.

    Regarding the other error, I can't reproduce it with Apache Pig. I am using
    Ant 1.8.2.

    Thanks,
    Cheolsoo



    On Thu, Apr 25, 2013 at 2:01 PM, Niels Basjes wrote:

    Hi Cheolsoo,

    On Wed, Apr 24, 2013 at 11:53 PM, Cheolsoo Park <piaozhexiu@gmail.com
    wrote:
    Please try:
    ant clean jar-all piggybank -Dhadoopversion=23
    Thanks, this really helped.
    Perhaps this should be part of a "Developer / Build Howto" ?


    I downloaded the current trunk of pig (via github) and ran into a
    completely different hurdle:
    With hadoopversion=23 this target is fired
    <target name="include-meta" if="isHadoop23">

    And even though I installed ant 1.9.0 (the latest) it causes an NPE in ant
    itself.

    *BUILD FAILED
    *
    */home/nbasjes/workspace/pig/build.xml:658: The following error occurred
    while executing this line:*
    */home/nbasjes/workspace/pig/build.xml:667: java.lang.NullPointerException*
    * at org.apache.tools.ant.taskdefs.Zip.zipFile(Zip.java:1827)*
    * at org.apache.tools.ant.taskdefs.Jar.zipFile(Jar.java:702)*
    * at org.apache.tools.ant.taskdefs.Zip.zipFile(Zip.java:1856)*
    *...*

    Has anyone seen this too? Is this a bug in ant or am I doing something
    wrong?

    --
    Best regards / Met vriendelijke groeten,

    Niels Basjes
  • Niels Basjes at May 8, 2013 at 2:59 pm
    Hi,
    On Fri, Apr 26, 2013 at 1:20 AM, Cheolsoo Park wrote:

    Perhaps this should be part of a "Developer / Build Howto" ?
    You probably want to ask it to CDH user group. I don't think Apache
    projects document about specific distros.
    After your first email I kicked the cloudera sources and went for the plain
    pig sources instead.


    Regarding the other error, I can't reproduce it with Apache Pig. I am using
    Ant 1.8.2.
    Hmm, I should try this with 1.8.2 then. Perhaps there is a bug in ant 1.9.0


    --
    Best regards / Met vriendelijke groeten,

    Niels Basjes

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedApr 24, '13 at 8:46p
activeMay 8, '13 at 2:59p
posts5
users2
websitepig.apache.org

2 users in discussion

Niels Basjes: 3 posts Cheolsoo Park: 2 posts

People

Translate

site design / logo © 2021 Grokbase