Grokbase Groups Pig user August 2010
FAQ
I keep getting this error, it seems pig cannot locate the jar that
contains my udf ImageProcessor.How do i build a jar that contains my udf
(p.s the UDF documentation wasnt that helpful



laptop:~/pig-0.7.0/trunk$ javac -cp $CLASSPATH:pig.jar ImageProcessor.java
laptop:~/pig-0.7.0/trunk$ bin/pig specialscript.pig
log4j:WARN No appenders could be found for logger
(org.apache.hadoop.conf.Configuration).
log4j:WARN Please initialize the log4j system properly.
2010-08-11 11:54:15,820 [main] INFO org.apache.pig.Main - Logging error
messages to: /home/hadoop/pig-0.7.0/trunk/pig_1281542055806.log
2010-08-11 11:54:16,005 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
Connecting to hadoop file system at: hdfs://localhost:54310
2010-08-11 11:54:16,179 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
Connecting to map-reduce job tracker at: localhost:54311
2010-08-11 11:54:16,384 [main] ERROR org.apache.pig.tools.grunt.Grunt -
ERROR 1070: Could not resolve ImageProcessor using imports: [,
org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Details at logfile: /home/hadoop/pig-0.7.0/trunk/pig_1281542055806.log
laptop:~/pig-0.7.0/trunk$

Search Discussions

  • Ashutosh Chauhan at Aug 11, 2010 at 4:47 pm
    Just compiling it isn't enough. You need to package it in a jar and then
    register that jar in your pig script.
    Documentation: http://hadoop.apache.org/pig/docs/r0.7.0/udf.html

    Ashutosh
    On Wed, Aug 11, 2010 at 09:06, Ifeanyichukwu Osuji wrote:

    I keep getting this error, it seems pig cannot locate the jar that
    contains my udf ImageProcessor.How do i build a jar that contains my udf
    (p.s the UDF documentation wasnt that helpful



    laptop:~/pig-0.7.0/trunk$ javac -cp $CLASSPATH:pig.jar ImageProcessor.java
    laptop:~/pig-0.7.0/trunk$ bin/pig specialscript.pig
    log4j:WARN No appenders could be found for logger
    (org.apache.hadoop.conf.Configuration).
    log4j:WARN Please initialize the log4j system properly.
    2010-08-11 11:54:15,820 [main] INFO org.apache.pig.Main - Logging error
    messages to: /home/hadoop/pig-0.7.0/trunk/pig_1281542055806.log
    2010-08-11 11:54:16,005 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
    Connecting to hadoop file system at: hdfs://localhost:54310
    2010-08-11 11:54:16,179 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
    Connecting to map-reduce job tracker at: localhost:54311
    2010-08-11 11:54:16,384 [main] ERROR org.apache.pig.tools.grunt.Grunt -
    ERROR 1070: Could not resolve ImageProcessor using imports: [,
    org.apache.pig.builtin., org.apache.pig.impl.builtin.]
    Details at logfile: /home/hadoop/pig-0.7.0/trunk/pig_1281542055806.log
    laptop:~/pig-0.7.0/trunk$

  • Dave Wellman at Aug 11, 2010 at 5:28 pm
    feanyichukwu Osuji,

    Not sure if this help you or not, but we had a similar problem where dependent jars where not included in the deployed jar, so we created a mvn that included the dependencies into the final jar.

    If that is your issue the following pom.xml snippet should help

    <plugin>
    <artifactId>maven-assembly-plugin</artifactId>
    <configuration>
    <descriptorRefs>
    <descriptorRef>jar-with-dependencies</descriptorRef>
    </descriptorRefs>
    </configuration>
    <executions>
    <execution>
    <id>make-assembly</id>
    <phase>package</phase>
    <goals>
    <goal>attached</goal>
    </goals>
    </execution>
    </executions>
    </plugin>


    Then you can register your jar with the pig command:

    REGISTER /my/local/path/to/the/jars/my.jar

    Define the method you want to call like:

    DEFINE MY_STUFF com.mycompany.ufs.MyStuff();

    Define the method you want to call like:

    x = FOREACH y GENERATE MY_STUFF(x);
    On Aug 11, 2010, at 10:06 AM, Ifeanyichukwu Osuji wrote:

    I keep getting this error, it seems pig cannot locate the jar that
    contains my udf ImageProcessor.How do i build a jar that contains my udf
    (p.s the UDF documentation wasnt that helpful



    laptop:~/pig-0.7.0/trunk$ javac -cp $CLASSPATH:pig.jar ImageProcessor.java
    laptop:~/pig-0.7.0/trunk$ bin/pig specialscript.pig
    log4j:WARN No appenders could be found for logger
    (org.apache.hadoop.conf.Configuration).
    log4j:WARN Please initialize the log4j system properly.
    2010-08-11 11:54:15,820 [main] INFO org.apache.pig.Main - Logging error
    messages to: /home/hadoop/pig-0.7.0/trunk/pig_1281542055806.log
    2010-08-11 11:54:16,005 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
    Connecting to hadoop file system at: hdfs://localhost:54310
    2010-08-11 11:54:16,179 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
    Connecting to map-reduce job tracker at: localhost:54311
    2010-08-11 11:54:16,384 [main] ERROR org.apache.pig.tools.grunt.Grunt -
    ERROR 1070: Could not resolve ImageProcessor using imports: [,
    org.apache.pig.builtin., org.apache.pig.impl.builtin.]
    Details at logfile: /home/hadoop/pig-0.7.0/trunk/pig_1281542055806.log
    laptop:~/pig-0.7.0/trunk$

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedAug 11, '10 at 4:47p
activeAug 11, '10 at 5:28p
posts3
users3
websitepig.apache.org

People

Translate

site design / logo © 2021 Grokbase