Grokbase Groups Pig user October 2010
FAQ
have had some success creating and registering one single jar with dependencies. downside - the jar can be quite big. If you use maven to build your jar files just the following to the plugin section of your pom.xml


<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>attached</goal>
</goals>
</execution>
</executions>
</plugin>


On Oct 21, 2010, at 6:08 PM, Yong-gang Cao wrote:

Hi,
I met a headache about using UDFs with many dependence, adding them using
register command is very painful and not extensible. I can make
self-contained jar for hadoop job using maven (a jar with a lib directory
which contains all jars will be used for class look up), but it seems
doesn't work for pig. pig just treat that jar as a regular jar and try to
find classes directly inside it instead of inside those embedded jars.
Is there a way to make pig do the hadoop way of looking into the
self-contained big jar for class loading?
Thanks!

--
Regards,

Yong-gang Cao
Seattle,WA,98104

Search Discussions

Discussion Posts

Previous

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 3 of 3 | next ›
Discussion Overview
groupuser @
categoriespig, hadoop
postedOct 22, '10 at 12:09a
activeOct 22, '10 at 3:28p
posts3
users3
websitepig.apache.org

People

Translate

site design / logo © 2021 Grokbase