Grokbase Groups Pig user January 2011
FAQ
Hey Guys,

I am running into a bit of trouble, and I know its something that must be
commonly done. I have created a loader function which uses external JARs,
which is fine when ran in local mode; the Pig job is also scripted in
embedded mode (so included in native java). The JARs aren't required by my
Pig script directly, but required by the UDF that the Pig script uses, how
do I distribute the JARs in mapreduce mode and how do I setup my native java
code to access it ?

Many Thanks,

Robert.

Search Discussions

  • Dmitriy Ryaboy at Jan 7, 2011 at 10:11 pm
    You just need to:
    1) make sure they are on the classpath when you call pig (you can add them
    to PIG_CLASSPATH)
    2) add "register /path/to/myrequired.jar" at the top of your script for
    every required jar.

    -D
    On Fri, Jan 7, 2011 at 2:00 PM, robert wrote:

    Hey Guys,

    I am running into a bit of trouble, and I know its something that must be
    commonly done. I have created a loader function which uses external JARs,
    which is fine when ran in local mode; the Pig job is also scripted in
    embedded mode (so included in native java). The JARs aren't required by my
    Pig script directly, but required by the UDF that the Pig script uses, how
    do I distribute the JARs in mapreduce mode and how do I setup my native
    java
    code to access it ?

    Many Thanks,

    Robert.
  • Robert at Jan 7, 2011 at 10:37 pm
    Dmitriy,

    many thanks, that seemed to stop it complaining about classes not being
    found.

    Robert.
    On Fri, Jan 7, 2011 at 10:10 PM, Dmitriy Ryaboy wrote:

    You just need to:
    1) make sure they are on the classpath when you call pig (you can add them
    to PIG_CLASSPATH)
    2) add "register /path/to/myrequired.jar" at the top of your script for
    every required jar.

    -D
    On Fri, Jan 7, 2011 at 2:00 PM, robert wrote:

    Hey Guys,

    I am running into a bit of trouble, and I know its something that must be
    commonly done. I have created a loader function which uses external JARs,
    which is fine when ran in local mode; the Pig job is also scripted in
    embedded mode (so included in native java). The JARs aren't required by my
    Pig script directly, but required by the UDF that the Pig script uses, how
    do I distribute the JARs in mapreduce mode and how do I setup my native
    java
    code to access it ?

    Many Thanks,

    Robert.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedJan 7, '11 at 10:00p
activeJan 7, '11 at 10:37p
posts3
users2
websitepig.apache.org

2 users in discussion

Robert: 2 posts Dmitriy Ryaboy: 1 post

People

Translate

site design / logo © 2022 Grokbase