FAQ
Hi All,

We have a use-case where we want to automatically register certain jars for
command-line users. I tried using ­jar, but this switch seems to do
absolutely nothing.

How do we go about auto-registering jars using pig? Any help is much
appreciated.

Thanks in advance!
Chris

Search Discussions

  • Ashutosh Chauhan at Feb 5, 2010 at 6:47 pm
    'register' is your friend here

    http://hadoop.apache.org/pig/docs/r0.5.0/piglatin_reference.html#REGISTER

    Ashutosh
    On Thu, Feb 4, 2010 at 23:25, Chris Riccomini wrote:
    Hi All,

    We have a use-case where we want to automatically register certain jars for
    command-line users. I tried using ­jar, but this switch seems to do
    absolutely nothing.

    How do we go about auto-registering jars using pig? Any help is much
    appreciated.

    Thanks in advance!
    Chris
  • Chris Riccomini at Feb 5, 2010 at 7:00 pm
    What I want to do is register the jar WITHOUT register. That is, when I
    issue my pig commandline call to start pig, it will register it at that
    point rather than in my pig script.

    This will save our users from all having to register 3 or 4 jars that they
    always will need to use pig effectively on our Hadoop cluster.

    Thanks,
    Chris

    On 2/5/10 10:46 AM, "Ashutosh Chauhan" wrote:

    'register' is your friend here

    http://hadoop.apache.org/pig/docs/r0.5.0/piglatin_reference.html#REGISTER

    Ashutosh
    On Thu, Feb 4, 2010 at 23:25, Chris Riccomini wrote:
    Hi All,

    We have a use-case where we want to automatically register certain jars for
    command-line users. I tried using ­jar, but this switch seems to do
    absolutely nothing.

    How do we go about auto-registering jars using pig? Any help is much
    appreciated.

    Thanks in advance!
    Chris
  • Zaki Rahaman at Feb 5, 2010 at 7:37 pm
    I've never tried this but putting te jars on your pig classpath should
    save you from having to register as I believe pig looks here for jars
    by default

    Sent from my iPhone
    On Feb 5, 2010, at 1:58 PM, Chris Riccomini wrote:

    What I want to do is register the jar WITHOUT register. That is,
    when I
    issue my pig commandline call to start pig, it will register it at
    that
    point rather than in my pig script.

    This will save our users from all having to register 3 or 4 jars
    that they
    always will need to use pig effectively on our Hadoop cluster.

    Thanks,
    Chris

    On 2/5/10 10:46 AM, "Ashutosh Chauhan" wrote:

    'register' is your friend here

    http://hadoop.apache.org/pig/docs/r0.5.0/piglatin_reference.html#REGISTER

    Ashutosh

    On Thu, Feb 4, 2010 at 23:25, Chris Riccomini <criccomini@linkedin.com
    wrote:
    Hi All,

    We have a use-case where we want to automatically register certain
    jars for
    command-line users. I tried using –jar, but this switch seems t
    o do
    absolutely nothing.

    How do we go about auto-registering jars using pig? Any help is much
    appreciated.

    Thanks in advance!
    Chris
  • Alan Gates at Feb 5, 2010 at 7:47 pm
    Putting the jars on your classpath works as long as the classes you
    need are directly referenced in your script. So:

    B = foreach A generate com.mycompany.myudf($0);

    If myudf is in a jar somewhere in your classpath then it will be
    picked up. If myudf depends on the class myudfsupport which is also
    in a jar on your classpath, myudfsupport will not be picked up. It
    must be explicitly registered.

    Alan.
    On Feb 5, 2010, at 11:36 AM, Zaki Rahaman wrote:

    I've never tried this but putting te jars on your pig classpath
    should save you from having to register as I believe pig looks here
    for jars by default

    Sent from my iPhone

    On Feb 5, 2010, at 1:58 PM, Chris Riccomini
    wrote:
    What I want to do is register the jar WITHOUT register. That is,
    when I
    issue my pig commandline call to start pig, it will register it at
    that
    point rather than in my pig script.

    This will save our users from all having to register 3 or 4 jars
    that they
    always will need to use pig effectively on our Hadoop cluster.

    Thanks,
    Chris


    On 2/5/10 10:46 AM, "Ashutosh Chauhan" <ashutosh.chauhan@gmail.com>
    wrote:
    'register' is your friend here

    http://hadoop.apache.org/pig/docs/r0.5.0/piglatin_reference.html#REGISTER

    Ashutosh

    On Thu, Feb 4, 2010 at 23:25, Chris Riccomini <criccomini@linkedin.com
    wrote:
    Hi All,

    We have a use-case where we want to automatically register
    certain jars for
    command-line users. I tried using –jar, but this switch seems to do
    absolutely nothing.

    How do we go about auto-registering jars using pig? Any help is
    much
    appreciated.

    Thanks in advance!
    Chris
  • Zaki rahaman at Feb 5, 2010 at 8:09 pm
    Alan,

    Thanks for the clarification. This thread raises an important issue. I think
    it would be worthwhile to be able to set up some mechanism for setting up a
    "script environment." Maybe an option to be able to register jars on the
    command line or maybe some way of including/requiring dummy scripts (with
    register commands) as part of other scripts. Thoughts?
    On Fri, Feb 5, 2010 at 2:46 PM, Alan Gates wrote:

    Putting the jars on your classpath works as long as the classes you need
    are directly referenced in your script. So:

    B = foreach A generate com.mycompany.myudf($0);

    If myudf is in a jar somewhere in your classpath then it will be picked up.
    If myudf depends on the class myudfsupport which is also in a jar on your
    classpath, myudfsupport will not be picked up. It must be explicitly
    registered.

    Alan.


    On Feb 5, 2010, at 11:36 AM, Zaki Rahaman wrote:

    I've never tried this but putting te jars on your pig classpath should
    save you from having to register as I believe pig looks here for jars by
    default

    Sent from my iPhone

    On Feb 5, 2010, at 1:58 PM, Chris Riccomini <criccomini@linkedin.com>
    wrote:

    What I want to do is register the jar WITHOUT register. That is, when I
    issue my pig commandline call to start pig, it will register it at that
    point rather than in my pig script.

    This will save our users from all having to register 3 or 4 jars that
    they
    always will need to use pig effectively on our Hadoop cluster.

    Thanks,
    Chris


    On 2/5/10 10:46 AM, "Ashutosh Chauhan" <ashutosh.chauhan@gmail.com>
    wrote:

    'register' is your friend here

    http://hadoop.apache.org/pig/docs/r0.5.0/piglatin_reference.html#REGISTER

    Ashutosh

    On Thu, Feb 4, 2010 at 23:25, Chris Riccomini <criccomini@linkedin.com>
    wrote:
    Hi All,

    We have a use-case where we want to automatically register certain jars
    for
    command-line users. I tried using –jar, but this switch seems to do
    absolutely nothing.

    How do we go about auto-registering jars using pig? Any help is much
    appreciated.

    Thanks in advance!
    Chris

    --
    Zaki Rahaman
  • Chris Riccomini at Feb 5, 2010 at 9:31 pm
    I would love to be able to register jars when starting the commandline. This
    is exactly what I need. I think we just need to create a -r switch in Main
    that calls pigContext.addJar(). Is this right?

    Thanks,
    Chris

    On 2/5/10 12:09 PM, "zaki rahaman" wrote:

    Alan,

    Thanks for the clarification. This thread raises an important issue. I think
    it would be worthwhile to be able to set up some mechanism for setting up a
    "script environment." Maybe an option to be able to register jars on the
    command line or maybe some way of including/requiring dummy scripts (with
    register commands) as part of other scripts. Thoughts?
    On Fri, Feb 5, 2010 at 2:46 PM, Alan Gates wrote:

    Putting the jars on your classpath works as long as the classes you need
    are directly referenced in your script. So:

    B = foreach A generate com.mycompany.myudf($0);

    If myudf is in a jar somewhere in your classpath then it will be picked up.
    If myudf depends on the class myudfsupport which is also in a jar on your
    classpath, myudfsupport will not be picked up. It must be explicitly
    registered.

    Alan.


    On Feb 5, 2010, at 11:36 AM, Zaki Rahaman wrote:

    I've never tried this but putting te jars on your pig classpath should
    save you from having to register as I believe pig looks here for jars by
    default

    Sent from my iPhone

    On Feb 5, 2010, at 1:58 PM, Chris Riccomini <criccomini@linkedin.com>
    wrote:

    What I want to do is register the jar WITHOUT register. That is, when I
    issue my pig commandline call to start pig, it will register it at that
    point rather than in my pig script.

    This will save our users from all having to register 3 or 4 jars that
    they
    always will need to use pig effectively on our Hadoop cluster.

    Thanks,
    Chris


    On 2/5/10 10:46 AM, "Ashutosh Chauhan" <ashutosh.chauhan@gmail.com>
    wrote:

    'register' is your friend here

    http://hadoop.apache.org/pig/docs/r0.5.0/piglatin_reference.html#REGISTER

    Ashutosh

    On Thu, Feb 4, 2010 at 23:25, Chris Riccomini <criccomini@linkedin.com>
    wrote:
    Hi All,

    We have a use-case where we want to automatically register certain jars
    for
    command-line users. I tried using ­jar, but this switch seems to do
    absolutely nothing.

    How do we go about auto-registering jars using pig? Any help is much
    appreciated.

    Thanks in advance!
    Chris
  • Alan Gates at Feb 5, 2010 at 10:15 pm
    Being able to add jars from the command line seems like a useful thing
    to be able to do. I've created https://issues.apache.org/jira/browse/PIG-1226
    to track this.

    Alan.
    On Feb 5, 2010, at 1:29 PM, Chris Riccomini wrote:

    I would love to be able to register jars when starting the
    commandline. This
    is exactly what I need. I think we just need to create a -r switch
    in Main
    that calls pigContext.addJar(). Is this right?

    Thanks,
    Chris

    On 2/5/10 12:09 PM, "zaki rahaman" wrote:

    Alan,

    Thanks for the clarification. This thread raises an important
    issue. I think
    it would be worthwhile to be able to set up some mechanism for
    setting up a
    "script environment." Maybe an option to be able to register jars
    on the
    command line or maybe some way of including/requiring dummy scripts
    (with
    register commands) as part of other scripts. Thoughts?

    On Fri, Feb 5, 2010 at 2:46 PM, Alan Gates <gates@yahoo-inc.com>
    wrote:
    Putting the jars on your classpath works as long as the classes
    you need
    are directly referenced in your script. So:

    B = foreach A generate com.mycompany.myudf($0);

    If myudf is in a jar somewhere in your classpath then it will be
    picked up.
    If myudf depends on the class myudfsupport which is also in a jar
    on your
    classpath, myudfsupport will not be picked up. It must be
    explicitly
    registered.

    Alan.


    On Feb 5, 2010, at 11:36 AM, Zaki Rahaman wrote:

    I've never tried this but putting te jars on your pig classpath
    should
    save you from having to register as I believe pig looks here for
    jars by
    default

    Sent from my iPhone

    On Feb 5, 2010, at 1:58 PM, Chris Riccomini <criccomini@linkedin.com
    wrote:

    What I want to do is register the jar WITHOUT register. That is,
    when I
    issue my pig commandline call to start pig, it will register it
    at that
    point rather than in my pig script.

    This will save our users from all having to register 3 or 4 jars
    that
    they
    always will need to use pig effectively on our Hadoop cluster.

    Thanks,
    Chris


    On 2/5/10 10:46 AM, "Ashutosh Chauhan"
    <ashutosh.chauhan@gmail.com>
    wrote:

    'register' is your friend here

    http://hadoop.apache.org/pig/docs/r0.5.0/piglatin_reference.html#REGISTER

    Ashutosh

    On Thu, Feb 4, 2010 at 23:25, Chris Riccomini <criccomini@linkedin.com
    wrote:
    Hi All,

    We have a use-case where we want to automatically register
    certain jars
    for
    command-line users. I tried using –jar, but this switch seems
    to do
    absolutely nothing.

    How do we go about auto-registering jars using pig? Any help
    is much
    appreciated.

    Thanks in advance!
    Chris

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedFeb 5, '10 at 7:27a
activeFeb 5, '10 at 10:15p
posts8
users4
websitepig.apache.org

People

Translate

site design / logo © 2022 Grokbase