Grokbase Groups Pig user April 2013
FAQ
Hi ,

I need a way to invoke pig script from a java program and capture the output returned by the pig script. I was looking at the PigRunner api, however did not get much examples.
Is there any way how this can be achieved?Any working examples?

Thanks,
Sid

DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.

Search Discussions

  • Prashant Kommireddi at Apr 23, 2013 at 6:58 am
    Have you looked at the PigServer API? Running scripts should be pretty
    straightforward using it. Let us know if you face any issues.

    http://pig.apache.org/docs/r0.11.1/api/org/apache/pig/PigServer.html#registerScript%28java.lang.String%29

    On Mon, Apr 22, 2013 at 11:50 PM, Siddhi Borkar wrote:

    Hi ,

    I need a way to invoke pig script from a java program and capture the
    output returned by the pig script. I was looking at the PigRunner api,
    however did not get much examples.
    Is there any way how this can be achieved?Any working examples?

    Thanks,
    Sid

    DISCLAIMER
    ==========
    This e-mail may contain privileged and confidential information which is
    the property of Persistent Systems Ltd. It is intended only for the use of
    the individual or entity to which it is addressed. If you are not the
    intended recipient, you are not authorized to read, retain, copy, print,
    distribute or use this message. If you have received this communication in
    error, please notify the sender and delete all copies of this message.
    Persistent Systems Ltd. does not accept any liability for virus infected
    mails.
  • Siddhi Borkar at Apr 23, 2013 at 8:18 am
    Any idea what is the difference between Pigrunner and pigserver?

    -----Original Message-----
    From: Prashant Kommireddi
    Sent: Tuesday, April 23, 2013 12:28 PM
    To: user@pig.apache.org
    Subject: Re: Invoking pig scripts from java

    Have you looked at the PigServer API? Running scripts should be pretty straightforward using it. Let us know if you face any issues.

    http://pig.apache.org/docs/r0.11.1/api/org/apache/pig/PigServer.html#registerScript%28java.lang.String%29

    On Mon, Apr 22, 2013 at 11:50 PM, Siddhi Borkar wrote:

    Hi ,

    I need a way to invoke pig script from a java program and capture the
    output returned by the pig script. I was looking at the PigRunner api,
    however did not get much examples.
    Is there any way how this can be achieved?Any working examples?

    Thanks,
    Sid

    DISCLAIMER
    ==========
    This e-mail may contain privileged and confidential information which
    is the property of Persistent Systems Ltd. It is intended only for the
    use of the individual or entity to which it is addressed. If you are
    not the intended recipient, you are not authorized to read, retain,
    copy, print, distribute or use this message. If you have received this
    communication in error, please notify the sender and delete all copies of this message.
    Persistent Systems Ltd. does not accept any liability for virus
    infected mails.
    DISCLAIMER
    ==========
    This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.
  • Prashant Kommireddi at Apr 23, 2013 at 8:34 am
    Both are used to run pig scripts. PigRunner uses a more direct way in using
    the exact grunt code path to execute a script.

    PigServer on the other hand provides a bunch of helper methods that could
    be useful (for eg, setting a job name, deleting/renaming files, job
    priority, running individual queries etc). So you don't have to set them as
    properties within your script.

    You would use PigRunner exactly the same way as running pig via CLI passing
    the args to suggest pig script name etc, additionally it takes in
    PigProgressNotificationListener as an argument if one is interested in
    providing custom implementations of it.

    On Tue, Apr 23, 2013 at 1:17 AM, Siddhi Borkar wrote:

    Any idea what is the difference between Pigrunner and pigserver?

    -----Original Message-----
    From: Prashant Kommireddi
    Sent: Tuesday, April 23, 2013 12:28 PM
    To: user@pig.apache.org
    Subject: Re: Invoking pig scripts from java

    Have you looked at the PigServer API? Running scripts should be pretty
    straightforward using it. Let us know if you face any issues.


    http://pig.apache.org/docs/r0.11.1/api/org/apache/pig/PigServer.html#registerScript%28java.lang.String%29


    On Mon, Apr 22, 2013 at 11:50 PM, Siddhi Borkar <
    siddhi_borkar@persistent.co.in> wrote:
    Hi ,

    I need a way to invoke pig script from a java program and capture the
    output returned by the pig script. I was looking at the PigRunner api,
    however did not get much examples.
    Is there any way how this can be achieved?Any working examples?

    Thanks,
    Sid

    DISCLAIMER
    ==========
    This e-mail may contain privileged and confidential information which
    is the property of Persistent Systems Ltd. It is intended only for the
    use of the individual or entity to which it is addressed. If you are
    not the intended recipient, you are not authorized to read, retain,
    copy, print, distribute or use this message. If you have received this
    communication in error, please notify the sender and delete all copies
    of this message.
    Persistent Systems Ltd. does not accept any liability for virus
    infected mails.
    DISCLAIMER
    ==========
    This e-mail may contain privileged and confidential information which is
    the property of Persistent Systems Ltd. It is intended only for the use of
    the individual or entity to which it is addressed. If you are not the
    intended recipient, you are not authorized to read, retain, copy, print,
    distribute or use this message. If you have received this communication in
    error, please notify the sender and delete all copies of this message.
    Persistent Systems Ltd. does not accept any liability for virus infected
    mails.
  • Siddhi Borkar at Apr 23, 2013 at 10:16 am
    Thanks a lot Prashant.
    Is it possible to redirect the output generated by the pig script to a file? I need to do this in the java code and not in the pig scripts. I have some customer provided pig scripts which I am not allowed to modify,
    Hence I need to do this in java. Is there any way of doing this?


    -----Original Message-----
    From: Prashant Kommireddi
    Sent: Tuesday, April 23, 2013 2:03 PM
    To: user@pig.apache.org
    Subject: Re: Invoking pig scripts from java

    Both are used to run pig scripts. PigRunner uses a more direct way in using the exact grunt code path to execute a script.

    PigServer on the other hand provides a bunch of helper methods that could be useful (for eg, setting a job name, deleting/renaming files, job priority, running individual queries etc). So you don't have to set them as properties within your script.

    You would use PigRunner exactly the same way as running pig via CLI passing the args to suggest pig script name etc, additionally it takes in PigProgressNotificationListener as an argument if one is interested in providing custom implementations of it.

    On Tue, Apr 23, 2013 at 1:17 AM, Siddhi Borkar wrote:

    Any idea what is the difference between Pigrunner and pigserver?

    -----Original Message-----
    From: Prashant Kommireddi
    Sent: Tuesday, April 23, 2013 12:28 PM
    To: user@pig.apache.org
    Subject: Re: Invoking pig scripts from java

    Have you looked at the PigServer API? Running scripts should be pretty
    straightforward using it. Let us know if you face any issues.


    http://pig.apache.org/docs/r0.11.1/api/org/apache/pig/PigServer.html#r
    egisterScript%28java.lang.String%29


    On Mon, Apr 22, 2013 at 11:50 PM, Siddhi Borkar <
    siddhi_borkar@persistent.co.in> wrote:
    Hi ,

    I need a way to invoke pig script from a java program and capture
    the output returned by the pig script. I was looking at the
    PigRunner api, however did not get much examples.
    Is there any way how this can be achieved?Any working examples?

    Thanks,
    Sid

    DISCLAIMER
    ==========
    This e-mail may contain privileged and confidential information
    which is the property of Persistent Systems Ltd. It is intended only
    for the use of the individual or entity to which it is addressed. If
    you are not the intended recipient, you are not authorized to read,
    retain, copy, print, distribute or use this message. If you have
    received this communication in error, please notify the sender and
    delete all copies
    of this message.
    Persistent Systems Ltd. does not accept any liability for virus
    infected mails.
    DISCLAIMER
    ==========
    This e-mail may contain privileged and confidential information which
    is the property of Persistent Systems Ltd. It is intended only for the
    use of the individual or entity to which it is addressed. If you are
    not the intended recipient, you are not authorized to read, retain,
    copy, print, distribute or use this message. If you have received this
    communication in error, please notify the sender and delete all copies of this message.
    Persistent Systems Ltd. does not accept any liability for virus
    infected mails.
    DISCLAIMER
    ==========
    This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.
  • Prashant Kommireddi at Apr 23, 2013 at 6:37 pm
    How do you plan on running the java process? You can redirect the output to
    a file?

    On Tue, Apr 23, 2013 at 3:15 AM, Siddhi Borkar wrote:

    Thanks a lot Prashant.
    Is it possible to redirect the output generated by the pig script to a
    file? I need to do this in the java code and not in the pig scripts. I have
    some customer provided pig scripts which I am not allowed to modify,
    Hence I need to do this in java. Is there any way of doing this?


    -----Original Message-----
    From: Prashant Kommireddi
    Sent: Tuesday, April 23, 2013 2:03 PM
    To: user@pig.apache.org
    Subject: Re: Invoking pig scripts from java

    Both are used to run pig scripts. PigRunner uses a more direct way in
    using the exact grunt code path to execute a script.

    PigServer on the other hand provides a bunch of helper methods that could
    be useful (for eg, setting a job name, deleting/renaming files, job
    priority, running individual queries etc). So you don't have to set them as
    properties within your script.

    You would use PigRunner exactly the same way as running pig via CLI
    passing the args to suggest pig script name etc, additionally it takes in
    PigProgressNotificationListener as an argument if one is interested in
    providing custom implementations of it.


    On Tue, Apr 23, 2013 at 1:17 AM, Siddhi Borkar <
    siddhi_borkar@persistent.co.in> wrote:
    Any idea what is the difference between Pigrunner and pigserver?

    -----Original Message-----
    From: Prashant Kommireddi
    Sent: Tuesday, April 23, 2013 12:28 PM
    To: user@pig.apache.org
    Subject: Re: Invoking pig scripts from java

    Have you looked at the PigServer API? Running scripts should be pretty
    straightforward using it. Let us know if you face any issues.


    http://pig.apache.org/docs/r0.11.1/api/org/apache/pig/PigServer.html#r
    egisterScript%28java.lang.String%29


    On Mon, Apr 22, 2013 at 11:50 PM, Siddhi Borkar <
    siddhi_borkar@persistent.co.in> wrote:
    Hi ,

    I need a way to invoke pig script from a java program and capture
    the output returned by the pig script. I was looking at the
    PigRunner api, however did not get much examples.
    Is there any way how this can be achieved?Any working examples?

    Thanks,
    Sid

    DISCLAIMER
    ==========
    This e-mail may contain privileged and confidential information
    which is the property of Persistent Systems Ltd. It is intended only
    for the use of the individual or entity to which it is addressed. If
    you are not the intended recipient, you are not authorized to read,
    retain, copy, print, distribute or use this message. If you have
    received this communication in error, please notify the sender and
    delete all copies
    of this message.
    Persistent Systems Ltd. does not accept any liability for virus
    infected mails.
    DISCLAIMER
    ==========
    This e-mail may contain privileged and confidential information which
    is the property of Persistent Systems Ltd. It is intended only for the
    use of the individual or entity to which it is addressed. If you are
    not the intended recipient, you are not authorized to read, retain,
    copy, print, distribute or use this message. If you have received this
    communication in error, please notify the sender and delete all copies
    of this message.
    Persistent Systems Ltd. does not accept any liability for virus
    infected mails.
    DISCLAIMER
    ==========
    This e-mail may contain privileged and confidential information which is
    the property of Persistent Systems Ltd. It is intended only for the use of
    the individual or entity to which it is addressed. If you are not the
    intended recipient, you are not authorized to read, retain, copy, print,
    distribute or use this message. If you have received this communication in
    error, please notify the sender and delete all copies of this message.
    Persistent Systems Ltd. does not accept any liability for virus infected
    mails.
  • Siddhi Borkar at Apr 24, 2013 at 4:34 am
    My java program is as follows

    PigServer pigServer = new PigServer(ExecType.LOCAL);
    pigServer.registerScript("processor.pig");

    I need to capture the output (the stack trace ) generated by the pig script and save it in a file. Is there any way to get this done?

    -----Original Message-----
    From: Prashant Kommireddi
    Sent: Wednesday, April 24, 2013 12:07 AM
    To: user@pig.apache.org
    Subject: Re: Invoking pig scripts from java

    How do you plan on running the java process? You can redirect the output to a file?

    On Tue, Apr 23, 2013 at 3:15 AM, Siddhi Borkar wrote:

    Thanks a lot Prashant.
    Is it possible to redirect the output generated by the pig script to a
    file? I need to do this in the java code and not in the pig scripts. I
    have some customer provided pig scripts which I am not allowed to
    modify, Hence I need to do this in java. Is there any way of doing this?


    -----Original Message-----
    From: Prashant Kommireddi
    Sent: Tuesday, April 23, 2013 2:03 PM
    To: user@pig.apache.org
    Subject: Re: Invoking pig scripts from java

    Both are used to run pig scripts. PigRunner uses a more direct way in
    using the exact grunt code path to execute a script.

    PigServer on the other hand provides a bunch of helper methods that
    could be useful (for eg, setting a job name, deleting/renaming files,
    job priority, running individual queries etc). So you don't have to
    set them as properties within your script.

    You would use PigRunner exactly the same way as running pig via CLI
    passing the args to suggest pig script name etc, additionally it takes
    in PigProgressNotificationListener as an argument if one is interested
    in providing custom implementations of it.


    On Tue, Apr 23, 2013 at 1:17 AM, Siddhi Borkar <
    siddhi_borkar@persistent.co.in> wrote:
    Any idea what is the difference between Pigrunner and pigserver?

    -----Original Message-----
    From: Prashant Kommireddi
    Sent: Tuesday, April 23, 2013 12:28 PM
    To: user@pig.apache.org
    Subject: Re: Invoking pig scripts from java

    Have you looked at the PigServer API? Running scripts should be
    pretty straightforward using it. Let us know if you face any issues.


    http://pig.apache.org/docs/r0.11.1/api/org/apache/pig/PigServer.html
    #r
    egisterScript%28java.lang.String%29


    On Mon, Apr 22, 2013 at 11:50 PM, Siddhi Borkar <
    siddhi_borkar@persistent.co.in> wrote:
    Hi ,

    I need a way to invoke pig script from a java program and capture
    the output returned by the pig script. I was looking at the
    PigRunner api, however did not get much examples.
    Is there any way how this can be achieved?Any working examples?

    Thanks,
    Sid

    DISCLAIMER
    ==========
    This e-mail may contain privileged and confidential information
    which is the property of Persistent Systems Ltd. It is intended
    only for the use of the individual or entity to which it is
    addressed. If you are not the intended recipient, you are not
    authorized to read, retain, copy, print, distribute or use this
    message. If you have received this communication in error, please
    notify the sender and delete all copies
    of this message.
    Persistent Systems Ltd. does not accept any liability for virus
    infected mails.
    DISCLAIMER
    ==========
    This e-mail may contain privileged and confidential information
    which is the property of Persistent Systems Ltd. It is intended only
    for the use of the individual or entity to which it is addressed. If
    you are not the intended recipient, you are not authorized to read,
    retain, copy, print, distribute or use this message. If you have
    received this communication in error, please notify the sender and
    delete all copies
    of this message.
    Persistent Systems Ltd. does not accept any liability for virus
    infected mails.
    DISCLAIMER
    ==========
    This e-mail may contain privileged and confidential information which
    is the property of Persistent Systems Ltd. It is intended only for the
    use of the individual or entity to which it is addressed. If you are
    not the intended recipient, you are not authorized to read, retain,
    copy, print, distribute or use this message. If you have received this
    communication in error, please notify the sender and delete all copies of this message.
    Persistent Systems Ltd. does not accept any liability for virus
    infected mails.
    DISCLAIMER
    ==========
    This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedApr 23, '13 at 6:51a
activeApr 24, '13 at 4:34a
posts7
users2
websitepig.apache.org

People

Translate

site design / logo © 2021 Grokbase