Grokbase Groups Pig user March 2011
FAQ
I want to do something really simple: I want to pass a string
into a Pig script. The string is either "stdout" or some target
file name. Say the string gets bound to $OUTPUT. That all
works fine.

After the script has computed a result R, depending on the
value of $OUTPUT, I want either to do a dump R (if
$OUTPUT == "stdout"), or a STORE of R into the given
file name that's held in $OUTPUT.

How do I do that conditional? I played with bincond, but
got stuck.

Thanks!

Andreas


Andreas

Search Discussions

  • Alan Gates at Mar 15, 2011 at 2:23 pm
    If you know before you start the script which you want, you can use
    parameter substitution:

    A = load 'foo';
    ...
    Z = foreach Y generate ...;
    $DO_OUTPUT

    Then, depending on which you want, run pig with

    pig -pDO_OUTPUT='dump Z;';

    or

    pig -pDO_OUTPUT="store Z into 'outfile';"

    If you want to decide which to do based on results of the script, then
    you'll need the new control flow integration features available in
    0.9. These aren't yet released, but they are in trunk.

    Alan.
    On Mar 14, 2011, at 10:40 PM, Andreas Paepcke wrote:

    I want to do something really simple: I want to pass a string
    into a Pig script. The string is either "stdout" or some target
    file name. Say the string gets bound to $OUTPUT. That all
    works fine.

    After the script has computed a result R, depending on the
    value of $OUTPUT, I want either to do a dump R (if
    $OUTPUT == "stdout"), or a STORE of R into the given
    file name that's held in $OUTPUT.

    How do I do that conditional? I played with bincond, but
    got stuck.

    Thanks!

    Andreas


    Andreas
  • Andreas Paepcke at Mar 15, 2011 at 3:20 pm
    Great, Alan. I do know ahead of time which
    output I want, so your solution will work. Didn't
    know you could bind Pig statements to variables
    and execute them.

    Nonetheless: looking forward to the control flow
    feature.

    Andreas

    On Tue, Mar 15, 2011 at 7:23 AM, Alan Gates wrote:

    If you know before you start the script which you want, you can use
    parameter substitution:

    A = load 'foo';
    ...
    Z = foreach Y generate ...;
    $DO_OUTPUT

    Then, depending on which you want, run pig with

    pig -pDO_OUTPUT='dump Z;';

    or

    pig -pDO_OUTPUT="store Z into 'outfile';"

    If you want to decide which to do based on results of the script, then
    you'll need the new control flow integration features available in 0.9.
    These aren't yet released, but they are in trunk.

    Alan.


    On Mar 14, 2011, at 10:40 PM, Andreas Paepcke wrote:

    I want to do something really simple: I want to pass a string
    into a Pig script. The string is either "stdout" or some target
    file name. Say the string gets bound to $OUTPUT. That all
    works fine.

    After the script has computed a result R, depending on the
    value of $OUTPUT, I want either to do a dump R (if
    $OUTPUT == "stdout"), or a STORE of R into the given
    file name that's held in $OUTPUT.

    How do I do that conditional? I played with bincond, but
    got stuck.

    Thanks!

    Andreas


    Andreas
  • Andreas Paepcke at Apr 19, 2011 at 1:26 pm
    I'm still struggling with parameter substitution.
    Below are six examples. Two work, the others don't.
    When they don't, I get this error message:

    ERROR org.apache.pig.Main - ERROR 2999: Unexpected internal error.
    Encountered unexpected arguments on command line - please check the
    command line.

    $ pig -param COLOR=blue script.pig -- Works
    $ pig -param COLOR="blue-green" script.pig -- Works
    $ pig -param COLOR="blue green" script.pig -- Fails
    $ pig -param COLOR="'blue green'" script.pig -- Fails
    $ pig -param COLOR="\'blue green\'" script.pig -- Fails
    $ pig -param COLOR="blue\ green" script.pig -- Fails

    How do I protect spaces so that I can pass multi-word
    parameters into Pig?

    Thanks,

    Andreas

    On Tue, Mar 15, 2011 at 7:23 AM, Alan Gates wrote:

    If you know before you start the script which you want, you can use
    parameter substitution:

    A = load 'foo';
    ...
    Z = foreach Y generate ...;
    $DO_OUTPUT

    Then, depending on which you want, run pig with

    pig -pDO_OUTPUT='dump Z;';

    or

    pig -pDO_OUTPUT="store Z into 'outfile';"

    If you want to decide which to do based on results of the script, then
    you'll need the new control flow integration features available in 0.9.
    These aren't yet released, but they are in trunk.

    Alan.


    On Mar 14, 2011, at 10:40 PM, Andreas Paepcke wrote:

    I want to do something really simple: I want to pass a string
    into a Pig script. The string is either "stdout" or some target
    file name. Say the string gets bound to $OUTPUT. That all
    works fine.

    After the script has computed a result R, depending on the
    value of $OUTPUT, I want either to do a dump R (if
    $OUTPUT == "stdout"), or a STORE of R into the given
    file name that's held in $OUTPUT.

    How do I do that conditional? I played with bincond, but
    got stuck.

    Thanks!

    Andreas


    Andreas
  • Romain Rigaux at Apr 20, 2011 at 7:29 pm
    For me in bash it works:

    -param COLOR="blue green"

    But you can try with \u0020 instead of space:

    -param COLOR="blue\u0020green"

    Romain
    On Mon, Apr 18, 2011 at 4:59 PM, Andreas Paepcke wrote:

    I'm still struggling with parameter substitution.
    Below are six examples. Two work, the others don't.
    When they don't, I get this error message:

    ERROR org.apache.pig.Main - ERROR 2999: Unexpected internal error.
    Encountered unexpected arguments on command line - please check the
    command line.

    $ pig -param COLOR=blue script.pig -- Works
    $ pig -param COLOR="blue-green" script.pig -- Works
    $ pig -param COLOR="blue green" script.pig -- Fails
    $ pig -param COLOR="'blue green'" script.pig -- Fails
    $ pig -param COLOR="\'blue green\'" script.pig -- Fails
    $ pig -param COLOR="blue\ green" script.pig -- Fails

    How do I protect spaces so that I can pass multi-word
    parameters into Pig?

    Thanks,

    Andreas

    On Tue, Mar 15, 2011 at 7:23 AM, Alan Gates wrote:

    If you know before you start the script which you want, you can use
    parameter substitution:

    A = load 'foo';
    ...
    Z = foreach Y generate ...;
    $DO_OUTPUT

    Then, depending on which you want, run pig with

    pig -pDO_OUTPUT='dump Z;';

    or

    pig -pDO_OUTPUT="store Z into 'outfile';"

    If you want to decide which to do based on results of the script, then
    you'll need the new control flow integration features available in 0.9.
    These aren't yet released, but they are in trunk.

    Alan.


    On Mar 14, 2011, at 10:40 PM, Andreas Paepcke wrote:

    I want to do something really simple: I want to pass a string
    into a Pig script. The string is either "stdout" or some target
    file name. Say the string gets bound to $OUTPUT. That all
    works fine.

    After the script has computed a result R, depending on the
    value of $OUTPUT, I want either to do a dump R (if
    $OUTPUT == "stdout"), or a STORE of R into the given
    file name that's held in $OUTPUT.

    How do I do that conditional? I played with bincond, but
    got stuck.

    Thanks!

    Andreas


    Andreas
  • Mridul Muralidharan at Apr 20, 2011 at 9:14 pm
    You could try using property file instead of cli param to pass the
    name/value ...


    Regards,
    Mridul
    On Tuesday 19 April 2011 05:29 AM, Andreas Paepcke wrote:
    I'm still struggling with parameter substitution.
    Below are six examples. Two work, the others don't.
    When they don't, I get this error message:

    ERROR org.apache.pig.Main - ERROR 2999: Unexpected internal error.
    Encountered unexpected arguments on command line - please check the
    command line.

    $ pig -param COLOR=blue script.pig -- Works
    $ pig -param COLOR="blue-green" script.pig -- Works
    $ pig -param COLOR="blue green" script.pig -- Fails
    $ pig -param COLOR="'blue green'" script.pig -- Fails
    $ pig -param COLOR="\'blue green\'" script.pig -- Fails
    $ pig -param COLOR="blue\ green" script.pig -- Fails

    How do I protect spaces so that I can pass multi-word
    parameters into Pig?

    Thanks,

    Andreas


    On Tue, Mar 15, 2011 at 7:23 AM, Alan Gateswrote:
    If you know before you start the script which you want, you can use
    parameter substitution:

    A = load 'foo';
    ...
    Z = foreach Y generate ...;
    $DO_OUTPUT

    Then, depending on which you want, run pig with

    pig -pDO_OUTPUT='dump Z;';

    or

    pig -pDO_OUTPUT="store Z into 'outfile';"

    If you want to decide which to do based on results of the script, then
    you'll need the new control flow integration features available in 0.9.
    These aren't yet released, but they are in trunk.

    Alan.


    On Mar 14, 2011, at 10:40 PM, Andreas Paepcke wrote:

    I want to do something really simple: I want to pass a string
    into a Pig script. The string is either "stdout" or some target
    file name. Say the string gets bound to $OUTPUT. That all
    works fine.

    After the script has computed a result R, depending on the
    value of $OUTPUT, I want either to do a dump R (if
    $OUTPUT == "stdout"), or a STORE of R into the given
    file name that's held in $OUTPUT.

    How do I do that conditional? I played with bincond, but
    got stuck.

    Thanks!

    Andreas


    Andreas

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedMar 15, '11 at 5:41a
activeApr 20, '11 at 9:14p
posts6
users4
websitepig.apache.org

People

Translate

site design / logo © 2021 Grokbase