Grokbase Groups Pig user May 2013
FAQ
Hi,

I am using PigTest in order to verify a script reading and storing data in
avro format.
However, at the moment, the script fails due to the optimisation rule
ColumnMapKeyPrune.
I known I can disable it using the -optimizer_off flag. But is there a way
to do that using PigTest?

It seems to me that the option is only for the command line, that there is
no way to provide it with PigTest and that there is no way to turn off the
rule from the script itself.

Could someone confirm that?

Regards

Bertrand

PS : I am using a Cloudera 0.10 pig version but the question is not
specific to Cloudera.

Search Discussions

  • Cheolsoo Park at May 13, 2013 at 5:29 pm
    Hi Bertand,

    I don't think there is a way to do it now.

    But PIG-3317 will allow you to disable optimization rules via Pig
    properties:
    https://issues.apache.org/jira/browse/PIG-3317

    So you will be able to set the properties in PigContext and pass it to
    PigServer.

    The patch is not committed yet, but it's likely to be in next release.

    Thanks,
    Cheolsoo


    On Mon, May 13, 2013 at 2:25 AM, Bertrand Dechoux wrote:

    Hi,

    I am using PigTest in order to verify a script reading and storing data in
    avro format.
    However, at the moment, the script fails due to the optimisation rule
    ColumnMapKeyPrune.
    I known I can disable it using the -optimizer_off flag. But is there a way
    to do that using PigTest?

    It seems to me that the option is only for the command line, that there is
    no way to provide it with PigTest and that there is no way to turn off the
    rule from the script itself.

    Could someone confirm that?

    Regards

    Bertrand

    PS : I am using a Cloudera 0.10 pig version but the question is not
    specific to Cloudera.
  • Bertrand Dechoux at May 13, 2013 at 5:50 pm
    Thanks for the information!

    Bertrand

    On Mon, May 13, 2013 at 6:27 PM, Cheolsoo Park wrote:

    Hi Bertand,

    I don't think there is a way to do it now.

    But PIG-3317 will allow you to disable optimization rules via Pig
    properties:
    https://issues.apache.org/jira/browse/PIG-3317

    So you will be able to set the properties in PigContext and pass it to
    PigServer.

    The patch is not committed yet, but it's likely to be in next release.

    Thanks,
    Cheolsoo



    On Mon, May 13, 2013 at 2:25 AM, Bertrand Dechoux <dechouxb@gmail.com
    wrote:
    Hi,

    I am using PigTest in order to verify a script reading and storing data in
    avro format.
    However, at the moment, the script fails due to the optimisation rule
    ColumnMapKeyPrune.
    I known I can disable it using the -optimizer_off flag. But is there a way
    to do that using PigTest?

    It seems to me that the option is only for the command line, that there is
    no way to provide it with PigTest and that there is no way to turn off the
    rule from the script itself.

    Could someone confirm that?

    Regards

    Bertrand

    PS : I am using a Cloudera 0.10 pig version but the question is not
    specific to Cloudera.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedMay 13, '13 at 9:25a
activeMay 13, '13 at 5:50p
posts3
users2
websitepig.apache.org

2 users in discussion

Bertrand Dechoux: 2 posts Cheolsoo Park: 1 post

People

Translate

site design / logo © 2021 Grokbase