Grokbase Groups Pig dev August 2010
FAQ
Code discovering UDFs in the script has a bug in a order by case
----------------------------------------------------------------

Key: PIG-1534
URL: https://issues.apache.org/jira/browse/PIG-1534
Project: Pig
Issue Type: Bug
Affects Versions: 0.7.0
Reporter: Pradeep Kamath
Fix For: 0.8.0


Consider the following commandline:
{noformat}
java -cp /tmp/svncheckout/pig.jar:udf.jar:clusterdir org.apache.pig.Main -e "a = load 'studenttab' using udf.MyPigStorage(); b = order a by $0; dump b;"
{noformat}

Notice there is no "register udf.jar", instead udf.jar (which contains udf.MyPigStorage) is in the classpath. Pig handles this case by shipping udf.jar to the backend. However the above script with order by triggers the bug with the following error message:
ERROR 2997: Unable to recreate exception from backed error: java.lang.RuntimeException: could not instantiate 'org.apache.pig.impl.builtin.RandomSampleLoader' with arguments '[udf.MyPigStorage, 100]'


--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Pradeep Kamath (JIRA) at Aug 4, 2010 at 7:13 pm
    [ https://issues.apache.org/jira/browse/PIG-1534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pradeep Kamath updated PIG-1534:
    --------------------------------

    Status: Patch Available (was: Open)
    Assignee: Pradeep Kamath
    Code discovering UDFs in the script has a bug in a order by case
    ----------------------------------------------------------------

    Key: PIG-1534
    URL: https://issues.apache.org/jira/browse/PIG-1534
    Project: Pig
    Issue Type: Bug
    Affects Versions: 0.7.0
    Reporter: Pradeep Kamath
    Assignee: Pradeep Kamath
    Fix For: 0.8.0

    Attachments: PIG-1534.patch


    Consider the following commandline:
    {noformat}
    java -cp /tmp/svncheckout/pig.jar:udf.jar:clusterdir org.apache.pig.Main -e "a = load 'studenttab' using udf.MyPigStorage(); b = order a by $0; dump b;"
    {noformat}
    Notice there is no "register udf.jar", instead udf.jar (which contains udf.MyPigStorage) is in the classpath. Pig handles this case by shipping udf.jar to the backend. However the above script with order by triggers the bug with the following error message:
    ERROR 2997: Unable to recreate exception from backed error: java.lang.RuntimeException: could not instantiate 'org.apache.pig.impl.builtin.RandomSampleLoader' with arguments '[udf.MyPigStorage, 100]'
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Pradeep Kamath (JIRA) at Aug 4, 2010 at 7:13 pm
    [ https://issues.apache.org/jira/browse/PIG-1534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pradeep Kamath updated PIG-1534:
    --------------------------------

    Attachment: PIG-1534.patch

    Patch fixes SampleOptimizer to add the loadFunc funcspecs into the Mapreduce operators after optimization - this fixes the above order by error.

    Here are results from running the test-patch target locally
    [exec] -1 overall.
    [exec]
    [exec] +1 @author. The patch does not contain any @author tags.
    [exec]
    [exec] +1 tests included. The patch appears to include 3 new or modified tests.
    [exec]
    [exec] -1 javadoc. The javadoc tool appears to have generated 1 warning messages.
    [exec]
    [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
    [exec]
    [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
    [exec]
    [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
    [exec]

    The javadoc warning is present on trunk and not related to this patch:
    {noformat}
    ...
    [javadoc] Standard Doclet version 1.6.0_01
    [javadoc] Building tree for all the packages and classes...
    [javadoc] /tmp/svncheckout/src/org/apache/pig/newplan/logical/expression/ProjectExpression.java:192: warning - @param argument "currentOp" is not a parameter name.
    [javadoc] Building index for all the packages and classes...
    ...
    {noformat}
    Will run unit tests locally and update with results.
    Code discovering UDFs in the script has a bug in a order by case
    ----------------------------------------------------------------

    Key: PIG-1534
    URL: https://issues.apache.org/jira/browse/PIG-1534
    Project: Pig
    Issue Type: Bug
    Affects Versions: 0.7.0
    Reporter: Pradeep Kamath
    Fix For: 0.8.0

    Attachments: PIG-1534.patch


    Consider the following commandline:
    {noformat}
    java -cp /tmp/svncheckout/pig.jar:udf.jar:clusterdir org.apache.pig.Main -e "a = load 'studenttab' using udf.MyPigStorage(); b = order a by $0; dump b;"
    {noformat}
    Notice there is no "register udf.jar", instead udf.jar (which contains udf.MyPigStorage) is in the classpath. Pig handles this case by shipping udf.jar to the backend. However the above script with order by triggers the bug with the following error message:
    ERROR 2997: Unable to recreate exception from backed error: java.lang.RuntimeException: could not instantiate 'org.apache.pig.impl.builtin.RandomSampleLoader' with arguments '[udf.MyPigStorage, 100]'
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Pradeep Kamath (JIRA) at Aug 5, 2010 at 2:45 am
    [ https://issues.apache.org/jira/browse/PIG-1534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12895522#action_12895522 ]

    Pradeep Kamath commented on PIG-1534:
    -------------------------------------

    Ran all unit tests - TestScriptUDF fails but the failure is unrelated to the change in this patch and the failure occurs even with a fresh svn checkout.

    Patch is ready for review.
    Code discovering UDFs in the script has a bug in a order by case
    ----------------------------------------------------------------

    Key: PIG-1534
    URL: https://issues.apache.org/jira/browse/PIG-1534
    Project: Pig
    Issue Type: Bug
    Affects Versions: 0.7.0
    Reporter: Pradeep Kamath
    Assignee: Pradeep Kamath
    Fix For: 0.8.0

    Attachments: PIG-1534.patch


    Consider the following commandline:
    {noformat}
    java -cp /tmp/svncheckout/pig.jar:udf.jar:clusterdir org.apache.pig.Main -e "a = load 'studenttab' using udf.MyPigStorage(); b = order a by $0; dump b;"
    {noformat}
    Notice there is no "register udf.jar", instead udf.jar (which contains udf.MyPigStorage) is in the classpath. Pig handles this case by shipping udf.jar to the backend. However the above script with order by triggers the bug with the following error message:
    ERROR 2997: Unable to recreate exception from backed error: java.lang.RuntimeException: could not instantiate 'org.apache.pig.impl.builtin.RandomSampleLoader' with arguments '[udf.MyPigStorage, 100]'
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Daniel Dai (JIRA) at Aug 5, 2010 at 5:21 pm
    [ https://issues.apache.org/jira/browse/PIG-1534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12895742#action_12895742 ]

    Daniel Dai commented on PIG-1534:
    ---------------------------------

    +1. Verified that also solve skewed join case.
    Code discovering UDFs in the script has a bug in a order by case
    ----------------------------------------------------------------

    Key: PIG-1534
    URL: https://issues.apache.org/jira/browse/PIG-1534
    Project: Pig
    Issue Type: Bug
    Affects Versions: 0.7.0
    Reporter: Pradeep Kamath
    Assignee: Pradeep Kamath
    Fix For: 0.8.0

    Attachments: PIG-1534.patch


    Consider the following commandline:
    {noformat}
    java -cp /tmp/svncheckout/pig.jar:udf.jar:clusterdir org.apache.pig.Main -e "a = load 'studenttab' using udf.MyPigStorage(); b = order a by $0; dump b;"
    {noformat}
    Notice there is no "register udf.jar", instead udf.jar (which contains udf.MyPigStorage) is in the classpath. Pig handles this case by shipping udf.jar to the backend. However the above script with order by triggers the bug with the following error message:
    ERROR 2997: Unable to recreate exception from backed error: java.lang.RuntimeException: could not instantiate 'org.apache.pig.impl.builtin.RandomSampleLoader' with arguments '[udf.MyPigStorage, 100]'
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Pradeep Kamath (JIRA) at Aug 5, 2010 at 7:26 pm
    [ https://issues.apache.org/jira/browse/PIG-1534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12895769#action_12895769 ]

    Pradeep Kamath commented on PIG-1534:
    -------------------------------------

    Thanks for the review Daniel, patch committed to trunk.
    Code discovering UDFs in the script has a bug in a order by case
    ----------------------------------------------------------------

    Key: PIG-1534
    URL: https://issues.apache.org/jira/browse/PIG-1534
    Project: Pig
    Issue Type: Bug
    Affects Versions: 0.7.0
    Reporter: Pradeep Kamath
    Assignee: Pradeep Kamath
    Fix For: 0.8.0

    Attachments: PIG-1534.patch


    Consider the following commandline:
    {noformat}
    java -cp /tmp/svncheckout/pig.jar:udf.jar:clusterdir org.apache.pig.Main -e "a = load 'studenttab' using udf.MyPigStorage(); b = order a by $0; dump b;"
    {noformat}
    Notice there is no "register udf.jar", instead udf.jar (which contains udf.MyPigStorage) is in the classpath. Pig handles this case by shipping udf.jar to the backend. However the above script with order by triggers the bug with the following error message:
    ERROR 2997: Unable to recreate exception from backed error: java.lang.RuntimeException: could not instantiate 'org.apache.pig.impl.builtin.RandomSampleLoader' with arguments '[udf.MyPigStorage, 100]'
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Pradeep Kamath (JIRA) at Aug 5, 2010 at 7:26 pm
    [ https://issues.apache.org/jira/browse/PIG-1534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pradeep Kamath updated PIG-1534:
    --------------------------------

    Status: Resolved (was: Patch Available)
    Hadoop Flags: [Reviewed]
    Resolution: Fixed
    Code discovering UDFs in the script has a bug in a order by case
    ----------------------------------------------------------------

    Key: PIG-1534
    URL: https://issues.apache.org/jira/browse/PIG-1534
    Project: Pig
    Issue Type: Bug
    Affects Versions: 0.7.0
    Reporter: Pradeep Kamath
    Assignee: Pradeep Kamath
    Fix For: 0.8.0

    Attachments: PIG-1534.patch


    Consider the following commandline:
    {noformat}
    java -cp /tmp/svncheckout/pig.jar:udf.jar:clusterdir org.apache.pig.Main -e "a = load 'studenttab' using udf.MyPigStorage(); b = order a by $0; dump b;"
    {noformat}
    Notice there is no "register udf.jar", instead udf.jar (which contains udf.MyPigStorage) is in the classpath. Pig handles this case by shipping udf.jar to the backend. However the above script with order by triggers the bug with the following error message:
    ERROR 2997: Unable to recreate exception from backed error: java.lang.RuntimeException: could not instantiate 'org.apache.pig.impl.builtin.RandomSampleLoader' with arguments '[udf.MyPigStorage, 100]'
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categoriespig, hadoop
postedAug 4, '10 at 5:29p
activeAug 5, '10 at 7:26p
posts7
users1
websitepig.apache.org

1 user in discussion

Pradeep Kamath (JIRA): 7 posts

People

Translate

site design / logo © 2022 Grokbase