Grokbase Groups Pig dev August 2008
FAQ
limit interfers with types in filter
------------------------------------

Key: PIG-399
URL: https://issues.apache.org/jira/browse/PIG-399
Project: Pig
Issue Type: Bug
Reporter: Olga Natkovich


The following script fails:

A = load 'data' as (name: chararray, age: chararray, gpa: float);
B = limit A 40;
C = filter B by age eq '40';
D = group C by name;
E = foreach D generate group, COUNT(C);
dump E;

If limit is removed, the script runs without any problems. The explain shows that filter looses type of the non-constant operand.

The errror is

8/08/25 13:34:15 INFO mapReduceLayer.MapReduceLauncher: Unsuccessful attempt. Completed 0.0% of the job
08/08/25 13:34:15 ERROR mapReduceLayer.Launcher: Error message from task (reduce) tip_200808131548_2163_r_000000java.lang.ClassCastException: java.lang.String cannot be cast to org.apache.pig.data.DataByteArray
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.ConstantExpression.getNext(ConstantExpression.java:111)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.EqualToExpr.getNext(EqualToExpr.java:68)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POFilter.getNext(POFilter.java:147)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:169)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:87)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:391)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)




--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Olga Natkovich (JIRA) at Aug 25, 2008 at 8:36 pm
    [ https://issues.apache.org/jira/browse/PIG-399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Olga Natkovich reassigned PIG-399:
    ----------------------------------

    Assignee: Alan Gates
    limit interfers with types in filter
    ------------------------------------

    Key: PIG-399
    URL: https://issues.apache.org/jira/browse/PIG-399
    Project: Pig
    Issue Type: Bug
    Reporter: Olga Natkovich
    Assignee: Alan Gates

    The following script fails:
    A = load 'data' as (name: chararray, age: chararray, gpa: float);
    B = limit A 40;
    C = filter B by age eq '40';
    D = group C by name;
    E = foreach D generate group, COUNT(C);
    dump E;
    If limit is removed, the script runs without any problems. The explain shows that filter looses type of the non-constant operand.
    The errror is
    8/08/25 13:34:15 INFO mapReduceLayer.MapReduceLauncher: Unsuccessful attempt. Completed 0.0% of the job
    08/08/25 13:34:15 ERROR mapReduceLayer.Launcher: Error message from task (reduce) tip_200808131548_2163_r_000000java.lang.ClassCastException: java.lang.String cannot be cast to org.apache.pig.data.DataByteArray
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.ConstantExpression.getNext(ConstantExpression.java:111)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.EqualToExpr.getNext(EqualToExpr.java:68)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POFilter.getNext(POFilter.java:147)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:169)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:87)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:391)
    at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Alan Gates (JIRA) at Aug 27, 2008 at 11:58 pm
    [ https://issues.apache.org/jira/browse/PIG-399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Alan Gates updated PIG-399:
    ---------------------------

    Attachment: PIG-399.patch

    Added limit to visitors in the optimizer so that it would be properly handled when we're rearranging the tree.
    limit interfers with types in filter
    ------------------------------------

    Key: PIG-399
    URL: https://issues.apache.org/jira/browse/PIG-399
    Project: Pig
    Issue Type: Bug
    Reporter: Olga Natkovich
    Assignee: Alan Gates
    Attachments: PIG-399.patch


    The following script fails:
    A = load 'data' as (name: chararray, age: chararray, gpa: float);
    B = limit A 40;
    C = filter B by age eq '40';
    D = group C by name;
    E = foreach D generate group, COUNT(C);
    dump E;
    If limit is removed, the script runs without any problems. The explain shows that filter looses type of the non-constant operand.
    The errror is
    8/08/25 13:34:15 INFO mapReduceLayer.MapReduceLauncher: Unsuccessful attempt. Completed 0.0% of the job
    08/08/25 13:34:15 ERROR mapReduceLayer.Launcher: Error message from task (reduce) tip_200808131548_2163_r_000000java.lang.ClassCastException: java.lang.String cannot be cast to org.apache.pig.data.DataByteArray
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.ConstantExpression.getNext(ConstantExpression.java:111)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.EqualToExpr.getNext(EqualToExpr.java:68)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POFilter.getNext(POFilter.java:147)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:169)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:87)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:391)
    at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Alan Gates (JIRA) at Aug 28, 2008 at 12:00 am
    [ https://issues.apache.org/jira/browse/PIG-399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Alan Gates updated PIG-399:
    ---------------------------

    Status: Patch Available (was: Open)
    limit interfers with types in filter
    ------------------------------------

    Key: PIG-399
    URL: https://issues.apache.org/jira/browse/PIG-399
    Project: Pig
    Issue Type: Bug
    Reporter: Olga Natkovich
    Assignee: Alan Gates
    Attachments: PIG-399.patch


    The following script fails:
    A = load 'data' as (name: chararray, age: chararray, gpa: float);
    B = limit A 40;
    C = filter B by age eq '40';
    D = group C by name;
    E = foreach D generate group, COUNT(C);
    dump E;
    If limit is removed, the script runs without any problems. The explain shows that filter looses type of the non-constant operand.
    The errror is
    8/08/25 13:34:15 INFO mapReduceLayer.MapReduceLauncher: Unsuccessful attempt. Completed 0.0% of the job
    08/08/25 13:34:15 ERROR mapReduceLayer.Launcher: Error message from task (reduce) tip_200808131548_2163_r_000000java.lang.ClassCastException: java.lang.String cannot be cast to org.apache.pig.data.DataByteArray
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.ConstantExpression.getNext(ConstantExpression.java:111)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.EqualToExpr.getNext(EqualToExpr.java:68)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POFilter.getNext(POFilter.java:147)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:169)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:87)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:391)
    at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Alan Gates (JIRA) at Aug 28, 2008 at 12:12 am
    [ https://issues.apache.org/jira/browse/PIG-399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Alan Gates updated PIG-399:
    ---------------------------

    Resolution: Fixed
    Status: Resolved (was: Patch Available)

    Patch checked in.
    limit interfers with types in filter
    ------------------------------------

    Key: PIG-399
    URL: https://issues.apache.org/jira/browse/PIG-399
    Project: Pig
    Issue Type: Bug
    Reporter: Olga Natkovich
    Assignee: Alan Gates
    Attachments: PIG-399.patch


    The following script fails:
    A = load 'data' as (name: chararray, age: chararray, gpa: float);
    B = limit A 40;
    C = filter B by age eq '40';
    D = group C by name;
    E = foreach D generate group, COUNT(C);
    dump E;
    If limit is removed, the script runs without any problems. The explain shows that filter looses type of the non-constant operand.
    The errror is
    8/08/25 13:34:15 INFO mapReduceLayer.MapReduceLauncher: Unsuccessful attempt. Completed 0.0% of the job
    08/08/25 13:34:15 ERROR mapReduceLayer.Launcher: Error message from task (reduce) tip_200808131548_2163_r_000000java.lang.ClassCastException: java.lang.String cannot be cast to org.apache.pig.data.DataByteArray
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.ConstantExpression.getNext(ConstantExpression.java:111)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.EqualToExpr.getNext(EqualToExpr.java:68)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POFilter.getNext(POFilter.java:147)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:169)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:87)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:391)
    at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categoriespig, hadoop
postedAug 25, '08 at 8:36p
activeAug 28, '08 at 12:12a
posts5
users1
websitepig.apache.org

1 user in discussion

Alan Gates (JIRA): 5 posts

People

Translate

site design / logo © 2022 Grokbase