FAQ
error with complex nested plan
------------------------------

Key: PIG-421
URL: https://issues.apache.org/jira/browse/PIG-421
Project: Pig
Issue Type: Bug
Affects Versions: types_branch
Reporter: Olga Natkovich
Assignee: Santhosh Srinivasan
Fix For: types_branch


Even after applying patch for PIG-398, the following query still fails:

a = load 'studenttab10k' as (name, age, gpa);
b = filter a by age < 20;
c = group b by age;
d = foreach c {
cf = filter b by gpa < 3.0;
cp = cf.gpa;
cd = distinct cp;
co = order cd by $0;
generate group, flatten(co);
}
store d into 'output';

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Santhosh Srinivasan (JIRA) at Sep 8, 2008 at 10:56 pm
    [ https://issues.apache.org/jira/browse/PIG-421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12629325#action_12629325 ]

    Santhosh Srinivasan commented on PIG-421:
    -----------------------------------------

    With Pig-398, the issue with LOVisitor not walking all inner plans correctly was resolved. That has uncovered other issues. One of which is the following:

    During type checking, schemas are computed for a collection of operators. Later, the optimizer, resets the schemas and calls the getSchema() method on the operators. Instead of returning the schemas computed by the type checker (which are now lost), the schemas computed by the individual operator are returned. In order to resolve this, we need to move the schema computation out of the type checker and into a common place (getSchema() method of the operators).

    The query in the bug report generates the following stack trace:

    {code}
    384826 [main] ERROR org.apache.pig.tools.grunt.GruntParser - java.io.IOException: Unable to explain alias d [Can't overwrite cause]
    at java.lang.Throwable.initCause(Throwable.java:320)
    at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.resolveLOProjectType(TypeCheckingVisitor.java:212)
    at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.checkInnerPlan(TypeCheckingVisitor.java:2298)
    at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:1848)
    at org.apache.pig.impl.logicalLayer.LOFilter.visit(LOFilter.java:102)
    {code}
    error with complex nested plan
    ------------------------------

    Key: PIG-421
    URL: https://issues.apache.org/jira/browse/PIG-421
    Project: Pig
    Issue Type: Bug
    Affects Versions: types_branch
    Reporter: Olga Natkovich
    Assignee: Santhosh Srinivasan
    Fix For: types_branch


    Even after applying patch for PIG-398, the following query still fails:
    a = load 'studenttab10k' as (name, age, gpa);
    b = filter a by age < 20;
    c = group b by age;
    d = foreach c {
    cf = filter b by gpa < 3.0;
    cp = cf.gpa;
    cd = distinct cp;
    co = order cd by $0;
    generate group, flatten(co);
    }
    store d into 'output';
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Santhosh Srinivasan (JIRA) at Sep 11, 2008 at 12:53 am
    [ https://issues.apache.org/jira/browse/PIG-421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Santhosh Srinivasan updated PIG-421:
    ------------------------------------

    Attachment: PIG-421.patch

    The attached patch (PIG-421.patch) does the following:

    1. Moved all the schema computation that existed within the type checker to the appropriate operators.
    2. The getType() method for all expressions will now call getFieldSchema() and return the type appropriately
    3. Renamed the resetSchema methods to unsetSchema which was an existing method in LogicalOperator

    Unit tests that still fail are:

    [junit] Running org.apache.pig.test.TestMRCompiler
    [junit] Tests run: 16, Failures: 1, Errors: 0, Time elapsed: 0.34 sec
    [junit] Test org.apache.pig.test.TestMRCompiler FAILED

    [junit] Running org.apache.pig.test.TestSplitStore
    [junit] Tests run: 9, Failures: 0, Errors: 2, Time elapsed: 216.145 sec
    [junit] Test org.apache.pig.test.TestSplitStore FAILED

    TestSplitStore has two test cases failing test6 and test7. These tests fail due to the issue reported in PIG-407.

    error with complex nested plan
    ------------------------------

    Key: PIG-421
    URL: https://issues.apache.org/jira/browse/PIG-421
    Project: Pig
    Issue Type: Bug
    Affects Versions: types_branch
    Reporter: Olga Natkovich
    Assignee: Santhosh Srinivasan
    Fix For: types_branch

    Attachments: PIG-421.patch


    Even after applying patch for PIG-398, the following query still fails:
    a = load 'studenttab10k' as (name, age, gpa);
    b = filter a by age < 20;
    c = group b by age;
    d = foreach c {
    cf = filter b by gpa < 3.0;
    cp = cf.gpa;
    cd = distinct cp;
    co = order cd by $0;
    generate group, flatten(co);
    }
    store d into 'output';
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Santhosh Srinivasan (JIRA) at Sep 12, 2008 at 10:58 pm
    [ https://issues.apache.org/jira/browse/PIG-421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Santhosh Srinivasan updated PIG-421:
    ------------------------------------

    Attachment: PIG-421_1.patch

    Attaching a new patch (PIG-421_1.patch). In addition to the previous patch, this patch fixes:

    1. Fixes the 2 failing unit test cases. Removed the 2 gold files for the Logical to Physical translation
    2. Fixed a bug in LOVisitor where LOSplit was visiting the LOSplitOutputs. The walker was also visiting LOSplitOutputs

    All unit test cases pass.
    error with complex nested plan
    ------------------------------

    Key: PIG-421
    URL: https://issues.apache.org/jira/browse/PIG-421
    Project: Pig
    Issue Type: Bug
    Affects Versions: types_branch
    Reporter: Olga Natkovich
    Assignee: Santhosh Srinivasan
    Fix For: types_branch

    Attachments: PIG-421.patch, PIG-421_1.patch


    Even after applying patch for PIG-398, the following query still fails:
    a = load 'studenttab10k' as (name, age, gpa);
    b = filter a by age < 20;
    c = group b by age;
    d = foreach c {
    cf = filter b by gpa < 3.0;
    cp = cf.gpa;
    cd = distinct cp;
    co = order cd by $0;
    generate group, flatten(co);
    }
    store d into 'output';
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Olga Natkovich (JIRA) at Sep 13, 2008 at 4:15 pm
    [ https://issues.apache.org/jira/browse/PIG-421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Olga Natkovich resolved PIG-421.
    --------------------------------

    Resolution: Fixed

    patch committed. thanks, santhosh!
    error with complex nested plan
    ------------------------------

    Key: PIG-421
    URL: https://issues.apache.org/jira/browse/PIG-421
    Project: Pig
    Issue Type: Bug
    Affects Versions: types_branch
    Reporter: Olga Natkovich
    Assignee: Santhosh Srinivasan
    Fix For: types_branch

    Attachments: PIG-421.patch, PIG-421_1.patch


    Even after applying patch for PIG-398, the following query still fails:
    a = load 'studenttab10k' as (name, age, gpa);
    b = filter a by age < 20;
    c = group b by age;
    d = foreach c {
    cf = filter b by gpa < 3.0;
    cp = cf.gpa;
    cd = distinct cp;
    co = order cd by $0;
    generate group, flatten(co);
    }
    store d into 'output';
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categoriespig, hadoop
postedSep 8, '08 at 9:34p
activeSep 13, '08 at 4:15p
posts5
users1
websitepig.apache.org

1 user in discussion

Olga Natkovich (JIRA): 5 posts

People

Translate

site design / logo © 2022 Grokbase