Grokbase Groups Pig dev February 2009
FAQ
UNION doesn't work in the latest code
-------------------------------------

Key: PIG-690
URL: https://issues.apache.org/jira/browse/PIG-690
Project: Pig
Issue Type: Bug
Environment: mapred mode, trunk. It doesn't affect local mode.
Reporter: Amir Youssefi


grunt> a = load 'tmp/f1' using BinStorage();
grunt> b = load 'tmp/f2' using BinStorage();
grunt> describe a;
a: {int,chararray,int,{(int,chararray,chararray)}}
grunt> describe b;
b: {int,chararray,int,{(int,chararray,chararray)}}
grunt> c = union a,b;
grunt> describe c;
2009-02-27 11:51:46,012 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1052: Cannot cast bag with schema bag({(int,chararray,chararray)}) to tuple with schema tuple
Details at logfile: /homes/amiry/pig_1235735380348.log

dump a and dump b work fine.

Sample data provided to dev team in an e-mail.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Amir Youssefi (JIRA) at Feb 27, 2009 at 12:11 pm
    [ https://issues.apache.org/jira/browse/PIG-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Amir Youssefi updated PIG-690:
    ------------------------------

    Environment:
    mapred mode. local mode.has the same problem under linux.
    code is taken from trunk

    was:mapred mode, trunk. It doesn't affect local mode.

    UNION doesn't work in the latest code
    -------------------------------------

    Key: PIG-690
    URL: https://issues.apache.org/jira/browse/PIG-690
    Project: Pig
    Issue Type: Bug
    Environment: mapred mode. local mode.has the same problem under linux.
    code is taken from trunk
    Reporter: Amir Youssefi

    grunt> a = load 'tmp/f1' using BinStorage();
    grunt> b = load 'tmp/f2' using BinStorage();
    grunt> describe a;
    a: {int,chararray,int,{(int,chararray,chararray)}}
    grunt> describe b;
    b: {int,chararray,int,{(int,chararray,chararray)}}
    grunt> c = union a,b;
    grunt> describe c;
    2009-02-27 11:51:46,012 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1052: Cannot cast bag with schema bag({(int,chararray,chararray)}) to tuple with schema tuple
    Details at logfile: /homes/amiry/pig_1235735380348.log
    dump a and dump b work fine.
    Sample data provided to dev team in an e-mail.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Pradeep Kamath (JIRA) at Mar 3, 2009 at 2:33 am
    [ https://issues.apache.org/jira/browse/PIG-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pradeep Kamath updated PIG-690:
    -------------------------------

    Fix Version/s: types_branch
    Assignee: Pradeep Kamath
    Affects Version/s: types_branch
    Status: Patch Available (was: Open)

    The root cause of the issue is while merging schemas, the code recursively merges subschemas if a field is a tuple or a bag. At that point, it does not properly attribute the type to be bag if that was the case. It always marks the type as tuple whenever the field schema is of type bag or tuple. This is fixed in the patch and a unit test case has been added which tries to union two relations which have a bag field.

    UNION doesn't work in the latest code
    -------------------------------------

    Key: PIG-690
    URL: https://issues.apache.org/jira/browse/PIG-690
    Project: Pig
    Issue Type: Bug
    Affects Versions: types_branch
    Environment: mapred mode. local mode.has the same problem under linux.
    code is taken from trunk
    Reporter: Amir Youssefi
    Assignee: Pradeep Kamath
    Fix For: types_branch

    Attachments: PIG-690.patch


    grunt> a = load 'tmp/f1' using BinStorage();
    grunt> b = load 'tmp/f2' using BinStorage();
    grunt> describe a;
    a: {int,chararray,int,{(int,chararray,chararray)}}
    grunt> describe b;
    b: {int,chararray,int,{(int,chararray,chararray)}}
    grunt> c = union a,b;
    grunt> describe c;
    2009-02-27 11:51:46,012 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1052: Cannot cast bag with schema bag({(int,chararray,chararray)}) to tuple with schema tuple
    Details at logfile: /homes/amiry/pig_1235735380348.log
    dump a and dump b work fine.
    Sample data provided to dev team in an e-mail.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Mridul Muralidharan at Mar 4, 2009 at 7:14 am
    Great, thanks !
    I am assuming this might also fix load related schema issues too (with
    BinStorage) ? Looked kind of similar issue as I reported in pig usergroup.

    - Mridul

    Pradeep Kamath (JIRA) wrote:
    [ https://issues.apache.org/jira/browse/PIG-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pradeep Kamath updated PIG-690:
    -------------------------------

    Fix Version/s: types_branch
    Assignee: Pradeep Kamath
    Affects Version/s: types_branch
    Status: Patch Available (was: Open)

    The root cause of the issue is while merging schemas, the code recursively merges subschemas if a field is a tuple or a bag. At that point, it does not properly attribute the type to be bag if that was the case. It always marks the type as tuple whenever the field schema is of type bag or tuple. This is fixed in the patch and a unit test case has been added which tries to union two relations which have a bag field.

    UNION doesn't work in the latest code
    -------------------------------------

    Key: PIG-690
    URL: https://issues.apache.org/jira/browse/PIG-690
    Project: Pig
    Issue Type: Bug
    Affects Versions: types_branch
    Environment: mapred mode. local mode.has the same problem under linux.
    code is taken from trunk
    Reporter: Amir Youssefi
    Assignee: Pradeep Kamath
    Fix For: types_branch

    Attachments: PIG-690.patch


    grunt> a = load 'tmp/f1' using BinStorage();
    grunt> b = load 'tmp/f2' using BinStorage();
    grunt> describe a;
    a: {int,chararray,int,{(int,chararray,chararray)}}
    grunt> describe b;
    b: {int,chararray,int,{(int,chararray,chararray)}}
    grunt> c = union a,b;
    grunt> describe c;
    2009-02-27 11:51:46,012 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1052: Cannot cast bag with schema bag({(int,chararray,chararray)}) to tuple with schema tuple
    Details at logfile: /homes/amiry/pig_1235735380348.log
    dump a and dump b work fine.
    Sample data provided to dev team in an e-mail.
  • Pradeep Kamath (JIRA) at Mar 3, 2009 at 2:33 am
    [ https://issues.apache.org/jira/browse/PIG-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pradeep Kamath updated PIG-690:
    -------------------------------

    Attachment: PIG-690.patch
    UNION doesn't work in the latest code
    -------------------------------------

    Key: PIG-690
    URL: https://issues.apache.org/jira/browse/PIG-690
    Project: Pig
    Issue Type: Bug
    Affects Versions: types_branch
    Environment: mapred mode. local mode.has the same problem under linux.
    code is taken from trunk
    Reporter: Amir Youssefi
    Assignee: Pradeep Kamath
    Fix For: types_branch

    Attachments: PIG-690.patch


    grunt> a = load 'tmp/f1' using BinStorage();
    grunt> b = load 'tmp/f2' using BinStorage();
    grunt> describe a;
    a: {int,chararray,int,{(int,chararray,chararray)}}
    grunt> describe b;
    b: {int,chararray,int,{(int,chararray,chararray)}}
    grunt> c = union a,b;
    grunt> describe c;
    2009-02-27 11:51:46,012 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1052: Cannot cast bag with schema bag({(int,chararray,chararray)}) to tuple with schema tuple
    Details at logfile: /homes/amiry/pig_1235735380348.log
    dump a and dump b work fine.
    Sample data provided to dev team in an e-mail.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Santhosh Srinivasan (JIRA) at Mar 4, 2009 at 12:03 am
    [ https://issues.apache.org/jira/browse/PIG-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12678519#action_12678519 ]

    Santhosh Srinivasan commented on PIG-690:
    -----------------------------------------

    I am reviewing this patch.
    UNION doesn't work in the latest code
    -------------------------------------

    Key: PIG-690
    URL: https://issues.apache.org/jira/browse/PIG-690
    Project: Pig
    Issue Type: Bug
    Affects Versions: types_branch
    Environment: mapred mode. local mode.has the same problem under linux.
    code is taken from trunk
    Reporter: Amir Youssefi
    Assignee: Pradeep Kamath
    Fix For: types_branch

    Attachments: PIG-690.patch


    grunt> a = load 'tmp/f1' using BinStorage();
    grunt> b = load 'tmp/f2' using BinStorage();
    grunt> describe a;
    a: {int,chararray,int,{(int,chararray,chararray)}}
    grunt> describe b;
    b: {int,chararray,int,{(int,chararray,chararray)}}
    grunt> c = union a,b;
    grunt> describe c;
    2009-02-27 11:51:46,012 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1052: Cannot cast bag with schema bag({(int,chararray,chararray)}) to tuple with schema tuple
    Details at logfile: /homes/amiry/pig_1235735380348.log
    dump a and dump b work fine.
    Sample data provided to dev team in an e-mail.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Santhosh Srinivasan (JIRA) at Mar 4, 2009 at 1:47 am
    [ https://issues.apache.org/jira/browse/PIG-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Santhosh Srinivasan updated PIG-690:
    ------------------------------------

    Resolution: Fixed
    Hadoop Flags: [Reviewed]
    Status: Resolved (was: Patch Available)

    Patch has been committed. Thanks for fixing this issue Pradeep.
    UNION doesn't work in the latest code
    -------------------------------------

    Key: PIG-690
    URL: https://issues.apache.org/jira/browse/PIG-690
    Project: Pig
    Issue Type: Bug
    Affects Versions: types_branch
    Environment: mapred mode. local mode.has the same problem under linux.
    code is taken from trunk
    Reporter: Amir Youssefi
    Assignee: Pradeep Kamath
    Fix For: types_branch

    Attachments: PIG-690.patch


    grunt> a = load 'tmp/f1' using BinStorage();
    grunt> b = load 'tmp/f2' using BinStorage();
    grunt> describe a;
    a: {int,chararray,int,{(int,chararray,chararray)}}
    grunt> describe b;
    b: {int,chararray,int,{(int,chararray,chararray)}}
    grunt> c = union a,b;
    grunt> describe c;
    2009-02-27 11:51:46,012 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1052: Cannot cast bag with schema bag({(int,chararray,chararray)}) to tuple with schema tuple
    Details at logfile: /homes/amiry/pig_1235735380348.log
    dump a and dump b work fine.
    Sample data provided to dev team in an e-mail.
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categoriespig, hadoop
postedFeb 27, '09 at 12:00p
activeMar 4, '09 at 7:14a
posts7
users2
websitepig.apache.org

People

Translate

site design / logo © 2022 Grokbase