FAQ
Loading data with a schema that has a bag as a field results in error
---------------------------------------------------------------------

Key: PIG-464
URL: https://issues.apache.org/jira/browse/PIG-464
Project: Pig
Issue Type: Bug
Affects Versions: types_branch
Reporter: Pradeep Kamath
Assignee: Pradeep Kamath
Fix For: types_branch


Script:
{code}
a = load '/user/pig/tests/data/singlefile/studenttab10k' using PigStorage() as (name:chararray, age:long, gpa: float);
b = group a by name;
c = foreach b generate a, (1,2,3), ['key1'#'value1','key2'#'value2'];
-- store the bag, tuple and map
store c into '/tmp/intermediate' using BinStorage();
d = load '/tmp/intermediate' using BinStorage() as (b:bag{t:tuple(x,y,z)}, t2:tuple(a,b,c), m:map[]);
e = foreach d generate COUNT(b), t2.a, t2.b, t2.c, m#'key1', m#'key2';
dump e;
{code}

Error:
{noformat}
java.io.IOException: Unable to open iterator for alias: e [Unable to store for alias: e [Unable to insert type casts into plan]]
at org.apache.pig.impl.logicalLayer.optimizer.TypeCastInserter.transform(TypeCastInserter.java:150)
at org.apache.pig.impl.plan.optimizer.PlanOptimizer.optimize(PlanOptimizer.java:65)
at org.apache.pig.PigServer.compileLp(PigServer.java:723)
at org.apache.pig.PigServer.compileLp(PigServer.java:655)
at org.apache.pig.PigServer.store(PigServer.java:433)
at org.apache.pig.PigServer.store(PigServer.java:421)
at org.apache.pig.PigServer.openIterator(PigServer.java:384)
at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:269)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:178)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:84)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:64)
at org.apache.pig.Main.main(Main.java:306)
Caused by: java.io.IOException: Unable to store for alias: e [Unable to insert type casts into plan]
... 12 more
Caused by: org.apache.pig.impl.plan.optimizer.OptimizerException: Unable to insert type casts into plan
... 12 more
Caused by: java.lang.NullPointerException
at java.util.ArrayList.(MultiMap.java:82)
at org.apache.pig.impl.logicalLayer.schema.Schema.clone(Schema.java:725)
at org.apache.pig.impl.logicalLayer.schema.Schema$FieldSchema.clone(Schema.java:348)
at org.apache.pig.impl.logicalLayer.optimizer.TypeCastInserter.transform(TypeCastInserter.java:131)

{noformat}

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

  • Pradeep Kamath (JIRA) at Oct 3, 2008 at 12:12 am
    [ https://issues.apache.org/jira/browse/PIG-464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pradeep Kamath updated PIG-464:
    -------------------------------

    Status: Patch Available (was: Open)

    Attached patch with unit test. The change was to use canonical name of FieldSchema as the key in the mFieldSchemas multimap. Earlier the FieldSchema object was the key and this caused problems when the the FieldSchema object was changed (say the schema or some other member was changed) while in the multimap. This caused the hashcode of the FieldSchema object to change and hence corrupt the multimap.
    Loading data with a schema that has a bag as a field results in error
    ---------------------------------------------------------------------

    Key: PIG-464
    URL: https://issues.apache.org/jira/browse/PIG-464
    Project: Pig
    Issue Type: Bug
    Affects Versions: types_branch
    Reporter: Pradeep Kamath
    Assignee: Pradeep Kamath
    Fix For: types_branch

    Attachments: PIG-464.patch


    Script:
    {code}
    a = load '/user/pig/tests/data/singlefile/studenttab10k' using PigStorage() as (name:chararray, age:long, gpa: float);
    b = group a by name;
    c = foreach b generate a, (1,2,3), ['key1'#'value1','key2'#'value2'];
    -- store the bag, tuple and map
    store c into '/tmp/intermediate' using BinStorage();
    d = load '/tmp/intermediate' using BinStorage() as (b:bag{t:tuple(x,y,z)}, t2:tuple(a,b,c), m:map[]);
    e = foreach d generate COUNT(b), t2.a, t2.b, t2.c, m#'key1', m#'key2';
    dump e;
    {code}
    Error:
    {noformat}
    java.io.IOException: Unable to open iterator for alias: e [Unable to store for alias: e [Unable to insert type casts into plan]]
    at org.apache.pig.impl.logicalLayer.optimizer.TypeCastInserter.transform(TypeCastInserter.java:150)
    at org.apache.pig.impl.plan.optimizer.PlanOptimizer.optimize(PlanOptimizer.java:65)
    at org.apache.pig.PigServer.compileLp(PigServer.java:723)
    at org.apache.pig.PigServer.compileLp(PigServer.java:655)
    at org.apache.pig.PigServer.store(PigServer.java:433)
    at org.apache.pig.PigServer.store(PigServer.java:421)
    at org.apache.pig.PigServer.openIterator(PigServer.java:384)
    at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:269)
    at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:178)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:84)
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:64)
    at org.apache.pig.Main.main(Main.java:306)
    Caused by: java.io.IOException: Unable to store for alias: e [Unable to insert type casts into plan]
    ... 12 more
    Caused by: org.apache.pig.impl.plan.optimizer.OptimizerException: Unable to insert type casts into plan
    ... 12 more
    Caused by: java.lang.NullPointerException
    at java.util.ArrayList.<init>(ArrayList.java:133)
    at org.apache.pig.impl.util.MultiMap.put(MultiMap.java:82)
    at org.apache.pig.impl.logicalLayer.schema.Schema.clone(Schema.java:725)
    at org.apache.pig.impl.logicalLayer.schema.Schema$FieldSchema.clone(Schema.java:348)
    at org.apache.pig.impl.logicalLayer.optimizer.TypeCastInserter.transform(TypeCastInserter.java:131)
    {noformat}
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Pradeep Kamath (JIRA) at Oct 3, 2008 at 12:12 am
    [ https://issues.apache.org/jira/browse/PIG-464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Pradeep Kamath updated PIG-464:
    -------------------------------

    Attachment: PIG-464.patch
    Loading data with a schema that has a bag as a field results in error
    ---------------------------------------------------------------------

    Key: PIG-464
    URL: https://issues.apache.org/jira/browse/PIG-464
    Project: Pig
    Issue Type: Bug
    Affects Versions: types_branch
    Reporter: Pradeep Kamath
    Assignee: Pradeep Kamath
    Fix For: types_branch

    Attachments: PIG-464.patch


    Script:
    {code}
    a = load '/user/pig/tests/data/singlefile/studenttab10k' using PigStorage() as (name:chararray, age:long, gpa: float);
    b = group a by name;
    c = foreach b generate a, (1,2,3), ['key1'#'value1','key2'#'value2'];
    -- store the bag, tuple and map
    store c into '/tmp/intermediate' using BinStorage();
    d = load '/tmp/intermediate' using BinStorage() as (b:bag{t:tuple(x,y,z)}, t2:tuple(a,b,c), m:map[]);
    e = foreach d generate COUNT(b), t2.a, t2.b, t2.c, m#'key1', m#'key2';
    dump e;
    {code}
    Error:
    {noformat}
    java.io.IOException: Unable to open iterator for alias: e [Unable to store for alias: e [Unable to insert type casts into plan]]
    at org.apache.pig.impl.logicalLayer.optimizer.TypeCastInserter.transform(TypeCastInserter.java:150)
    at org.apache.pig.impl.plan.optimizer.PlanOptimizer.optimize(PlanOptimizer.java:65)
    at org.apache.pig.PigServer.compileLp(PigServer.java:723)
    at org.apache.pig.PigServer.compileLp(PigServer.java:655)
    at org.apache.pig.PigServer.store(PigServer.java:433)
    at org.apache.pig.PigServer.store(PigServer.java:421)
    at org.apache.pig.PigServer.openIterator(PigServer.java:384)
    at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:269)
    at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:178)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:84)
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:64)
    at org.apache.pig.Main.main(Main.java:306)
    Caused by: java.io.IOException: Unable to store for alias: e [Unable to insert type casts into plan]
    ... 12 more
    Caused by: org.apache.pig.impl.plan.optimizer.OptimizerException: Unable to insert type casts into plan
    ... 12 more
    Caused by: java.lang.NullPointerException
    at java.util.ArrayList.<init>(ArrayList.java:133)
    at org.apache.pig.impl.util.MultiMap.put(MultiMap.java:82)
    at org.apache.pig.impl.logicalLayer.schema.Schema.clone(Schema.java:725)
    at org.apache.pig.impl.logicalLayer.schema.Schema$FieldSchema.clone(Schema.java:348)
    at org.apache.pig.impl.logicalLayer.optimizer.TypeCastInserter.transform(TypeCastInserter.java:131)
    {noformat}
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.
  • Olga Natkovich (JIRA) at Oct 3, 2008 at 2:35 am
    [ https://issues.apache.org/jira/browse/PIG-464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Olga Natkovich updated PIG-464:
    -------------------------------

    Resolution: Fixed
    Status: Resolved (was: Patch Available)

    patch comitted; thanks pradeep
    Loading data with a schema that has a bag as a field results in error
    ---------------------------------------------------------------------

    Key: PIG-464
    URL: https://issues.apache.org/jira/browse/PIG-464
    Project: Pig
    Issue Type: Bug
    Affects Versions: types_branch
    Reporter: Pradeep Kamath
    Assignee: Pradeep Kamath
    Fix For: types_branch

    Attachments: PIG-464.patch


    Script:
    {code}
    a = load '/user/pig/tests/data/singlefile/studenttab10k' using PigStorage() as (name:chararray, age:long, gpa: float);
    b = group a by name;
    c = foreach b generate a, (1,2,3), ['key1'#'value1','key2'#'value2'];
    -- store the bag, tuple and map
    store c into '/tmp/intermediate' using BinStorage();
    d = load '/tmp/intermediate' using BinStorage() as (b:bag{t:tuple(x,y,z)}, t2:tuple(a,b,c), m:map[]);
    e = foreach d generate COUNT(b), t2.a, t2.b, t2.c, m#'key1', m#'key2';
    dump e;
    {code}
    Error:
    {noformat}
    java.io.IOException: Unable to open iterator for alias: e [Unable to store for alias: e [Unable to insert type casts into plan]]
    at org.apache.pig.impl.logicalLayer.optimizer.TypeCastInserter.transform(TypeCastInserter.java:150)
    at org.apache.pig.impl.plan.optimizer.PlanOptimizer.optimize(PlanOptimizer.java:65)
    at org.apache.pig.PigServer.compileLp(PigServer.java:723)
    at org.apache.pig.PigServer.compileLp(PigServer.java:655)
    at org.apache.pig.PigServer.store(PigServer.java:433)
    at org.apache.pig.PigServer.store(PigServer.java:421)
    at org.apache.pig.PigServer.openIterator(PigServer.java:384)
    at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:269)
    at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:178)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:84)
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:64)
    at org.apache.pig.Main.main(Main.java:306)
    Caused by: java.io.IOException: Unable to store for alias: e [Unable to insert type casts into plan]
    ... 12 more
    Caused by: org.apache.pig.impl.plan.optimizer.OptimizerException: Unable to insert type casts into plan
    ... 12 more
    Caused by: java.lang.NullPointerException
    at java.util.ArrayList.<init>(ArrayList.java:133)
    at org.apache.pig.impl.util.MultiMap.put(MultiMap.java:82)
    at org.apache.pig.impl.logicalLayer.schema.Schema.clone(Schema.java:725)
    at org.apache.pig.impl.logicalLayer.schema.Schema$FieldSchema.clone(Schema.java:348)
    at org.apache.pig.impl.logicalLayer.optimizer.TypeCastInserter.transform(TypeCastInserter.java:131)
    {noformat}
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categoriespig, hadoop
postedSep 29, '08 at 6:21p
activeOct 3, '08 at 2:35a
posts4
users1
websitepig.apache.org

1 user in discussion

Olga Natkovich (JIRA): 4 posts

People

Translate

site design / logo © 2022 Grokbase