Grokbase Groups Pig user March 2010
FAQ
I'm using the 0.6.0 build and I was trying out the ILLUSTRATE command the
example given in the Pig Latin Reference Manual 2 for JOIN (inner).

I have the following pig script:
A = LOAD 'data1' AS (a1:int,a2:int,a3:int);
B = LOAD 'data2' AS (b1:int,b2:int);
X = JOIN A BY a1, B BY b1;
ILLUSTRATE X;

Which produces one of two null pointer exceptions:

1)
Pig Stack Trace
---------------
ERROR 2999: Unexpected internal error. null

java.lang.NullPointerException
at
org.apache.pig.pen.LineageTrimmingVisitor.PruneBaseDataConstrainedCoverage(LineageTrimmingVisitor.java:297)
at
org.apache.pig.pen.LineageTrimmingVisitor.checkCompleteness(LineageTrimmingVisitor.java:466)
at
org.apache.pig.pen.LineageTrimmingVisitor.processOperator(LineageTrimmingVisitor.java:424)
at
org.apache.pig.pen.LineageTrimmingVisitor.visit(LineageTrimmingVisitor.java:255)
at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:130)
at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:45)
at
org.apache.pig.pen.util.PreOrderDepthFirstWalker.depthFirst(PreOrderDepthFirstWalker.java:70)
at
org.apache.pig.pen.util.PreOrderDepthFirstWalker.depthFirst(PreOrderDepthFirstWalker.java:72)
at
org.apache.pig.pen.util.PreOrderDepthFirstWalker.walk(PreOrderDepthFirstWalker.java:55)
at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
at
org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:108)
at org.apache.pig.PigServer.getExamples(PigServer.java:731)
at
org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:549)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:246)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
at org.apache.pig.Main.main(Main.java:386)
================================================================================

2)
java.lang.NullPointerException
at
org.apache.pig.pen.util.DisplayExamples.MakeArray(DisplayExamples.java:183)
at
org.apache.pig.pen.util.DisplayExamples.printTabular(DisplayExamples.java:86)
at
org.apache.pig.pen.util.DisplayExamples.printTabular(DisplayExamples.java:69)
at
org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:144)
at org.apache.pig.PigServer.getExamples(PigServer.java:731)
at
org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:549)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:246)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
at org.apache.pig.Main.main(Main.java:386)


Any insight is greatly appreciated.

- Ming-Hay Luk

Search Discussions

  • Rekha Joshi at Mar 17, 2010 at 9:17 am
    If you get the same after filtering possible null values, you might hitting the known issues with illustrate - PIG-1066, PIG-534 to name a few.
    I think the fix would be coming in later versions, the workaround is to do without illustrate after join..describe/explain work fine.

    Cheers,
    /R

    On 3/17/10 2:27 PM, "Ming-Hay Luk" wrote:

    I'm using the 0.6.0 build and I was trying out the ILLUSTRATE command the
    example given in the Pig Latin Reference Manual 2 for JOIN (inner).

    I have the following pig script:
    A = LOAD 'data1' AS (a1:int,a2:int,a3:int);
    B = LOAD 'data2' AS (b1:int,b2:int);
    X = JOIN A BY a1, B BY b1;
    ILLUSTRATE X;

    Which produces one of two null pointer exceptions:

    1)
    Pig Stack Trace
    ---------------
    ERROR 2999: Unexpected internal error. null

    java.lang.NullPointerException
    at
    org.apache.pig.pen.LineageTrimmingVisitor.PruneBaseDataConstrainedCoverage(LineageTrimmingVisitor.java:297)
    at
    org.apache.pig.pen.LineageTrimmingVisitor.checkCompleteness(LineageTrimmingVisitor.java:466)
    at
    org.apache.pig.pen.LineageTrimmingVisitor.processOperator(LineageTrimmingVisitor.java:424)
    at
    org.apache.pig.pen.LineageTrimmingVisitor.visit(LineageTrimmingVisitor.java:255)
    at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:130)
    at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:45)
    at
    org.apache.pig.pen.util.PreOrderDepthFirstWalker.depthFirst(PreOrderDepthFirstWalker.java:70)
    at
    org.apache.pig.pen.util.PreOrderDepthFirstWalker.depthFirst(PreOrderDepthFirstWalker.java:72)
    at
    org.apache.pig.pen.util.PreOrderDepthFirstWalker.walk(PreOrderDepthFirstWalker.java:55)
    at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
    at
    org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:108)
    at org.apache.pig.PigServer.getExamples(PigServer.java:731)
    at
    org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:549)
    at
    org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:246)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144)
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
    at org.apache.pig.Main.main(Main.java:386)
    ================================================================================

    2)
    java.lang.NullPointerException
    at
    org.apache.pig.pen.util.DisplayExamples.MakeArray(DisplayExamples.java:183)
    at
    org.apache.pig.pen.util.DisplayExamples.printTabular(DisplayExamples.java:86)
    at
    org.apache.pig.pen.util.DisplayExamples.printTabular(DisplayExamples.java:69)
    at
    org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:144)
    at org.apache.pig.PigServer.getExamples(PigServer.java:731)
    at
    org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:549)
    at
    org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:246)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144)
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
    at org.apache.pig.Main.main(Main.java:386)


    Any insight is greatly appreciated.

    - Ming-Hay Luk

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedMar 17, '10 at 8:58a
activeMar 17, '10 at 9:17a
posts2
users2
websitepig.apache.org

2 users in discussion

Rekha Joshi: 1 post Ming-Hay Luk: 1 post

People

Translate

site design / logo © 2021 Grokbase