Grokbase Groups Pig user May 2013
FAQ
I'm new to pig and I'm getting a ClassCastException when I try to run the following script in pig 0.11.1:


A = LOAD 'test.log' AS (timestamp:long, pk_id:int, array_field:chararray, fk_id:int);

B = FOREACH A GENERATE timestamp, pk_id, FLATTEN(STRSPLIT(array_field,',')) AS (field1:chararray, field2:chararray, age:long), fk_id;

C = FOREACH B GENERATE timestamp-(age IS NULL ? (long) 0 : age) AS adjusted_timestamp:long, pk_id, fk_id;

DUMP C;


test.log contains only one row (I replaced tabs with \t):
1362178873552\t15404\ta,b,5\t6

I'm able to do a dump on B and get the results I expect, but when I do a dump/store on C I get this error:
java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Number
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Subtract.genericGetNext(Subtract.java:96)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Subtract.getNext(Subtract.java:119)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:348)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:372)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:297)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:283)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:278)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)

I've tried casting every timestamp and age to a long in the definition of C and it still doesn't work.

Any ideas on how to fix the script?

Thanks,
Peter

Search Discussions

  • Johnny Zhang at May 5, 2013 at 6:57 am
    Hi, Peter:
    http://pig.apache.org/docs/r0.11.1/basic.html#cast
    cast from String to long is not supported.

    Johnny

    On Fri, May 3, 2013 at 12:01 PM, Peter Connolly wrote:

    I'm new to pig and I'm getting a ClassCastException when I try to run the
    following script in pig 0.11.1:


    A = LOAD 'test.log' AS (timestamp:long, pk_id:int, array_field:chararray,
    fk_id:int);

    B = FOREACH A GENERATE timestamp, pk_id,
    FLATTEN(STRSPLIT(array_field,',')) AS (field1:chararray, field2:chararray,
    age:long), fk_id;

    C = FOREACH B GENERATE timestamp-(age IS NULL ? (long) 0 : age) AS
    adjusted_timestamp:long, pk_id, fk_id;

    DUMP C;


    test.log contains only one row (I replaced tabs with \t):
    1362178873552\t15404\ta,b,5\t6

    I'm able to do a dump on B and get the results I expect, but when I do a
    dump/store on C I get this error:
    java.lang.ClassCastException: java.lang.String cannot be cast to
    java.lang.Number
    at
    org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Subtract.genericGetNext(Subtract.java:96)
    at
    org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Subtract.getNext(Subtract.java:119)
    at
    org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:348)
    at
    org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:372)
    at
    org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:297)
    at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:283)
    at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:278)
    at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
    at
    org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)

    I've tried casting every timestamp and age to a long in the definition of
    C and it still doesn't work.

    Any ideas on how to fix the script?

    Thanks,
    Peter

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedMay 3, '13 at 7:01p
activeMay 5, '13 at 6:57a
posts2
users2
websitepig.apache.org

2 users in discussion

Peter Connolly: 1 post Johnny Zhang: 1 post

People

Translate

site design / logo © 2021 Grokbase