Grokbase Groups Pig user April 2011
FAQ
I have a python script which does some date/time epoch conversion which
is sent to a pig job. However, it seems to error out all the time now.
I'm sort of confused you could say

The pig command being run is : pig -f succinct-gcb-epoch.pig -param
epoch=1302842158 -param sdate=2011-04-15 -param edate=2011-04-15 -param
reportName=last10hours-outgoing-all-min5 -param min=5

There is one line that I think is the culprit, which is the epoch
parameter. It looks like so:
filtered = FILTER log BY module !='ARP' AND
com.site.pig.ExtractEpochSeconds(created_at) > $epoch;

Any help would be appreciated. Stack trace below.


Backend error message
---------------------
java.lang.ClassCastException: org.apache.pig.impl.io.NullableText cannot
be cast to org.apache.pig.impl.io.NullableLongWritable
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigLongRawC
omparator.compare(PigLongRawComparator.java:94)
at java.util.Arrays.binarySearch0(Arrays.java:2105)
at java.util.Arrays.binarySearch(Arrays.java:2043)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioner
s.WeightedRangePartitioner.getPartition(WeightedRangePartitioner.java:65
)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioner
s.WeightedRangePartitioner.getPartition(WeightedRangePartitioner.java:56
)
at
org.apache.hadoop.mapred.MapTask$OldOutputCollector.collect(MapTask.java
:466)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduc
e$Map.collect(PigMapReduce.java:109)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.
runPipeline(PigMapBase.java:255)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.
map(PigMapBase.java:244)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduc
e$Map.map(PigMapReduce.java:94)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)

Pig Stack Trace
---------------
ERROR 6015: During execution, encountered a Hadoop error.

org.apache.pig.backend.executionengine.ExecException: ERROR 6015: During
execution, encountered a Hadoop error.
at
.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigLongRawComp
arator.compare(PigLongRawComparator.java:94)
at .util.Arrays.binarySearch0(Arrays.java:2105)
at .util.Arrays.binarySearch(Arrays.java:2043)
at
.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.W
eightedRangePartitioner.getPartition(WeightedRangePartitioner.java:65)
at
.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.W
eightedRangePartitioner.getPartition(WeightedRangePartitioner.java:56)
at
.apache.hadoop.mapred.MapTask$OldOutputCollector.collect(MapTask.java:46
6)
at
.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$M
ap.collect(PigMapReduce.java:109)
at
.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.run
Pipeline(PigMapBase.java:255)
at
.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map
(PigMapBase.java:244)
at
.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$M
ap.map(PigMapReduce.java:94)
at .apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at .apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at .apache.hadoop.mapred.MapTask.run(MapTask.java:307)
Caused by: java.lang.ClassCastException:
org.apache.pig.impl.io.NullableText cannot be cast to
org.apache.pig.impl.io.NullableLongWritable
... 13 more
========================================================================
========

Search Discussions

  • Daniel Dai at Apr 15, 2011 at 6:53 pm
    From the stack, it seems the exception is thrown in "order by"
    statement. Can you post your complete script?

    Daniel
    On 04/15/2011 07:46 AM, Brian Adams wrote:
    I have a python script which does some date/time epoch conversion which
    is sent to a pig job. However, it seems to error out all the time now.
    I'm sort of confused you could say

    The pig command being run is : pig -f succinct-gcb-epoch.pig -param
    epoch=1302842158 -param sdate=2011-04-15 -param edate=2011-04-15 -param
    reportName=last10hours-outgoing-all-min5 -param min=5

    There is one line that I think is the culprit, which is the epoch
    parameter. It looks like so:
    filtered = FILTER log BY module !='ARP' AND
    com.site.pig.ExtractEpochSeconds(created_at)> $epoch;

    Any help would be appreciated. Stack trace below.


    Backend error message
    ---------------------
    java.lang.ClassCastException: org.apache.pig.impl.io.NullableText cannot
    be cast to org.apache.pig.impl.io.NullableLongWritable
    at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigLongRawC
    omparator.compare(PigLongRawComparator.java:94)
    at java.util.Arrays.binarySearch0(Arrays.java:2105)
    at java.util.Arrays.binarySearch(Arrays.java:2043)
    at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioner
    s.WeightedRangePartitioner.getPartition(WeightedRangePartitioner.java:65
    )
    at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioner
    s.WeightedRangePartitioner.getPartition(WeightedRangePartitioner.java:56
    )
    at
    org.apache.hadoop.mapred.MapTask$OldOutputCollector.collect(MapTask.java
    :466)
    at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduc
    e$Map.collect(PigMapReduce.java:109)
    at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.
    runPipeline(PigMapBase.java:255)
    at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.
    map(PigMapBase.java:244)
    at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduc
    e$Map.map(PigMapReduce.java:94)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
    at
    org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
    at org.apache.hadoop.mapred.Child.main(Child.java:170)

    Pig Stack Trace
    ---------------
    ERROR 6015: During execution, encountered a Hadoop error.

    org.apache.pig.backend.executionengine.ExecException: ERROR 6015: During
    execution, encountered a Hadoop error.
    at
    .apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigLongRawComp
    arator.compare(PigLongRawComparator.java:94)
    at .util.Arrays.binarySearch0(Arrays.java:2105)
    at .util.Arrays.binarySearch(Arrays.java:2043)
    at
    .apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.W
    eightedRangePartitioner.getPartition(WeightedRangePartitioner.java:65)
    at
    .apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.W
    eightedRangePartitioner.getPartition(WeightedRangePartitioner.java:56)
    at
    .apache.hadoop.mapred.MapTask$OldOutputCollector.collect(MapTask.java:46
    6)
    at
    .apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$M
    ap.collect(PigMapReduce.java:109)
    at
    .apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.run
    Pipeline(PigMapBase.java:255)
    at
    .apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map
    (PigMapBase.java:244)
    at
    .apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$M
    ap.map(PigMapReduce.java:94)
    at .apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
    at .apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
    at .apache.hadoop.mapred.MapTask.run(MapTask.java:307)
    Caused by: java.lang.ClassCastException:
    org.apache.pig.impl.io.NullableText cannot be cast to
    org.apache.pig.impl.io.NullableLongWritable
    ... 13 more
    ========================================================================
    ========

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedApr 15, '11 at 2:47p
activeApr 15, '11 at 6:53p
posts2
users2
websitepig.apache.org

2 users in discussion

Brian Adams: 1 post Daniel Dai: 1 post

People

Translate

site design / logo © 2021 Grokbase