Grokbase Groups Pig user June 2011
FAQ
Hi,

Previously we had cassandra-0.7.6-2 setup installed in a cluster and
used PIG Scripts for reading data from cassandra DB.

Right now we changed our Cassandra version from Cassandra-0.7.6-2 to
Cassandra-0.8.0 and the cassandra cluster working fine now, but while
reading from cassandra using Pig-0.8.1 is throwing below error.

"2011-06-23 03:06:58,629 [Thread-13] WARN
org.apache.hadoop.mapred.LocalJobRunner - job_local_0001
java.lang.RuntimeException: Unexpected data type -1 found in stream.
at org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:478)
at org.apache.pig.data.BinInterSedes.writeTuple(BinInterSedes.java:541)
at org.apache.pig.data.BinInterSedes.writeBag(BinInterSedes.java:522)
at org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:361)
at org.apache.pig.data.BinInterSedes.writeTuple(BinInterSedes.java:541)
at org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:357)
at
org.apache.pig.impl.io.InterRecordWriter.write(InterRecordWriter.java:73)
at org.apache.pig.impl.io.InterStorage.putNext(InterStorage.java:87)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:138)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:97)
at
org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:498)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.collect(PigMapOnly.java:48)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:242)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:58)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)"

please advice.

thanks,
venkat

Search Discussions

  • Matt Davies at Jun 23, 2011 at 1:09 pm
    I've seen this before when some data was a null, and I wasn't checking for
    it. Have you tried stepping through the pig script by

    1. pig -r ...... (generates script with any variables replaced)
    2. copy / paste lines into a grunt shell

    This will tell you (most of the time) where the problem lies in your
    script.

    -Matt
    On Thu, Jun 23, 2011 at 5:26 AM, venkat wrote:

    Hi,

    Previously we had cassandra-0.7.6-2 setup installed in a cluster and used
    PIG Scripts for reading data from cassandra DB.

    Right now we changed our Cassandra version from Cassandra-0.7.6-2 to
    Cassandra-0.8.0 and the cassandra cluster working fine now, but while
    reading from cassandra using Pig-0.8.1 is throwing below error.

    "2011-06-23 03:06:58,629 [Thread-13] WARN org.apache.hadoop.mapred.**LocalJobRunner
    - job_local_0001
    java.lang.RuntimeException: Unexpected data type -1 found in stream.
    at org.apache.pig.data.**BinInterSedes.writeDatum(**
    BinInterSedes.java:478)
    at org.apache.pig.data.**BinInterSedes.writeTuple(**
    BinInterSedes.java:541)
    at org.apache.pig.data.**BinInterSedes.writeBag(**
    BinInterSedes.java:522)
    at org.apache.pig.data.**BinInterSedes.writeDatum(**
    BinInterSedes.java:361)
    at org.apache.pig.data.**BinInterSedes.writeTuple(**
    BinInterSedes.java:541)
    at org.apache.pig.data.**BinInterSedes.writeDatum(**
    BinInterSedes.java:357)
    at org.apache.pig.impl.io.**InterRecordWriter.write(**
    InterRecordWriter.java:73)
    at org.apache.pig.impl.io.**InterStorage.putNext(**
    InterStorage.java:87)
    at org.apache.pig.backend.hadoop.**executionengine.**mapReduceLayer.**
    PigOutputFormat$**PigRecordWriter.write(**PigOutputFormat.java:138)
    at org.apache.pig.backend.hadoop.**executionengine.**mapReduceLayer.**
    PigOutputFormat$**PigRecordWriter.write(**PigOutputFormat.java:97)
    at org.apache.hadoop.mapred.**MapTask$**NewDirectOutputCollector.**
    write(MapTask.java:498)
    at org.apache.hadoop.mapreduce.**TaskInputOutputContext.write(**
    TaskInputOutputContext.java:**80)
    at org.apache.pig.backend.hadoop.**executionengine.**
    mapReduceLayer.PigMapOnly$Map.**collect(PigMapOnly.java:48)
    at org.apache.pig.backend.hadoop.**executionengine.**
    mapReduceLayer.PigMapBase.map(**PigMapBase.java:242)
    at org.apache.pig.backend.hadoop.**executionengine.**
    mapReduceLayer.PigMapBase.map(**PigMapBase.java:58)
    at org.apache.hadoop.mapreduce.**Mapper.run(Mapper.java:144)
    at org.apache.hadoop.mapred.**MapTask.runNewMapper(MapTask.**java:621)
    at org.apache.hadoop.mapred.**MapTask.run(MapTask.java:305)
    at org.apache.hadoop.mapred.**LocalJobRunner$Job.run(**
    LocalJobRunner.java:177)"

    please advice.

    thanks,
    venkat
  • Daniel Dai at Jun 23, 2011 at 5:11 pm
    One possibility is that you produced some unknown data type in your UDF.
    Check https://issues.apache.org/jira/browse/PIG-1826.

    Daniel
    On 06/23/2011 06:08 AM, Matt Davies wrote:
    I've seen this before when some data was a null, and I wasn't checking for
    it. Have you tried stepping through the pig script by

    1. pig -r ...... (generates script with any variables replaced)
    2. copy / paste lines into a grunt shell

    This will tell you (most of the time) where the problem lies in your
    script.

    -Matt

    On Thu, Jun 23, 2011 at 5:26 AM, venkatwrote:
    Hi,

    Previously we had cassandra-0.7.6-2 setup installed in a cluster and used
    PIG Scripts for reading data from cassandra DB.

    Right now we changed our Cassandra version from Cassandra-0.7.6-2 to
    Cassandra-0.8.0 and the cassandra cluster working fine now, but while
    reading from cassandra using Pig-0.8.1 is throwing below error.

    "2011-06-23 03:06:58,629 [Thread-13] WARN org.apache.hadoop.mapred.**LocalJobRunner
    - job_local_0001
    java.lang.RuntimeException: Unexpected data type -1 found in stream.
    at org.apache.pig.data.**BinInterSedes.writeDatum(**
    BinInterSedes.java:478)
    at org.apache.pig.data.**BinInterSedes.writeTuple(**
    BinInterSedes.java:541)
    at org.apache.pig.data.**BinInterSedes.writeBag(**
    BinInterSedes.java:522)
    at org.apache.pig.data.**BinInterSedes.writeDatum(**
    BinInterSedes.java:361)
    at org.apache.pig.data.**BinInterSedes.writeTuple(**
    BinInterSedes.java:541)
    at org.apache.pig.data.**BinInterSedes.writeDatum(**
    BinInterSedes.java:357)
    at org.apache.pig.impl.io.**InterRecordWriter.write(**
    InterRecordWriter.java:73)
    at org.apache.pig.impl.io.**InterStorage.putNext(**
    InterStorage.java:87)
    at org.apache.pig.backend.hadoop.**executionengine.**mapReduceLayer.**
    PigOutputFormat$**PigRecordWriter.write(**PigOutputFormat.java:138)
    at org.apache.pig.backend.hadoop.**executionengine.**mapReduceLayer.**
    PigOutputFormat$**PigRecordWriter.write(**PigOutputFormat.java:97)
    at org.apache.hadoop.mapred.**MapTask$**NewDirectOutputCollector.**
    write(MapTask.java:498)
    at org.apache.hadoop.mapreduce.**TaskInputOutputContext.write(**
    TaskInputOutputContext.java:**80)
    at org.apache.pig.backend.hadoop.**executionengine.**
    mapReduceLayer.PigMapOnly$Map.**collect(PigMapOnly.java:48)
    at org.apache.pig.backend.hadoop.**executionengine.**
    mapReduceLayer.PigMapBase.map(**PigMapBase.java:242)
    at org.apache.pig.backend.hadoop.**executionengine.**
    mapReduceLayer.PigMapBase.map(**PigMapBase.java:58)
    at org.apache.hadoop.mapreduce.**Mapper.run(Mapper.java:144)
    at org.apache.hadoop.mapred.**MapTask.runNewMapper(MapTask.**java:621)
    at org.apache.hadoop.mapred.**MapTask.run(MapTask.java:305)
    at org.apache.hadoop.mapred.**LocalJobRunner$Job.run(**
    LocalJobRunner.java:177)"

    please advice.

    thanks,
    venkat

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedJun 23, '11 at 11:27a
activeJun 23, '11 at 5:11p
posts3
users3
websitepig.apache.org

3 users in discussion

Daniel Dai: 1 post Matt Davies: 1 post Venkat: 1 post

People

Translate

site design / logo © 2021 Grokbase