FAQ
Hello,

I got the following exception when I made a simple query in my table.
I made the table to store email header data with the following command:

CREATE TABLE mytable (xfrom STRING, xto STRING, subject STRING, cc STRING,
xgeoip STRING, receivedspf STRING, xoriginatingip STRING, messageid STRING,
replyto STRING, xenvelopefrom STRING, contenttype STRING, xoriginatingip2
STRING, version STRING, fromaddr STRING, recaddr STRING) PARTITIONED BY
(year int,month int,day int,dat INT) ROW FORMAT SERDE
'org.apache.hadoop.hive.contrib.serde2.JsonSerde';

then I insert some data using LOAD DATA LOCAL INPATH "/mypipe" INTO TABLE
mytable PARTITION(....); in my java app.
the pipe contains json data of course;

Everything works fine, but when I want to get some data:
hive>add jar hive-json-serde-0.1.jar;
hive>from mytable select xfrom where year=2010 month=9 day=30;
or
hive>from mytable select subject where year=2010 month=9 day=30;
these works fine.
BUT
hive>from mytable select xto where year=2010 month=9 day=30;
fails with the following error:
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.ExecDriver
in the log:

java.lang.RuntimeException: java.lang.Double cannot be cast to java.lang.String
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:188)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)

at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.ClassCastException: java.lang.Double cannot be
cast to java.lang.String

at org.apache.hadoop.hive.serde2.objectinspector.primitive.JavaStringObjectInspector.getPrimitiveWritableObject(JavaStringObjectInspector.java:35)
at org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.serialize(BinarySortableSerDe.java:506)

at org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.serialize(BinarySortableSerDe.java:399)
at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:162)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)

at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)
at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:81)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)

at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)
at org.apache.hadoop.hive.ql.exec.FilterOperator.processOp(FilterOperator.java:73)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)

at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)
at org.apache.hadoop.hive.ql.exec.FilterOperator.processOp(FilterOperator.java:73)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)

at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)
at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:43)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)

at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:350)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:171)

... 4 more


I don't know where is this Double variable, everything must be stored as
String.
xto field contanins the To field of the email header.


Thanks
Zoltan

Search Discussions

  • Steven Wong at Sep 30, 2010 at 10:26 pm
    I am not familiar with JsonSerde, but your data might contain doubles that you didn't know about. LOAD does not check whether the data's actual types match the types specified in the schema.


    From: Hock Zoltán
    Sent: Thursday, September 30, 2010 8:46 AM
    To: hive-user@hadoop.apache.org
    Subject: java.lang.RuntimeException: java.lang.Double cannot be cast to java.lang.String

    Hello,

    I got the following exception when I made a simple query in my table.
    I made the table to store email header data with the following command:

    CREATE TABLE mytable (xfrom STRING, xto STRING, subject STRING, cc STRING, xgeoip STRING, receivedspf STRING, xoriginatingip STRING, messageid STRING, replyto STRING, xenvelopefrom STRING, contenttype STRING, xoriginatingip2 STRING, version STRING, fromaddr STRING, recaddr STRING) PARTITIONED BY (year int,month int,day int,dat INT) ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.JsonSerde';

    then I insert some data using LOAD DATA LOCAL INPATH "/mypipe" INTO TABLE mytable PARTITION(....); in my java app.
    the pipe contains json data of course;

    Everything works fine, but when I want to get some data:
    hive>add jar hive-json-serde-0.1.jar;
    hive>from mytable select xfrom where year=2010 month=9 day=30;
    or
    hive>from mytable select subject where year=2010 month=9 day=30;
    these works fine.
    BUT
    hive>from mytable select xto where year=2010 month=9 day=30;
    fails with the following error:
    FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.ExecDriver
    in the log:

    java.lang.RuntimeException: java.lang.Double cannot be cast to java.lang.String

    at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:188)

    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)







    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)

    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)

    at org.apache.hadoop.mapred.Child.main(Child.java:170)

    Caused by: java.lang.ClassCastException: java.lang.Double cannot be cast to java.lang.String







    at org.apache.hadoop.hive.serde2.objectinspector.primitive.JavaStringObjectInspector.getPrimitiveWritableObject(JavaStringObjectInspector.java:35)

    at org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.serialize(BinarySortableSerDe.java:506)







    at org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.serialize(BinarySortableSerDe.java:399)

    at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:162)

    at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)







    at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)

    at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:81)

    at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)







    at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)

    at org.apache.hadoop.hive.ql.exec.FilterOperator.processOp(FilterOperator.java:73)

    at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)







    at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)

    at org.apache.hadoop.hive.ql.exec.FilterOperator.processOp(FilterOperator.java:73)

    at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)







    at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)

    at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:43)

    at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)







    at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)

    at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:350)

    at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:171)







    ... 4 more

    I don't know where is this Double variable, everything must be stored as String.
    xto field contanins the To field of the email header.


    Thanks
    Zoltan
  • Hock Zoltán at Oct 1, 2010 at 8:06 am
    Thank you for answering,
    Is there any way to check and fix types?

    Zoltan

    2010/10/1 Steven Wong <swong@netflix.com>
    I am not familiar with JsonSerde, but your data might contain doubles
    that you didn’t know about. LOAD does not check whether the data’s actual
    types match the types specified in the schema.





    *From:* Hock Zoltán
    *Sent:* Thursday, September 30, 2010 8:46 AM
    *To:* hive-user@hadoop.apache.org
    *Subject:* java.lang.RuntimeException: java.lang.Double cannot be cast to
    java.lang.String



    Hello,

    I got the following exception when I made a simple query in my table.
    I made the table to store email header data with the following command:

    CREATE TABLE mytable (xfrom STRING, xto STRING, subject STRING, cc STRING,
    xgeoip STRING, receivedspf STRING, xoriginatingip STRING, messageid STRING,
    replyto STRING, xenvelopefrom STRING, contenttype STRING, xoriginatingip2
    STRING, version STRING, fromaddr STRING, recaddr STRING) PARTITIONED BY
    (year int,month int,day int,dat INT) ROW FORMAT SERDE
    'org.apache.hadoop.hive.contrib.serde2.JsonSerde';

    then I insert some data using LOAD DATA LOCAL INPATH "/mypipe" INTO TABLE
    mytable PARTITION(....); in my java app.
    the pipe contains json data of course;

    Everything works fine, but when I want to get some data:
    hive>add jar hive-json-serde-0.1.jar;
    hive>from mytable select xfrom where year=2010 month=9 day=30;
    or
    hive>from mytable select subject where year=2010 month=9 day=30;
    these works fine.
    BUT
    hive>from mytable select xto where year=2010 month=9 day=30;
    fails with the following error:
    FAILED: Execution Error, return code 2 from
    org.apache.hadoop.hive.ql.exec.ExecDriver
    in the log:

    java.lang.RuntimeException: java.lang.Double cannot be cast to java.lang.String

    at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:188)

    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)



    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)

    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)

    at org.apache.hadoop.mapred.Child.main(Child.java:170)

    Caused by: java.lang.ClassCastException: java.lang.Double cannot be cast to java.lang.String



    at org.apache.hadoop.hive.serde2.objectinspector.primitive.JavaStringObjectInspector.getPrimitiveWritableObject(JavaStringObjectInspector.java:35)

    at org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.serialize(BinarySortableSerDe.java:506)



    at org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.serialize(BinarySortableSerDe.java:399)

    at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:162)

    at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)



    at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)

    at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:81)

    at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)



    at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)

    at org.apache.hadoop.hive.ql.exec.FilterOperator.processOp(FilterOperator.java:73)

    at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)



    at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)

    at org.apache.hadoop.hive.ql.exec.FilterOperator.processOp(FilterOperator.java:73)

    at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)



    at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)

    at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:43)

    at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)



    at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)

    at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:350)

    at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:171)



    ... 4 more


    I don't know where is this Double variable, everything must be stored as
    String.
    xto field contanins the To field of the email header.


    Thanks
    Zoltan
  • Steven Wong at Oct 1, 2010 at 6:15 pm
    One way is to inspect the data files in the table to see if there's any wrongly typed data in them. First, find the directory of your table where the files are, using Hive's "describe extended <tableName>". Then, use hadoop fs -ls to find the files and hadoop fs -cat (or -get or whatever your favorite way is) to get at the file contents.


    From: Hock Zoltán
    Sent: Friday, October 01, 2010 1:06 AM
    To: hive-user@hadoop.apache.org
    Subject: Re: java.lang.RuntimeException: java.lang.Double cannot be cast to java.lang.String

    Thank you for answering,
    Is there any way to check and fix types?

    Zoltan
    2010/10/1 Steven Wong <swong@netflix.com
    I am not familiar with JsonSerde, but your data might contain doubles that you didn't know about. LOAD does not check whether the data's actual types match the types specified in the schema.


    From: Hock Zoltán
    Sent: Thursday, September 30, 2010 8:46 AM
    To: hive-user@hadoop.apache.org
    Subject: java.lang.RuntimeException: java.lang.Double cannot be cast to java.lang.String

    Hello,

    I got the following exception when I made a simple query in my table.
    I made the table to store email header data with the following command:

    CREATE TABLE mytable (xfrom STRING, xto STRING, subject STRING, cc STRING, xgeoip STRING, receivedspf STRING, xoriginatingip STRING, messageid STRING, replyto STRING, xenvelopefrom STRING, contenttype STRING, xoriginatingip2 STRING, version STRING, fromaddr STRING, recaddr STRING) PARTITIONED BY (year int,month int,day int,dat INT) ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.JsonSerde';

    then I insert some data using LOAD DATA LOCAL INPATH "/mypipe" INTO TABLE mytable PARTITION(....); in my java app.
    the pipe contains json data of course;

    Everything works fine, but when I want to get some data:
    hive>add jar hive-json-serde-0.1.jar;
    hive>from mytable select xfrom where year=2010 month=9 day=30;
    or
    hive>from mytable select subject where year=2010 month=9 day=30;
    these works fine.
    BUT
    hive>from mytable select xto where year=2010 month=9 day=30;
    fails with the following error:
    FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.ExecDriver
    in the log:

    java.lang.RuntimeException: java.lang.Double cannot be cast to java.lang.String





    at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:188)





    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)












    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)





    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)





    at org.apache.hadoop.mapred.Child.main(Child.java:170)





    Caused by: java.lang.ClassCastException: java.lang.Double cannot be cast to java.lang.String












    at org.apache.hadoop.hive.serde2.objectinspector.primitive.JavaStringObjectInspector.getPrimitiveWritableObject(JavaStringObjectInspector.java:35)





    at org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.serialize(BinarySortableSerDe.java:506)












    at org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.serialize(BinarySortableSerDe.java:399)





    at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:162)





    at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)












    at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)





    at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:81)





    at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)












    at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)





    at org.apache.hadoop.hive.ql.exec.FilterOperator.processOp(FilterOperator.java:73)





    at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)












    at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)





    at org.apache.hadoop.hive.ql.exec.FilterOperator.processOp(FilterOperator.java:73)





    at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)












    at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)





    at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:43)





    at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)












    at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)





    at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:350)





    at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:171)












    ... 4 more

    I don't know where is this Double variable, everything must be stored as String.
    xto field contanins the To field of the email header.


    Thanks
    Zoltan

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshive, hadoop
postedSep 30, '10 at 3:46p
activeOct 1, '10 at 6:15p
posts4
users2
websitehive.apache.org

2 users in discussion

Hock Zoltán: 2 posts Steven Wong: 2 posts

People

Translate

site design / logo © 2022 Grokbase