Grokbase › Groups › Hive › user › June 2010
FAQ
I've run into a problem using the Hive JDBC driver, with tables that contain
floats. The following two statements work fine in the hive shell, but break
if I execute them through the JDBC driver.

I can create a table like this:

CREATE TABLE simple (
id int,
value string,
amt float
)
ROW FORMAT DELIMITED
STORED AS TEXTFILE

Which works ok. But then I try this:

select * from simple

This fails with an exception

java.sql.SQLException: Could not create ResultSet:
java.lang.RuntimeException: specifying type float which has not been defined
at
org.apache.hadoop.hive.jdbc.HiveResultSet.initDynamicSerde(HiveResultSet.java:120)
at
org.apache.hadoop.hive.jdbc.HiveResultSet.(HiveStatement.java:178)
...

I'm using Hive 0.5.0, but it's the Cloudera distribution (0.5.0+20). This
is all running locally on my mac on a standalone cluster.

Also, if I use type "double" instead of "float", then it works. Which is a
suitable work around for my needs, but I'm wondering what's going on.

Marc

Search Discussions

  • Zhou Shuaifeng at Jun 24, 2010 at 10:34 am ⇧
    Float is not supported because it's not supported by Thrift, and Hive uses
    Thrift a lot for internal data serialization/deserialization.
    See http://wiki.apache.org/hadoop/Hive/HiveQL/Types

    Shuaifeng
    _____

    From: Marc Limotte
    Sent: Thursday, June 24, 2010 10:11 AM
    To: hive-user@hadoop.apache.org
    Subject: Hive problem when using JDBC driver and type float


    I've run into a problem using the Hive JDBC driver, with tables that contain
    floats. The following two statements work fine in the hive shell, but break
    if I execute them through the JDBC driver.

    I can create a table like this:


    CREATE TABLE simple (
    id int,
    value string,
    amt float
    )
    ROW FORMAT DELIMITED
    STORED AS TEXTFILE


    Which works ok. But then I try this:


    select * from simple


    This fails with an exception


    java.sql.SQLException: Could not create ResultSet:
    java.lang.RuntimeException: specifying type float which has not been defined
    at
    org.apache.hadoop.hive.jdbc.HiveResultSet.initDynamicSerde(HiveResultSet.jav
    a:120)
    at
    org.apache.hadoop.hive.jdbc.HiveResultSet.(HiveStatement.java:17
    8)
    ...


    I'm using Hive 0.5.0, but it's the Cloudera distribution (0.5.0+20). This
    is all running locally on my mac on a standalone cluster.

    Also, if I use type "double" instead of "float", then it works. Which is a
    suitable work around for my needs, but I'm wondering what's going on.

    Marc
  • Ashutosh Chauhan at Jun 24, 2010 at 5:04 pm ⇧
    Hi Shuaifeng,
    Float is not supported because it's not supported by Thrift,
    But I see float listed as a valid primitive type on
    http://wiki.apache.org/hadoop/Hive/HiveQL and also in various places
    in source tree. (e.g.,
    o.a.h.h.serde2.objectinspector.PrimitiveObjectInspector) But wiki page
    that you listed indeed doesnt have it. Looking further in Thrift in
    o.a.thrift.protocol.TType float is not listed as a type. So, I am now
    confused whether it really is one of the supported types or not. Is it
    supported only in case where Thrift is not used for ser-deser ? Can
    someone please clarify.

    Thanks,
    Ashutosh
    On Thu, Jun 24, 2010 at 03:33, Zhou Shuaifeng wrote:
    Float is not supported because it's not supported by Thrift, and Hive uses
    Thrift a lot for internal data serialization/deserialization.
    See http://wiki.apache.org/hadoop/Hive/HiveQL/Types

    Shuaifeng
    ________________________________
    From: Marc Limotte
    Sent: Thursday, June 24, 2010 10:11 AM
    To: hive-user@hadoop.apache.org
    Subject: Hive problem when using JDBC driver and type float

    I've run into a problem using the Hive JDBC driver, with tables that contain
    floats.  The following two statements work fine in the hive shell, but break
    if I execute them through the JDBC driver.

    I can create a table like this:

    CREATE TABLE simple (
    id int,
    value string,
    amt float
    )
    ROW FORMAT DELIMITED
    STORED AS TEXTFILE

    Which works ok.  But then I try this:

    select * from simple

    This fails with an exception

    java.sql.SQLException: Could not create ResultSet:
    java.lang.RuntimeException: specifying type float which has not been defined
    at
    org.apache.hadoop.hive.jdbc.HiveResultSet.initDynamicSerde(HiveResultSet.java:120)
    at
    org.apache.hadoop.hive.jdbc.HiveResultSet.<init>(HiveResultSet.java:74)
    at
    org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:178)
    ...

    I'm using Hive 0.5.0, but it's the Cloudera distribution (0.5.0+20).  This
    is all running locally on my mac on a standalone cluster.

    Also, if I use type "double" instead of "float", then it works.  Which is a
    suitable work around for my needs, but I'm wondering what's going on.

    Marc

  • Zhou Shuaifeng at Jun 25, 2010 at 1:08 am ⇧
    Hi Marc,

    Yes, there seems conflict between these two pages. I can't explain it also.
    Obviously it's a bug need to be improved. May be you can send mail to the
    hive-dev mail list to feed back this problem and find answers.
    And you may use double instead of float temporarily.

    Regards,
    Shuaifeng

    -----Original Message-----
    From: Ashutosh Chauhan
    Sent: Friday, June 25, 2010 1:03 AM
    To: hive-user@hadoop.apache.org
    Subject: Re: Hive problem when using JDBC driver and type float

    Hi Shuaifeng,
    Float is not supported because it's not supported by Thrift,
    But I see float listed as a valid primitive type on
    http://wiki.apache.org/hadoop/Hive/HiveQL and also in various places in
    source tree. (e.g.,
    o.a.h.h.serde2.objectinspector.PrimitiveObjectInspector) But wiki page that
    you listed indeed doesnt have it. Looking further in Thrift in
    o.a.thrift.protocol.TType float is not listed as a type. So, I am now
    confused whether it really is one of the supported types or not. Is it
    supported only in case where Thrift is not used for ser-deser ? Can someone
    please clarify.

    Thanks,
    Ashutosh
    On Thu, Jun 24, 2010 at 03:33, Zhou Shuaifeng wrote:
    Float is not supported because it's not supported by Thrift, and Hive
    uses Thrift a lot for internal data serialization/deserialization.
    See http://wiki.apache.org/hadoop/Hive/HiveQL/Types

    Shuaifeng
    ________________________________
    From: Marc Limotte
    Sent: Thursday, June 24, 2010 10:11 AM
    To: hive-user@hadoop.apache.org
    Subject: Hive problem when using JDBC driver and type float

    I've run into a problem using the Hive JDBC driver, with tables that
    contain floats.  The following two statements work fine in the hive
    shell, but break if I execute them through the JDBC driver.

    I can create a table like this:

    CREATE TABLE simple (
    id int,
    value string,
    amt float
    )
    ROW FORMAT DELIMITED
    STORED AS TEXTFILE

    Which works ok.  But then I try this:

    select * from simple

    This fails with an exception

    java.sql.SQLException: Could not create ResultSet:
    java.lang.RuntimeException: specifying type float which has not been
    defined
    at
    org.apache.hadoop.hive.jdbc.HiveResultSet.initDynamicSerde(HiveResultS
    et.java:120)
    at
    org.apache.hadoop.hive.jdbc.HiveResultSet.<init>(HiveResultSet.java:74
    )
    at
    org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.j
    ava:178)
    ...

    I'm using Hive 0.5.0, but it's the Cloudera distribution (0.5.0+20).
    This is all running locally on my mac on a standalone cluster.

    Also, if I use type "double" instead of "float", then it works.  Which
    is a suitable work around for my needs, but I'm wondering what's going on.

    Marc

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshive, hadoop
postedJun 24, '10 at 2:11a
activeJun 25, '10 at 1:08a
posts4
users3
websitehive.apache.org

3 users in discussion

Zhou Shuaifeng: 2 posts Ashutosh Chauhan: 1 post Marc Limotte: 1 post

Content

People

Support

Translate

site design / logo © 2022 Grokbase