Grokbase Groups Hive user June 2009
FAQ
I am using a Hive instance using MySQL as the meta-data store. The cluster
works fine in command line mode.

The problem I am having is that when I attempt to connect using the Hive
JDBC connector I get the following error:

java.sql.SQLException:
org.jpox.jdo.exceptions.TransactionNotReadableException: Cant read fields
outside of transactions. You may want to set 'NontransactionalRead=true'.
[java] FailedObject:1[OID]org.apache.hadoop.hive.metastore.model.MDatabase
[java] at org.apache.hadoop.hive.jdbc.HiveDriver.connect(HiveDriver.java:71)
[java] at java.sql.DriverManager.getConnection(DriverManager.java:582)
[java] at java.sql.DriverManager.getConnection(DriverManager.java:185)
[java] at com.adknowledge.HiveDriver.HiveDrvr.<init>(HiveDrvr.java:27)
[java] at com.adknowledge.HiveDriver.HiveDrvr.main(HiveDrvr.java:46)
[java] java.lang.NullPointerException
[java] at com.adknowledge.HiveDriver.HiveDrvr.go(HiveDrvr.java:36)
[java] at com.adknowledge.HiveDriver.HiveDrvr.main(HiveDrvr.java:47)

This is after the basic connection to the meta-store has worked.
.
.
.
. [java] 09/06/01 17:08:41 INFO Datastore.Schema: Validating 1 foreign
key(s) for table `PARTITION_KEY_VALS`
[java] 09/06/01 17:08:41 INFO Datastore.Schema: Validating 1 unique key(s)
for table `PARTITION_KEY_VALS`
[java] 09/06/01 17:08:41 INFO Datastore.Schema: Catalog "hive", Schema ""
initialised - managing 14 classes
[java] 09/06/01 17:08:41 INFO metastore.ObjectStore: Initialized ObjectStore

.
.Has anyone tried this combination before or am I in uncharted waters?

Search Discussions

  • Jeff Hammerbacher at Jun 1, 2009 at 11:31 pm
    Hey,

    The current state of the JDBC driver for Hive is probably best described as
    "busted". See the following thread for more:
    http://markmail.org/thread/w6if4fqxgdzxq4r5.

    If you have significant expertise with JDBC, drop me a line off list.

    Regards,
    Jeff
    On Mon, Jun 1, 2009 at 3:14 PM, wrote:

    I am using a Hive instance using MySQL as the meta-data store. The cluster
    works fine in command line mode.

    The problem I am having is that when I attempt to connect using the Hive
    JDBC connector I get the following error:

    java.sql.SQLException:
    org.jpox.jdo.exceptions.TransactionNotReadableException: Cant read fields
    outside of transactions. You may want to set 'NontransactionalRead=true'.
    [java] FailedObject:1[OID]org.apache.hadoop.hive.metastore.model.MDatabase
    [java] at
    org.apache.hadoop.hive.jdbc.HiveDriver.connect(HiveDriver.java:71)
    [java] at java.sql.DriverManager.getConnection(DriverManager.java:582)
    [java] at java.sql.DriverManager.getConnection(DriverManager.java:185)
    [java] at com.adknowledge.HiveDriver.HiveDrvr.<init>(HiveDrvr.java:27)
    [java] at com.adknowledge.HiveDriver.HiveDrvr.main(HiveDrvr.java:46)
    [java] java.lang.NullPointerException
    [java] at com.adknowledge.HiveDriver.HiveDrvr.go(HiveDrvr.java:36)
    [java] at com.adknowledge.HiveDriver.HiveDrvr.main(HiveDrvr.java:47)

    This is after the basic connection to the meta-store has worked.
    .
    .
    .
    . [java] 09/06/01 17:08:41 INFO Datastore.Schema: Validating 1 foreign
    key(s) for table `PARTITION_KEY_VALS`
    [java] 09/06/01 17:08:41 INFO Datastore.Schema: Validating 1 unique key(s)
    for table `PARTITION_KEY_VALS`
    [java] 09/06/01 17:08:41 INFO Datastore.Schema: Catalog "hive", Schema ""
    initialised - managing 14 classes
    [java] 09/06/01 17:08:41 INFO metastore.ObjectStore: Initialized
    ObjectStore

    .
    .Has anyone tried this combination before or am I in uncharted waters?
  • Bill Graham at Jun 2, 2009 at 12:28 am
    I've been able to get the Hive JDBC client working with a Hive server (I'd
    reply to the referenced thread, but I joined the list since then so I don't
    have it). I haven't tried using MySQL yet for the metastore, but this is
    what my setup looks like. Give this a shot, then you should be able to swap
    out MySQL for Derby on the Hive server with a setup similar to this one:

    Hadoop setup: 4 node clister running version 0.18.3
    Hive install: hive-0.3.0-hadoop-0.18.0-bin

    - Hive Server:
    I'm running the Hive server on the same host as my namenode, using the
    default hive configs. (I have HADOOP_HOME set, but not HADOOP_VERSION or
    ANT_HOME.) From my Hive directory I use this command starts the server
    listening on the default port 10000:
    bin/hive --service hiveserver
    Since I'm using the default configs, this is using the servers local derby
    metastore. Initially I was running the JDBC client and the server on the
    same host and was running into all sorts of issues. I suspect this was due
    to conflicts with the JDBC client and the server accessing some of the same
    local Hive resources??

    - JDBC client
    I've got a groovy test script that tests the basic functionality (add/drop
    table, load, select) running on a separate host. Here's a snippet of the
    connection initialization:

    String uri = "jdbc:hive://myhostname.com:10000/default"; // for standalone
    mode
    String driverName = "org.apache.hadoop.hive.jdbc.HiveDriver";
    Class.forName(driverName);
    Connection con = DriverManager.getConnection(uri, "", "");

    This worked for me with the default hadoop JDBC jar, but there was a bug
    where the first item in the results set was returning in an infinite loop.
    This has been fixed on the hive trunk. After checking out the trunk and
    building, the new jar fixed this issue. I run the groovy script with the
    following bash script, using the patched jar:

    export HIVE_HOME=/path/to/hive
    export
    CLASSPATH=lib-ext/hive_jdbc_snapshot_778799.jar:$HIVE_HOME/lib/hive_exec.jar:$HIVE_HOME/lib/hive_service.jar:$HIVE_HOME/lib/hive_metastore.jar:$HIVE_HOME/lib/libfb303.jar:$HIVE_HOME/lib/log4j-1.2.15.jar:lib-ext/hadoop-0.18.3-core.jar
    groovy test_hive_jdbc.groovy

    If you want to swap in MySQL for derby for the matastore, you can change the
    configs on the server and leave the client as is.

    HTH.

    Bill
    On Mon, Jun 1, 2009 at 4:31 PM, Jeff Hammerbacher wrote:

    Hey,

    The current state of the JDBC driver for Hive is probably best described as
    "busted". See the following thread for more:
    http://markmail.org/thread/w6if4fqxgdzxq4r5.

    If you have significant expertise with JDBC, drop me a line off list.

    Regards,
    Jeff

    On Mon, Jun 1, 2009 at 3:14 PM, wrote:

    I am using a Hive instance using MySQL as the meta-data store. The cluster
    works fine in command line mode.

    The problem I am having is that when I attempt to connect using the Hive
    JDBC connector I get the following error:

    java.sql.SQLException:
    org.jpox.jdo.exceptions.TransactionNotReadableException: Cant read fields
    outside of transactions. You may want to set 'NontransactionalRead=true'.
    [java] FailedObject:1[OID]org.apache.hadoop.hive.metastore.model.MDatabase
    [java] at
    org.apache.hadoop.hive.jdbc.HiveDriver.connect(HiveDriver.java:71)
    [java] at java.sql.DriverManager.getConnection(DriverManager.java:582)
    [java] at java.sql.DriverManager.getConnection(DriverManager.java:185)
    [java] at com.adknowledge.HiveDriver.HiveDrvr.<init>(HiveDrvr.java:27)
    [java] at com.adknowledge.HiveDriver.HiveDrvr.main(HiveDrvr.java:46)
    [java] java.lang.NullPointerException
    [java] at com.adknowledge.HiveDriver.HiveDrvr.go(HiveDrvr.java:36)
    [java] at com.adknowledge.HiveDriver.HiveDrvr.main(HiveDrvr.java:47)

    This is after the basic connection to the meta-store has worked.
    .
    .
    .
    . [java] 09/06/01 17:08:41 INFO Datastore.Schema: Validating 1 foreign
    key(s) for table `PARTITION_KEY_VALS`
    [java] 09/06/01 17:08:41 INFO Datastore.Schema: Validating 1 unique key(s)
    for table `PARTITION_KEY_VALS`
    [java] 09/06/01 17:08:41 INFO Datastore.Schema: Catalog "hive", Schema ""
    initialised - managing 14 classes
    [java] 09/06/01 17:08:41 INFO metastore.ObjectStore: Initialized
    ObjectStore

    .
    .Has anyone tried this combination before or am I in uncharted waters?
  • Bill Craig at Jun 4, 2009 at 7:21 pm
    After I got a seemingly good connect.

    13:39:07 INFO hive.metastore: Connected to metastore.

    I am getting the following error when I try to use Hive JDBC

    FAILED: Error in metadata:
    com.facebook.thrift.transport.TTransportException: Cannot read. Remote
    side has closed. Tried to read 872415232 bytes, but only got 52 bytes.

    I apologize if this a resend but for some reason my longer post keeps
    being bounced.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshive, hadoop
postedJun 1, '09 at 10:15p
activeJun 4, '09 at 7:21p
posts4
users3
websitehive.apache.org

People

Translate

site design / logo © 2022 Grokbase