[Hive-user] Problems using hive JDBC driver on Windows (with Squirrel SQL Client)

Hi Vijay,

Check your classpath to make sure you've got the correct hive-jdbc.jar
version built using either the trunk or the current 4.0 branch.
HiveStatement.java:390 used to throw 'java.sql.SQLException: Method not
supported' before HIVE-679 was committed. In the current code base after the
commit, the setMaxRows method is on lines 422-425.

thanks,
BIll

Thanks a lot Bill! That was my problem. I thought my code base was pretty
recent. Apparently not!!

I have a follow up question. Why does the hive jdbc driver need all of the
jar files under lib *and* the hadoop core jar file? It seems like the jdbc
driver is primarily acting as a thrift client to the thrift server.
Shouldn't the requirements then be much more light-weight?

Good question. The JDBC package relies on the hive serde2 package, which has
at least the following Hadoop dependencies:

org.apache.hadoop.conf.Configuration;
org.apache.hadoop.io.*;
org.apache.hadoop.util.StringUtils;

Someone more familiar with the serde2 package of the Hive codebase could
comment better as to why these deps exist.

SerDe may be needed to deserialize the result from Hive server. But I also thought the results are in delimited form (LazySimpleSerDe or MetadataTypedColumnSetSerDe or something like that) so it is possible to decouple hadoop jar but will take some work.


________________________________
From: Bill Graham <billgraham@gmail.com>
Reply-To: <hive-user@hadoop.apache.org>, <billgraham@gmail.com>
Date: Tue, 15 Sep 2009 22:54:50 -0700
To: Vijay <techvd@gmail.com>
Cc: <hive-user@hadoop.apache.org>
Subject: Re: Problems using hive JDBC driver on Windows (with Squirrel SQL Client)

Good question. The JDBC package relies on the hive serde2 package, which has at least the following Hadoop dependencies:

org.apache.hadoop.conf.Configuration;
org.apache.hadoop.io.*;
org.apache.hadoop.util.StringUtils;

Someone more familiar with the serde2 package of the Hive codebase could comment better as to why these deps exist.



On Tue, Sep 15, 2009 at 3:46 PM, Vijay wrote:
Thanks a lot Bill! That was my problem. I thought my code base was pretty recent. Apparently not!!

I have a follow up question. Why does the hive jdbc driver need all of the jar files under lib *and* the hadoop core jar file? It seems like the jdbc driver is primarily acting as a thrift client to the thrift server. Shouldn't the requirements then be much more light-weight?


On Tue, Sep 15, 2009 at 2:31 PM, Bill Graham wrote:
Hi Vijay,

Check your classpath to make sure you've got the correct hive-jdbc.jar version built using either the trunk or the current 4.0 branch. HiveStatement.java:390 used to throw 'java.sql.SQLException: Method not supported' before HIVE-679 was committed. In the current code base after the commit, the setMaxRows method is on lines 422-425.

thanks,
BIll


On Tue, Sep 15, 2009 at 2:13 PM, Vijay wrote:
I'm having trouble getting the hive jdbc driver to work on Windows. I'm following the Squirrel SQL Client setup from the Hive/HiveJDBCInterface wiki page. Everything works great when Squirrel SQL Client is running on Linux but on Windows it doesn't. Squirrel can connect to the hive server fine so the setup is alright. However, when I issue a query, it doesn't seem to execute at all. I see this exception in the Squirrel client:

2009-09-15 14:10:35,268 [Thread-5] ERROR net.sourceforge.squirrel_sql.client.session.SQLExecuterTask - Can't Set MaxRows
java.sql.SQLException: Method not supported
at org.apache.hadoop.hive.jdbc.HiveStatement.setMaxRows(HiveStatement.java:390)
at net.sourceforge.squirrel_sql.client.session.SQLExecuterTask.setMaxRows(SQLExecuterTask.java:318)
at net.sourceforge.squirrel_sql.client.session.SQLExecuterTask.run(SQLExecuterTask.java:157)
at net.sourceforge.squirrel_sql.fw.util.TaskExecuter.run(TaskExecuter.java:82)
at java.lang.Thread.run(Thread.java:619)

I don't seem to get this exception on Linux. I can't get the Squirrel client to not set max rows but I'm not entirely sure that's the real problem.

Any help is appreciated.
Vijay

This may be something i'd be interested in working on. My idea in general
would be to make the jdbc driver as thin and full featured as reasonable.
I'm convinced this is the best way to integrate hive with many of the
excellent tools available out thee.

Although I could get the jdbc driver to work with a couple of these tools
I'm having trouble to get it working with many others. Especially on windows
it is slightly more painful. Is there some work in progress along these
lines? Any thoughts or pointers?

Thanks in advance,
Vijay

On Sep 15, 2009 10:59 PM, "Prasad Chakka" wrote:

SerDe may be needed to deserialize the result from Hive server. But I also
thought the results are in delimited form (LazySimpleSerDe or
MetadataTypedColumnSetSerDe or something like that) so it is possible to
decouple hadoop jar but will take some work.


------------------------------
*From: *Bill Graham <billgraham@gmail.com>
*Reply-To: *<hive-user@hadoop.apache.org>, <billgraham@gmail.com>
*Date: *Tue, 15 Sep 2009 22:54:50 -0700
*To: *Vijay <techvd@gmail.com>
*Cc: *<hive-user@hadoop.apache.org>
*Subject: *Re: Problems using hive JDBC driver on Windows (with Squirrel SQL
Client)

Good question. The JDBC package relies on the hive serde2 package, which has
at least the followin...
pretty recent. Apparently not...

Additional help could certainly be used on the JDBC driver. I agree that
getting the JDBC support full-featured is a good place to focus your energy.
Guys correct me if I'm wrong, but the approach has generally been to pick a
SQL client desktop app and implement the JDBC methods need to support it.
Raghu did that with Pentaho and I did the same for Squirrel.

I don't normally use Squirrel, but I implemented to it since it's open
source and it gives great visibility in the logs/admin window into which
JDBC methods are being called that are not yet implemented and it's fairly
forgiving. (I'm a fan of DBVisualizer, but it doesn't give very much useful
info when something in the driver isn't implemented, plus it tends to
fail-fast when an optional metadata methods fails.)

Regarding what's currently being worked on in the driver, I'm working on
HIVE-795 to get better error messaing and SQLStates passed from the Hive
server to the JDBC driver. Others can comment on what else is active, but
I'd recommend searching for open JIRAs where Component/s is 'Client'. This
issue has also become somewhat of an umbrella bug for JDBC, and contains a
number of TODOs:

https://issues.apache.org/jira/browse/HIVE-576

HTH.

thanks,
BIll