FAQ
Hi All,

We've setup CDH4 at our site at University of Ghent but I have an issue
when I try to connect Pentaho to Hive through JDBC. It just hangs and when
I look into the Hiveserver log I see the following:

2013-04-09 15:02:30,215 INFO
org.apache.hive.service.cli.thrift.ThriftCLIService: ThriftCLIService
listening on 0.0.0.0/0.0.0.0:10000
2013-04-09 15:05:31,105 DEBUG
org.apache.thrift.transport.TSaslServerTransport: transport map does not
contain key
2013-04-09 15:05:31,108 DEBUG org.apache.thrift.transport.TSaslTransport:
opening transport org.apache.thrift.transport.TSaslServerTransport@a7e8c4d
transport map does not contain key

Any ideas?

Many thanks!

Cedric

--

Search Discussions

  • Hendra Budiawan at Apr 10, 2013 at 2:50 am
    Did you have start hive thrift server ? if not, you must start it first,

    hive --service hive-server

    if you want run on background process try this

    nohup HIVE_PORT=10000 hive --service hive-server &

    To check service of hive-server use this
    netstat | grep 10000

    Good Luck,
    Regarsd.

    On Tue, Apr 9, 2013 at 7:17 PM, Cedric De Vroey wrote:

    Hi All,

    We've setup CDH4 at our site at University of Ghent but I have an issue
    when I try to connect Pentaho to Hive through JDBC. It just hangs and when
    I look into the Hiveserver log I see the following:

    2013-04-09 15:02:30,215 INFO
    org.apache.hive.service.cli.thrift.ThriftCLIService: ThriftCLIService
    listening on 0.0.0.0/0.0.0.0:10000
    2013-04-09 15:05:31,105 DEBUG
    org.apache.thrift.transport.TSaslServerTransport: transport map does not
    contain key
    2013-04-09 15:05:31,108 DEBUG org.apache.thrift.transport.TSaslTransport:
    opening transport org.apache.thrift.transport.TSaslServerTransport@a7e8c4d
    transport map does not contain key

    Any ideas?

    Many thanks!

    Cedric

    --


    --
  • Hendra Budiawan at Apr 10, 2013 at 3:06 am
    Sorry Cedric, this the right command to start hive trift server

    hive --service hiveserver
    In background use this
    nohup HIVE_PORT=10000 hive --service hiveserver &

    And try again to connect from pentaho to hive server using port 10000

    On Wed, Apr 10, 2013 at 9:50 AM, Hendra Budiawan wrote:

    Did you have start hive thrift server ? if not, you must start it first,

    hive --service hive-server

    if you want run on background process try this

    nohup HIVE_PORT=10000 hive --service hive-server &

    To check service of hive-server use this
    netstat | grep 10000

    Good Luck,
    Regarsd.

    On Tue, Apr 9, 2013 at 7:17 PM, Cedric De Vroey wrote:

    Hi All,

    We've setup CDH4 at our site at University of Ghent but I have an issue
    when I try to connect Pentaho to Hive through JDBC. It just hangs and when
    I look into the Hiveserver log I see the following:

    2013-04-09 15:02:30,215 INFO
    org.apache.hive.service.cli.thrift.ThriftCLIService: ThriftCLIService
    listening on 0.0.0.0/0.0.0.0:10000
    2013-04-09 15:05:31,105 DEBUG
    org.apache.thrift.transport.TSaslServerTransport: transport map does not
    contain key
    2013-04-09 15:05:31,108 DEBUG
    org.apache.thrift.transport.TSaslTransport: opening transport
    org.apache.thrift.transport.TSaslServerTransport@a7e8c4d
    transport map does not contain key

    Any ideas?

    Many thanks!

    Cedric

    --


    --
  • Cedric De Vroey at Apr 10, 2013 at 11:26 am
    Hi Hendra,

    Thank you for your help! Yes, the hiveserver is running:

    hive 22485 0.0 1.6 1007220 132516 ? Sl Apr09 0:37
    /usr/lib/jvm/j2sdk1.6-oracle/bin/java -Xmx256m
    -Djava.net.preferIPv4Stack=true
    -Dhadoop.log.dir=/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/hadoop/logs
    -Dhadoop.log.file=hadoop.log
    -Dhadoop.home.dir=/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/hadoop
    -Dhadoop.id.str= -Dhadoop.root.logger=INFO,console
    -Djava.library.path=/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/hadoop/lib/native
    -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true
    -Djava.net.preferIPv4Stack=true -Xmx268435456
    -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar
    /opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/hive/lib/hive-service-0.10.0-cdh4.2.0.jar
    org.apache.hive.service.server.HiveServer2
    hive 22539 0.0 1.7 1007492 141268 ? Sl Apr09 0:42
    /usr/lib/jvm/j2sdk1.6-oracle/bin/java -Xmx256m
    -Djava.net.preferIPv4Stack=true
    -Dhadoop.log.dir=/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/hadoop/logs
    -Dhadoop.log.file=hadoop.log
    -Dhadoop.home.dir=/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/hadoop
    -Dhadoop.id.str= -Dhadoop.root.logger=INFO,console
    -Djava.library.path=/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/hadoop/lib/native
    -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true
    -Djava.net.preferIPv4Stack=true -Xmx268435456
    -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar
    /opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/hive/lib/hive-service-0.10.0-cdh4.2.0.jar
    org.apache.hadoop.hive.metastore.HiveMetaStore -p 9083
    124 22677 0.0 0.1 133224 10320 ? Ss Apr09 0:00 postgres:
    hive hive 10.11.12.200(24901) idle

    124 22678 0.0 0.0 129464 5984 ? Ss Apr09 0:00 postgres:
    hive hive 10.11.12.200(24902) idle

    We use Cloudera Manager to configure it and we used Hiveserver2.

    When I connect using a JDBC connection in Kettle, the client just hangs
    until I restart the hiveserver2 instance. Then it throws me this error:

    Error connecting to database: (using class
    org.apache.hadoop.hive.jdbc.HiveDriver)
    org.apache.thrift.transport.TTransportException

    org.pentaho.di.core.exception.KettleDatabaseException:
    Error occured while trying to connect to the database

    Error connecting to database: (using class
    org.apache.hadoop.hive.jdbc.HiveDriver)
    org.apache.thrift.transport.TTransportException

    at
    org.pentaho.di.core.database.Database.normalConnect(Database.java:366)
    .....

    at org.pentaho.commons.launcher.Launcher.main(Launcher.java:134)

    Caused by: org.pentaho.di.core.exception.KettleDatabaseException:

    Error connecting to database: (using class
    org.apache.hadoop.hive.jdbc.HiveDriver)
    org.apache.thrift.transport.TTransportException

    at
    org.pentaho.di.core.database.Database.connectUsingClass(Database.java:502)
    at org.pentaho.di.core.database.Database.normalConnect(Database.java:350)

    ... 44 more

    Caused by: java.sql.SQLException:
    org.apache.thrift.transport.TTransportException at
    org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:191) at
    org.apache.hadoop.hive.jdbc.HiveStatement.execute(HiveStatement.java:127) at
    org.apache.hadoop.hive.jdbc.HiveConnection.configureConnection(HiveConnection.java:136) at
    org.apache.hadoop.hive.jdbc.HiveConnection.<init>(HiveConnection.java:131)
    at org.apache.hadoop.hive.jdbc.HiveDriver.connect(HiveDriver.java:104)

    at org.apache.hadoop.hive.jdbc.HiveDriver$1.call(HiveDriver.java:126)

    at org.apache.hadoop.hive.jdbc.HiveDriver$1.call(HiveDriver.java:123)

    at
    org.apache.hadoop.hive.jdbc.JDBCDriverCallable.callWithDriver(JDBCDriverCallable.java:57) at
    org.apache.hadoop.hive.jdbc.HiveDriver.callWithActiveDriver(HiveDriver.java:112)
    at org.apache.hadoop.hive.jdbc.HiveDriver.connect(HiveDriver.java:123)

    at java.sql.DriverManager.getConnection(Unknown Source)

    at java.sql.DriverManager.getConnection(Unknown Source)

    at
    org.pentaho.di.core.database.Database.connectUsingClass(Database.java:488)
    ... 45 more

    Hostname : localhost
    Port : 10000

    Database name : default
    FYI: I'm connecting through an ssh-connection, so on the client I connect
    to localhost:10000 which is tunneled to cdh400:10000 on the other side of
    the ssh connection.



    On Wed, Apr 10, 2013 at 5:06 AM, Hendra Budiawan wrote:

    Sorry Cedric, this the right command to start hive trift server

    hive --service hiveserver
    In background use this
    nohup HIVE_PORT=10000 hive --service hiveserver &

    And try again to connect from pentaho to hive server using port 10000


    On Wed, Apr 10, 2013 at 9:50 AM, Hendra Budiawan <
    hendra.budiawan.01@gmail.com> wrote:
    Did you have start hive thrift server ? if not, you must start it first,

    hive --service hive-server

    if you want run on background process try this

    nohup HIVE_PORT=10000 hive --service hive-server &

    To check service of hive-server use this
    netstat | grep 10000

    Good Luck,
    Regarsd.


    On Tue, Apr 9, 2013 at 7:17 PM, Cedric De Vroey <cedric.devroey@gmail.com
    wrote:
    Hi All,

    We've setup CDH4 at our site at University of Ghent but I have an issue
    when I try to connect Pentaho to Hive through JDBC. It just hangs and when
    I look into the Hiveserver log I see the following:

    2013-04-09 15:02:30,215 INFO
    org.apache.hive.service.cli.thrift.ThriftCLIService: ThriftCLIService
    listening on 0.0.0.0/0.0.0.0:10000
    2013-04-09 15:05:31,105 DEBUG
    org.apache.thrift.transport.TSaslServerTransport: transport map does not
    contain key
    2013-04-09 15:05:31,108 DEBUG
    org.apache.thrift.transport.TSaslTransport: opening transport
    org.apache.thrift.transport.TSaslServerTransport@a7e8c4d
    transport map does not contain key

    Any ideas?

    Many thanks!

    Cedric

    --


    --




    --
    Cedric De Vroey

    --
  • Cedric De Vroey at Apr 10, 2013 at 11:37 am
    Maybe something in the hiveserver2-log that you can spot? I've attached
    it...


    On Wed, Apr 10, 2013 at 1:25 PM, Cedric De Vroey
    wrote:
    Hi Hendra,

    Thank you for your help! Yes, the hiveserver is running:

    hive 22485 0.0 1.6 1007220 132516 ? Sl Apr09 0:37
    /usr/lib/jvm/j2sdk1.6-oracle/bin/java -Xmx256m
    -Djava.net.preferIPv4Stack=true
    -Dhadoop.log.dir=/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/hadoop/logs
    -Dhadoop.log.file=hadoop.log
    -Dhadoop.home.dir=/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/hadoop
    -Dhadoop.id.str= -Dhadoop.root.logger=INFO,console
    -Djava.library.path=/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/hadoop/lib/native
    -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true
    -Djava.net.preferIPv4Stack=true -Xmx268435456
    -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar
    /opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/hive/lib/hive-service-0.10.0-cdh4.2.0.jar
    org.apache.hive.service.server.HiveServer2
    hive 22539 0.0 1.7 1007492 141268 ? Sl Apr09 0:42
    /usr/lib/jvm/j2sdk1.6-oracle/bin/java -Xmx256m
    -Djava.net.preferIPv4Stack=true
    -Dhadoop.log.dir=/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/hadoop/logs
    -Dhadoop.log.file=hadoop.log
    -Dhadoop.home.dir=/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/hadoop
    -Dhadoop.id.str= -Dhadoop.root.logger=INFO,console
    -Djava.library.path=/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/hadoop/lib/native
    -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true
    -Djava.net.preferIPv4Stack=true -Xmx268435456
    -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar
    /opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/hive/lib/hive-service-0.10.0-cdh4.2.0.jar
    org.apache.hadoop.hive.metastore.HiveMetaStore -p 9083
    124 22677 0.0 0.1 133224 10320 ? Ss Apr09 0:00
    postgres: hive hive 10.11.12.200(24901) idle

    124 22678 0.0 0.0 129464 5984 ? Ss Apr09 0:00
    postgres: hive hive 10.11.12.200(24902) idle

    We use Cloudera Manager to configure it and we used Hiveserver2.

    When I connect using a JDBC connection in Kettle, the client just hangs
    until I restart the hiveserver2 instance. Then it throws me this error:

    Error connecting to database: (using class
    org.apache.hadoop.hive.jdbc.HiveDriver)
    org.apache.thrift.transport.TTransportException

    org.pentaho.di.core.exception.KettleDatabaseException:
    Error occured while trying to connect to the database

    Error connecting to database: (using class
    org.apache.hadoop.hive.jdbc.HiveDriver)
    org.apache.thrift.transport.TTransportException

    at
    org.pentaho.di.core.database.Database.normalConnect(Database.java:366)
    .....

    at org.pentaho.commons.launcher.Launcher.main(Launcher.java:134)

    Caused by: org.pentaho.di.core.exception.KettleDatabaseException:

    Error connecting to database: (using class
    org.apache.hadoop.hive.jdbc.HiveDriver)
    org.apache.thrift.transport.TTransportException

    at
    org.pentaho.di.core.database.Database.connectUsingClass(Database.java:502) at
    org.pentaho.di.core.database.Database.normalConnect(Database.java:350)
    ... 44 more

    Caused by: java.sql.SQLException:
    org.apache.thrift.transport.TTransportException at
    org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:191) at
    org.apache.hadoop.hive.jdbc.HiveStatement.execute(HiveStatement.java:127) at
    org.apache.hadoop.hive.jdbc.HiveConnection.configureConnection(HiveConnection.java:136) at
    org.apache.hadoop.hive.jdbc.HiveConnection.<init>(HiveConnection.java:131)
    at org.apache.hadoop.hive.jdbc.HiveDriver.connect(HiveDriver.java:104)

    at org.apache.hadoop.hive.jdbc.HiveDriver$1.call(HiveDriver.java:126)

    at org.apache.hadoop.hive.jdbc.HiveDriver$1.call(HiveDriver.java:123)

    at
    org.apache.hadoop.hive.jdbc.JDBCDriverCallable.callWithDriver(JDBCDriverCallable.java:57) at
    org.apache.hadoop.hive.jdbc.HiveDriver.callWithActiveDriver(HiveDriver.java:112)
    at org.apache.hadoop.hive.jdbc.HiveDriver.connect(HiveDriver.java:123)

    at java.sql.DriverManager.getConnection(Unknown Source)

    at java.sql.DriverManager.getConnection(Unknown Source)

    at
    org.pentaho.di.core.database.Database.connectUsingClass(Database.java:488)
    ... 45 more

    Hostname : localhost
    Port : 10000

    Database name : default
    FYI: I'm connecting through an ssh-connection, so on the client I connect
    to localhost:10000 which is tunneled to cdh400:10000 on the other side of
    the ssh connection.




    On Wed, Apr 10, 2013 at 5:06 AM, Hendra Budiawan <
    hendra.budiawan.01@gmail.com> wrote:
    Sorry Cedric, this the right command to start hive trift server

    hive --service hiveserver
    In background use this
    nohup HIVE_PORT=10000 hive --service hiveserver &

    And try again to connect from pentaho to hive server using port 10000


    On Wed, Apr 10, 2013 at 9:50 AM, Hendra Budiawan <
    hendra.budiawan.01@gmail.com> wrote:
    Did you have start hive thrift server ? if not, you must start it first,

    hive --service hive-server

    if you want run on background process try this

    nohup HIVE_PORT=10000 hive --service hive-server &

    To check service of hive-server use this
    netstat | grep 10000

    Good Luck,
    Regarsd.


    On Tue, Apr 9, 2013 at 7:17 PM, Cedric De Vroey <
    cedric.devroey@gmail.com> wrote:
    Hi All,

    We've setup CDH4 at our site at University of Ghent but I have an issue
    when I try to connect Pentaho to Hive through JDBC. It just hangs and when
    I look into the Hiveserver log I see the following:

    2013-04-09 15:02:30,215 INFO
    org.apache.hive.service.cli.thrift.ThriftCLIService: ThriftCLIService
    listening on 0.0.0.0/0.0.0.0:10000
    2013-04-09 15:05:31,105 DEBUG
    org.apache.thrift.transport.TSaslServerTransport: transport map does not
    contain key
    2013-04-09 15:05:31,108 DEBUG
    org.apache.thrift.transport.TSaslTransport: opening transport
    org.apache.thrift.transport.TSaslServerTransport@a7e8c4d
    transport map does not contain key

    Any ideas?

    Many thanks!

    Cedric

    --


    --




    --
    Cedric De Vroey


    --
    Cedric De Vroey

    --

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcdh-user @
categorieshadoop
postedApr 9, '13 at 12:17p
activeApr 10, '13 at 11:37a
posts5
users2
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase