FAQ
Hi,

Hue support for Hive Server 2 will come in the next 2.3 release of Hue:
https://issues.cloudera.org/browse/HUE-1075

In the meantime, Beeswax is the only alternative for submitting Hive or
Impala queries with Hue.

Romain
On Wed, Mar 20, 2013 at 6:44 AM, Serega Sheypak wrote:

Hi, I'm using CM 4.1.3 and CDH 4.1.3
I've tried to migrate to HiveServer2.

Here are my settings:

[devops@uat-beeswax ~]$ sudo yum install hive-server2
[devops@uat-beeswax ~]$ sudo yum install hive-metastore

Cloudera manager:
Beeswax Server -> Database ->
Beeswax Hive Metastore Database Type = mysql
Beeswax Hive Metastore Database Name = hivemetastoredb
Beeswax Hive Metastore Database Host = uat-beeswax.lol.ru
Beeswax Hive Metastore Database Port = 3306
Beeswax Hive Metastore Database User = hive
Beeswax Hive Metastore Database Password = hive
Ports and Addresses ->
Beeswax Metaserver Port = 9083
Beeswax Server Port = 8002
Advanced ->
Automatically Restart Process = true
Hive System Group = hive
Hive Configuration Safety Valve =
<property>
<name>hive.support.concurrency</name>
<description>Enable Hive's Table Lock Manager Service</description>
<value>true</value>
</property>

<property>
<name>hive.zookeeper.quorum</name>
<description>Zookeeper quorum used by Hive's Table Lock
Manager</description>
<value>node01.lol.ru,node02.lol,node03.lol.ru</value>
</property>


<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://uat-beeswax.lol.ru/hivemetastoredb</value>
<description>the URL of the MySQL database</description>
</property>

<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>

<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hive</value>
</property>

<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>hive</value>
</property>

<property>
<name>datanucleus.autoCreateSchema</name>
<value>false</value>
</property>

<property>
<name>datanucleus.fixedDatastore</name>
<value>true</value>
</property>

<property>
<name>hive.metastore.uris</name>
<value>thrift://uat-beeswax.lol.ru:9083</value>
<description>IP address (or fully-qualified domain name) and port of
the metastore host</description>
</property>


<property>
<name>hive.server2.thrift.min.worker.threads</name>
<value>5</value>
</property>

<property>
<name>hive.server2.thrift.max.worker.threads</name>
<value>100</value>
</property>

<property>
<name>hive.server2.thrift.port</name>
<value>10001</value>
</property>

<property>
<name>hive.server2.thrift.bind.host</name>
<value>uat-beeswax.lol.ru</value>
</property>

<property>
<name>hive.server2.authentication</name>
<value>NONE</value>
</property>

Hue, Beeswax are restarted. I'm trying to execute a query from Beeswax we
console:


MetaException(message:file:/user/hive/warehouse/scoring is not a directory
or unable to create one)



3/03/20 06:32:54 INFO parse.ParseDriver: Parse Completed
13/03/20 06:32:54 INFO parse.SemanticAnalyzer: Starting Semantic Analysis
13/03/20 06:32:54 INFO parse.SemanticAnalyzer: Creating table scoring position=22
13/03/20 06:32:54 INFO ql.Driver: Semantic Analysis Completed
13/03/20 06:32:54 INFO ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null)
13/03/20 06:32:54 INFO ql.Driver: </PERFLOG method=compile start=1363786374850 end=1363786374855 duration=5>
Hive history file=/tmp/hue/hive_job_log_hue_201303200632_619558420.txt
13/03/20 06:32:54 INFO exec.HiveHistory: Hive history file=/tmp/hue/hive_job_log_hue_201303200632_619558420.txt
13/03/20 06:32:54 ERROR conf.HiveConf: Found both hive.metastore.uris and javax.jdo.option.ConnectionURL Recommended to have exactly one of those config keyin configuration
13/03/20 06:32:54 INFO ql.Driver: <PERFLOG method=Driver.execute>
13/03/20 06:32:54 INFO ql.Driver: Starting command: CREATE EXTERNAL TABLE scoring
(
requestTime BIGINT,
contentId STRING,
hitCount BIGINT
)
PARTITIONED BY(year STRING, month STRING, day STRING, hour STRING)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'
STORED AS TEXTFILE
13/03/20 06:32:54 INFO ql.Driver: </PERFLOG method=TimeToSubmit end=1363786374885>
13/03/20 06:32:54 INFO exec.DDLTask: Default to LazySimpleSerDe for table scoring
13/03/20 06:32:54 INFO hive.log: DDL: struct scoring { i32 regionid, i32 resourcetypeid, i32 resourceid, i64 requesttime, string contentid, i64 hitcount}
13/03/20 06:32:54 INFO hive.metastore: Trying to connect to metastore with URI thrift://uat-beeswax.kyc.megafon.ru:9083
13/03/20 06:32:54 INFO hive.metastore: Connected to metastore.
FAILED: Error in metadata: MetaException(message:file:/user/hive/warehouse/scoring is not a directory or unable to create one)
13/03/20 06:32:54 ERROR exec.Task: FAILED: Error in metadata: MetaException(message:file:/user/hive/warehouse/scoring is not a directory or unable to create one)
org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:file:/user/hive/warehouse/scoring is not a directory or unable to create one)
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:545)
at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3397)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:243)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1317)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1103)
at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:312)
at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:577)
at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:566)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:337)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1312)
at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:566)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: MetaException(message:file:/user/hive/warehouse/scoring is not a directory or unable to create one)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$create_table_result.read(ThriftHiveMetastore.java:17151)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_create_table(ThriftHiveMetastore.java:694)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.create_table(ThriftHiveMetastore.java:681)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:408)
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:539)
... 19 more

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
13/03/20 06:32:54 ERROR ql.Driver: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
13/03/20 06:32:54 INFO ql.Driver: </PERFLOG method=Driver.execute start=1363786374884 end=1363786374903 duration=19>
13/03/20 06:32:54 ERROR beeswax.BeeswaxServiceImpl: Exception while processing query
BeeswaxException(message:Driver returned: 1. Errors: Hive history file=/tmp/hue/hive_job_log_hue_201303200632_619558420.txt
FAILED: Error in metadata: MetaException(message:file:/user/hive/warehouse/scoring is not a directory or unable to create one)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

What do I do wrong???
Why I can't create a table from beeswax UI?

--


Search Discussions

  • Serega Sheypak at Mar 21, 2013 at 6:42 am
    Dear Romain, please review this configuration.
    I suppose that:
    HiveServer is working on 10000 (thed efault one, as it was)
    hive-metastore works on 9083 (thrift interface)
    HiveServer2 is working on 10001 and using hive-metastore
    Metastore data itself is in MySQL DB
    beeswax is working and using hive-metastore

    1. Install packages

    [devops@uat-beeswax ~]$ sudo yum install hive-server2
    [devops@uat-beeswax ~]$ sudo yum install hive-metastore

    *(mysql, db, user have been installed/created using puppet)*
    * /etc/hive/conf/hive-site.xml *


    <?xml version="1.0"?>
    <!--
    Licensed to the Apache Software Foundation (ASF) under one or more
    contributor license agreements. See the NOTICE file distributed with
    this work for additional information regarding copyright ownership.
    The ASF licenses this file to You under the Apache License, Version 2.0
    (the "License"); you may not use this file except in compliance with
    the License. You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License.
    -->
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

    <configuration>

    <!-- Hive Configuration can either be stored in this file or in the hadoop
    configuration files -->
    <!-- that are implied by Hadoop setup variables. -->
    <!-- Aside from Hadoop setup variables - this file is provided as a
    convenience so that Hive -->
    <!-- users do not have to edit hadoop configuration files (that may be
    managed as a centralized -->
    <!-- resource). -->

    <!-- Hive Execution Parameters -->

    <property>
    <name>hive.metastore.warehouse.dir</name>
    <value>hdfs://node01.lol.ru:8020/user/hive/warehouse</value>
    </property>


    <property>
    <name>hive.metastore.local</name>
    <value>false</value>
    </property>

    <property>
    <name>hive.support.concurrency</name>
    <description>Enable Hive's Table Lock Manager Service</description>
    <value>true</value>
    </property>

    <property>
    <name>hive.zookeeper.quorum</name>
    <description>Zookeeper quorum used by Hive's Table Lock Manager</description
    >
    <value>node01.lol.ru,node02.lol.ru,node03.lol.ru</value>
    </property>

    <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://localhost:3306/hivemetastoredb</value>
    <description>the URL of the MySQL database</description>
    </property>

    <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    </property>

    <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>root</value>
    </property>

    <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>Qwe123</value>
    </property>

    <property>
    <name>datanucleus.autoCreateSchema</name>
    <value>false</value>
    </property>

    <property>
    <name>datanucleus.fixedDatastore</name>
    <value>true</value>
    </property>

    <property>
    <name>hive.metastore.uris</name>
    <value>thrift://uat-beeswax.lol.ru:9083</value>
    <description>IP address (or fully-qualified domain name) and port of the
    metastore host</description>
    </property>


    <property>
    <name>hive.server2.thrift.min.worker.threads</name>
    <value>5</value>
    </property>

    <property>
    <name>hive.server2.thrift.max.worker.threads</name>
    <value>100</value>
    </property>

    <property>
    <name>hive.server2.thrift.port</name>
    <value>10001</value>
    </property>

    <property>
    <name>hive.server2.thrift.bind.host</name>
    <value>uat-beeswax.lol.ru</value>
    </property>

    <property>
    <name>hive.server2.authentication</name>
    <value>NONE</value>
    </property>

    </configuration>


    Cloudera manager settings:

    Beeswax Server -> Database ->
    Beeswax Hive Metastore
    Database Type = mysql
    Beeswax Hive Metastore
    Database Name = hivemetastoredb
    Beeswax Hive Metastore
    Database Host = uat-beeswax.lol.ru
    Beeswax Hive Metastore
    Database Port = 3306
    Beeswax Hive Metastore
    Database User = hive
    Beeswax Hive Metastore
    Database Password = hive
    Ports and Addresses ->
    Beeswax Metaserver Port = *9083
    *
    Beeswax Server Port = 8002

    Advanced ->
    Automatically Restart Process
    = true
    Hive System Group = hive
    Hive Configuration Safety
    Valve = *take from /etc/hive/conf/hive-site.xml*

    Run services

    [devops@uat-beeswax ~]$ sudo service hive-metastore start
    [devops@uat-beeswax ~]$ sudo service hive-server2 start


    Now I can create and query tables from beeswax UI.

    2013/3/20 Romain Rigaux <romain@cloudera.com>
    Hi,

    Hue support for Hive Server 2 will come in the next 2.3 release of Hue:
    https://issues.cloudera.org/browse/HUE-1075

    In the meantime, Beeswax is the only alternative for submitting Hive or
    Impala queries with Hue.

    Romain

    On Wed, Mar 20, 2013 at 6:44 AM, Serega Sheypak wrote:

    Hi, I'm using CM 4.1.3 and CDH 4.1.3
    I've tried to migrate to HiveServer2.

    Here are my settings:

    [devops@uat-beeswax ~]$ sudo yum install hive-server2
    [devops@uat-beeswax ~]$ sudo yum install hive-metastore

    Cloudera manager:
    Beeswax Server -> Database ->
    Beeswax Hive Metastore Database Type = mysql
    Beeswax Hive Metastore Database Name = hivemetastoredb
    Beeswax Hive Metastore Database Host = uat-beeswax.lol.ru
    Beeswax Hive Metastore Database Port = 3306
    Beeswax Hive Metastore Database User = hive
    Beeswax Hive Metastore Database Password = hive
    Ports and Addresses ->
    Beeswax Metaserver Port = 9083
    Beeswax Server Port = 8002
    Advanced ->
    Automatically Restart Process = true
    Hive System Group = hive
    Hive Configuration Safety Valve =
    <property>
    <name>hive.support.concurrency</name>
    <description>Enable Hive's Table Lock Manager Service</description>
    <value>true</value>
    </property>

    <property>
    <name>hive.zookeeper.quorum</name>
    <description>Zookeeper quorum used by Hive's Table Lock
    Manager</description>
    <value>node01.lol.ru,node02.lol,node03.lol.ru</value>
    </property>


    <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://uat-beeswax.lol.ru/hivemetastoredb</value>
    <description>the URL of the MySQL database</description>
    </property>

    <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    </property>

    <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>hive</value>
    </property>

    <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>hive</value>
    </property>

    <property>
    <name>datanucleus.autoCreateSchema</name>
    <value>false</value>
    </property>

    <property>
    <name>datanucleus.fixedDatastore</name>
    <value>true</value>
    </property>

    <property>
    <name>hive.metastore.uris</name>
    <value>thrift://uat-beeswax.lol.ru:9083</value>
    <description>IP address (or fully-qualified domain name) and port of
    the metastore host</description>
    </property>


    <property>
    <name>hive.server2.thrift.min.worker.threads</name>
    <value>5</value>
    </property>

    <property>
    <name>hive.server2.thrift.max.worker.threads</name>
    <value>100</value>
    </property>

    <property>
    <name>hive.server2.thrift.port</name>
    <value>10001</value>
    </property>

    <property>
    <name>hive.server2.thrift.bind.host</name>
    <value>uat-beeswax.lol.ru</value>
    </property>

    <property>
    <name>hive.server2.authentication</name>
    <value>NONE</value>
    </property>

    Hue, Beeswax are restarted. I'm trying to execute a query from Beeswax we
    console:


    MetaException(message:file:/user/hive/warehouse/scoring is not a
    directory or unable to create one)


    3/03/20 06:32:54 INFO parse.ParseDriver: Parse Completed
    13/03/20 06:32:54 INFO parse.SemanticAnalyzer: Starting Semantic Analysis
    13/03/20 06:32:54 INFO parse.SemanticAnalyzer: Creating table scoring position=22
    13/03/20 06:32:54 INFO ql.Driver: Semantic Analysis Completed
    13/03/20 06:32:54 INFO ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null)
    13/03/20 06:32:54 INFO ql.Driver: </PERFLOG method=compile start=1363786374850 end=1363786374855 duration=5>
    Hive history file=/tmp/hue/hive_job_log_hue_201303200632_619558420.txt
    13/03/20 06:32:54 INFO exec.HiveHistory: Hive history file=/tmp/hue/hive_job_log_hue_201303200632_619558420.txt
    13/03/20 06:32:54 ERROR conf.HiveConf: Found both hive.metastore.uris and javax.jdo.option.ConnectionURL Recommended to have exactly one of those config keyin configuration
    13/03/20 06:32:54 INFO ql.Driver: <PERFLOG method=Driver.execute>
    13/03/20 06:32:54 INFO ql.Driver: Starting command: CREATE EXTERNAL TABLE scoring
    (
    requestTime BIGINT,
    contentId STRING,
    hitCount BIGINT
    )
    PARTITIONED BY(year STRING, month STRING, day STRING, hour STRING)
    ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'
    STORED AS TEXTFILE
    13/03/20 06:32:54 INFO ql.Driver: </PERFLOG method=TimeToSubmit end=1363786374885>
    13/03/20 06:32:54 INFO exec.DDLTask: Default to LazySimpleSerDe for table scoring
    13/03/20 06:32:54 INFO hive.log: DDL: struct scoring { i32 regionid, i32 resourcetypeid, i32 resourceid, i64 requesttime, string contentid, i64 hitcount}
    13/03/20 06:32:54 INFO hive.metastore: Trying to connect to metastore with URI thrift://uat-beeswax.lol.ru:9083 <http://uat-beeswax.kyc.megafon.ru:9083>
    13/03/20 06:32:54 INFO hive.metastore: Connected to metastore.
    FAILED: Error in metadata: MetaException(message:file:/user/hive/warehouse/scoring is not a directory or unable to create one)
    13/03/20 06:32:54 ERROR exec.Task: FAILED: Error in metadata: MetaException(message:file:/user/hive/warehouse/scoring is not a directory or unable to create one)
    org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:file:/user/hive/warehouse/scoring is not a directory or unable to create one)
    at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:545)
    at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3397)
    at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:243)
    at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
    at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1317)
    at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1103)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:312)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:577)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:566)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:337)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1312)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:566)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
    Caused by: MetaException(message:file:/user/hive/warehouse/scoring is not a directory or unable to create one)
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$create_table_result.read(ThriftHiveMetastore.java:17151)
    at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_create_table(ThriftHiveMetastore.java:694)
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.create_table(ThriftHiveMetastore.java:681)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:408)
    at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:539)
    ... 19 more

    FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
    13/03/20 06:32:54 ERROR ql.Driver: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
    13/03/20 06:32:54 INFO ql.Driver: </PERFLOG method=Driver.execute start=1363786374884 end=1363786374903 duration=19>
    13/03/20 06:32:54 ERROR beeswax.BeeswaxServiceImpl: Exception while processing query
    BeeswaxException(message:Driver returned: 1. Errors: Hive history file=/tmp/hue/hive_job_log_hue_201303200632_619558420.txt
    FAILED: Error in metadata: MetaException(message:file:/user/hive/warehouse/scoring is not a directory or unable to create one)
    FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

    What do I do wrong???
    Why I can't create a table from beeswax UI?

    --


    --


Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouphue-user @
categorieshadoop
postedMar 20, '13 at 5:33p
activeMar 21, '13 at 6:42a
posts2
users2
websitecloudera.com
irc#hadoop

2 users in discussion

Romain Rigaux: 1 post Serega Sheypak: 1 post

People

Translate

site design / logo © 2022 Grokbase