FAQ
googling again, found the following issue matching my situation:

— Impala fails to refresh the Hive metastore if a Hive temporary
configuration file is removed

Impala is impacted by Hive bug
HIVE-3596<https://issues.apache.org/jira/browse/HIVE-3596> which
may cause metastore refreshes to fail if a Hive temporary configuration
file is deleted (normally located at /tmp/hive-<user>-<tmp_number>.xml).
Additionally, the impala-shell will incorrectly report that the failed
metadata refresh completed successfully.

*Severity:* Medium
*Anticipated Resolution:* To be fixed in a future release
*Workaround:* Restart the impalad service. Use the impalad log to check for
metadata refresh errors.

After restarting impalad this problem was resolved.

[MYIP] > refresh;
Successfully refreshed catalog
[MYIP] > show tables;
Query: show tables
Query finished, fetching results ...
mbt
mbt_ext
numbers
Returned 3 row(s) in 0.98s


2013/2/20 Stephen Boesch <javadba@gmail.com>

I found some docs on this: they say to 'refresh' in impala.

Doing so causes a configuration related exception. Is this because
impala is not properly pointing to the hive configuration? If that were
the case, then how would it have been able to find the "numbers" table?

Returned 1 row(s) in 0.10s
[MYIP] > *refresh;*
RPC Error: java.lang.RuntimeException: java.io.FileNotFoundException:
/tmp/hive-default-6589356860794670563.xml (No such file or directory)
at
org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1993)
at
org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1861)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1769)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:861)
at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:895)
at org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:846)
at com.cloudera.impala.catalog.Catalog.<init>(Catalog.java:205)
at com.cloudera.impala.service.Frontend.resetCatalog(Frontend.java:100)
at
com.cloudera.impala.service.JniFrontend.resetCatalog(JniFrontend.java:230)
Caused by: java.io.FileNotFoundException:
/tmp/hive-default-6589356860794670563.xml (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:120)
at java.io.FileInputStream.<init>(FileInputStream.java:79)
at
sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:70)
at
sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:161)
at java.net.URL.openStream(URL.java:1010)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:1827)
at
org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1895)
... 8 more

[10.196.12.144:21000] >



2013/2/20 Stephen Boesch <javadba@gmail.com>
Hi,
I am puzzled as to why I can access one table in impala shared with
hive, but another newly created one is not visible. The old table I did an
insert on it *after *creating the new table and I see the effect
immediately in impala (avg is 2.5 which is correct with the new data). So
that tells me i'm connecting to the right place from impala.

As you can see below all of the tables have same owner. The offending
table is "mbt" whereas the properly functioning one is "numbers"

Here are details.

HIVE metastore:

mysql> select DB_ID, OWNER, TBL_NAME, TBL_TYPE from TBLS;
+-------+--------+----------+----------------+
DB_ID | OWNER | TBL_NAME | TBL_TYPE |
+-------+--------+----------+----------------+
1 | impala | numbers | MANAGED_TABLE |
1 | *impala | mbt | MANAGED_TABLE |*
+-------+--------+----------+----------------+

hive> desc mbt;
OK
billingdate string
netqty smallint
revenue int
customer_key int
product string
version string
prodlang string
customer string
state string
country string
Time taken: 0.191 seconds

impala-shell on same machine

[MYSERVERIP] > connect MYSERVERIP;
Connected to MYSERVERIP
[MYSERVERIP] > select avg(num) from numbers;
Query: select avg(num) from numbers
Query finished, fetching results ...
2.5
Returned 1 row(s) in 3.27s
[MYSERVERIP] > describe mbt;
Query: describe mbt
ERROR: com.cloudera.impala.common.AnalysisException: Unknown table:
default.mbt
at com.cloudera.impala.service.Frontend.describeTable(Frontend.java:194)
at
com.cloudera.impala.service.JniFrontend.describeTable(JniFrontend.java:179)

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupimpala-user @
categorieshadoop
postedFeb 21, '13 at 8:09a
activeFeb 21, '13 at 8:09a
posts1
users1
websitecloudera.com
irc#hadoop

1 user in discussion

Stephen Boesch: 1 post

People

Translate

site design / logo © 2022 Grokbase