Re: Impala shell behaving differently when services are started as root/impala

On Wed, Jan 23, 2013 at 9:22 PM, Manisha Agrawal wrote:

Hi all,

I have CDH 4.1.2 running in pseudo mode on a RHEL 64 bit machine.
Mysql is configured as Hive's remote datasore.
Hive succsessfully reads metadata and fetches correct results.

Impala is running on top of it. I am unable to excute queries on impala.

I have a external table named sample and permissions on file are as
follows:
[[email protected] conf]# sudo -u hdfs hadoop fs -ls -R
/user/impala
-rw-r--r-- 2 impala hdfs 31 2013-01-23 23:10
/user/impala/test_ext.out


This is how I start statestored and impalad services

GLOG_v=1 /usr/bin/statestored -state_store_port=24000 &
GLOG_v=1 /usr/bin/impalad -state_store_host=localhost -nn=localhost
-nn_port=8020 -hostname=localhost -ipaddress=10.100.255.207 &

When I start services as *root user* , hdfs is unreadable.
and impala-shell throws below error:

readDirect: FSDataInputStream#read error:
org.apache.hadoop.hdfs.**BlockMissingException: Could not obtain block:
BP-344648192-10.100.255.207-**1358752982703:blk_**183363845701608725_1522
file=/user/impala/test_ext.out
at org.apache.hadoop.hdfs.**DFSInputStream.chooseDataNode(**
DFSInputStream.java:734)
at org.apache.hadoop.hdfs.**DFSInputStream.blockSeekTo(**
DFSInputStream.java:448)
at org.apache.hadoop.hdfs.**DFSInputStream.**readWithStrategy(**
DFSInputStream.java:645)
at org.apache.hadoop.hdfs.**DFSInputStream.read(**
DFSInputStream.java:696)
at org.apache.hadoop.fs.**FSDataInputStream.read(**
FSDataInputStream.java:123)
hdfsOpenFile(hdfs://localhost:**8020/user/impala/test_ext.out)**: WARN:
Unexpected error 255 when testing for direct read compatibility


On starting services as *impala user,* metadata is unreadable
error in impala-shell is as below:

Query: select * from sample
ERROR: com.cloudera.impala.common.**AnalysisException: Analysis
exception (in select * from sample)
at com.cloudera.impala.analysis.**AnalysisContext.analyze(**
AnalysisContext.java:133)
at com.cloudera.impala.service.**Frontend.createExecRequest(**
Frontend.java:216)
at com.cloudera.impala.service.**JniFrontend.createExecRequest(*
*JniFrontend.java:86)
Caused by: com.cloudera.impala.common.**AnalysisException: Unknown
table: 'sample'
at com.cloudera.impala.analysis.**Analyzer.registerBaseTableRef(
**Analyzer.java:178)
at com.cloudera.impala.analysis.**BaseTableRef.analyze(**
BaseTableRef.java:51)
at com.cloudera.impala.analysis.**SelectStmt.analyze(SelectStmt.
**java:115)
at com.cloudera.impala.analysis.**AnalysisContext.analyze(**
AnalysisContext.java:130)
... 2 more

Please provide suggestions to resolve the issue.

Thanks in advance
Manisha
==============================**==============================**
==============================
*core-site.xml*

<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:8020</**value>
</property>
<!-- OOZIE proxy user setting -->
<property>
<name>hadoop.proxyuser.oozie.**hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.oozie.**groups</name>
<value>*</value>
</property>
<!-- HTTPFS proxy user setting -->
<property>
<name>hadoop.proxyuser.httpfs.**hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.httpfs.**groups</name>
<value>*</value>
</property>
<property>
<name>dfs.client.read.**shortcircuit</name>
<value>true</value>
</property>
</configuration>

==============================**==============================**
==============================

*hdfs-site.xml*

<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<!-- Immediately exit safemode as soon as one DataNode checks in.
On a multi-node cluster, these configurations must be removed.
-->
<property>
<name>dfs.safemode.extension</**name>
<value>0</value>
</property>
<property>
<name>dfs.safemode.min.**datanodes</name>
<value>1</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/var/lib/hadoop-hdfs/**cache/${user.name}</value>
</property>
<property>
<name>dfs.namenode.name.dir</**name>
<value>/var/lib/hadoop-hdfs/**cache/${user.name}/dfs/name</**value>
</property>
<property>
<name>dfs.namenode.checkpoint.**dir</name>
<value>/var/lib/hadoop-hdfs/**cache/${user.name}/dfs/**
namesecondary</value>
</property>
<property>
<name>dfs.datanode.data.dir</**name>
<value>/var/lib/hadoop-hdfs/**cache/${user.name}/dfs/data</**value>
</property>
<property>
<name>dfs.datanode.data.dir.**perm</name>
<value>750</value>
</property>
<property>
<name>dfs.block.local-path-**access.user</name>
<value>impala,hdfs,mapred</**value>
</property>
<property>
<name>dfs.datanode.hdfs-**blocks-metadata.enabled</name>
<value>true</value>
</property>
</configuration>
==============================**==============================**
==============================

*hive-site.xml*

<configuration>
<property>
<name>javax.jdo.option.**ConnectionURL</name>
<value>jdbc:mysql://localhost/**metastore</value>
<description>JDBC connect string for a JDBC metastore</description>
</property>
<property>
<name>javax.jdo.option.**ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</**value>
<description>Driver class name for a JDBC metastore</description>
</property>
<property>
<name>javax.jdo.option.**ConnectionUserName</name>
<value>hive</value>
</property>

<property>
<name>javax.jdo.option.**ConnectionPassword</name>
<value>hive</value>
</property>

<property>
<name>hive.metastore.local</**name>
<value>true</value>
</property>
</configuration>
=============================**==============================**
==============================**=============




--