1. I already did as you guided. The directory is *210-impala-STATESTORE. *Its
content is in attach file
*$ sudo ls -l /run/cloudera-scm-agent/process/*
*total 0*
*drwxr-x--x 3 hdfs hdfs 220 Aug 8 06:28
188-hdfs-SECONDARYNAMENODE*
*drwxr-x--x 3 hdfs hdfs 220 Aug 8 06:28 191-hdfs-DATANODE*
*drwxr-x--x 3 hdfs hdfs 360 Aug 8 06:28 192-hdfs-NAMENODE*
*drwxr-x--x 3 mapred hadoop 380 Aug 8 06:28 193-mapreduce-JOBTRACKER*
*drwxr-x--x 3 mapred hadoop 240 Aug 8 06:28 196-mapreduce-TASKTRACKER
*
*drwxr-x--x 3 zookeeper zookeeper 220 Aug 8 06:28 197-zookeeper-server*
*drwxr-x--x 4 hive hive 160 Aug 8 06:28 198-hive-HIVEMETASTORE*
*drwxr-x--x 6 impala impala 180 Aug 8 07:56 199-impala-IMPALAD*
*drwxr-x--x 4 impala impala 140 Aug 8 07:57 202-impala-STATESTORE*
*drwxr-x--x 6 impala impala 180 Aug 8 07:59 203-impala-IMPALAD*
*drwxr-x--x 4 impala impala 140 Aug 8 07:59 206-impala-STATESTORE*
*drwxr-x--x 6 impala impala 180 Aug 8 08:13 207-impala-IMPALAD*
*drwxr-x--x 4 impala impala 140 Aug 8 08:13 210-impala-STATESTORE*
2. It works! How can I run without using Debug Build?
Thanks,
Andy
On Friday, August 9, 2013 2:15:14 AM UTC+8, vikas wrote:
Thanks Andy, the libraries seem to be ok. CM adds the libhdfs to the
LD_LIBRARY_PATH so that is fine.
The core dump part is interesting. There are few things we can try:
1. Collect the core dump for debugging. In the CM UI, got to Impala
configuration -> Service Wide -> Enable core dump. Check the check-box and
save the configuration. Then restart the statestore and this time it will
core dump, the dump should go to its cwd, which will be under '/run/cloudera-scm-agent/process/'
directory (the latest created directory with STATESTORE in its name).
Please see if you have a hs_err_<pid>.log there too. Please send us these.
2. To get things working, try running debug build. You can go to CM
console and select the Impala configuration option -> Sevice Wide -> Use
Debug Build. Check this option and save configuration. Try restarting
service now and see if it helps.
You may be hitting this: https://issues.cloudera.org/browse/IMPALA-160,
please look at output of "stdout" and see if the error matches.
Vikas
On Thu, Aug 8, 2013 at 11:13 AM, Andy <[email protected] <javascript:>>wrote:
Thanks Andy, the libraries seem to be ok. CM adds the libhdfs to the
LD_LIBRARY_PATH so that is fine.
The core dump part is interesting. There are few things we can try:
1. Collect the core dump for debugging. In the CM UI, got to Impala
configuration -> Service Wide -> Enable core dump. Check the check-box and
save the configuration. Then restart the statestore and this time it will
core dump, the dump should go to its cwd, which will be under '/run/cloudera-scm-agent/process/'
directory (the latest created directory with STATESTORE in its name).
Please see if you have a hs_err_<pid>.log there too. Please send us these.
2. To get things working, try running debug build. You can go to CM
console and select the Impala configuration option -> Sevice Wide -> Use
Debug Build. Check this option and save configuration. Try restarting
service now and see if it helps.
You may be hitting this: https://issues.cloudera.org/browse/IMPALA-160,
please look at output of "stdout" and see if the error matches.
Vikas
On Thu, Aug 8, 2013 at 11:13 AM, Andy <[email protected] <javascript:>>wrote:
Hi Vikas,
The output is:
*$
/opt/cloudera/parcels/IMPALA-1.1-1.p0.8/lib/impala/../../bin/statestored*
*Illegal instruction (core dumped)*
*$ ldd
/opt/cloudera/parcels/IMPALA-1.1-1.p0.8/lib/impala/sbin-retail/statestored
*
* linux-vdso.so.1 => (0x00007fff7413c000)*
* libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
(0x00007f5c3a8e6000)*
* libhdfs.so.0.0.0 => not found*
* librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f5c3a6dd000)*
* libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f5c3a4d9000)*
* libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0
(0x00007f5c3a27d000)*
* libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
(0x00007f5c39f7c000)*
* libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f5c39c80000)*
* libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
(0x00007f5c39a6a000)*
* libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f5c396aa000)*
* /lib64/ld-linux-x86-64.so.2 (0x00007f5c3ab0a000)*
* libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
(0x00007f5c392e2000)*
* libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f5c390ca000)*
*
*
*$ locate libhdfs.so.0.0.0*
*/opt/cloudera/parcels/CDH-4.3.0-1.cdh4.3.0.p0.22/lib/libhdfs.so.0.0.0*
*/opt/cloudera/parcels/IMPALA-1.1-1.p0.8/lib/impala/lib/libhdfs.so.0.0.0*
Thanks,
Andy
The output is:
*$
/opt/cloudera/parcels/IMPALA-1.1-1.p0.8/lib/impala/../../bin/statestored*
*Illegal instruction (core dumped)*
*$ ldd
/opt/cloudera/parcels/IMPALA-1.1-1.p0.8/lib/impala/sbin-retail/statestored
*
* linux-vdso.so.1 => (0x00007fff7413c000)*
* libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
(0x00007f5c3a8e6000)*
* libhdfs.so.0.0.0 => not found*
* librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f5c3a6dd000)*
* libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f5c3a4d9000)*
* libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0
(0x00007f5c3a27d000)*
* libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
(0x00007f5c39f7c000)*
* libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f5c39c80000)*
* libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
(0x00007f5c39a6a000)*
* libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f5c396aa000)*
* /lib64/ld-linux-x86-64.so.2 (0x00007f5c3ab0a000)*
* libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
(0x00007f5c392e2000)*
* libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f5c390ca000)*
*
*
*$ locate libhdfs.so.0.0.0*
*/opt/cloudera/parcels/CDH-4.3.0-1.cdh4.3.0.p0.22/lib/libhdfs.so.0.0.0*
*/opt/cloudera/parcels/IMPALA-1.1-1.p0.8/lib/impala/lib/libhdfs.so.0.0.0*
Thanks,
Andy
On Friday, August 9, 2013 1:39:06 AM UTC+8, vikas wrote:
Hi Andy,
So either the binaries are terminating very early (even before opening a
log), or we are not able to exec them (some dependent libraries are
missing). What happens if you run them command directly from the
commandline:
*/opt/cloudera/parcels/IMPALA-1.1-1.p0.8/lib/impala/../../
bin/statestored*
*
*
*Try using ldd on **/opt/cloudera/parcels/IMPALA-1.1-1.p0.8/lib/impala/sbin-retail/statestored
to see if it shows up something.*
*
*
*- Vikas*
Hi Andy,
So either the binaries are terminating very early (even before opening a
log), or we are not able to exec them (some dependent libraries are
missing). What happens if you run them command directly from the
commandline:
*/opt/cloudera/parcels/IMPALA-1.1-1.p0.8/lib/impala/../../
bin/statestored*
*
*
*Try using ldd on **/opt/cloudera/parcels/IMPALA-1.1-1.p0.8/lib/impala/sbin-retail/statestored
to see if it shows up something.*
*
*
*- Vikas*
On Thu, Aug 8, 2013 at 10:18 AM, Andy wrote:
Hi Vikas,
There's nothing in */var/log/impalad *and */var/log/statestore *directories.
This is their permission:
*
*
*drwxr-xr-x 2 impala impala 4096 Aug 4 12:05 impalad/*
*drwxr-xr-x 2 impala impala 4096 Aug 4 12:16
statestore/*
Thanks,
Andy
Hi Vikas,
There's nothing in */var/log/impalad *and */var/log/statestore *directories.
This is their permission:
*
*
*drwxr-xr-x 2 impala impala 4096 Aug 4 12:05 impalad/*
*drwxr-xr-x 2 impala impala 4096 Aug 4 12:16
statestore/*
Thanks,
Andy
On Thursday, August 8, 2013 11:44:57 PM UTC+8, vikas wrote:
Hi Andy,
Can you get the impalad and statestore log and provide it to us. The
stderr/stdout log is from the bash file that launches impalad/statestore.
The log from impalad/statestore will tell us more on why we can't start
these daemons.
Thanks,
Vikas
Hi Andy,
Can you get the impalad and statestore log and provide it to us. The
stderr/stdout log is from the bash file that launches impalad/statestore.
The log from impalad/statestore will tell us more on why we can't start
these daemons.
Thanks,
Vikas
On Thu, Aug 8, 2013 at 8:17 AM, Andy wrote:
Hi all,
I install Impala on Ubuntu server 12.04 using Cloudera
Manager 4.6.2, Parcels. Then I add services: HDFS, Hive, MapReduce, Impala.
HDFS, Hive, MapReduce services work fine, the status is Good health.
But I can't start Impala, the error is:
*Supervisor returned FATAL. Please check the role log file, stderr,
or stdout.*
*
*
The error log is:
*+ HADOOP_CONF_DIR=/run/cloudera-scm-agent/process/208-impala-I
MPALAD/hadoop-conf*
*+ export HIVE_CONF_DIR=/run/cloudera-scm-agent/process/208-impala-
IMPALAD/hive-conf*
*+ HIVE_CONF_DIR=/run/cloudera-scm-agent/process/208-impala-IMP
ALAD/hive-conf*
*+ export HBASE_CONF_DIR=/run/cloudera-scm-agent/process/208-impala-
IMPALAD/hbase-conf*
*+ HBASE_CONF_DIR=/run/cloudera-scm-agent/process/208-impala-IM
PALAD/hbase-conf*
*+ JDBC_JARS=/usr/share/java/mysql-connector-java.jar:/usr/shar
e/cmf/lib/postgresql-9.0-801.jdbc4.jar:/usr/share/java/orac
le-connector-java.jar*
*+ [[ -z '' ]]*
*+ export AUX_CLASSPATH=/usr/share/java/
mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-9.0-8
01.jdbc4.jar:/usr/share/java/oracle-connector-java.jar*
*+ AUX_CLASSPATH=/usr/share/java/mysql-connector-java.jar:/usr/
share/cmf/lib/postgresql-9.0-801.jdbc4.jar:/usr/share/java/o
racle-connector-java.jar*
*+ [[ -z '' ]]*
*+ export CLASSPATH=/usr/share/java/mysql-connector-java.jar:/usr/
share/cmf/lib/postgresql-9.0-801.jdbc4.jar:/usr/share/java/orac
le-connector-java.jar*
*+ CLASSPATH=/usr/share/java/mysql-connector-java.jar:/usr/shar
e/cmf/lib/postgresql-9.0-801.jdbc4.jar:/usr/share/java/orac
le-connector-java.jar*
*+ FLAG_FILE=/run/cloudera-scm-agent/process/208-impala-IMPALAD
/impala-conf/impalad_flags*
*+ USE_DEBUG_BUILD=false*
*+ perl -pi -e 's#{{CMF_CONF_DIR}}#/run/cloudera-scm-agent/process/
208-impala-IMPALAD#g' /run/cloudera-scm-agent/proces
s/208-impala-IMPALAD/impala-conf/impalad_flags*
*+ '[' -f /run/cloudera-scm-agent/process/208-impala-IMPALAD/impala-conf/.htpasswd
']'*
*+ chmod 600 /run/cloudera-scm-agent/process/208-impala-IMPALAD/
impala-conf/.htpasswd*
*+ false*
*+ export IMPALA_BIN=/opt/cloudera/parcels/IMPALA-1.1-1.p0.8/lib/
impala/sbin-retail*
*+ IMPALA_BIN=/opt/cloudera/parcels/IMPALA-1.1-1.p0.8/lib/impal
a/sbin-retail*
*+ '[' impalad = impalad ']'*
*+ exec /opt/cloudera/parcels/IMPALA-1.1-1.p0.8/lib/impala/../../bin/impalad
--flagfile=/run/cloudera-scm-agent/process/208-impala-IMPALA
D/impala-conf/impalad_flags*
*
*
Are there anyone know how to fix this problem? Thank you.
*
*
Hi all,
I install Impala on Ubuntu server 12.04 using Cloudera
Manager 4.6.2, Parcels. Then I add services: HDFS, Hive, MapReduce, Impala.
HDFS, Hive, MapReduce services work fine, the status is Good health.
But I can't start Impala, the error is:
*Supervisor returned FATAL. Please check the role log file, stderr,
or stdout.*
*
*
The error log is:
*+ HADOOP_CONF_DIR=/run/cloudera-scm-agent/process/208-impala-I
MPALAD/hadoop-conf*
*+ export HIVE_CONF_DIR=/run/cloudera-scm-agent/process/208-impala-
IMPALAD/hive-conf*
*+ HIVE_CONF_DIR=/run/cloudera-scm-agent/process/208-impala-IMP
ALAD/hive-conf*
*+ export HBASE_CONF_DIR=/run/cloudera-scm-agent/process/208-impala-
IMPALAD/hbase-conf*
*+ HBASE_CONF_DIR=/run/cloudera-scm-agent/process/208-impala-IM
PALAD/hbase-conf*
*+ JDBC_JARS=/usr/share/java/mysql-connector-java.jar:/usr/shar
e/cmf/lib/postgresql-9.0-801.jdbc4.jar:/usr/share/java/orac
le-connector-java.jar*
*+ [[ -z '' ]]*
*+ export AUX_CLASSPATH=/usr/share/java/
mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-9.0-8
01.jdbc4.jar:/usr/share/java/oracle-connector-java.jar*
*+ AUX_CLASSPATH=/usr/share/java/mysql-connector-java.jar:/usr/
share/cmf/lib/postgresql-9.0-801.jdbc4.jar:/usr/share/java/o
racle-connector-java.jar*
*+ [[ -z '' ]]*
*+ export CLASSPATH=/usr/share/java/mysql-connector-java.jar:/usr/
share/cmf/lib/postgresql-9.0-801.jdbc4.jar:/usr/share/java/orac
le-connector-java.jar*
*+ CLASSPATH=/usr/share/java/mysql-connector-java.jar:/usr/shar
e/cmf/lib/postgresql-9.0-801.jdbc4.jar:/usr/share/java/orac
le-connector-java.jar*
*+ FLAG_FILE=/run/cloudera-scm-agent/process/208-impala-IMPALAD
/impala-conf/impalad_flags*
*+ USE_DEBUG_BUILD=false*
*+ perl -pi -e 's#{{CMF_CONF_DIR}}#/run/cloudera-scm-agent/process/
208-impala-IMPALAD#g' /run/cloudera-scm-agent/proces
s/208-impala-IMPALAD/impala-conf/impalad_flags*
*+ '[' -f /run/cloudera-scm-agent/process/208-impala-IMPALAD/impala-conf/.htpasswd
']'*
*+ chmod 600 /run/cloudera-scm-agent/process/208-impala-IMPALAD/
impala-conf/.htpasswd*
*+ false*
*+ export IMPALA_BIN=/opt/cloudera/parcels/IMPALA-1.1-1.p0.8/lib/
impala/sbin-retail*
*+ IMPALA_BIN=/opt/cloudera/parcels/IMPALA-1.1-1.p0.8/lib/impal
a/sbin-retail*
*+ '[' impalad = impalad ']'*
*+ exec /opt/cloudera/parcels/IMPALA-1.1-1.p0.8/lib/impala/../../bin/impalad
--flagfile=/run/cloudera-scm-agent/process/208-impala-IMPALA
D/impala-conf/impalad_flags*
*
*
Are there anyone know how to fix this problem? Thank you.
*
*
