FAQ
Hi Vikas,

There's nothing in '*/var/crash'* directory:
*$ /opt/cloudera/parcels/IMPALA-1.1-1.p0.8/lib/impala/../../bin/statestored*
*Illegal instruction (core dumped)*
*$ ls /var/crash/*
*$ *
*
*
I can't change the value of '*/proc/sys/kernel/core_pattern*':
*$ sudo echo core > /proc/sys/kernel/core_pattern *
*-bash: /proc/sys/kernel/core_pattern: Permission denied*

Andy.
On Friday, August 9, 2013 4:14:25 AM UTC+8, vikas wrote:

Hi Andy,

So you have Ubuntu apport enabled to capture dumps. The crash file may be
in '/var/crash'.

If you can't find the core there, try setting core_pattern to simple
'core' (echo core > /proc/sys/kernel/core_pattern) and then it should dump
the core in the current working directory.

Vikas


On Thu, Aug 8, 2013 at 12:59 PM, Andy <nguyend...@gmail.com <javascript:>>wrote:
Hi Vikas,

1. The value of */proc/sys/kernel/core_pattern *is:
*|/usr/share/apport/apport %p %s %c*
*
*
2. Thanks, I'll try CentOS.

Andy.
On Friday, August 9, 2013 3:16:46 AM UTC+8, vikas wrote:

Hi Andy,

1. Unfortunately there is no core present. By default the core get
generated in the current working directory of the process. What is the
value of /proc/sys/kernel/core_**pattern on your setup?
2. To run retail build, if possible try using some other os than Ubuntu
12.04 till we get this issue sorted out.

- Vikas

On Thu, Aug 8, 2013 at 11:36 AM, Andy wrote:

Hi Vikas,

1. I already did as you guided. The directory is *210-impala-STATESTORE.
*Its content is in attach file
*$ sudo ls -l /run/cloudera-scm-agent/process/*
*total 0*
*drwxr-x--x 3 hdfs hdfs 220 Aug 8 06:28
188-hdfs-SECONDARYNAMENODE*
*drwxr-x--x 3 hdfs hdfs 220 Aug 8 06:28 191-hdfs-DATANODE*
*drwxr-x--x 3 hdfs hdfs 360 Aug 8 06:28 192-hdfs-NAMENODE*
*drwxr-x--x 3 mapred hadoop 380 Aug 8 06:28
193-mapreduce-JOBTRACKER*
*drwxr-x--x 3 mapred hadoop 240 Aug 8 06:28
196-mapreduce-TASKTRACKER*
*drwxr-x--x 3 zookeeper zookeeper 220 Aug 8 06:28 197-zookeeper-server
*
*drwxr-x--x 4 hive hive 160 Aug 8 06:28
198-hive-HIVEMETASTORE*
*drwxr-x--x 6 impala impala 180 Aug 8 07:56 199-impala-IMPALAD*
*drwxr-x--x 4 impala impala 140 Aug 8 07:57
202-impala-STATESTORE*
*drwxr-x--x 6 impala impala 180 Aug 8 07:59 203-impala-IMPALAD*
*drwxr-x--x 4 impala impala 140 Aug 8 07:59
206-impala-STATESTORE*
*drwxr-x--x 6 impala impala 180 Aug 8 08:13 207-impala-IMPALAD*
*drwxr-x--x 4 impala impala 140 Aug 8 08:13
210-impala-STATESTORE*

2. It works! How can I run without using Debug Build?

Thanks,
Andy
On Friday, August 9, 2013 2:15:14 AM UTC+8, vikas wrote:

Thanks Andy, the libraries seem to be ok. CM adds the libhdfs to the
LD_LIBRARY_PATH so that is fine.

The core dump part is interesting. There are few things we can try:

1. Collect the core dump for debugging. In the CM UI, got to Impala
configuration -> Service Wide -> Enable core dump. Check the check-box and
save the configuration. Then restart the statestore and this time it will
core dump, the dump should go to its cwd, which will be under '
/run/cloudera-scm-agent/**proce**ss/' directory (the latest created
directory with STATESTORE in its name). Please see if you have a
hs_err_<pid>.log there too. Please send us these.

2. To get things working, try running debug build. You can go to CM
console and select the Impala configuration option -> Sevice Wide -> Use
Debug Build. Check this option and save configuration. Try restarting
service now and see if it helps.

You may be hitting this: https://issues.cloudera.****
org/browse/IMPALA-160 <https://issues.cloudera.org/browse/IMPALA-160>,
please look at output of "stdout" and see if the error matches.

Vikas

On Thu, Aug 8, 2013 at 11:13 AM, Andy wrote:

Hi Vikas,

The output is:
*$ /opt/cloudera/parcels/IMPALA-1.1-1.p0.8/lib/impala/../../bin
/statestored*
*Illegal instruction (core dumped)*

*$ ldd /opt/cloudera/parcels/IMPALA-1.1-1.p0.8/lib/impala/sbin-reta
il/statestored*
* linux-vdso.so.1 => (0x00007fff7413c000)*
* libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
(0x00007f5c3a8e6000)*
* libhdfs.so.0.0.0 => not found*
* librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f5c3a6dd000)
*
* libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f5c3a4d9000)
*
* libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0
(0x00007f5c3a27d000)*
* libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
(0x00007f5c39f7c000)*
* libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f5c39c80000)*
* libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
(0x00007f5c39a6a000)*
* libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f5c396aa000)*
* /lib64/ld-linux-x86-64.so.2 (0x00007f5c3ab0a000)*
* libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
(0x00007f5c392e2000)*
* libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f5c390ca000)*
*
*
*$ locate libhdfs.so.0.0.0*
*/opt/cloudera/parcels/CDH-4.3.0-1.cdh4.3.0.p0.22/lib/libhdfs
.so.0.0.0*
*/opt/cloudera/parcels/IMPALA-1.1-1.p0.8/lib/impala/lib/libhd
fs.so.0.0.0*

Thanks,
Andy
On Friday, August 9, 2013 1:39:06 AM UTC+8, vikas wrote:

Hi Andy,

So either the binaries are terminating very early (even before
opening a log), or we are not able to exec them (some dependent libraries
are missing). What happens if you run them command directly from the
commandline:

*/opt/cloudera/parcels/IMPALA-1.1-1.p0.8/lib/impala/../../bin
/statestored*
*
*
*Try using ldd on **/opt/cloudera/parcels/IMPAL
A-1.1-1.p0.8/lib/impala/sbin-retail/statestored to see if it shows
up something.*
*
*
*- Vikas*

On Thu, Aug 8, 2013 at 10:18 AM, Andy wrote:

Hi Vikas,

There's nothing in */var/log/impalad *and */var/log/statestore *directories.
This is their permission:
*
*
*drwxr-xr-x 2 impala impala 4096 Aug 4 12:05
impalad/*
*drwxr-xr-x 2 impala impala 4096 Aug 4 12:16
statestore/*

Thanks,
Andy
On Thursday, August 8, 2013 11:44:57 PM UTC+8, vikas wrote:

Hi Andy,

Can you get the impalad and statestore log and provide it to us.
The stderr/stdout log is from the bash file that launches
impalad/statestore. The log from impalad/statestore will tell us more on
why we can't start these daemons.

Thanks,
Vikas

On Thu, Aug 8, 2013 at 8:17 AM, Andy wrote:

Hi all,

I install Impala on Ubuntu server 12.04 using Cloudera
Manager 4.6.2, Parcels. Then I add services: HDFS, Hive, MapReduce, Impala.

HDFS, Hive, MapReduce services work fine, the status is Good
health. But I can't start Impala, the error is:

*Supervisor returned FATAL. Please check the role log file,
stderr, or stdout.*
*
*
The error log is:

*+ HADOOP_CONF_DIR=/run/cloudera-scm-agent/process/208-impala-I
MPALAD/hadoop-conf*
*+ export HIVE_CONF_DIR=/run/cloudera-sc
m-agent/process/208-impala-IMPALAD/hive-conf*
*+ HIVE_CONF_DIR=/run/cloudera-scm-agent/process/208-impala-IMP
ALAD/hive-conf*
*+ export HBASE_CONF_DIR=/run/cloudera-s
cm-agent/process/208-impala-IMPALAD/hbase-conf*
*+ HBASE_CONF_DIR=/run/cloudera-scm-agent/process/208-impala-IM
PALAD/hbase-conf*
*+ JDBC_JARS=/usr/share/java/mysql-connector-java.jar:/usr/shar
e/cmf/lib/postgresql-9.0-801.jdbc4.jar:/usr/share/java/oracl
e-connector-java.jar*
*+ [[ -z '' ]]*
*+ export AUX_CLASSPATH=/usr/share/java/
mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-9.0-8
01.jdbc4.jar:/usr/share/java/oracle-connector-java.jar*
*+ AUX_CLASSPATH=/usr/share/java/mysql-connector-java.jar:/usr/
share/cmf/lib/postgresql-9.0-801.jdbc4.jar:/usr/share/java/o
racle-connector-java.jar*
*+ [[ -z '' ]]*
*+ export CLASSPATH=/usr/share/java/mysq
l-connector-java.jar:/usr/share/cmf/lib/postgresql-9.0-801.j
dbc4.jar:/usr/share/java/oracle-connector-java.jar*
*+ CLASSPATH=/usr/share/java/mysql-connector-java.jar:/usr/shar
e/cmf/lib/postgresql-9.0-801.jdbc4.jar:/usr/share/java/oracl
e-connector-java.jar*
*+ FLAG_FILE=/run/cloudera-scm-agent/process/208-impala-IMPALAD
/impala-conf/impalad_flags*
*+ USE_DEBUG_BUILD=false*
*+ perl -pi -e 's#{{CMF_CONF_DIR}}#/run/cloud
era-scm-agent/process/208-impala-IMPALAD#g'
/run/cloudera-scm-agent/process/208-impala-IMPALAD/impala-co
nf/impalad_flags*
*+ '[' -f /run/cloudera-scm-agent/process/208-impala-IMPALAD/
impala-conf/.htpasswd ']'*
*+ chmod 600 /run/cloudera-scm-agent/process/208-impala-IMPALAD/
impala-conf/.htpasswd*
*+ false*
*+ export IMPALA_BIN=/opt/cloudera/parcels/IMPALA-1.1-1.p0.8/lib/
impala/sbin-retail*
*+ IMPALA_BIN=/opt/cloudera/parcels/IMPALA-1.1-1.p0.8/lib/impal
a/sbin-retail*
*+ '[' impalad = impalad ']'*
*+ exec /opt/cloudera/parcels/IMPALA-1.1-1.p0.8/lib/impala/../../
bin/impalad --flagfile=/run/cloudera-scm-a
gent/process/208-impala-IMPALAD/impala-conf/impalad_flags*
*
*
Are there anyone know how to fix this problem? Thank you.
*
*

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 10 of 12 | next ›
Discussion Overview
groupimpala-user @
categorieshadoop
postedAug 8, '13 at 3:17p
activeSep 4, '13 at 4:45p
posts12
users3
websitecloudera.com
irc#hadoop

3 users in discussion

Andy: 6 posts Vikas Singh: 5 posts Kieran Evans: 1 post

People

Translate

site design / logo © 2022 Grokbase