FAQ
Hi Colin,

Impala requires Oracle Java, not OpenJDK. Would you mind trying it on
Oracle Java 1.6 and see if it solves the problem?

Thanks,
Alan

On Wed, Oct 2, 2013 at 3:13 PM, Colin Marc wrote:

I have an existing Impala 1.1.1 cluster (made up of, as far as I can tell,
identical servers with identical environments) that works perfectly fine.
However, I added 12 new nodes to the cluster yesterday and on all of them,
I get an immediate segfault when trying to run impalad. It doesn't appear
to log anything useful before crashing:

impalad -logtostderr
I1002 22:08:51.493381 30540 daemon.cc:34] impalad version 1.1.1 RELEASE
(build 83d5868f005966883a918a819a449f636a5b3d5f)
Built on Fri, 23 Aug 2013 17:52:24 PST
I1002 22:08:51.493577 30540 daemon.cc:35] Using hostname:
bigdata15.stripe.io
I1002 22:08:51.494186 30540 logging.cc:100] Flags (see also /varz are on
debug webserver):
--dump_ir=false
--module_output=
--be_port=22000
--enable_process_lifetime_heap_profiling=false
--heap_profile_dir=
--hostname=bigdata15.stripe.io
--keytab_file=
--mem_limit=80%
--principal=
--exchg_node_buffer_size_bytes=10485760
--max_row_batches=0
--num_disks=0
--num_threads_per_disk=0
--read_size=8388608
--enable_webserver=true
--state_store_host=localhost
--state_store_subscriber_port=23000
--use_statestore=true
--serialize_batch=false
--status_report_interval=5
--num_threads_per_core=3
--authorization_policy_file=

--authorization_policy_provider_class=org.apache.sentry.provider.file.HadoopGroupResourceAuthorizationProvider
--load_catalog_at_startup=false
--server_name=
--abort_on_config_error=true
--abort_on_failed_audit_event=true
--audit_event_log_dir=
--be_service_threads=64
--beeswax_port=21000
--cancellation_thread_pool_size=5
--default_query_options=
--fe_service_threads=64
--hs2_port=21050
--log_mem_usage_interval=0
--log_query_to_file=true
--max_audit_event_log_file_size=5000
--max_profile_log_file_size=5000
--profile_log_dir=
--query_log_size=25
--statestore_subscriber_timeout_seconds=10
--state_store_port=24000
--statestore_max_missed_heartbeats=5
--statestore_num_heartbeat_threads=10
--statestore_suspect_heartbeats=2
--kerberos_reinit_interval=60

--sasl_path=/usr/lib/sasl2:/usr/lib64/sasl2:/usr/local/lib/sasl2:/usr/lib/x86_64-linux-gnu/sasl2
--num_cores=0
--web_log_bytes=1048576
--log_filename=impalad
--periodic_counter_update_period_ms=500
--rpc_cnxn_attempts=10
--rpc_cnxn_retry_interval_ms=2000
--enable_webserver_doc_root=true
--webserver_authentication_domain=
--webserver_certificate_file=
--webserver_doc_root=/usr/lib/impala
--webserver_interface=
--webserver_password_file=
--webserver_port=25000
--flagfile=
--fromenv=
--tryfromenv=
--undefok=
--tab_completion_columns=80
--tab_completion_word=
--help=false
--helpfull=false
--helpmatch=
--helpon=
--helppackage=false
--helpshort=false
--helpxml=false
--version=false
--alsologtoemail=
--alsologtostderr=false
--drop_log_memory=true
--log_backtrace_at=
--log_dir=/tmp
--log_link=
--log_prefix=true
--logbuflevel=0
--logbufsecs=30
--logemaillevel=999
--logmailer=/bin/mail
--logtostderr=true
--max_log_size=1800
--minloglevel=0
--stderrthreshold=2
--stop_logging_if_full_disk=false
--symbolize_stacktrace=true
--v=0
--vmodule=
I1002 22:08:51.495079 30540 mem-info.cc:66] Physical Memory: 14.63 GB
I1002 22:08:51.495136 30540 daemon.cc:43] Cpu Info:
Model: Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz
Cores: 4
L1 Cache: 32.00 KB
L2 Cache: 256.00 KB
L3 Cache: 20.00 MB
Hardware Supports:
ssse3
sse4_1
sse4_2
popcnt
I1002 22:08:51.495182 30540 daemon.cc:44] Disk Info:
Num disks 6:
xvda (rotational=true)
xvdb (rotational=false)
xvdc (rotational=false)
xvdd (rotational=false)
xvde (rotational=false)
dm- (rotational=true)
I1002 22:08:51.495216 30540 daemon.cc:45] Mem Info: 14.63 GB
Segmentation fault (core dumped)

Here's the backtrace from the core dump:

#0 0x00007ffff60b22a6 in JNI_GetCreatedJavaVMs () from
/usr/lib/jvm/default-java/jre/lib/amd64/cacao/libjvm.so
#1 0x00007ffff77b54a5 in getGlobalJNIEnv () at
/root/impala-hadoop-update/cdh/build/cdh4/hadoop/2.0.0-cdh4.3.0/source/hadoop-hdfs-project/hadoop-hdfs/src/main/native/libhdfs/jni_helper.c:445
#2 0x00007ffff77b580f in getJNIEnv () at
/root/impala-hadoop-update/cdh/build/cdh4/hadoop/2.0.0-cdh4.3.0/source/hadoop-hdfs-project/hadoop-hdfs/src/main/native/libhdfs/jni_helper.c:584
#3 0x00007ffff77b3b51 in hdfsBuilderConnect (bld=0x7fffffffc410)
at
/root/impala-hadoop-update/cdh/build/cdh4/hadoop/2.0.0-cdh4.3.0/source/hadoop-hdfs-project/hadoop-hdfs/src/main/native/libhdfs/hdfs.c:575
#4 0x000000000075d672 in impala::JniUtil::InitLibhdfs ()
at
/var/lib/jenkins/workspace/generic-package-ubuntu64-12-04-impala/Impala-Packaging-1.1-2013-08-23_17-27-47/impala-1.1.1-1.p0.21~precise/be/src/util/jni-util.cc:121
#5 0x000000000067bff7 in main (argc=<optimized out>, argv=<optimized
out>)
at
/var/lib/jenkins/workspace/generic-package-ubuntu64-12-04-impala/Impala-Packaging-1.1-2013-08-23_17-27-47/impala-1.1.1-1.p0.21~precise/be/src/service/impalad-main.cc:70

and here is the build version (installed from the deb packages):

$ impalad --version
impalad version 1.1.1 RELEASE (build
83d5868f005966883a918a819a449f636a5b3d5f)
Built on Fri, 23 Aug 2013 17:52:24 PST

And my java version:

$ java -version
java version "1.6.0_27"
OpenJDK Runtime Environment (IcedTea6 1.12.6)
(6b27-1.12.6-1ubuntu0.12.04.2)
OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)

I'm on Ubuntu 12.04 precise.

Thanks in advance for any insights!

To unsubscribe from this group and stop receiving emails from it, send an
email to impala-user+unsubscribe@cloudera.org.
To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 2 of 4 | next ›
Discussion Overview
groupimpala-user @
categorieshadoop
postedOct 2, '13 at 10:13p
activeOct 2, '13 at 11:19p
posts4
users2
websitecloudera.com
irc#hadoop

2 users in discussion

Colin Marc: 3 posts Alan Choi: 1 post

People

Translate

site design / logo © 2022 Grokbase