FAQ
I can try that, but it works fine on the existing servers (which are all
running the same exact java, OpenJDK 1.6).
On Wednesday, October 2, 2013 3:19:02 PM UTC-7, Alan wrote:

Hi Colin,

Impala requires Oracle Java, not OpenJDK. Would you mind trying it on
Oracle Java 1.6 and see if it solves the problem?

Thanks,
Alan


On Wed, Oct 2, 2013 at 3:13 PM, Colin Marc <coli...@gmail.com<javascript:>
wrote:
I have an existing Impala 1.1.1 cluster (made up of, as far as I can
tell, identical servers with identical environments) that works perfectly
fine. However, I added 12 new nodes to the cluster yesterday and on all of
them, I get an immediate segfault when trying to run impalad. It doesn't
appear to log anything useful before crashing:

impalad -logtostderr
I1002 22:08:51.493381 30540 daemon.cc:34] impalad version 1.1.1 RELEASE
(build 83d5868f005966883a918a819a449f636a5b3d5f)
Built on Fri, 23 Aug 2013 17:52:24 PST
I1002 22:08:51.493577 30540 daemon.cc:35] Using hostname:
bigdata15.stripe.io
I1002 22:08:51.494186 30540 logging.cc:100] Flags (see also /varz are on
debug webserver):
--dump_ir=false
--module_output=
--be_port=22000
--enable_process_lifetime_heap_profiling=false
--heap_profile_dir=
--hostname=bigdata15.stripe.io
--keytab_file=
--mem_limit=80%
--principal=
--exchg_node_buffer_size_bytes=10485760
--max_row_batches=0
--num_disks=0
--num_threads_per_disk=0
--read_size=8388608
--enable_webserver=true
--state_store_host=localhost
--state_store_subscriber_port=23000
--use_statestore=true
--serialize_batch=false
--status_report_interval=5
--num_threads_per_core=3
--authorization_policy_file=

--authorization_policy_provider_class=org.apache.sentry.provider.file.HadoopGroupResourceAuthorizationProvider
--load_catalog_at_startup=false
--server_name=
--abort_on_config_error=true
--abort_on_failed_audit_event=true
--audit_event_log_dir=
--be_service_threads=64
--beeswax_port=21000
--cancellation_thread_pool_size=5
--default_query_options=
--fe_service_threads=64
--hs2_port=21050
--log_mem_usage_interval=0
--log_query_to_file=true
--max_audit_event_log_file_size=5000
--max_profile_log_file_size=5000
--profile_log_dir=
--query_log_size=25
--statestore_subscriber_timeout_seconds=10
--state_store_port=24000
--statestore_max_missed_heartbeats=5
--statestore_num_heartbeat_threads=10
--statestore_suspect_heartbeats=2
--kerberos_reinit_interval=60

--sasl_path=/usr/lib/sasl2:/usr/lib64/sasl2:/usr/local/lib/sasl2:/usr/lib/x86_64-linux-gnu/sasl2
--num_cores=0
--web_log_bytes=1048576
--log_filename=impalad
--periodic_counter_update_period_ms=500
--rpc_cnxn_attempts=10
--rpc_cnxn_retry_interval_ms=2000
--enable_webserver_doc_root=true
--webserver_authentication_domain=
--webserver_certificate_file=
--webserver_doc_root=/usr/lib/impala
--webserver_interface=
--webserver_password_file=
--webserver_port=25000
--flagfile=
--fromenv=
--tryfromenv=
--undefok=
--tab_completion_columns=80
--tab_completion_word=
--help=false
--helpfull=false
--helpmatch=
--helpon=
--helppackage=false
--helpshort=false
--helpxml=false
--version=false
--alsologtoemail=
--alsologtostderr=false
--drop_log_memory=true
--log_backtrace_at=
--log_dir=/tmp
--log_link=
--log_prefix=true
--logbuflevel=0
--logbufsecs=30
--logemaillevel=999
--logmailer=/bin/mail
--logtostderr=true
--max_log_size=1800
--minloglevel=0
--stderrthreshold=2
--stop_logging_if_full_disk=false
--symbolize_stacktrace=true
--v=0
--vmodule=
I1002 22:08:51.495079 30540 mem-info.cc:66] Physical Memory: 14.63 GB
I1002 22:08:51.495136 30540 daemon.cc:43] Cpu Info:
Model: Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz
Cores: 4
L1 Cache: 32.00 KB
L2 Cache: 256.00 KB
L3 Cache: 20.00 MB
Hardware Supports:
ssse3
sse4_1
sse4_2
popcnt
I1002 22:08:51.495182 30540 daemon.cc:44] Disk Info:
Num disks 6:
xvda (rotational=true)
xvdb (rotational=false)
xvdc (rotational=false)
xvdd (rotational=false)
xvde (rotational=false)
dm- (rotational=true)
I1002 22:08:51.495216 30540 daemon.cc:45] Mem Info: 14.63 GB
Segmentation fault (core dumped)

Here's the backtrace from the core dump:

#0 0x00007ffff60b22a6 in JNI_GetCreatedJavaVMs () from
/usr/lib/jvm/default-java/jre/lib/amd64/cacao/libjvm.so
#1 0x00007ffff77b54a5 in getGlobalJNIEnv () at
/root/impala-hadoop-update/cdh/build/cdh4/hadoop/2.0.0-cdh4.3.0/source/hadoop-hdfs-project/hadoop-hdfs/src/main/native/libhdfs/jni_helper.c:445
#2 0x00007ffff77b580f in getJNIEnv () at
/root/impala-hadoop-update/cdh/build/cdh4/hadoop/2.0.0-cdh4.3.0/source/hadoop-hdfs-project/hadoop-hdfs/src/main/native/libhdfs/jni_helper.c:584
#3 0x00007ffff77b3b51 in hdfsBuilderConnect (bld=0x7fffffffc410)
at
/root/impala-hadoop-update/cdh/build/cdh4/hadoop/2.0.0-cdh4.3.0/source/hadoop-hdfs-project/hadoop-hdfs/src/main/native/libhdfs/hdfs.c:575
#4 0x000000000075d672 in impala::JniUtil::InitLibhdfs ()
at
/var/lib/jenkins/workspace/generic-package-ubuntu64-12-04-impala/Impala-Packaging-1.1-2013-08-23_17-27-47/impala-1.1.1-1.p0.21~precise/be/src/util/jni-util.cc:121
#5 0x000000000067bff7 in main (argc=<optimized out>, argv=<optimized
out>)
at
/var/lib/jenkins/workspace/generic-package-ubuntu64-12-04-impala/Impala-Packaging-1.1-2013-08-23_17-27-47/impala-1.1.1-1.p0.21~precise/be/src/service/impalad-main.cc:70

and here is the build version (installed from the deb packages):

$ impalad --version
impalad version 1.1.1 RELEASE (build
83d5868f005966883a918a819a449f636a5b3d5f)
Built on Fri, 23 Aug 2013 17:52:24 PST

And my java version:

$ java -version
java version "1.6.0_27"
OpenJDK Runtime Environment (IcedTea6 1.12.6)
(6b27-1.12.6-1ubuntu0.12.04.2)
OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)

I'm on Ubuntu 12.04 precise.

Thanks in advance for any insights!

To unsubscribe from this group and stop receiving emails from it, send an
email to impala-user...@cloudera.org <javascript:>.
To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 3 of 4 | next ›
Discussion Overview
groupimpala-user @
categorieshadoop
postedOct 2, '13 at 10:13p
activeOct 2, '13 at 11:19p
posts4
users2
websitecloudera.com
irc#hadoop

2 users in discussion

Colin Marc: 3 posts Alan Choi: 1 post

People

Translate

site design / logo © 2022 Grokbase