FAQ
hi,all

  I want to start impala with kerberos, I have done these things;
   1) start hadoop with kerberos and it work
   2) enable kerberos in hive metastore according to the instruction of
cloudera

http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Security-Guide/cdh4sg_topic_9_2.html

  now i found that :
  1)i just enable the enable kerberos in hive metastore and not in
impala-server(not configure any flag about kerberos in /etc/default/impala)
and i start the client with command: impala-shell (without -k)
       and it work , i can show, create ,drop table and load data)
  2) when i start the client with command: impala-shell -k , then it suppended
and can not do anything, but the porcess:
        /usr/lib/impala-shell/impala_shell.py -k -s hadoop is still
running, and i did not found any error log in log file
  3 ) and if i configure the principal and keytab_file
in /etc/default/impala and restart the server according to the cloudera
instruction and restart the server , the impalad started successfully.
       but the frontend did not start, not listen on 21000, impala-shell can
not connect of course
       and i check the log in impalad.ERROE impalad.INFO and
impala-server.log, but can not find any error information, just:

13/05/30 17:06:58 WARN conf.HiveConf: DEPRECATED: Configuration property
hive.metastore.local no longer has any effect. Make sure to provide a valid
value for hive.metastore.uris if you are connecting to a remote metastore.
13/05/30 17:06:58 INFO hive.metastore: Trying to connect to metastore with
URI thrift://XXXX:9083
13/05/30 17:06:59 INFO hive.metastore: Waiting 1 seconds before next
connection attempt.
13/05/30 17:07:00 INFO hive.metastore: Connected to metastore.
13/05/30 17:07:00 INFO hive.metastore: Trying to connect to metastore with
URI thrift://XXXXX:9083
13/05/30 17:07:00 INFO hive.metastore: Waiting 1 seconds before next
connection attempt.
13/05/30 17:07:01 INFO hive.metastore: Connected to metastore.
13/05/30 17:07:01 INFO hive.metastore: Trying to connect to metastore with
URI thrift://XXXXXX:9083
13/05/30 17:07:01 INFO hive.metastore: Waiting 1 seconds before next
connection attempt.
13/05/30 17:07:02 INFO hive.metastore: Connected to metastore.
13/05/30 17:07:02 INFO hive.metastore: Trying to connect to metastore with
URI thrift://XXXXXX:9083
13/05/30 17:07:02 INFO hive.metastore: Waiting 1 seconds before next
connection attempt.
13/05/30 17:07:03 INFO hive.metastore: Connected to metastore.
and i also check the hive-log, it show that the impala should connect to
the metastore(no error information)

so, it is hard to understand ,any suggestions will be useful, thanks!

Search Discussions

  • Lenni Kuff at May 30, 2013 at 2:54 pm
    Hi Jianan,
    The log snippet you sent shows Impala was able to connect to the Hive
    Metastore (which is a good start). To further investigate this problem,
    could you send us:

    1) The full list of flags impalad was started with. Also remember the same
    flags need to be applied to the statestore so Impala is able to communicate
    with it in a secure environment.
    2) The full impalad.INFO log file. Specifically, any text that is
    *after*the snippet you sent. If Kerberos is configured properly I
    would expect to
    see something like the following in the impalad logs. Please let me know
    how your log differs.

    INFO0516 17:42:21.434000 main hive.metastore] Trying to connect to
    metastore with URI thrift://XXX:9083
    INFO0516 17:42:21.442000 main hive.metastore] Waiting 1 seconds before next
    connection attempt.
    INFO0516 17:42:22.443000 main hive.metastore] Connected to metastore.
    I0516 17:42:24.802597 13796 impala-server.cc:2015] Impala Beeswax Service
    listening on 21000
    I0516 17:42:24.803232 13796 impala-server.cc:2026] Impala HiveServer2
    Service listening on 21050
    I0516 17:42:24.803282 13796 impala-server.cc:2034] ImpalaInternalService
    listening on 22000
    I0516 17:42:24.804029 13796 thrift-server.cc:365] ThriftServer 'backend'
    started on port: 22000
    I0516 17:42:24.804069 13796 exec-env.cc:146] Starting global services
    I0516 17:42:24.804112 13796 exec-env.cc:167] Using global memory limit:
    40.00 GB
    I0516 17:42:24.804147 13796 webserver.cc:126] Starting webserver on
    0.0.0.0:25000
    I0516 17:42:24.804167 13796 webserver.cc:139] Document root: /usr/lib/impala
    I0516 17:42:24.831971 13796 webserver.cc:178] Webserver started
    I0516 17:42:24.832020 13796 simple-scheduler.cc:103] Starting simple
    scheduler
    I0516 17:42:24.832772 13796 simple-scheduler.cc:149] Simple-scheduler
    using XXX as IP address
    I0516 17:42:24.832810 13796 state-store-subscriber.cc:137] Starting
    subscriber
    I0516 17:42:24.833181 13796 thrift-server.cc:365] ThriftServer
    'StateStoreSubscriber' started on port: 23000
    I0516 17:42:24.833256 13796 client-cache.cc:98] CreateClient(): adding new
    client for XXX:24000
    I0516 17:42:24.913975 13796 thrift-server.cc:365] ThriftServer
    'beeswax-frontend' started on port: 21000
    I0516 17:42:24.916090 13796 thrift-server.cc:365] ThriftServer
    'hiveServer2-frontend' started on port: 21050
    I0516 17:42:24.916127 13796 impalad-main.cc:102] Impala has started.
    I0516 17:42:26.122102 13885 authorization.cc:153] Kerberos User: impala/XXX
    for: impala/XXX from
    INFO0516 17:54:13.401000 TGT Renewer for impala/
    xxx.hal.cloudera.com@xxx.cloudera.comorg.apache.hadoop.security.usergroupinformation]
    Initiating logout for
    impala/XXX@XXX.CLOUDERA.COM


    Thanks,
    Lenni
    On Thu, May 30, 2013 at 3:44 AM, jianan Mo wrote:

    hi,all

    I want to start impala with kerberos, I have done these things;
    1) start hadoop with kerberos and it work
    2) enable kerberos in hive metastore according to the instruction of
    cloudera

    http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Security-Guide/cdh4sg_topic_9_2.html

    now i found that :
    1)i just enable the enable kerberos in hive metastore and not in
    impala-server(not configure any flag about kerberos in /etc/default/impala)
    and i start the client with command: impala-shell (without -k)
    and it work , i can show, create ,drop table and load data)
    2) when i start the client with command: impala-shell -k , then it suppended
    and can not do anything, but the porcess:
    /usr/lib/impala-shell/impala_shell.py -k -s hadoop is still
    running, and i did not found any error log in log file
    3 ) and if i configure the principal and keytab_file
    in /etc/default/impala and restart the server according to the cloudera
    instruction and restart the server , the impalad started successfully.
    but the frontend did not start, not listen on 21000, impala-shell
    can not connect of course
    and i check the log in impalad.ERROE impalad.INFO and
    impala-server.log, but can not find any error information, just:

    13/05/30 17:06:58 WARN conf.HiveConf: DEPRECATED: Configuration property
    hive.metastore.local no longer has any effect. Make sure to provide a valid
    value for hive.metastore.uris if you are connecting to a remote metastore.
    13/05/30 17:06:58 INFO hive.metastore: Trying to connect to metastore with
    URI thrift://XXXX:9083
    13/05/30 17:06:59 INFO hive.metastore: Waiting 1 seconds before next
    connection attempt.
    13/05/30 17:07:00 INFO hive.metastore: Connected to metastore.
    13/05/30 17:07:00 INFO hive.metastore: Trying to connect to metastore with
    URI thrift://XXXXX:9083
    13/05/30 17:07:00 INFO hive.metastore: Waiting 1 seconds before next
    connection attempt.
    13/05/30 17:07:01 INFO hive.metastore: Connected to metastore.
    13/05/30 17:07:01 INFO hive.metastore: Trying to connect to metastore with
    URI thrift://XXXXXX:9083
    13/05/30 17:07:01 INFO hive.metastore: Waiting 1 seconds before next
    connection attempt.
    13/05/30 17:07:02 INFO hive.metastore: Connected to metastore.
    13/05/30 17:07:02 INFO hive.metastore: Trying to connect to metastore with
    URI thrift://XXXXXX:9083
    13/05/30 17:07:02 INFO hive.metastore: Waiting 1 seconds before next
    connection attempt.
    13/05/30 17:07:03 INFO hive.metastore: Connected to metastore.
    and i also check the hive-log, it show that the impala should connect to
    the metastore(no error information)

    so, it is hard to understand ,any suggestions will be useful, thanks!

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupimpala-user @
categorieshadoop
postedMay 30, '13 at 10:44a
activeMay 30, '13 at 2:54p
posts2
users2
websitecloudera.com
irc#hadoop

2 users in discussion

Lenni Kuff: 1 post jianan Mo: 1 post

People

Translate

site design / logo © 2022 Grokbase