Grokbase Groups Hive user August 2011
FAQ
Hi all,

I've been struggling with getting Hive authorization to work for a few
hours, and I really hope someone can help me. I installed Hive 0.7.1
on top of Hadoop 0.20.203. I'm using mysql for the metastore, and
configured Hive to enable authorization:

<property>
<name>hive.security.authorization.enabled</name>
<value>true</value>
<description>enable or disable the hive client authorization</description>
</property>

I kept all the other Hive security configs with their default settings.

I'm running in pseudo-distributed mode on a single node. HDFS, the Hive
metastore and the Hive CLI are all running as the same user (the HDFS
superuser). Here are the sequence of steps that are causing me issues.
Without authorization everything works perfectly (creating, loading, selecting).
I've also tried creating and loading the table without authorization, granting
the select privilege at various levels (global, table, database), turning on
auth and performing the select, resulting in the same exception.

Any help with this would be greatly appreciated!

Thanks,
Alex

--

[hduser@aholmes-desktop ~]$ hive
Hive history file=/tmp/hduser/hive_job_log_hduser_201108162158_1976573160.txt
hive> set hive.security.authorization.enabled=false;
hive> grant all to user hduser;
OK
Time taken: 0.233 seconds
hive> set hive.security.authorization.enabled=true;
hive> CREATE TABLE pokes3 (foo INT, bar STRING);
FAILED: Hive Internal Error:
org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
get_privilege_set failed: unknown result)
org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.thrift.TApplicationException: get_privilege_set failed:
unknown result
at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:433)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: org.apache.thrift.TApplicationException: get_privilege_set
failed: unknown result
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
... 14 more

Search Discussions

  • Yongqiang he at Aug 23, 2011 at 5:27 pm
    Have you created the metastore mysql tables for authorization? Can u
    do a show grant?

    thanks
    yongqiang
    On Tue, Aug 16, 2011 at 2:55 PM, Alex Holmes wrote:
    Hi all,

    I've been struggling with getting Hive authorization to work for a few
    hours, and I really hope someone can help me.  I installed Hive 0.7.1
    on top of Hadoop 0.20.203.  I'm using mysql for the metastore, and
    configured Hive to enable authorization:

    <property>
    <name>hive.security.authorization.enabled</name>
    <value>true</value>
    <description>enable or disable the hive client authorization</description>
    </property>

    I kept all the other Hive security configs with their default settings.

    I'm running in pseudo-distributed mode on a single node.  HDFS, the Hive
    metastore and the Hive CLI are all running as the same user (the HDFS
    superuser).  Here are the sequence of steps that are causing me issues.
    Without authorization everything works perfectly (creating, loading, selecting).
    I've also tried creating and loading the table without authorization, granting
    the select privilege at various levels (global, table, database), turning on
    auth and performing the select, resulting in the same exception.

    Any help with this would be greatly appreciated!

    Thanks,
    Alex

    --

    [hduser@aholmes-desktop ~]$ hive
    Hive history file=/tmp/hduser/hive_job_log_hduser_201108162158_1976573160.txt
    hive> set hive.security.authorization.enabled=false;
    hive> grant all to user hduser;
    OK
    Time taken: 0.233 seconds
    hive> set hive.security.authorization.enabled=true;
    hive> CREATE TABLE pokes3 (foo INT, bar STRING);
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
    at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:433)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
    Caused by: org.apache.thrift.TApplicationException: get_privilege_set
    failed: unknown result
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
    ... 14 more
  • Alex Holmes at Aug 24, 2011 at 1:25 pm
    I created the mysql database (with the simple create database command)
    and the remote metastore seemed to creat the mysql tables. Here's
    some grant information and what I see in the database:

    [hduser@aholmes-desktop conf]$ hive
    hive> grant all to user hduser;
    OK
    Time taken: 0.334 seconds
    hive> show grant user hduser;
    OK

    principalName hduser
    principalType USER
    privilege All
    grantTime 1314191500
    grantor hduser
    Time taken: 0.046 seconds
    hive> CREATE TABLE pokes (foo INT, bar STRING);
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
    ...

    mysql> use hive;
    Database changed
    mysql> select * from GLOBAL_PRIVS;
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    USER_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE
    PRINCIPAL_NAME | PRINCIPAL_TYPE | USER_PRIV |
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    1 | 1314191500 | 0 | hduser | USER
    hduser | USER | All |
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    1 row in set (0.00 sec)


    Thanks for your help,
    Alex
    On Tue, Aug 23, 2011 at 1:27 PM, yongqiang he wrote:
    Have you created the metastore mysql tables for authorization? Can u
    do a show grant?

    thanks
    yongqiang
    On Tue, Aug 16, 2011 at 2:55 PM, Alex Holmes wrote:
    Hi all,

    I've been struggling with getting Hive authorization to work for a few
    hours, and I really hope someone can help me.  I installed Hive 0.7.1
    on top of Hadoop 0.20.203.  I'm using mysql for the metastore, and
    configured Hive to enable authorization:

    <property>
    <name>hive.security.authorization.enabled</name>
    <value>true</value>
    <description>enable or disable the hive client authorization</description>
    </property>

    I kept all the other Hive security configs with their default settings.

    I'm running in pseudo-distributed mode on a single node.  HDFS, the Hive
    metastore and the Hive CLI are all running as the same user (the HDFS
    superuser).  Here are the sequence of steps that are causing me issues.
    Without authorization everything works perfectly (creating, loading, selecting).
    I've also tried creating and loading the table without authorization, granting
    the select privilege at various levels (global, table, database), turning on
    auth and performing the select, resulting in the same exception.

    Any help with this would be greatly appreciated!

    Thanks,
    Alex

    --

    [hduser@aholmes-desktop ~]$ hive
    Hive history file=/tmp/hduser/hive_job_log_hduser_201108162158_1976573160.txt
    hive> set hive.security.authorization.enabled=false;
    hive> grant all to user hduser;
    OK
    Time taken: 0.233 seconds
    hive> set hive.security.authorization.enabled=true;
    hive> CREATE TABLE pokes3 (foo INT, bar STRING);
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
    at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:433)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
    Caused by: org.apache.thrift.TApplicationException: get_privilege_set
    failed: unknown result
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
    ... 14 more
  • Yongqiang he at Aug 24, 2011 at 7:38 pm
    This is a bug. Will open a jira to fix this. and will backport it to 0.7.1.
    https://issues.apache.org/jira/browse/HIVE-2405

    thanks for reporting this one!
    On Wed, Aug 24, 2011 at 6:25 AM, Alex Holmes wrote:
    I created the mysql database (with the simple create database command)
    and the remote metastore seemed to creat the mysql tables.  Here's
    some grant information and what I see in the database:

    [hduser@aholmes-desktop conf]$ hive
    hive> grant all to user hduser;
    OK
    Time taken: 0.334 seconds
    hive> show grant user hduser;
    OK

    principalName   hduser
    principalType   USER
    privilege       All
    grantTime       1314191500
    grantor hduser
    Time taken: 0.046 seconds
    hive> CREATE TABLE pokes (foo INT, bar STRING);
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
    ...

    mysql> use hive;
    Database changed
    mysql> select * from GLOBAL_PRIVS;
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    USER_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE
    PRINCIPAL_NAME | PRINCIPAL_TYPE | USER_PRIV |
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    1 |  1314191500 |            0 | hduser  | USER
    hduser         | USER           | All       |
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    1 row in set (0.00 sec)


    Thanks for your help,
    Alex
    On Tue, Aug 23, 2011 at 1:27 PM, yongqiang he wrote:
    Have you created the metastore mysql tables for authorization? Can u
    do a show grant?

    thanks
    yongqiang
    On Tue, Aug 16, 2011 at 2:55 PM, Alex Holmes wrote:
    Hi all,

    I've been struggling with getting Hive authorization to work for a few
    hours, and I really hope someone can help me.  I installed Hive 0.7.1
    on top of Hadoop 0.20.203.  I'm using mysql for the metastore, and
    configured Hive to enable authorization:

    <property>
    <name>hive.security.authorization.enabled</name>
    <value>true</value>
    <description>enable or disable the hive client authorization</description>
    </property>

    I kept all the other Hive security configs with their default settings.

    I'm running in pseudo-distributed mode on a single node.  HDFS, the Hive
    metastore and the Hive CLI are all running as the same user (the HDFS
    superuser).  Here are the sequence of steps that are causing me issues.
    Without authorization everything works perfectly (creating, loading, selecting).
    I've also tried creating and loading the table without authorization, granting
    the select privilege at various levels (global, table, database), turning on
    auth and performing the select, resulting in the same exception.

    Any help with this would be greatly appreciated!

    Thanks,
    Alex

    --

    [hduser@aholmes-desktop ~]$ hive
    Hive history file=/tmp/hduser/hive_job_log_hduser_201108162158_1976573160.txt
    hive> set hive.security.authorization.enabled=false;
    hive> grant all to user hduser;
    OK
    Time taken: 0.233 seconds
    hive> set hive.security.authorization.enabled=true;
    hive> CREATE TABLE pokes3 (foo INT, bar STRING);
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
    at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:433)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
    Caused by: org.apache.thrift.TApplicationException: get_privilege_set
    failed: unknown result
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
    ... 14 more
  • Alex Holmes at Aug 24, 2011 at 8:59 pm
    Thanks for opening a ticket.

    Table-level grants aren't working for me either (HIVE-2405 suggests
    that the bug is only related to global grants).

    hive> set hive.security.authorization.enabled=false;
    hive> CREATE TABLE pokes (foo INT, bar STRING);
    OK
    Time taken: 1.245 seconds
    hive> LOAD DATA LOCAL INPATH 'hive1.in' OVERWRITE INTO TABLE pokes;
    FAILED: Error in semantic analysis: Line 1:23 Invalid path 'hive1.in':
    No files matching path file:/app/hadoop/hive-0.7.1/conf/hive1.in
    hive> LOAD DATA LOCAL INPATH '/app/hadoop/hive1.in' OVERWRITE INTO TABLE pokes;
    Copying data from file:/app/hadoop/hive1.in
    Copying file: file:/app/hadoop/hive1.in
    Loading data to table default.pokes
    Moved to trash: hdfs://localhost:54310/user/hive/warehouse/pokes
    OK
    Time taken: 0.33 seconds
    hive> select * from pokes;
    OK
    1 a
    2 b
    3 c
    Time taken: 0.095 seconds
    hive> grant select on table pokes to user hduser;
    OK
    Time taken: 0.251 seconds
    hive> set hive.security.authorization.enabled=true;
    hive> select * from pokes;
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    ...

    mysql> select * from TBL_PRIVS;
    +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
    TBL_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE |
    PRINCIPAL_NAME | PRINCIPAL_TYPE | TBL_PRIV | TBL_ID |
    +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
    1 | 1314219701 | 0 | hduser | USER |
    hduser | USER | Select | 1 |
    +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+

    Also, I noticed in HIVE-2405 that you get a meaningful error message:

    Authorization failed:No privilege 'Create' found for outputs {
    database:default}. Use show grant to get more details.

    Whereas I just get an exception (as you can see above). Were you also
    running with the remote metastore? I get these meaningful messages
    with the local metastore (and authorization on), but with the remote
    metastore with authorization turned on, I always get exceptions.

    Many thanks,
    Alex
    On Wed, Aug 24, 2011 at 3:38 PM, yongqiang he wrote:
    This is a bug. Will open a jira to fix this. and will backport it to 0.7.1.
    https://issues.apache.org/jira/browse/HIVE-2405

    thanks for reporting this one!
    On Wed, Aug 24, 2011 at 6:25 AM, Alex Holmes wrote:
    I created the mysql database (with the simple create database command)
    and the remote metastore seemed to creat the mysql tables.  Here's
    some grant information and what I see in the database:

    [hduser@aholmes-desktop conf]$ hive
    hive> grant all to user hduser;
    OK
    Time taken: 0.334 seconds
    hive> show grant user hduser;
    OK

    principalName   hduser
    principalType   USER
    privilege       All
    grantTime       1314191500
    grantor hduser
    Time taken: 0.046 seconds
    hive> CREATE TABLE pokes (foo INT, bar STRING);
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
    ...

    mysql> use hive;
    Database changed
    mysql> select * from GLOBAL_PRIVS;
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    USER_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE
    PRINCIPAL_NAME | PRINCIPAL_TYPE | USER_PRIV |
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    1 |  1314191500 |            0 | hduser  | USER
    hduser         | USER           | All       |
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    1 row in set (0.00 sec)


    Thanks for your help,
    Alex
    On Tue, Aug 23, 2011 at 1:27 PM, yongqiang he wrote:
    Have you created the metastore mysql tables for authorization? Can u
    do a show grant?

    thanks
    yongqiang
    On Tue, Aug 16, 2011 at 2:55 PM, Alex Holmes wrote:
    Hi all,

    I've been struggling with getting Hive authorization to work for a few
    hours, and I really hope someone can help me.  I installed Hive 0.7.1
    on top of Hadoop 0.20.203.  I'm using mysql for the metastore, and
    configured Hive to enable authorization:

    <property>
    <name>hive.security.authorization.enabled</name>
    <value>true</value>
    <description>enable or disable the hive client authorization</description>
    </property>

    I kept all the other Hive security configs with their default settings.

    I'm running in pseudo-distributed mode on a single node.  HDFS, the Hive
    metastore and the Hive CLI are all running as the same user (the HDFS
    superuser).  Here are the sequence of steps that are causing me issues.
    Without authorization everything works perfectly (creating, loading, selecting).
    I've also tried creating and loading the table without authorization, granting
    the select privilege at various levels (global, table, database), turning on
    auth and performing the select, resulting in the same exception.

    Any help with this would be greatly appreciated!

    Thanks,
    Alex

    --

    [hduser@aholmes-desktop ~]$ hive
    Hive history file=/tmp/hduser/hive_job_log_hduser_201108162158_1976573160.txt
    hive> set hive.security.authorization.enabled=false;
    hive> grant all to user hduser;
    OK
    Time taken: 0.233 seconds
    hive> set hive.security.authorization.enabled=true;
    hive> CREATE TABLE pokes3 (foo INT, bar STRING);
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
    at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:433)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
    Caused by: org.apache.thrift.TApplicationException: get_privilege_set
    failed: unknown result
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
    ... 14 more
  • Yongqiang he at Aug 24, 2011 at 9:21 pm
    I am using local metastore, and can not reproduce the problem.

    what message did you get when running local metastore?
    On Wed, Aug 24, 2011 at 1:58 PM, Alex Holmes wrote:
    Thanks for opening a ticket.

    Table-level grants aren't working for me either (HIVE-2405 suggests
    that the bug is only related to global grants).

    hive> set hive.security.authorization.enabled=false;
    hive> CREATE TABLE pokes (foo INT, bar STRING);
    OK
    Time taken: 1.245 seconds
    hive> LOAD DATA LOCAL INPATH 'hive1.in' OVERWRITE INTO TABLE pokes;
    FAILED: Error in semantic analysis: Line 1:23 Invalid path 'hive1.in':
    No files matching path file:/app/hadoop/hive-0.7.1/conf/hive1.in
    hive> LOAD DATA LOCAL INPATH '/app/hadoop/hive1.in' OVERWRITE INTO TABLE pokes;
    Copying data from file:/app/hadoop/hive1.in
    Copying file: file:/app/hadoop/hive1.in
    Loading data to table default.pokes
    Moved to trash: hdfs://localhost:54310/user/hive/warehouse/pokes
    OK
    Time taken: 0.33 seconds
    hive> select * from pokes;
    OK
    1       a
    2       b
    3       c
    Time taken: 0.095 seconds
    hive> grant select on table pokes to user hduser;
    OK
    Time taken: 0.251 seconds
    hive> set hive.security.authorization.enabled=true;
    hive> select * from pokes;
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    ...

    mysql> select * from TBL_PRIVS;
    +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
    TBL_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE |
    PRINCIPAL_NAME | PRINCIPAL_TYPE | TBL_PRIV | TBL_ID |
    +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
    1 |  1314219701 |            0 | hduser  | USER         |
    hduser         | USER           | Select   |      1 |
    +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+

    Also, I noticed in HIVE-2405 that you get a meaningful error message:

    Authorization failed:No privilege 'Create' found for outputs {
    database:default}. Use show grant to get more details.

    Whereas I just get an exception (as you can see above).  Were you also
    running with the remote metastore?  I get these meaningful messages
    with the local metastore (and authorization on), but with the remote
    metastore with authorization turned on, I always get exceptions.

    Many thanks,
    Alex
    On Wed, Aug 24, 2011 at 3:38 PM, yongqiang he wrote:
    This is a bug. Will open a jira to fix this. and will backport it to 0.7.1.
    https://issues.apache.org/jira/browse/HIVE-2405

    thanks for reporting this one!
    On Wed, Aug 24, 2011 at 6:25 AM, Alex Holmes wrote:
    I created the mysql database (with the simple create database command)
    and the remote metastore seemed to creat the mysql tables.  Here's
    some grant information and what I see in the database:

    [hduser@aholmes-desktop conf]$ hive
    hive> grant all to user hduser;
    OK
    Time taken: 0.334 seconds
    hive> show grant user hduser;
    OK

    principalName   hduser
    principalType   USER
    privilege       All
    grantTime       1314191500
    grantor hduser
    Time taken: 0.046 seconds
    hive> CREATE TABLE pokes (foo INT, bar STRING);
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
    ...

    mysql> use hive;
    Database changed
    mysql> select * from GLOBAL_PRIVS;
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    USER_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE
    PRINCIPAL_NAME | PRINCIPAL_TYPE | USER_PRIV |
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    1 |  1314191500 |            0 | hduser  | USER
    hduser         | USER           | All       |
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    1 row in set (0.00 sec)


    Thanks for your help,
    Alex
    On Tue, Aug 23, 2011 at 1:27 PM, yongqiang he wrote:
    Have you created the metastore mysql tables for authorization? Can u
    do a show grant?

    thanks
    yongqiang
    On Tue, Aug 16, 2011 at 2:55 PM, Alex Holmes wrote:
    Hi all,

    I've been struggling with getting Hive authorization to work for a few
    hours, and I really hope someone can help me.  I installed Hive 0.7.1
    on top of Hadoop 0.20.203.  I'm using mysql for the metastore, and
    configured Hive to enable authorization:

    <property>
    <name>hive.security.authorization.enabled</name>
    <value>true</value>
    <description>enable or disable the hive client authorization</description>
    </property>

    I kept all the other Hive security configs with their default settings.

    I'm running in pseudo-distributed mode on a single node.  HDFS, the Hive
    metastore and the Hive CLI are all running as the same user (the HDFS
    superuser).  Here are the sequence of steps that are causing me issues.
    Without authorization everything works perfectly (creating, loading, selecting).
    I've also tried creating and loading the table without authorization, granting
    the select privilege at various levels (global, table, database), turning on
    auth and performing the select, resulting in the same exception.

    Any help with this would be greatly appreciated!

    Thanks,
    Alex

    --

    [hduser@aholmes-desktop ~]$ hive
    Hive history file=/tmp/hduser/hive_job_log_hduser_201108162158_1976573160.txt
    hive> set hive.security.authorization.enabled=false;
    hive> grant all to user hduser;
    OK
    Time taken: 0.233 seconds
    hive> set hive.security.authorization.enabled=true;
    hive> CREATE TABLE pokes3 (foo INT, bar STRING);
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
    at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:433)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
    Caused by: org.apache.thrift.TApplicationException: get_privilege_set
    failed: unknown result
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
    ... 14 more
  • Alex Holmes at Aug 24, 2011 at 9:34 pm
    Authorization works for me with the local metastore. The remote
    metastore works with authorization turned off, but as soon as I turn
    it on and issue any commands I get these exceptions on the hive
    client.

    Could you also try the remote metastore please? I'm pretty sure that
    authorization does not work with it at all.

    Thanks,
    Alex
    On Wed, Aug 24, 2011 at 5:20 PM, yongqiang he wrote:
    I am using local metastore,  and can not reproduce the problem.

    what message did you get when running local metastore?
    On Wed, Aug 24, 2011 at 1:58 PM, Alex Holmes wrote:
    Thanks for opening a ticket.

    Table-level grants aren't working for me either (HIVE-2405 suggests
    that the bug is only related to global grants).

    hive> set hive.security.authorization.enabled=false;
    hive> CREATE TABLE pokes (foo INT, bar STRING);
    OK
    Time taken: 1.245 seconds
    hive> LOAD DATA LOCAL INPATH 'hive1.in' OVERWRITE INTO TABLE pokes;
    FAILED: Error in semantic analysis: Line 1:23 Invalid path 'hive1.in':
    No files matching path file:/app/hadoop/hive-0.7.1/conf/hive1.in
    hive> LOAD DATA LOCAL INPATH '/app/hadoop/hive1.in' OVERWRITE INTO TABLE pokes;
    Copying data from file:/app/hadoop/hive1.in
    Copying file: file:/app/hadoop/hive1.in
    Loading data to table default.pokes
    Moved to trash: hdfs://localhost:54310/user/hive/warehouse/pokes
    OK
    Time taken: 0.33 seconds
    hive> select * from pokes;
    OK
    1       a
    2       b
    3       c
    Time taken: 0.095 seconds
    hive> grant select on table pokes to user hduser;
    OK
    Time taken: 0.251 seconds
    hive> set hive.security.authorization.enabled=true;
    hive> select * from pokes;
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    ...

    mysql> select * from TBL_PRIVS;
    +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
    TBL_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE |
    PRINCIPAL_NAME | PRINCIPAL_TYPE | TBL_PRIV | TBL_ID |
    +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
    1 |  1314219701 |            0 | hduser  | USER         |
    hduser         | USER           | Select   |      1 |
    +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+

    Also, I noticed in HIVE-2405 that you get a meaningful error message:

    Authorization failed:No privilege 'Create' found for outputs {
    database:default}. Use show grant to get more details.

    Whereas I just get an exception (as you can see above).  Were you also
    running with the remote metastore?  I get these meaningful messages
    with the local metastore (and authorization on), but with the remote
    metastore with authorization turned on, I always get exceptions.

    Many thanks,
    Alex
    On Wed, Aug 24, 2011 at 3:38 PM, yongqiang he wrote:
    This is a bug. Will open a jira to fix this. and will backport it to 0.7.1.
    https://issues.apache.org/jira/browse/HIVE-2405

    thanks for reporting this one!
    On Wed, Aug 24, 2011 at 6:25 AM, Alex Holmes wrote:
    I created the mysql database (with the simple create database command)
    and the remote metastore seemed to creat the mysql tables.  Here's
    some grant information and what I see in the database:

    [hduser@aholmes-desktop conf]$ hive
    hive> grant all to user hduser;
    OK
    Time taken: 0.334 seconds
    hive> show grant user hduser;
    OK

    principalName   hduser
    principalType   USER
    privilege       All
    grantTime       1314191500
    grantor hduser
    Time taken: 0.046 seconds
    hive> CREATE TABLE pokes (foo INT, bar STRING);
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
    ...

    mysql> use hive;
    Database changed
    mysql> select * from GLOBAL_PRIVS;
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    USER_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE
    PRINCIPAL_NAME | PRINCIPAL_TYPE | USER_PRIV |
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    1 |  1314191500 |            0 | hduser  | USER
    hduser         | USER           | All       |
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    1 row in set (0.00 sec)


    Thanks for your help,
    Alex
    On Tue, Aug 23, 2011 at 1:27 PM, yongqiang he wrote:
    Have you created the metastore mysql tables for authorization? Can u
    do a show grant?

    thanks
    yongqiang
    On Tue, Aug 16, 2011 at 2:55 PM, Alex Holmes wrote:
    Hi all,

    I've been struggling with getting Hive authorization to work for a few
    hours, and I really hope someone can help me.  I installed Hive 0.7.1
    on top of Hadoop 0.20.203.  I'm using mysql for the metastore, and
    configured Hive to enable authorization:

    <property>
    <name>hive.security.authorization.enabled</name>
    <value>true</value>
    <description>enable or disable the hive client authorization</description>
    </property>

    I kept all the other Hive security configs with their default settings.

    I'm running in pseudo-distributed mode on a single node.  HDFS, the Hive
    metastore and the Hive CLI are all running as the same user (the HDFS
    superuser).  Here are the sequence of steps that are causing me issues.
    Without authorization everything works perfectly (creating, loading, selecting).
    I've also tried creating and loading the table without authorization, granting
    the select privilege at various levels (global, table, database), turning on
    auth and performing the select, resulting in the same exception.

    Any help with this would be greatly appreciated!

    Thanks,
    Alex

    --

    [hduser@aholmes-desktop ~]$ hive
    Hive history file=/tmp/hduser/hive_job_log_hduser_201108162158_1976573160.txt
    hive> set hive.security.authorization.enabled=false;
    hive> grant all to user hduser;
    OK
    Time taken: 0.233 seconds
    hive> set hive.security.authorization.enabled=true;
    hive> CREATE TABLE pokes3 (foo INT, bar STRING);
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
    at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:433)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
    Caused by: org.apache.thrift.TApplicationException: get_privilege_set
    failed: unknown result
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
    ... 14 more
  • Yongqiang he at Aug 24, 2011 at 10:07 pm
    this is what i have tried with a remote metastore:
    set hive.security.authorization.enabled=false;
    hive>
    >
    >
    drop table src2;
    OK
    Time taken: 1.002 seconds
    hive> create table src2 (key int, value string);
    OK
    Time taken: 0.03 seconds
    hive>
    >
    >
    set hive.security.authorization.enabled=true;
    hive> grant select on table src2 to user heyongqiang;
    OK
    Time taken: 0.113 seconds
    hive> select * from src2;
    OK
    Time taken: 0.188 seconds
    hive> show grant user heyongqiang on table src2;
    OK

    database default
    table src2
    principalName heyongqiang
    principalType USER
    privilege Select
    grantTime Wed Aug 24 15:03:51 PDT 2011
    grantor heyongqiang

    can u do a show grant?

    (But with remote metastore, i think hive should not return empty list
    instead of null for list_privileges etc.)


    On Wed, Aug 24, 2011 at 2:34 PM, Alex Holmes wrote:
    Authorization works for me with the local metastore.  The remote
    metastore works with authorization turned off, but as soon as I turn
    it on and issue any commands I get these exceptions on the hive
    client.

    Could you also try the remote metastore please?  I'm pretty sure that
    authorization does not work with it at all.

    Thanks,
    Alex
    On Wed, Aug 24, 2011 at 5:20 PM, yongqiang he wrote:
    I am using local metastore,  and can not reproduce the problem.

    what message did you get when running local metastore?
    On Wed, Aug 24, 2011 at 1:58 PM, Alex Holmes wrote:
    Thanks for opening a ticket.

    Table-level grants aren't working for me either (HIVE-2405 suggests
    that the bug is only related to global grants).

    hive> set hive.security.authorization.enabled=false;
    hive> CREATE TABLE pokes (foo INT, bar STRING);
    OK
    Time taken: 1.245 seconds
    hive> LOAD DATA LOCAL INPATH 'hive1.in' OVERWRITE INTO TABLE pokes;
    FAILED: Error in semantic analysis: Line 1:23 Invalid path 'hive1.in':
    No files matching path file:/app/hadoop/hive-0.7.1/conf/hive1.in
    hive> LOAD DATA LOCAL INPATH '/app/hadoop/hive1.in' OVERWRITE INTO TABLE pokes;
    Copying data from file:/app/hadoop/hive1.in
    Copying file: file:/app/hadoop/hive1.in
    Loading data to table default.pokes
    Moved to trash: hdfs://localhost:54310/user/hive/warehouse/pokes
    OK
    Time taken: 0.33 seconds
    hive> select * from pokes;
    OK
    1       a
    2       b
    3       c
    Time taken: 0.095 seconds
    hive> grant select on table pokes to user hduser;
    OK
    Time taken: 0.251 seconds
    hive> set hive.security.authorization.enabled=true;
    hive> select * from pokes;
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    ...

    mysql> select * from TBL_PRIVS;
    +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
    TBL_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE |
    PRINCIPAL_NAME | PRINCIPAL_TYPE | TBL_PRIV | TBL_ID |
    +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
    1 |  1314219701 |            0 | hduser  | USER         |
    hduser         | USER           | Select   |      1 |
    +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+

    Also, I noticed in HIVE-2405 that you get a meaningful error message:

    Authorization failed:No privilege 'Create' found for outputs {
    database:default}. Use show grant to get more details.

    Whereas I just get an exception (as you can see above).  Were you also
    running with the remote metastore?  I get these meaningful messages
    with the local metastore (and authorization on), but with the remote
    metastore with authorization turned on, I always get exceptions.

    Many thanks,
    Alex
    On Wed, Aug 24, 2011 at 3:38 PM, yongqiang he wrote:
    This is a bug. Will open a jira to fix this. and will backport it to 0.7.1.
    https://issues.apache.org/jira/browse/HIVE-2405

    thanks for reporting this one!
    On Wed, Aug 24, 2011 at 6:25 AM, Alex Holmes wrote:
    I created the mysql database (with the simple create database command)
    and the remote metastore seemed to creat the mysql tables.  Here's
    some grant information and what I see in the database:

    [hduser@aholmes-desktop conf]$ hive
    hive> grant all to user hduser;
    OK
    Time taken: 0.334 seconds
    hive> show grant user hduser;
    OK

    principalName   hduser
    principalType   USER
    privilege       All
    grantTime       1314191500
    grantor hduser
    Time taken: 0.046 seconds
    hive> CREATE TABLE pokes (foo INT, bar STRING);
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
    ...

    mysql> use hive;
    Database changed
    mysql> select * from GLOBAL_PRIVS;
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    USER_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE
    PRINCIPAL_NAME | PRINCIPAL_TYPE | USER_PRIV |
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    1 |  1314191500 |            0 | hduser  | USER
    hduser         | USER           | All       |
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    1 row in set (0.00 sec)


    Thanks for your help,
    Alex
    On Tue, Aug 23, 2011 at 1:27 PM, yongqiang he wrote:
    Have you created the metastore mysql tables for authorization? Can u
    do a show grant?

    thanks
    yongqiang
    On Tue, Aug 16, 2011 at 2:55 PM, Alex Holmes wrote:
    Hi all,

    I've been struggling with getting Hive authorization to work for a few
    hours, and I really hope someone can help me.  I installed Hive 0.7.1
    on top of Hadoop 0.20.203.  I'm using mysql for the metastore, and
    configured Hive to enable authorization:

    <property>
    <name>hive.security.authorization.enabled</name>
    <value>true</value>
    <description>enable or disable the hive client authorization</description>
    </property>

    I kept all the other Hive security configs with their default settings.

    I'm running in pseudo-distributed mode on a single node.  HDFS, the Hive
    metastore and the Hive CLI are all running as the same user (the HDFS
    superuser).  Here are the sequence of steps that are causing me issues.
    Without authorization everything works perfectly (creating, loading, selecting).
    I've also tried creating and loading the table without authorization, granting
    the select privilege at various levels (global, table, database), turning on
    auth and performing the select, resulting in the same exception.

    Any help with this would be greatly appreciated!

    Thanks,
    Alex

    --

    [hduser@aholmes-desktop ~]$ hive
    Hive history file=/tmp/hduser/hive_job_log_hduser_201108162158_1976573160.txt
    hive> set hive.security.authorization.enabled=false;
    hive> grant all to user hduser;
    OK
    Time taken: 0.233 seconds
    hive> set hive.security.authorization.enabled=true;
    hive> CREATE TABLE pokes3 (foo INT, bar STRING);
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
    at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:433)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
    Caused by: org.apache.thrift.TApplicationException: get_privilege_set
    failed: unknown result
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
    ... 14 more
  • Alex Holmes at Aug 26, 2011 at 12:14 am
    Hi,

    hive> CREATE TABLE pokes2 (foo INT, bar STRING);
    OK
    hive> LOAD DATA LOCAL INPATH '/app/hadoop/hive1.in' OVERWRITE INTO TABLE pokes2;
    OK
    hive> grant select on table pokes2 to user hduser;
    OK
    hive> set hive.security.authorization.enabled=true;
    hive> show grant user hduser on table pokes2;
    OK

    database default
    table pokes2
    principalName hduser
    principalType USER
    privilege Select
    grantTime 1314318185
    grantor hduser
    Time taken: 0.041 seconds

    hive> select * from pokes2;
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserDBAndTable(DefaultHiveAuthorizationProvider.java:259)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:159)
    at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:531)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
    Caused by: org.apache.thrift.TApplicationException: get_privilege_set
    failed: unknown result
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
    ... 15 more



    On Wed, Aug 24, 2011 at 6:06 PM, yongqiang he wrote:
    this is what i have tried with a remote metastore:
    set hive.security.authorization.enabled=false; hive>

    drop table src2;
    OK
    Time taken: 1.002 seconds
    hive> create table src2 (key int, value string);
    OK
    Time taken: 0.03 seconds
    hive>

    set hive.security.authorization.enabled=true;
    hive> grant select on table src2 to user heyongqiang;
    OK
    Time taken: 0.113 seconds
    hive> select * from src2;
    OK
    Time taken: 0.188 seconds
    hive> show grant user heyongqiang on table src2;
    OK

    database        default
    table   src2
    principalName   heyongqiang
    principalType   USER
    privilege       Select
    grantTime       Wed Aug 24 15:03:51 PDT 2011
    grantor heyongqiang

    can u do a show grant?

    (But with remote metastore, i think hive should not return empty list
    instead of null for list_privileges etc.)


    On Wed, Aug 24, 2011 at 2:34 PM, Alex Holmes wrote:
    Authorization works for me with the local metastore.  The remote
    metastore works with authorization turned off, but as soon as I turn
    it on and issue any commands I get these exceptions on the hive
    client.

    Could you also try the remote metastore please?  I'm pretty sure that
    authorization does not work with it at all.

    Thanks,
    Alex
    On Wed, Aug 24, 2011 at 5:20 PM, yongqiang he wrote:
    I am using local metastore,  and can not reproduce the problem.

    what message did you get when running local metastore?
    On Wed, Aug 24, 2011 at 1:58 PM, Alex Holmes wrote:
    Thanks for opening a ticket.

    Table-level grants aren't working for me either (HIVE-2405 suggests
    that the bug is only related to global grants).

    hive> set hive.security.authorization.enabled=false;
    hive> CREATE TABLE pokes (foo INT, bar STRING);
    OK
    Time taken: 1.245 seconds
    hive> LOAD DATA LOCAL INPATH 'hive1.in' OVERWRITE INTO TABLE pokes;
    FAILED: Error in semantic analysis: Line 1:23 Invalid path 'hive1.in':
    No files matching path file:/app/hadoop/hive-0.7.1/conf/hive1.in
    hive> LOAD DATA LOCAL INPATH '/app/hadoop/hive1.in' OVERWRITE INTO TABLE pokes;
    Copying data from file:/app/hadoop/hive1.in
    Copying file: file:/app/hadoop/hive1.in
    Loading data to table default.pokes
    Moved to trash: hdfs://localhost:54310/user/hive/warehouse/pokes
    OK
    Time taken: 0.33 seconds
    hive> select * from pokes;
    OK
    1       a
    2       b
    3       c
    Time taken: 0.095 seconds
    hive> grant select on table pokes to user hduser;
    OK
    Time taken: 0.251 seconds
    hive> set hive.security.authorization.enabled=true;
    hive> select * from pokes;
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    ...

    mysql> select * from TBL_PRIVS;
    +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
    TBL_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE |
    PRINCIPAL_NAME | PRINCIPAL_TYPE | TBL_PRIV | TBL_ID |
    +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
    1 |  1314219701 |            0 | hduser  | USER         |
    hduser         | USER           | Select   |      1 |
    +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+

    Also, I noticed in HIVE-2405 that you get a meaningful error message:

    Authorization failed:No privilege 'Create' found for outputs {
    database:default}. Use show grant to get more details.

    Whereas I just get an exception (as you can see above).  Were you also
    running with the remote metastore?  I get these meaningful messages
    with the local metastore (and authorization on), but with the remote
    metastore with authorization turned on, I always get exceptions.

    Many thanks,
    Alex
    On Wed, Aug 24, 2011 at 3:38 PM, yongqiang he wrote:
    This is a bug. Will open a jira to fix this. and will backport it to 0.7.1.
    https://issues.apache.org/jira/browse/HIVE-2405

    thanks for reporting this one!
    On Wed, Aug 24, 2011 at 6:25 AM, Alex Holmes wrote:
    I created the mysql database (with the simple create database command)
    and the remote metastore seemed to creat the mysql tables.  Here's
    some grant information and what I see in the database:

    [hduser@aholmes-desktop conf]$ hive
    hive> grant all to user hduser;
    OK
    Time taken: 0.334 seconds
    hive> show grant user hduser;
    OK

    principalName   hduser
    principalType   USER
    privilege       All
    grantTime       1314191500
    grantor hduser
    Time taken: 0.046 seconds
    hive> CREATE TABLE pokes (foo INT, bar STRING);
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
    ...

    mysql> use hive;
    Database changed
    mysql> select * from GLOBAL_PRIVS;
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    USER_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE
    PRINCIPAL_NAME | PRINCIPAL_TYPE | USER_PRIV |
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    1 |  1314191500 |            0 | hduser  | USER
    hduser         | USER           | All       |
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    1 row in set (0.00 sec)


    Thanks for your help,
    Alex
    On Tue, Aug 23, 2011 at 1:27 PM, yongqiang he wrote:
    Have you created the metastore mysql tables for authorization? Can u
    do a show grant?

    thanks
    yongqiang
    On Tue, Aug 16, 2011 at 2:55 PM, Alex Holmes wrote:
    Hi all,

    I've been struggling with getting Hive authorization to work for a few
    hours, and I really hope someone can help me.  I installed Hive 0.7.1
    on top of Hadoop 0.20.203.  I'm using mysql for the metastore, and
    configured Hive to enable authorization:

    <property>
    <name>hive.security.authorization.enabled</name>
    <value>true</value>
    <description>enable or disable the hive client authorization</description>
    </property>

    I kept all the other Hive security configs with their default settings.

    I'm running in pseudo-distributed mode on a single node.  HDFS, the Hive
    metastore and the Hive CLI are all running as the same user (the HDFS
    superuser).  Here are the sequence of steps that are causing me issues.
    Without authorization everything works perfectly (creating, loading, selecting).
    I've also tried creating and loading the table without authorization, granting
    the select privilege at various levels (global, table, database), turning on
    auth and performing the select, resulting in the same exception.

    Any help with this would be greatly appreciated!

    Thanks,
    Alex

    --

    [hduser@aholmes-desktop ~]$ hive
    Hive history file=/tmp/hduser/hive_job_log_hduser_201108162158_1976573160.txt
    hive> set hive.security.authorization.enabled=false;
    hive> grant all to user hduser;
    OK
    Time taken: 0.233 seconds
    hive> set hive.security.authorization.enabled=true;
    hive> CREATE TABLE pokes3 (foo INT, bar STRING);
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
    at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:433)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
    Caused by: org.apache.thrift.TApplicationException: get_privilege_set
    failed: unknown result
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
    ... 14 more
  • Alex Holmes at Aug 26, 2011 at 12:15 am
    Here's the hive-site.xml file (I use the same file for both the client
    and remote metastore). We're using mysql as the metastore DB.


    <?xml version="1.0"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    <configuration>
    <property>
    <name>hive.security.authorization.enabled</name>
    <value>true</value>
    </property>
    <property>
    <name>hive.metastore.local</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.metastore.uris</name>
    <value>thrift://localhost:9083</value>
    </property>
    <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://localhost/hive?createDatabaseIfNotExist=true</value>
    </property>
    <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    </property>
    <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>hive</value>
    </property>
    <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>secret</value>
    </property>
    </configuration>


    On Wed, Aug 24, 2011 at 6:06 PM, yongqiang he wrote:
    this is what i have tried with a remote metastore:
    set hive.security.authorization.enabled=false; hive>

    drop table src2;
    OK
    Time taken: 1.002 seconds
    hive> create table src2 (key int, value string);
    OK
    Time taken: 0.03 seconds
    hive>

    set hive.security.authorization.enabled=true;
    hive> grant select on table src2 to user heyongqiang;
    OK
    Time taken: 0.113 seconds
    hive> select * from src2;
    OK
    Time taken: 0.188 seconds
    hive> show grant user heyongqiang on table src2;
    OK

    database        default
    table   src2
    principalName   heyongqiang
    principalType   USER
    privilege       Select
    grantTime       Wed Aug 24 15:03:51 PDT 2011
    grantor heyongqiang

    can u do a show grant?

    (But with remote metastore, i think hive should not return empty list
    instead of null for list_privileges etc.)


    On Wed, Aug 24, 2011 at 2:34 PM, Alex Holmes wrote:
    Authorization works for me with the local metastore.  The remote
    metastore works with authorization turned off, but as soon as I turn
    it on and issue any commands I get these exceptions on the hive
    client.

    Could you also try the remote metastore please?  I'm pretty sure that
    authorization does not work with it at all.

    Thanks,
    Alex
    On Wed, Aug 24, 2011 at 5:20 PM, yongqiang he wrote:
    I am using local metastore,  and can not reproduce the problem.

    what message did you get when running local metastore?
    On Wed, Aug 24, 2011 at 1:58 PM, Alex Holmes wrote:
    Thanks for opening a ticket.

    Table-level grants aren't working for me either (HIVE-2405 suggests
    that the bug is only related to global grants).

    hive> set hive.security.authorization.enabled=false;
    hive> CREATE TABLE pokes (foo INT, bar STRING);
    OK
    Time taken: 1.245 seconds
    hive> LOAD DATA LOCAL INPATH 'hive1.in' OVERWRITE INTO TABLE pokes;
    FAILED: Error in semantic analysis: Line 1:23 Invalid path 'hive1.in':
    No files matching path file:/app/hadoop/hive-0.7.1/conf/hive1.in
    hive> LOAD DATA LOCAL INPATH '/app/hadoop/hive1.in' OVERWRITE INTO TABLE pokes;
    Copying data from file:/app/hadoop/hive1.in
    Copying file: file:/app/hadoop/hive1.in
    Loading data to table default.pokes
    Moved to trash: hdfs://localhost:54310/user/hive/warehouse/pokes
    OK
    Time taken: 0.33 seconds
    hive> select * from pokes;
    OK
    1       a
    2       b
    3       c
    Time taken: 0.095 seconds
    hive> grant select on table pokes to user hduser;
    OK
    Time taken: 0.251 seconds
    hive> set hive.security.authorization.enabled=true;
    hive> select * from pokes;
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    ...

    mysql> select * from TBL_PRIVS;
    +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
    TBL_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE |
    PRINCIPAL_NAME | PRINCIPAL_TYPE | TBL_PRIV | TBL_ID |
    +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
    1 |  1314219701 |            0 | hduser  | USER         |
    hduser         | USER           | Select   |      1 |
    +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+

    Also, I noticed in HIVE-2405 that you get a meaningful error message:

    Authorization failed:No privilege 'Create' found for outputs {
    database:default}. Use show grant to get more details.

    Whereas I just get an exception (as you can see above).  Were you also
    running with the remote metastore?  I get these meaningful messages
    with the local metastore (and authorization on), but with the remote
    metastore with authorization turned on, I always get exceptions.

    Many thanks,
    Alex
    On Wed, Aug 24, 2011 at 3:38 PM, yongqiang he wrote:
    This is a bug. Will open a jira to fix this. and will backport it to 0.7.1.
    https://issues.apache.org/jira/browse/HIVE-2405

    thanks for reporting this one!
    On Wed, Aug 24, 2011 at 6:25 AM, Alex Holmes wrote:
    I created the mysql database (with the simple create database command)
    and the remote metastore seemed to creat the mysql tables.  Here's
    some grant information and what I see in the database:

    [hduser@aholmes-desktop conf]$ hive
    hive> grant all to user hduser;
    OK
    Time taken: 0.334 seconds
    hive> show grant user hduser;
    OK

    principalName   hduser
    principalType   USER
    privilege       All
    grantTime       1314191500
    grantor hduser
    Time taken: 0.046 seconds
    hive> CREATE TABLE pokes (foo INT, bar STRING);
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
    ...

    mysql> use hive;
    Database changed
    mysql> select * from GLOBAL_PRIVS;
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    USER_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE
    PRINCIPAL_NAME | PRINCIPAL_TYPE | USER_PRIV |
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    1 |  1314191500 |            0 | hduser  | USER
    hduser         | USER           | All       |
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    1 row in set (0.00 sec)


    Thanks for your help,
    Alex
    On Tue, Aug 23, 2011 at 1:27 PM, yongqiang he wrote:
    Have you created the metastore mysql tables for authorization? Can u
    do a show grant?

    thanks
    yongqiang
    On Tue, Aug 16, 2011 at 2:55 PM, Alex Holmes wrote:
    Hi all,

    I've been struggling with getting Hive authorization to work for a few
    hours, and I really hope someone can help me.  I installed Hive 0.7.1
    on top of Hadoop 0.20.203.  I'm using mysql for the metastore, and
    configured Hive to enable authorization:

    <property>
    <name>hive.security.authorization.enabled</name>
    <value>true</value>
    <description>enable or disable the hive client authorization</description>
    </property>

    I kept all the other Hive security configs with their default settings.

    I'm running in pseudo-distributed mode on a single node.  HDFS, the Hive
    metastore and the Hive CLI are all running as the same user (the HDFS
    superuser).  Here are the sequence of steps that are causing me issues.
    Without authorization everything works perfectly (creating, loading, selecting).
    I've also tried creating and loading the table without authorization, granting
    the select privilege at various levels (global, table, database), turning on
    auth and performing the select, resulting in the same exception.

    Any help with this would be greatly appreciated!

    Thanks,
    Alex

    --

    [hduser@aholmes-desktop ~]$ hive
    Hive history file=/tmp/hduser/hive_job_log_hduser_201108162158_1976573160.txt
    hive> set hive.security.authorization.enabled=false;
    hive> grant all to user hduser;
    OK
    Time taken: 0.233 seconds
    hive> set hive.security.authorization.enabled=true;
    hive> CREATE TABLE pokes3 (foo INT, bar STRING);
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
    at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:433)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
    Caused by: org.apache.thrift.TApplicationException: get_privilege_set
    failed: unknown result
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
    ... 14 more
  • Yongqiang he at Aug 26, 2011 at 12:22 am
    what is your unix name on that machine? can u do a whoami?
    On Thu, Aug 25, 2011 at 5:15 PM, Alex Holmes wrote:
    Here's the hive-site.xml file (I use the same file for both the client
    and remote metastore).  We're using mysql as the metastore DB.


    <?xml version="1.0"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    <configuration>
    <property>
    <name>hive.security.authorization.enabled</name>
    <value>true</value>
    </property>
    <property>
    <name>hive.metastore.local</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.metastore.uris</name>
    <value>thrift://localhost:9083</value>
    </property>
    <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://localhost/hive?createDatabaseIfNotExist=true</value>
    </property>
    <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    </property>
    <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>hive</value>
    </property>
    <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>secret</value>
    </property>
    </configuration>


    On Wed, Aug 24, 2011 at 6:06 PM, yongqiang he wrote:
    this is what i have tried with a remote metastore:
    set hive.security.authorization.enabled=false; hive>

    drop table src2;
    OK
    Time taken: 1.002 seconds
    hive> create table src2 (key int, value string);
    OK
    Time taken: 0.03 seconds
    hive>

    set hive.security.authorization.enabled=true;
    hive> grant select on table src2 to user heyongqiang;
    OK
    Time taken: 0.113 seconds
    hive> select * from src2;
    OK
    Time taken: 0.188 seconds
    hive> show grant user heyongqiang on table src2;
    OK

    database        default
    table   src2
    principalName   heyongqiang
    principalType   USER
    privilege       Select
    grantTime       Wed Aug 24 15:03:51 PDT 2011
    grantor heyongqiang

    can u do a show grant?

    (But with remote metastore, i think hive should not return empty list
    instead of null for list_privileges etc.)


    On Wed, Aug 24, 2011 at 2:34 PM, Alex Holmes wrote:
    Authorization works for me with the local metastore.  The remote
    metastore works with authorization turned off, but as soon as I turn
    it on and issue any commands I get these exceptions on the hive
    client.

    Could you also try the remote metastore please?  I'm pretty sure that
    authorization does not work with it at all.

    Thanks,
    Alex
    On Wed, Aug 24, 2011 at 5:20 PM, yongqiang he wrote:
    I am using local metastore,  and can not reproduce the problem.

    what message did you get when running local metastore?
    On Wed, Aug 24, 2011 at 1:58 PM, Alex Holmes wrote:
    Thanks for opening a ticket.

    Table-level grants aren't working for me either (HIVE-2405 suggests
    that the bug is only related to global grants).

    hive> set hive.security.authorization.enabled=false;
    hive> CREATE TABLE pokes (foo INT, bar STRING);
    OK
    Time taken: 1.245 seconds
    hive> LOAD DATA LOCAL INPATH 'hive1.in' OVERWRITE INTO TABLE pokes;
    FAILED: Error in semantic analysis: Line 1:23 Invalid path 'hive1.in':
    No files matching path file:/app/hadoop/hive-0.7.1/conf/hive1.in
    hive> LOAD DATA LOCAL INPATH '/app/hadoop/hive1.in' OVERWRITE INTO TABLE pokes;
    Copying data from file:/app/hadoop/hive1.in
    Copying file: file:/app/hadoop/hive1.in
    Loading data to table default.pokes
    Moved to trash: hdfs://localhost:54310/user/hive/warehouse/pokes
    OK
    Time taken: 0.33 seconds
    hive> select * from pokes;
    OK
    1       a
    2       b
    3       c
    Time taken: 0.095 seconds
    hive> grant select on table pokes to user hduser;
    OK
    Time taken: 0.251 seconds
    hive> set hive.security.authorization.enabled=true;
    hive> select * from pokes;
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    ...

    mysql> select * from TBL_PRIVS;
    +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
    TBL_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE |
    PRINCIPAL_NAME | PRINCIPAL_TYPE | TBL_PRIV | TBL_ID |
    +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
    1 |  1314219701 |            0 | hduser  | USER         |
    hduser         | USER           | Select   |      1 |
    +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+

    Also, I noticed in HIVE-2405 that you get a meaningful error message:

    Authorization failed:No privilege 'Create' found for outputs {
    database:default}. Use show grant to get more details.

    Whereas I just get an exception (as you can see above).  Were you also
    running with the remote metastore?  I get these meaningful messages
    with the local metastore (and authorization on), but with the remote
    metastore with authorization turned on, I always get exceptions.

    Many thanks,
    Alex
    On Wed, Aug 24, 2011 at 3:38 PM, yongqiang he wrote:
    This is a bug. Will open a jira to fix this. and will backport it to 0.7.1.
    https://issues.apache.org/jira/browse/HIVE-2405

    thanks for reporting this one!
    On Wed, Aug 24, 2011 at 6:25 AM, Alex Holmes wrote:
    I created the mysql database (with the simple create database command)
    and the remote metastore seemed to creat the mysql tables.  Here's
    some grant information and what I see in the database:

    [hduser@aholmes-desktop conf]$ hive
    hive> grant all to user hduser;
    OK
    Time taken: 0.334 seconds
    hive> show grant user hduser;
    OK

    principalName   hduser
    principalType   USER
    privilege       All
    grantTime       1314191500
    grantor hduser
    Time taken: 0.046 seconds
    hive> CREATE TABLE pokes (foo INT, bar STRING);
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
    ...

    mysql> use hive;
    Database changed
    mysql> select * from GLOBAL_PRIVS;
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    USER_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE
    PRINCIPAL_NAME | PRINCIPAL_TYPE | USER_PRIV |
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    1 |  1314191500 |            0 | hduser  | USER
    hduser         | USER           | All       |
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    1 row in set (0.00 sec)


    Thanks for your help,
    Alex
    On Tue, Aug 23, 2011 at 1:27 PM, yongqiang he wrote:
    Have you created the metastore mysql tables for authorization? Can u
    do a show grant?

    thanks
    yongqiang
    On Tue, Aug 16, 2011 at 2:55 PM, Alex Holmes wrote:
    Hi all,

    I've been struggling with getting Hive authorization to work for a few
    hours, and I really hope someone can help me.  I installed Hive 0.7.1
    on top of Hadoop 0.20.203.  I'm using mysql for the metastore, and
    configured Hive to enable authorization:

    <property>
    <name>hive.security.authorization.enabled</name>
    <value>true</value>
    <description>enable or disable the hive client authorization</description>
    </property>

    I kept all the other Hive security configs with their default settings.

    I'm running in pseudo-distributed mode on a single node.  HDFS, the Hive
    metastore and the Hive CLI are all running as the same user (the HDFS
    superuser).  Here are the sequence of steps that are causing me issues.
    Without authorization everything works perfectly (creating, loading, selecting).
    I've also tried creating and loading the table without authorization, granting
    the select privilege at various levels (global, table, database), turning on
    auth and performing the select, resulting in the same exception.

    Any help with this would be greatly appreciated!

    Thanks,
    Alex

    --

    [hduser@aholmes-desktop ~]$ hive
    Hive history file=/tmp/hduser/hive_job_log_hduser_201108162158_1976573160.txt
    hive> set hive.security.authorization.enabled=false;
    hive> grant all to user hduser;
    OK
    Time taken: 0.233 seconds
    hive> set hive.security.authorization.enabled=true;
    hive> CREATE TABLE pokes3 (foo INT, bar STRING);
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
    at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:433)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
    Caused by: org.apache.thrift.TApplicationException: get_privilege_set
    failed: unknown result
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
    ... 14 more
  • Alex Holmes at Aug 26, 2011 at 12:04 pm
    HDFS, the Hive metastore and the hive client are all running as "hduser".
    On Thu, Aug 25, 2011 at 8:22 PM, yongqiang he wrote:
    what is your unix name on that machine? can u do a whoami?
    On Thu, Aug 25, 2011 at 5:15 PM, Alex Holmes wrote:
    Here's the hive-site.xml file (I use the same file for both the client
    and remote metastore).  We're using mysql as the metastore DB.


    <?xml version="1.0"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    <configuration>
    <property>
    <name>hive.security.authorization.enabled</name>
    <value>true</value>
    </property>
    <property>
    <name>hive.metastore.local</name>
    <value>false</value>
    </property>
    <property>
    <name>hive.metastore.uris</name>
    <value>thrift://localhost:9083</value>
    </property>
    <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://localhost/hive?createDatabaseIfNotExist=true</value>
    </property>
    <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    </property>
    <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>hive</value>
    </property>
    <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>secret</value>
    </property>
    </configuration>


    On Wed, Aug 24, 2011 at 6:06 PM, yongqiang he wrote:
    this is what i have tried with a remote metastore:
    set hive.security.authorization.enabled=false; hive>

    drop table src2;
    OK
    Time taken: 1.002 seconds
    hive> create table src2 (key int, value string);
    OK
    Time taken: 0.03 seconds
    hive>

    set hive.security.authorization.enabled=true;
    hive> grant select on table src2 to user heyongqiang;
    OK
    Time taken: 0.113 seconds
    hive> select * from src2;
    OK
    Time taken: 0.188 seconds
    hive> show grant user heyongqiang on table src2;
    OK

    database        default
    table   src2
    principalName   heyongqiang
    principalType   USER
    privilege       Select
    grantTime       Wed Aug 24 15:03:51 PDT 2011
    grantor heyongqiang

    can u do a show grant?

    (But with remote metastore, i think hive should not return empty list
    instead of null for list_privileges etc.)


    On Wed, Aug 24, 2011 at 2:34 PM, Alex Holmes wrote:
    Authorization works for me with the local metastore.  The remote
    metastore works with authorization turned off, but as soon as I turn
    it on and issue any commands I get these exceptions on the hive
    client.

    Could you also try the remote metastore please?  I'm pretty sure that
    authorization does not work with it at all.

    Thanks,
    Alex
    On Wed, Aug 24, 2011 at 5:20 PM, yongqiang he wrote:
    I am using local metastore,  and can not reproduce the problem.

    what message did you get when running local metastore?
    On Wed, Aug 24, 2011 at 1:58 PM, Alex Holmes wrote:
    Thanks for opening a ticket.

    Table-level grants aren't working for me either (HIVE-2405 suggests
    that the bug is only related to global grants).

    hive> set hive.security.authorization.enabled=false;
    hive> CREATE TABLE pokes (foo INT, bar STRING);
    OK
    Time taken: 1.245 seconds
    hive> LOAD DATA LOCAL INPATH 'hive1.in' OVERWRITE INTO TABLE pokes;
    FAILED: Error in semantic analysis: Line 1:23 Invalid path 'hive1.in':
    No files matching path file:/app/hadoop/hive-0.7.1/conf/hive1.in
    hive> LOAD DATA LOCAL INPATH '/app/hadoop/hive1.in' OVERWRITE INTO TABLE pokes;
    Copying data from file:/app/hadoop/hive1.in
    Copying file: file:/app/hadoop/hive1.in
    Loading data to table default.pokes
    Moved to trash: hdfs://localhost:54310/user/hive/warehouse/pokes
    OK
    Time taken: 0.33 seconds
    hive> select * from pokes;
    OK
    1       a
    2       b
    3       c
    Time taken: 0.095 seconds
    hive> grant select on table pokes to user hduser;
    OK
    Time taken: 0.251 seconds
    hive> set hive.security.authorization.enabled=true;
    hive> select * from pokes;
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    ...

    mysql> select * from TBL_PRIVS;
    +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
    TBL_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE |
    PRINCIPAL_NAME | PRINCIPAL_TYPE | TBL_PRIV | TBL_ID |
    +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+
    1 |  1314219701 |            0 | hduser  | USER         |
    hduser         | USER           | Select   |      1 |
    +--------------+-------------+--------------+---------+--------------+----------------+----------------+----------+--------+

    Also, I noticed in HIVE-2405 that you get a meaningful error message:

    Authorization failed:No privilege 'Create' found for outputs {
    database:default}. Use show grant to get more details.

    Whereas I just get an exception (as you can see above).  Were you also
    running with the remote metastore?  I get these meaningful messages
    with the local metastore (and authorization on), but with the remote
    metastore with authorization turned on, I always get exceptions.

    Many thanks,
    Alex
    On Wed, Aug 24, 2011 at 3:38 PM, yongqiang he wrote:
    This is a bug. Will open a jira to fix this. and will backport it to 0.7.1.
    https://issues.apache.org/jira/browse/HIVE-2405

    thanks for reporting this one!
    On Wed, Aug 24, 2011 at 6:25 AM, Alex Holmes wrote:
    I created the mysql database (with the simple create database command)
    and the remote metastore seemed to creat the mysql tables.  Here's
    some grant information and what I see in the database:

    [hduser@aholmes-desktop conf]$ hive
    hive> grant all to user hduser;
    OK
    Time taken: 0.334 seconds
    hive> show grant user hduser;
    OK

    principalName   hduser
    principalType   USER
    privilege       All
    grantTime       1314191500
    grantor hduser
    Time taken: 0.046 seconds
    hive> CREATE TABLE pokes (foo INT, bar STRING);
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
    ...

    mysql> use hive;
    Database changed
    mysql> select * from GLOBAL_PRIVS;
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    USER_GRANT_ID | CREATE_TIME | GRANT_OPTION | GRANTOR | GRANTOR_TYPE
    PRINCIPAL_NAME | PRINCIPAL_TYPE | USER_PRIV |
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    1 |  1314191500 |            0 | hduser  | USER
    hduser         | USER           | All       |
    +---------------+-------------+--------------+---------+--------------+----------------+----------------+-----------+
    1 row in set (0.00 sec)


    Thanks for your help,
    Alex
    On Tue, Aug 23, 2011 at 1:27 PM, yongqiang he wrote:
    Have you created the metastore mysql tables for authorization? Can u
    do a show grant?

    thanks
    yongqiang
    On Tue, Aug 16, 2011 at 2:55 PM, Alex Holmes wrote:
    Hi all,

    I've been struggling with getting Hive authorization to work for a few
    hours, and I really hope someone can help me.  I installed Hive 0.7.1
    on top of Hadoop 0.20.203.  I'm using mysql for the metastore, and
    configured Hive to enable authorization:

    <property>
    <name>hive.security.authorization.enabled</name>
    <value>true</value>
    <description>enable or disable the hive client authorization</description>
    </property>

    I kept all the other Hive security configs with their default settings.

    I'm running in pseudo-distributed mode on a single node.  HDFS, the Hive
    metastore and the Hive CLI are all running as the same user (the HDFS
    superuser).  Here are the sequence of steps that are causing me issues.
    Without authorization everything works perfectly (creating, loading, selecting).
    I've also tried creating and loading the table without authorization, granting
    the select privilege at various levels (global, table, database), turning on
    auth and performing the select, resulting in the same exception.

    Any help with this would be greatly appreciated!

    Thanks,
    Alex

    --

    [hduser@aholmes-desktop ~]$ hive
    Hive history file=/tmp/hduser/hive_job_log_hduser_201108162158_1976573160.txt
    hive> set hive.security.authorization.enabled=false;
    hive> grant all to user hduser;
    OK
    Time taken: 0.233 seconds
    hive> set hive.security.authorization.enabled=true;
    hive> CREATE TABLE pokes3 (foo INT, bar STRING);
    FAILED: Hive Internal Error:
    org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.thrift.TApplicationException:
    get_privilege_set failed: unknown result)
    org.apache.hadoop.hive.ql.metadata.HiveException:
    org.apache.thrift.TApplicationException: get_privilege_set failed:
    unknown result
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1617)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserPriv(DefaultHiveAuthorizationProvider.java:201)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorizeUserAndDBPriv(DefaultHiveAuthorizationProvider.java:226)
    at org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider.authorize(DefaultHiveAuthorizationProvider.java:89)
    at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:433)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:393)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
    Caused by: org.apache.thrift.TApplicationException: get_privilege_set
    failed: unknown result
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_privilege_set(ThriftHiveMetastore.java:2414)
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_privilege_set(ThriftHiveMetastore.java:2379)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.get_privilege_set(HiveMetaStoreClient.java:1042)
    at org.apache.hadoop.hive.ql.metadata.Hive.get_privilege_set(Hive.java:1615)
    ... 14 more

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshive, hadoop
postedAug 16, '11 at 9:55p
activeAug 26, '11 at 12:04p
posts12
users2
websitehive.apache.org

2 users in discussion

Alex Holmes: 7 posts Yongqiang he: 5 posts

People

Translate

site design / logo © 2022 Grokbase