FAQ
Hi,

when user droped the table in hive, it was still showing on hdfs where
table was removed from hive.. is there any Bug or any kind of
permission issue.

Permissions were set like this...

[root@edge001 ~]# hadoop fs -ls /user/hive
Found 1 items
drwxrwxrwx - hive supergroup 0 2012-12-03 13:37 /user/hive/warehouse

Thanks
Chittu

Search Discussions

  • Mark Grover at Dec 3, 2012 at 11:54 pm
    Chittu,
    What was the statement you used to create the table?

    Deleting an external tables only drops the metadata but not the HDFS data
    backing the table.

    Deleting a managed (a.k.a non-external) table drops both the metadata and
    HDFS data.

    Mark
    On Mon, Dec 3, 2012 at 3:52 PM, wrote:

    Hi,

    when user droped the table in hive, it was still showing on hdfs where
    table was removed from hive.. is there any Bug or any kind of
    permission issue.

    Permissions were set like this...

    [root@edge001 ~]# hadoop fs -ls /user/hive
    Found 1 items
    drwxrwxrwx - hive supergroup 0 2012-12-03 13:37 /user/hive/warehouse

    Thanks
    Chittu
  • Mark Grover at Dec 4, 2012 at 12:43 am
    That should have removed your HDFS directory as well. Can you check Hive
    logs and see if there you see any permission related errors there?

    What version of Hive are you using? Alternatively, you can specify what
    version of CDH you are using. And, you used CM to install your cluster?

    Well, hadoop fs -rm -r <path> would delete data in HDFS but like I said,
    Hive is supposed to do that for native tables.

    Mark
    On Mon, Dec 3, 2012 at 4:13 PM, wrote:

    We are using the following statements for creating/dropping tables.
    CREATE TABLE sample;
    drop table sample;

    Please let me know if there is any other way to delete in HDFS level?

    Thanks
    Chittu
    On Mon, Dec 3, 2012 at 3:54 PM, Mark Grover wrote:
    Chittu,
    What was the statement you used to create the table?

    Deleting an external tables only drops the metadata but not the HDFS data
    backing the table.

    Deleting a managed (a.k.a non-external) table drops both the metadata and
    HDFS data.

    Mark

    On Mon, Dec 3, 2012 at 3:52 PM, wrote:

    Hi,

    when user droped the table in hive, it was still showing on hdfs where
    table was removed from hive.. is there any Bug or any kind of
    permission issue.

    Permissions were set like this...

    [root@edge001 ~]# hadoop fs -ls /user/hive
    Found 1 items
    drwxrwxrwx - hive supergroup 0 2012-12-03 13:37 /user/hive/warehouse

    Thanks
    Chittu
  • Chittu546 at Dec 4, 2012 at 9:03 pm
    we are using following Hive..
    Hive Version:
    hive-server.noarch 0.9.0+155-1.cdh4.1.2.p0.21.el6
    cloudera-cdh4

    seeing the following error..

    2012-12-03 11:26:28,428 ERROR hive.log
    (MetaStoreUtils.java:logAndThrowMetaException(781)) - Got exception:
    org.apache.hadoop
    .security.AccessControlException Permission denied: user=st891x,
    access=ALL, inode="/user/hive/warehouse/test":jl3984:hdfs:dr
    wxr-xr-x
    On Mon, Dec 3, 2012 at 4:42 PM, Mark Grover wrote:
    That should have removed your HDFS directory as well. Can you check Hive
    logs and see if there you see any permission related errors there?

    What version of Hive are you using? Alternatively, you can specify what
    version of CDH you are using. And, you used CM to install your cluster?

    Well, hadoop fs -rm -r <path> would delete data in HDFS but like I said,
    Hive is supposed to do that for native tables.

    Mark
    On Mon, Dec 3, 2012 at 4:13 PM, wrote:

    We are using the following statements for creating/dropping tables.
    CREATE TABLE sample;
    drop table sample;

    Please let me know if there is any other way to delete in HDFS level?

    Thanks
    Chittu

    On Mon, Dec 3, 2012 at 3:54 PM, Mark Grover <grover.markgrover@gmail.com>
    wrote:
    Chittu,
    What was the statement you used to create the table?

    Deleting an external tables only drops the metadata but not the HDFS
    data
    backing the table.

    Deleting a managed (a.k.a non-external) table drops both the metadata
    and
    HDFS data.

    Mark

    On Mon, Dec 3, 2012 at 3:52 PM, wrote:

    Hi,

    when user droped the table in hive, it was still showing on hdfs where
    table was removed from hive.. is there any Bug or any kind of
    permission issue.

    Permissions were set like this...

    [root@edge001 ~]# hadoop fs -ls /user/hive
    Found 1 items
    drwxrwxrwx - hive supergroup 0 2012-12-03 13:37 /user/hive/warehouse

    Thanks
    Chittu
  • Mark Grover at Dec 4, 2012 at 9:15 pm
    Chittu,
    I'd recommend that the /user/hive/warehouse directory is owned by the user
    running Hive queries.

    Alternatively, run your Hive queries as the user who owns the
    /user/hive/warehouse directory on HDFS.

    Also, please ensure that you provide the following permissions to your /tmp
    and /user/hive/warehouse directories on HDFS:

    $ $HADOOP_HOME/bin/hadoop fs -mkdir /tmp
    $ $HADOOP_HOME/bin/hadoop fs -mkdir /user/hive/warehouse
    $ $HADOOP_HOME/bin/hadoop fs -chmod g+w /tmp
    $ $HADOOP_HOME/bin/hadoop fs -chmod g+w /user/hive/warehouse

    (reference:
    https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-RunningHive
    )
    On Tue, Dec 4, 2012 at 1:03 PM, wrote:

    we are using following Hive..
    Hive Version:
    hive-server.noarch 0.9.0+155-1.cdh4.1.2.p0.21.el6
    cloudera-cdh4

    seeing the following error..

    2012-12-03 11:26:28,428 ERROR hive.log
    (MetaStoreUtils.java:logAndThrowMetaException(781)) - Got exception:
    org.apache.hadoop
    .security.AccessControlException Permission denied: user=st891x,
    access=ALL, inode="/user/hive/warehouse/test":jl3984:hdfs:dr
    wxr-xr-x
    On Mon, Dec 3, 2012 at 4:42 PM, Mark Grover wrote:
    That should have removed your HDFS directory as well. Can you check Hive
    logs and see if there you see any permission related errors there?

    What version of Hive are you using? Alternatively, you can specify what
    version of CDH you are using. And, you used CM to install your cluster?

    Well, hadoop fs -rm -r <path> would delete data in HDFS but like I said,
    Hive is supposed to do that for native tables.

    Mark
    On Mon, Dec 3, 2012 at 4:13 PM, wrote:

    We are using the following statements for creating/dropping tables.
    CREATE TABLE sample;
    drop table sample;

    Please let me know if there is any other way to delete in HDFS level?

    Thanks
    Chittu

    On Mon, Dec 3, 2012 at 3:54 PM, Mark Grover <
    grover.markgrover@gmail.com>
    wrote:
    Chittu,
    What was the statement you used to create the table?

    Deleting an external tables only drops the metadata but not the HDFS
    data
    backing the table.

    Deleting a managed (a.k.a non-external) table drops both the metadata
    and
    HDFS data.

    Mark

    On Mon, Dec 3, 2012 at 3:52 PM, wrote:

    Hi,

    when user droped the table in hive, it was still showing on hdfs
    where
    table was removed from hive.. is there any Bug or any kind of
    permission issue.

    Permissions were set like this...

    [root@edge001 ~]# hadoop fs -ls /user/hive
    Found 1 items
    drwxrwxrwx - hive supergroup 0 2012-12-03 13:37 /user/hive/warehouse

    Thanks
    Chittu
  • Chittu546 at Dec 4, 2012 at 9:29 pm
    Thank you Mark for your reply.
    here we have multiple users to access hive..so we set permissions like
    this. and I do not understand what does "supergroup" refers here..I
    did not find the group "supergroup" in system level. /tmp was already
    created on part of cdh4 installation.

    [root@edge001 st891x]# hadoop fs -ls /user/hive/
    Found 1 items
    drwxrwxrwx - hive supergroup 0 2012-12-04 13:13 /user/hive/warehouse
    [root@edge003 tmp]# hadoop fs -ls /
    Found 3 items
    drwxr-xr-x - hbase hbase 0 2012-11-13 13:59 /hbase
    drwxrwxrwt - hdfs hdfs 0 2012-11-29 15:30 /tmp
    drwxr-xr-x - hdfs supergroup 0 2012-11-28 12:51 /user
    On Tue, Dec 4, 2012 at 1:14 PM, Mark Grover wrote:
    Chittu,
    I'd recommend that the /user/hive/warehouse directory is owned by the user
    running Hive queries.

    Alternatively, run your Hive queries as the user who owns the
    /user/hive/warehouse directory on HDFS.

    Also, please ensure that you provide the following permissions to your /tmp
    and /user/hive/warehouse directories on HDFS:

    $ $HADOOP_HOME/bin/hadoop fs -mkdir /tmp
    $ $HADOOP_HOME/bin/hadoop fs -mkdir /user/hive/warehouse
    $ $HADOOP_HOME/bin/hadoop fs -chmod g+w /tmp
    $ $HADOOP_HOME/bin/hadoop fs -chmod g+w /user/hive/warehouse

    (reference:
    https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-RunningHive)
    On Tue, Dec 4, 2012 at 1:03 PM, wrote:

    we are using following Hive..
    Hive Version:
    hive-server.noarch 0.9.0+155-1.cdh4.1.2.p0.21.el6
    cloudera-cdh4

    seeing the following error..

    2012-12-03 11:26:28,428 ERROR hive.log
    (MetaStoreUtils.java:logAndThrowMetaException(781)) - Got exception:
    org.apache.hadoop
    .security.AccessControlException Permission denied: user=st891x,
    access=ALL, inode="/user/hive/warehouse/test":jl3984:hdfs:dr
    wxr-xr-x

    On Mon, Dec 3, 2012 at 4:42 PM, Mark Grover <grover.markgrover@gmail.com>
    wrote:
    That should have removed your HDFS directory as well. Can you check Hive
    logs and see if there you see any permission related errors there?

    What version of Hive are you using? Alternatively, you can specify what
    version of CDH you are using. And, you used CM to install your cluster?

    Well, hadoop fs -rm -r <path> would delete data in HDFS but like I said,
    Hive is supposed to do that for native tables.

    Mark
    On Mon, Dec 3, 2012 at 4:13 PM, wrote:

    We are using the following statements for creating/dropping tables.
    CREATE TABLE sample;
    drop table sample;

    Please let me know if there is any other way to delete in HDFS level?

    Thanks
    Chittu

    On Mon, Dec 3, 2012 at 3:54 PM, Mark Grover
    <grover.markgrover@gmail.com>
    wrote:
    Chittu,
    What was the statement you used to create the table?

    Deleting an external tables only drops the metadata but not the HDFS
    data
    backing the table.

    Deleting a managed (a.k.a non-external) table drops both the metadata
    and
    HDFS data.

    Mark

    On Mon, Dec 3, 2012 at 3:52 PM, wrote:

    Hi,

    when user droped the table in hive, it was still showing on hdfs
    where
    table was removed from hive.. is there any Bug or any kind of
    permission issue.

    Permissions were set like this...

    [root@edge001 ~]# hadoop fs -ls /user/hive
    Found 1 items
    drwxrwxrwx - hive supergroup 0 2012-12-03 13:37 /user/hive/warehouse

    Thanks
    Chittu

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedDec 3, '12 at 11:52p
activeDec 4, '12 at 9:29p
posts6
users2
websitecloudera.com
irc#hadoop

2 users in discussion

Chittu546: 3 posts Mark Grover: 3 posts

People

Translate

site design / logo © 2022 Grokbase