Grokbase Groups Hive user August 2010
FAQ
Hello,

When the order of partitioning columns is different in create table and insert, I am not able to query for any data. However if the order is the same its possible to see data.

Should partitioning order be maintained through all inserts? As you see below, kv2.txt is still on HDFS and 2 different partition orders are created. Running off hive trunk.

hive> CREATE TABLE invites (foo INT, bar STRING) PARTITIONED BY (ds STRING, country STRING);
hive> LOAD DATA LOCAL INPATH '/tmp/kv2.txt' OVERWRITE INTO TABLE invites PARTITION (country='india', ds='2008-08-15');
hive> select * from invites;
hive>

[thiruvel@hive]$ hadoop fs -lsr /user/hive/warehouse/invites
drwxr-xr-x - thiruvel supergroup 0 2010-08-06 12:52 /user/hive/warehouse/invites/country=india
drwxr-xr-x - thiruvel supergroup 0 2010-08-06 12:52 /user/hive/warehouse/invites/country=india/ds=2008-08-15
-rw-r--r-- 1 thiruvel supergroup 5791 2010-08-06 12:52 /user/hive/warehouse/invites/country=india/ds=2008-08-15/kv2.txt
drwxr-xr-x - thiruvel supergroup 0 2010-08-06 12:52 /user/hive/warehouse/invites/ds=2008-08-15
drwxr-xr-x - thiruvel supergroup 0 2010-08-06 12:52 /user/hive/warehouse/invites/ds=2008-08-15/country=india
[thiruvel@ hive]$

Thanks,
Thiruvel

Search Discussions

  • Namit Jain at Aug 6, 2010 at 3:25 pm
    The order should be the same.

    Can you file a jira for this issue ? We should throw an error.


    Thanks,
    -namit

    ________________________________________
    From: Thiruvel Thirumoolan [thiruvel@yahoo-inc.com]
    Sent: Friday, August 06, 2010 12:29 AM
    To: hive-user@hadoop.apache.org
    Subject: Shud partition order be same in create and insert?

    Hello,

    When the order of partitioning columns is different in create table and insert, I am not able to query for any data. However if the order is the same its possible to see data.

    Should partitioning order be maintained through all inserts? As you see below, kv2.txt is still on HDFS and 2 different partition orders are created. Running off hive trunk.

    hive> CREATE TABLE invites (foo INT, bar STRING) PARTITIONED BY (ds STRING, country STRING);
    hive> LOAD DATA LOCAL INPATH '/tmp/kv2.txt' OVERWRITE INTO TABLE invites PARTITION (country='india', ds='2008-08-15');
    hive> select * from invites;
    hive>

    [thiruvel@hive]$ hadoop fs -lsr /user/hive/warehouse/invites
    drwxr-xr-x - thiruvel supergroup 0 2010-08-06 12:52 /user/hive/warehouse/invites/country=india
    drwxr-xr-x - thiruvel supergroup 0 2010-08-06 12:52 /user/hive/warehouse/invites/country=india/ds=2008-08-15
    -rw-r--r-- 1 thiruvel supergroup 5791 2010-08-06 12:52 /user/hive/warehouse/invites/country=india/ds=2008-08-15/kv2.txt
    drwxr-xr-x - thiruvel supergroup 0 2010-08-06 12:52 /user/hive/warehouse/invites/ds=2008-08-15
    drwxr-xr-x - thiruvel supergroup 0 2010-08-06 12:52 /user/hive/warehouse/invites/ds=2008-08-15/country=india
    [thiruvel@ hive]$

    Thanks,
    Thiruvel

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshive, hadoop
postedAug 6, '10 at 7:30a
activeAug 6, '10 at 3:25p
posts2
users2
websitehive.apache.org

People

Translate

site design / logo © 2021 Grokbase