Grokbase Groups Hive user May 2010
FAQ
I read the hive wiki, it says:

The output of each of the select statements is written to the chosen table
(or partition).

Currently the OVERWRITE keyword is mandatory and implies that the contents
of the chosen table or partition are replaced with the output of
corresponding select statement



can hive currently support appending data to the table?

Search Discussions

  • Ning Zhang at May 7, 2010 at 2:53 pm
    Hive does not support appending to exist partition ot table yet. For non bucketed table a workaround is to insert the new data into a temp table and move the files to the destination by hand.

    Of course you are welcome to add this to the project.
    Thanks,
    Ning
    ------
    Sent from my blackberry

    ________________________________
    From: luocanrao <luocan19826164@sohu.com>
    To: hive-user@hadoop.apache.org <hive-user@hadoop.apache.org>
    Sent: Fri May 07 07:22:38 2010
    Subject: can hive currently support appending data to the table?

    I read the hive wiki, it says:
    The output of each of the select statements is written to the chosen table (or partition).
    Currently the OVERWRITE keyword is mandatory and implies that the contents of the chosen table or partition are replaced with the output of corresponding select statement

    can hive currently support appending data to the table?
  • Zheng Shao at May 7, 2010 at 6:29 pm
    We can also add more partitions to a table (or a sub partition to a
    partition) to simulate appending to table/partition.

    There is no real efficiency loss except that we will create a new
    directory/file per append.

    Zheng
    On Fri, May 7, 2010 at 7:51 AM, Ning Zhang wrote:
    Hive does not support appending to exist partition ot table yet. For non
    bucketed table a workaround is to insert the new data into a temp table and
    move the files to the destination by hand.

    Of course you are welcome to add this to the project.
    Thanks,
    Ning
    ------
    Sent from my blackberry

    ________________________________
    From: luocanrao <luocan19826164@sohu.com>
    To: hive-user@hadoop.apache.org <hive-user@hadoop.apache.org>
    Sent: Fri May 07 07:22:38 2010
    Subject: can hive currently support appending data to the table?

    I read the hive wiki, it says:

    The output of each of the select statements is written to the chosen table
    (or partition).

    Currently the OVERWRITE keyword is mandatory and implies that the contents
    of the chosen table or partition are replaced with the output of
    corresponding select statement



    can hive currently support appending data to the table?


    --
    Yours,
    Zheng
    http://www.linkedin.com/in/zshao
  • Luocan19826164 at May 8, 2010 at 1:55 am
    Thanks for your reply, can you give me some example about adding a sub partition to a Partition?
    How can I achieve that creating a new directory/file per append
    <?xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" /><o:p>&nbsp;</o:p>
    -----邮件原件-----
    发件人: Zheng Shao
    发送时间: 2010年5月8日 2:29
    收件人: hive-user@hadoop.apache.org
    主题: Re: can hive currently support appending data to the table?
    <o:p>&nbsp;</o:p>
    We can also add more partitions to a table (or a sub partition to a
    partition) to simulate appending to table/partition.
    <o:p>&nbsp;</o:p>
    There is no real efficiency loss except that we will create a new directory/file per append.
    <o:p>&nbsp;</o:p>
    Zheng
    <o:p>&nbsp;</o:p>
    On Fri, May 7, 2010 at 7:51 AM, Ning Zhang &lt;nzhang@facebook.com&gt; wrote:
    &gt; Hive does not support appending to exist partition ot table yet. For
    &gt; non bucketed table a workaround is to insert the new data into a temp
    &gt; table and move the files to the destination by hand.
    &gt;<o:p>&nbsp;</o:p>
    &gt; Of course you are welcome to add this to the project.
    &gt; Thanks,
    &gt; Ning
    &gt; ------
    &gt; Sent from my blackberry
    &gt;<o:p>&nbsp;</o:p>
    &gt; ________________________________
    &gt; From: luocanrao &lt;luocan19826164@sohu.com&gt;
    &gt; To: hive-user@hadoop.apache.org &lt;hive-user@hadoop.apache.org&gt;
    &gt; Sent: Fri May 07 07:22:38 2010
    &gt; Subject: can hive currently support appending data to the table?
    &gt;<o:p>&nbsp;</o:p>
    &gt; I read the hive wiki, it says:
    &gt;<o:p>&nbsp;</o:p>
    &gt; The output of each of the select statements is written to the chosen
    &gt; table (or partition).
    &gt;<o:p>&nbsp;</o:p>
    &gt; Currently the OVERWRITE keyword is mandatory and implies that the
    &gt; contents of the chosen table or partition are replaced with the output
    &gt; of corresponding select statement
    &gt;<o:p>&nbsp;</o:p>
    &gt;<o:p>&nbsp;</o:p>
    &gt;<o:p>&nbsp;</o:p>
    &gt; can hive currently support appending data to the table?
    <o:p>&nbsp;</o:p>
    <o:p>&nbsp;</o:p>
    <o:p>&nbsp;</o:p>
  • Arvind Prabhakar at May 17, 2010 at 5:19 pm
    Please refer to the Hive Language Manual:
    http://wiki.apache.org/hadoop/Hive/LanguageManual/DML#Inserting_data_into_Hive_Tables_from_queries

    Loading data into a sub-partition of a table is the same as loading data
    into regular table with the partition predicate specified. For example:

    LOAD DATA LOCAL INPATH '/path/to/data/file' OVERWRITE INTO TABLE table_name
    partition *( part_column = part_column_value)*

    More information on manipulating partitions as a whole is available here:
    http://wiki.apache.org/hadoop/Hive/LanguageManual/DDL#Add_Partitions

    Hope this helps.
    -Arvind
    On Fri, May 7, 2010 at 6:54 PM, wrote:

    Thanks for your reply, can you give me some example about adding a sub
    partition to a Partition?

    How can I achieve that creating a new directory/file per append



    -----邮件原件-----

    发件人: Zheng Shao

    发送时间: 2010年5月8日 2:29

    收件人: hive-user@hadoop.apache.org

    主题: Re: can hive currently support appending data to the table?



    We can also add more partitions to a table (or a sub partition to a

    partition) to simulate appending to table/partition.



    There is no real efficiency loss except that we will create a new
    directory/file per append.



    Zheng


    On Fri, May 7, 2010 at 7:51 AM, Ning Zhang wrote:

    Hive does not support appending to exist partition ot table yet. For
    non bucketed table a workaround is to insert the new data into a temp
    table and move the files to the destination by hand.

    Of course you are welcome to add this to the project.
    Thanks,
    Ning
    ------
    Sent from my blackberry

    ________________________________
    From: luocanrao <luocan19826164@sohu.com>
    To: hive-user@hadoop.apache.org <hive-user@hadoop.apache.org>
    Sent: Fri May 07 07:22:38 2010
    Subject: can hive currently support appending data to the table?

    I read the hive wiki, it says:

    The output of each of the select statements is written to the chosen
    table (or partition).

    Currently the OVERWRITE keyword is mandatory and implies that the
    contents of the chosen table or partition are replaced with the output
    of corresponding select statement



    can hive currently support appending data to the table?






    --

    Yours,

    Zheng

    http://www.linkedin.com/in/zshao


    ------------------------------

    你的1G网络U盘真好用! <http://goto.mail.sohu.com/goto.php?code=udisk_zhujiao>
    查薪酬:对比同行工资!<http://sohu.ad-plus.cn/event.ng/Type=click&FlightID=201004&TargetID=sohu&Values=df789d86,92d3d91d,277177cc,c2935d8d&AdID=54157>

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshive, hadoop
postedMay 7, '10 at 2:23p
activeMay 17, '10 at 5:19p
posts5
users4
websitehive.apache.org

People

Translate

site design / logo © 2022 Grokbase