FAQ
I do have that.

However I am not writing directly to the table partition. Instead, I first write
my data in a tmp directory (eventually moved to the hdfs table partition) and
then publish that partition using alter table statement in metastore.

Something like this:

-- create table x ... stored as SeqFile
-- insert overwrite directory 'd' select * from table y
-- distcp 'd' x/dateint=.../hour=...
-- alter table x add partition ....

In the second step above I need to produce SeqFile.


Thanks for prompt reply.
Gaurav Jain


----- Original Message ----
From: Yang <teddyyyy123@gmail.com>
To: jainy_gaurav@yahoo.com
Sent: Wed, October 6, 2010 1:28:42 PM
Subject: Re: How to output SeqFile

Gaurav:

not sure if I understand your question correctly....
when you create the output table, that has an option to set the
output table SerDe

Regards
Yang
On Wed, Oct 6, 2010 at 1:18 PM, gaurav jain wrote:




How can I produce a sequence file from query

insert overwrite directory ....


I have set:

SET io.seqfile.compression.type=BLOCK;
SET hive.exec.compress.output=true;
set mapred.output.compression.type=BLOCK;
set mapred.output.compression.codec=org.apache.hadoop.io.compress.GzipCodec;



It seems to produce Text .gz format files.



Regards,
Gaurav Jain


Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 4 of 7 | next ›
Discussion Overview
groupuser @
categorieshive, hadoop
postedOct 6, '10 at 3:44a
activeOct 6, '10 at 8:52p
posts7
users3
websitehive.apache.org

People

Translate

site design / logo © 2022 Grokbase