I was hoping there would be a configuration where I can set the outputformat for
my query.

Gaurav Jain

----- Original Message ----
From: Jacob R Rideout <apache@jacobrideout.net>
To: hive-user@hadoop.apache.org
Sent: Wed, October 6, 2010 1:42:57 PM
Subject: Re: How to output SeqFile
On Wed, Oct 6, 2010 at 2:35 PM, gaurav jain wrote:
I do have that.

However I am not writing directly to the table partition. Instead, I first
my data in a tmp directory (eventually moved to the hdfs table partition) and
then publish that partition using alter table statement in metastore.

Something like this:

-- create table x ... stored as SeqFile
-- insert overwrite directory 'd' select * from table y
-- distcp 'd' x/dateint=.../hour=...
-- alter table x add partition ....

In the second step above I need to produce SeqFile.

Thanks for prompt reply.
Gaurav Jain

----- Original Message ----
From: Yang <teddyyyy123@gmail.com>
To: jainy_gaurav@yahoo.com
Sent: Wed, October 6, 2010 1:28:42 PM
Subject: Re: How to output SeqFile


not sure if I understand your question correctly....
when you create the output table, that has an option to set the
output table SerDe

On Wed, Oct 6, 2010 at 1:18 PM, gaurav jain wrote:

How can I produce a sequence file from query

insert overwrite directory ....

I have set:

SET io.seqfile.compression.type=BLOCK;
SET hive.exec.compress.output=true;
set mapred.output.compression.type=BLOCK;
set mapred.output.compression.codec=org.apache.hadoop.io.compress.GzipCodec;

It seems to produce Text .gz format files.

Gaurav Jain

if you are inserting into the directory rather than the table, hive
won't know to look at the metadata description of the table

you need something like:
insert overwrite table x select * from table y

Search Discussions

Discussion Posts


Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 6 of 7 | next ›
Discussion Overview
groupuser @
categorieshive, hadoop
postedOct 6, '10 at 3:44a
activeOct 6, '10 at 8:52p



site design / logo © 2022 Grokbase