Grokbase Groups Pig user August 2011
Pig by default use plain text file as input/output, unless you write a
custom LoadFunc/StoreFunc. There is no specific Pig storage format.
You can copy the file to local using copyToLocal. If you want to
export directly to SQL table, you need to write a StoreFunc. Pig work
on tuple rather than K,V pair, key in MR job is thrown away.

Also CC pig user group.

On Tue, Aug 23, 2011 at 3:04 PM, Keren Ouaknine wrote:

Pig generates data to HDFS and I would like to find a way to convert it to a
general format by either:
1. flatening the data (would copyToLocal work here?!)
2. export the data to SQL tables (or any other non specific Hadoop format)
3. generate K,V pairs of data (since Pig code is converted to a MR job which
take a K,V pair)

Which solution is feasible / prefered? Thanks for your help!

Keren Ouaknine
Cell: +972 54 2565404

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedAug 26, '11 at 9:37p
activeAug 26, '11 at 9:37p

1 user in discussion

Daniel Dai: 1 post



site design / logo © 2022 Grokbase