Grokbase Groups Pig user July 2009
PIG-252 ( address this issue.

Instead of using union, you can try this:

Raw = LOAD '$inputPathprefix{1,2,3,4}/*' using PigStorage('\t');

----- Original Message -----
From: "Palleti, Pallavi" <>
To: <>
Sent: Wednesday, July 08, 2009 8:34 PM
Subject: Specifying multiple input paths in LOAD command

Hi all,

We have a facility in hadoop where we can specify multiple input paths.
Does this exist in Pig? Essentially, Is it possible to specify multiple
paths in load command? For example, I have n number of input paths which
I need to load for processing. The only possibility that I can see right
now is to use n variables using n load commands and do an union at the

For ex:

Raw1 = LOAD '$inputPath1/*' using PigStorage('\t');

Raw2 = LOAD '$inputPath2/*' using PigStorage('\t');





Rawn = LOAD '$inputPathn/*' using PigStorage('\t');

Raw = UNION Raw1,Raw2,....RawN

Can anyone kindly let me know if there is a simpler way of doing it in
single LOAD line or something like that?



Search Discussions

Discussion Posts


Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 4 of 4 | next ›
Discussion Overview
groupuser @
categoriespig, hadoop
postedJul 9, '09 at 3:34a
activeJul 9, '09 at 2:48p



site design / logo © 2021 Grokbase