Hi all,
We have a facility in hadoop where we can specify multiple input paths.
Does this exist in Pig? Essentially, Is it possible to specify multiple
paths in load command? For example, I have n number of input paths which
I need to load for processing. The only possibility that I can see right
now is to use n variables using n load commands and do an union at the
end.
For ex:
Raw1 = LOAD '$inputPath1/*' using PigStorage('\t');
Raw2 = LOAD '$inputPath2/*' using PigStorage('\t');
.
.
.
.
Rawn = LOAD '$inputPathn/*' using PigStorage('\t');
Raw = UNION Raw1,Raw2,....RawN
Can anyone kindly let me know if there is a simpler way of doing it in
single LOAD line or something like that?
Thanks
Pallavi