If I have multiple files in a directory, how do I load this into Pig? I want to run Pig over an input directory, not an individual file.
%ls Data
myfile1.txt
myfile2.txt
myfile3.txt
myfile4.txt
myfile5.txt
thanks.
Also, if I run the sample Pig Latin commands, I keep getting errors saying "Unable to open iterator"
For example,
A = LOAD 'myfile.txt' USING PigStorage('\t') AS (f1,f2,f3);
dump A
Gives me correct:
<1, 2, 3>
<4, 2, 1>
<8, 3, 4>
<4, 3, 3>
<7, 2, 5>
<8, 4, 3>
but, then when I do the next sample,
Y = FILTER A BY f1 == '8';
dump Y
I get a bunch of parser errors then the Unable to open iterator Y.
This happens for most of the rest of the samples.
What's going on?