|| at Aug 19, 2010 at 8:50 am
I am afraid you should write your own LoadFunc to interpret the text.
From Pig 0.7, the local mode use the hadoop's standalone local mode,
so it will won't store all the data in memory, the data will been read
in stream mode, but this mode need more memory because each task is
executed in another jvm.
On Thu, Aug 19, 2010 at 12:48 AM, Defenestrator
What loader should I use on csv files with quoted strings that contain
embedded commas? (i.e. Embedded commas should not be a separator.)
And when LOADing large files in local mode, does Pig just store it all
in memory? Or does it have memory management ala buffer managers in