Grokbase Groups Pig user August 2009
FAQ
Which version of hadoop support hadoop globbing? or Do i have to apply patch
for it? and Ya will it be compatible with Pig 0.3.0? has anyone tested it?

Pankil

On Wed, Aug 26, 2009 at 3:08 PM, Mridul Muralidharan
wrote:
Hi Pankil,

As thejas pointed out in the other thread, you can use globbing that
hadoop supports :
http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/fs/FileSy
stem.html#globStatus(org.apache.hadoop.fs.Path)


Regards,
Mridul


Pankil Doshi wrote:
Hello Everyone,

I am trying to write Pig scripts for my project. Problem I ma facing is I
want to load different files to same variable .Can it be possible to do
without modifying the Loader. I read about Hadoop globbing . Does anyone
have solution to these.

I know I can load all files of a given directory to single variable.
But is it possible to load specific files from that directory? Or specific
files from different directories to same load variable?

I also know about UNION strategy but that increase one map-reduce job and
I
want to avoid that.

Any kind of suggestions are welcomed.

Pankil

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 4 of 7 | next ›
Discussion Overview
groupuser @
categoriespig, hadoop
postedAug 26, '09 at 5:22p
activeSep 3, '09 at 2:03p
posts7
users5
websitepig.apache.org

People

Translate

site design / logo © 2021 Grokbase