Grokbase Groups Pig user August 2009
FAQ

Pankil Doshi wrote:
Which version of hadoop support hadoop globbing? or Do i have to apply patch
for it? and Ya will it be compatible with Pig 0.3.0? has anyone tested it?

Someone from pig team can give details of actual versions.
But I have been using globbing for quite a while now, and I think all
versions of pig which you can get your hands on should be able to
support it !

Regards,
Mridul

PS: iirc there are difference between hadoop globbing and bash globbing,
so you might want to look at the javadoc.
Pankil

On Wed, Aug 26, 2009 at 3:08 PM, Mridul Muralidharan
wrote:
Hi Pankil,

As thejas pointed out in the other thread, you can use globbing that
hadoop supports :
http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/fs/FileSy
stem.html#globStatus(org.apache.hadoop.fs.Path)


Regards,
Mridul


Pankil Doshi wrote:
Hello Everyone,

I am trying to write Pig scripts for my project. Problem I ma facing is I
want to load different files to same variable .Can it be possible to do
without modifying the Loader. I read about Hadoop globbing . Does anyone
have solution to these.

I know I can load all files of a given directory to single variable.
But is it possible to load specific files from that directory? Or specific
files from different directories to same load variable?

I also know about UNION strategy but that increase one map-reduce job and
I
want to avoid that.

Any kind of suggestions are welcomed.

Pankil

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 6 of 7 | next ›
Discussion Overview
groupuser @
categoriespig, hadoop
postedAug 26, '09 at 5:22p
activeSep 3, '09 at 2:03p
posts7
users5
websitepig.apache.org

People

Translate

site design / logo © 2021 Grokbase