Grokbase Groups Pig dev January 2009
FAQ
[ https://issues.apache.org/jira/browse/PIG-252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Daniel Dai updated PIG-252:
---------------------------

Attachment: (was: localglobbing.patch)
Allow multiple paths in the load statement
------------------------------------------

Key: PIG-252
URL: https://issues.apache.org/jira/browse/PIG-252
Project: Pig
Issue Type: Improvement
Reporter: Olga Natkovich

From Tom White:
I;m having a problem loading data from multiple paths in Pig. What I'm trying to do is to load data from a range of dates, so I would like to specify an input of two globbed paths:
x = LOAD '2008/05/{26,27,28,29,30,31},2008/06/{1,2}'
Pig doesn't seem to like this though as it's trying to interpret it as a single path. The best I can do it to use UNION:
x1 = LOAD '2008/05/{26,27,28,29,30,31}'
x2 = LOAD '2008/06/{1,2}'
x = UNION x1, x2
The downside to this is that I want to parameterize my paths, and having separate script for each number of paths in the input is cumbersome.
Is there a better way of doing this? Are there any plans to support multiple paths, and/or PathFilters?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupdev @
categoriespig, hadoop
postedJan 26, '09 at 5:55a
activeJan 26, '09 at 5:55a
posts1
users1
websitepig.apache.org

1 user in discussion

Daniel Dai (JIRA): 1 post

People

Translate

site design / logo © 2021 Grokbase