FAQ
Thanks guys!

When I think about it, it may be good enough to do this at the CLI level as
that is probably the most common use case for this (in most of the other
"API" style modes the apps can dynamically generate queries as they need).
That way the parser does not have to be overloaded with too many assumptions
around this concept.

We should take a look at current "Parameter
Substitution<http://hadoop.apache.org/pig/docs/r0.5.0/piglatin_reference.html#Parameter+Substitution>"
feature in Pig. It seems pretty comprehensive. I'm not familiar enough with
the code to even venture a guess as to how much of that code would be
reusable.
On Wed, Dec 30, 2009 at 7:31 AM, Edward Capriolo wrote:

I see two ways to do this. we can do the variable substitution at the
CLI level. Or we can do this at the query processor level.

In each case the variables would be set into the SessionState and the
respective component could do the substitution.

I think having the query processor handle this would be better.

If we don't here back in a few I will gladly do this as I can leverage
this as well.

Edward
On Tue, Dec 29, 2009 at 8:57 PM, Zheng Shao wrote:
Hi Vijay,

I sent out an inquiry to the guys at aws on 12/21. There is no reply
yet. It might be that people are on vacation.
Let's wait a bit to see if they can contribute that back to open-source.

Zheng
On Tue, Dec 29, 2009 at 5:10 PM, Vijay wrote:
Sorry to bump the thread again. I thought this was lost during the
holidays.
Anybody have any ideas about this?
On Sat, Dec 19, 2009 at 6:29 PM, Vijay wrote:

Amazon Elastic MapReduce version of Hive seems to have a nice feature
called "Variables." Basically you can define a variable via
command-line
while invoking hive with -d DT=2009-12-09 and then refer to the
variable via
${DT} within the hive queries. This could be extremely useful. I can't
seem
to find this feature even on trunk. Is this feature currently anywhere
in
the roadmap?


--
Yours,
Zheng

Search Discussions

  • Thejas Nair at Jan 4, 2010 at 3:06 pm
    The parameter substitution in pig is done using a query pre-processor, this
    code is mostly independent of rest of pig code, so it can be understood in
    isolation. It uses javacc.
    The code is in the package - org.apache.pig.tools.parameters (
    http://svn.apache.org/viewvc/hadoop/pig/trunk/src/org/apache/pig/tools/param
    eters/)

    -Thejas
    On 12/30/09 12:03 PM, "Vijay" wrote:

    Thanks guys!

    When I think about it, it may be good enough to do this at the CLI level as
    that is probably the most common use case for this (in most of the other
    "API" style modes the apps can dynamically generate queries as they need).
    That way the parser does not have to be overloaded with too many assumptions
    around this concept.

    We should take a look at current "Parameter
    Substitution<http://hadoop.apache.org/pig/docs/r0.5.0/piglatin_reference.html#
    Parameter+Substitution>"
    feature in Pig. It seems pretty comprehensive. I'm not familiar enough with
    the code to even venture a guess as to how much of that code would be
    reusable.
    On Wed, Dec 30, 2009 at 7:31 AM, Edward Capriolo wrote:

    I see two ways to do this. we can do the variable substitution at the
    CLI level. Or we can do this at the query processor level.

    In each case the variables would be set into the SessionState and the
    respective component could do the substitution.

    I think having the query processor handle this would be better.

    If we don't here back in a few I will gladly do this as I can leverage
    this as well.

    Edward
    On Tue, Dec 29, 2009 at 8:57 PM, Zheng Shao wrote:
    Hi Vijay,

    I sent out an inquiry to the guys at aws on 12/21. There is no reply
    yet. It might be that people are on vacation.
    Let's wait a bit to see if they can contribute that back to open-source.

    Zheng
    On Tue, Dec 29, 2009 at 5:10 PM, Vijay wrote:
    Sorry to bump the thread again. I thought this was lost during the
    holidays.
    Anybody have any ideas about this?
    On Sat, Dec 19, 2009 at 6:29 PM, Vijay wrote:

    Amazon Elastic MapReduce version of Hive seems to have a nice feature
    called "Variables." Basically you can define a variable via
    command-line
    while invoking hive with -d DT=2009-12-09 and then refer to the
    variable via
    ${DT} within the hive queries. This could be extremely useful. I can't
    seem
    to find this feature even on trunk. Is this feature currently anywhere
    in
    the roadmap?


    --
    Yours,
    Zheng

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedDec 30, '09 at 8:04p
activeJan 4, '10 at 3:06p
posts2
users2
websitepig.apache.org

2 users in discussion

Thejas Nair: 1 post Vijay: 1 post

People

Translate

site design / logo © 2021 Grokbase