FAQ
It looks like the way to use muti-query from Java is as follows:

1. pigServer.setBatchOn();
2. register your queries with pigServer
3. List<ExecJob> jobs = pigServer.executeBatch();
4. for (ExecJob job : jobs) { Iterator<Tuple> results = job.getResults(); }

This will cause all stores to get evaluated in a single batch. Is
there a way to know which job corresponds to which store?
Experimentally, they seem to be in reverse order of appearance. Is
that guaranteed, or is the order basically undefined and we need to
add getAlias() to ExecJob?

Also, it's inconvenient to need to store to disk in order to take
advantage of multi-query -- when executing from Java, I may just want
several iterators. Any thoughts on the advisability of adding
pigServer.executeBatch(List<String> aliases) ?

Thanks,
-Dmitriy

Search Discussions

  • Thejas Nair at Dec 15, 2009 at 9:40 pm
    I don't think the order of the jobs is guaranteed. Yes, api's need to be
    added to support the association of job to store . ExecJob should return
    alias or the FileSpec of the store .
    To to this in sql implementation, I added a ExecJob.getFileSpec() . But
    getAlias() might be more suitable for the use case you have mentioned.

    -Thejas


    On 12/15/09 12:44 PM, "Dmitriy Ryaboy" wrote:

    It looks like the way to use muti-query from Java is as follows:

    1. pigServer.setBatchOn();
    2. register your queries with pigServer
    3. List<ExecJob> jobs = pigServer.executeBatch();
    4. for (ExecJob job : jobs) { Iterator<Tuple> results = job.getResults(); }

    This will cause all stores to get evaluated in a single batch. Is
    there a way to know which job corresponds to which store?
    Experimentally, they seem to be in reverse order of appearance. Is
    that guaranteed, or is the order basically undefined and we need to
    add getAlias() to ExecJob?

    Also, it's inconvenient to need to store to disk in order to take
    advantage of multi-query -- when executing from Java, I may just want
    several iterators. Any thoughts on the advisability of adding
    pigServer.executeBatch(List<String> aliases) ?

    Thanks,
    -Dmitriy

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedDec 15, '09 at 8:45p
activeDec 15, '09 at 9:40p
posts2
users2
websitepig.apache.org

2 users in discussion

Thejas Nair: 1 post Dmitriy Ryaboy: 1 post

People

Translate

site design / logo © 2022 Grokbase