FAQ
Hi everyone,

I would like to execute a Hive query on the server in an asynchronous
manner. The Hive query will likely take a long time to complete, so I would
prefer not to block on the call. I am currently using Thirft to make a
blocking call (blocks on client.execute()), but I have not seen an example
of how to make a non-blocking call. Here is the blocking code:

TSocket transport = new TSocket("hive.example.com", 10000);
transport.setTimeout(999999999);
TBinaryProtocol protocol = new TBinaryProtocol(transport);
Client client = new ThriftHive.Client(protocol);
transport.open();
client.execute(hql); // Omitted HQL

List<String> rows;
while ((rows = client.fetchN(1000)) != null) {
for (String row : rows) {
// Do stuff with row
}
}

transport.close();

The code above is missing try/catch blocks to keep it short.

I can see that Thirft has things like "TNonblockingSocket" but I cannot find
any examples of how to use it, or if Hive even supports it. Does anyone have
any ideas how to do an async call? Is there a better way?

Thanks!

Kind regards,
Peter Sankauskas

Search Discussions

  • Zheng Shao at Feb 3, 2010 at 9:11 am
    Hi Peter,

    Currently there is no such method in Hive. It should be pretty
    straightforward to do it with a thread.
    I would encourage you to open a JIRA and post a patch if you are
    interested in making it work.

    Zheng
    On Tue, Feb 2, 2010 at 9:42 AM, Peter Sankauskas wrote:
    Hi everyone,

    I would like to execute a Hive query on the server in an asynchronous
    manner. The Hive query will likely take a long time to complete, so I would
    prefer not to block on the call. I am currently using Thirft to make a
    blocking call (blocks on client.execute()), but I have not seen an example
    of how to make a non-blocking call. Here is the blocking code:

    TSocket transport = new TSocket("hive.example.com", 10000);
    transport.setTimeout(999999999);
    TBinaryProtocol protocol = new TBinaryProtocol(transport);
    Client client = new ThriftHive.Client(protocol);
    transport.open();
    client.execute(hql);  // Omitted HQL

    List<String> rows;
    while ((rows = client.fetchN(1000)) != null) {
    for (String row : rows) {
    // Do stuff with row
    }
    }

    transport.close();

    The code above is missing try/catch blocks to keep it short.

    I can see that Thirft has things like "TNonblockingSocket" but I cannot find
    any examples of how to use it, or if Hive even supports it. Does anyone have
    any ideas how to do an async call? Is there a better way?

    Thanks!

    Kind regards,
    Peter Sankauskas


    --
    Yours,
    Zheng

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshive, hadoop
postedFeb 2, '10 at 5:43p
activeFeb 3, '10 at 9:11a
posts2
users2
websitehive.apache.org

2 users in discussion

Zheng Shao: 1 post Peter Sankauskas: 1 post

People

Translate

site design / logo © 2022 Grokbase