FAQ
Hi,

I would appreciate if someone can provide answers to the following -

1) Is impalad process multi-threaded? From the functionality it seems to me
(heartbeat thread talks with statestored, thread(s) for planner, thread(s)
for coordinator, thread(s) from executor )

2) Does coordinator wait for each fragment to return the whole data before
performing aggregation/sorting/topn operations? Or it starts fetching data
as generated by the fragments running on another node or coordinator node
starts pulling data as it is generated by fragments on another node? Or it
is dependent on the operators created by the planner in the query plan?

Thanks in advance.

Search Discussions

  • Nong at Feb 17, 2013 at 7:33 pm
    Inline
    On Wednesday, February 13, 2013 5:15:00 PM UTC-8, Deepsoft wrote:

    Hi,

    I would appreciate if someone can provide answers to the following -

    1) Is impalad process multi-threaded? From the functionality it seems to
    me (heartbeat thread talks with statestored, thread(s) for planner,
    thread(s) for coordinator, thread(s) from executor )
    The process is multithreaded. Each query fragment (including each
    coordinator fragment) runs on it's own thread. Planning happens on the
    coordinator. The scanning portion of plan fragments are also
    multithreaded. The IO and network systems are highly multithreaded.

    2) Does coordinator wait for each fragment to return the whole data before
    performing aggregation/sorting/topn operations? Or it starts fetching data
    as generated by the fragments running on another node or coordinator node
    starts pulling data as it is generated by fragments on another node? Or it
    is dependent on the operators created by the planner in the query plan?
    In general, the execution is pipelined as much as possible. In the
    aggregation example, the coordinator fragment starts doing the final
    aggregation as soon as the preaggregation on the other fragments has
    started to return results.
    Thanks in advance.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupimpala-user @
categorieshadoop
postedFeb 14, '13 at 1:15a
activeFeb 17, '13 at 7:33p
posts2
users2
websitecloudera.com
irc#hadoop

2 users in discussion

Deepsoft: 1 post Nong: 1 post

People

Translate

site design / logo © 2021 Grokbase