FAQ
Hi,

this is a screenshot from the Impala Query Details page in Cloudera Manager
(in German, sorry):

<https://lh3.googleusercontent.com/-qb7uJf14Fy8/UtlUkhmjJoI/AAAAAAAAH5M/V8B-JvMVnlQ/s1600/Impala_Query_Result.jpg>

We are confused about start time (Startzeit), end time (Endzeit) and
duration (Dauer).
If we compare start and end time we see that the query took *3min 26s* to
complete but what does duration then mean? We expected duration to be the
query runtime.
The output in Cloudera Manager is just a formatted output of the query
PROFILE (see the full output of PROFILE attached).
The value of duration seems to correspond to the TotalTime in Execution
Profile but the query actually took over 3min to complete.

Can somebody help us to understand the output of Cloudera Manager or
PROFILE, respectively?
Is there somewhere a documentation how to interpret the output of PROFILE,
the Cloudera documentation about it is not really useful:
http://www.cloudera.com/content/cloudera-content/cloudera-docs/Impala/latest/Installing-and-Using-Impala/ciiu_performance.html#perf_profile_unique_1

Thx,
Alex

To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.

Search Discussions

  • Chris Leroy at Jan 17, 2014 at 7:16 pm
    Alex,

    There is a bug in CM's computation of the duration. You are correct that it
    is simply reporting the total execution time and does not include other
    portions of the query lifecycle. A fix for this will be in the next version
    of CM (5.0 and 4.x).

    chris




    On Fri, Jan 17, 2014 at 8:22 AM, Alexander Schätzle wrote:

    Hi,

    this is a screenshot from the Impala Query Details page in Cloudera
    Manager (in German, sorry):


    <https://lh3.googleusercontent.com/-qb7uJf14Fy8/UtlUkhmjJoI/AAAAAAAAH5M/V8B-JvMVnlQ/s1600/Impala_Query_Result.jpg>

    We are confused about start time (Startzeit), end time (Endzeit) and
    duration (Dauer).
    If we compare start and end time we see that the query took *3min 26s* to
    complete but what does duration then mean? We expected duration to be the
    query runtime.
    The output in Cloudera Manager is just a formatted output of the query
    PROFILE (see the full output of PROFILE attached).
    The value of duration seems to correspond to the TotalTime in Execution
    Profile but the query actually took over 3min to complete.

    Can somebody help us to understand the output of Cloudera Manager or
    PROFILE, respectively?
    Is there somewhere a documentation how to interpret the output of PROFILE,
    the Cloudera documentation about it is not really useful:

    http://www.cloudera.com/content/cloudera-content/cloudera-docs/Impala/latest/Installing-and-Using-Impala/ciiu_performance.html#perf_profile_unique_1

    Thx,
    Alex

    To unsubscribe from this group and stop receiving emails from it, send an
    email to impala-user+unsubscribe@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.
  • Alexander Schätzle at Jan 20, 2014 at 9:31 am
    Thx Chris for clarification.

    Is there a more detailed documentation about EXPLAIN and PROFILE somewhere?
    It is hard to understand without any kind of documentation and sometimes
    you may misinterpret the values.

    Thx,
    Alex

    Am Freitag, 17. Januar 2014 20:16:11 UTC+1 schrieb Chris Leroy:
    Alex,

    There is a bug in CM's computation of the duration. You are correct that
    it is simply reporting the total execution time and does not include other
    portions of the query lifecycle. A fix for this will be in the next version
    of CM (5.0 and 4.x).

    chris





    On Fri, Jan 17, 2014 at 8:22 AM, Alexander Schätzle <
    schaetzle...@gmail.com <javascript:>> wrote:
    Hi,

    this is a screenshot from the Impala Query Details page in Cloudera
    Manager (in German, sorry):


    <https://lh3.googleusercontent.com/-qb7uJf14Fy8/UtlUkhmjJoI/AAAAAAAAH5M/V8B-JvMVnlQ/s1600/Impala_Query_Result.jpg>

    We are confused about start time (Startzeit), end time (Endzeit) and
    duration (Dauer).
    If we compare start and end time we see that the query took *3min 26s* to
    complete but what does duration then mean? We expected duration to be the
    query runtime.
    The output in Cloudera Manager is just a formatted output of the query
    PROFILE (see the full output of PROFILE attached).
    The value of duration seems to correspond to the TotalTime in Execution
    Profile but the query actually took over 3min to complete.

    Can somebody help us to understand the output of Cloudera Manager or
    PROFILE, respectively?
    Is there somewhere a documentation how to interpret the output of
    PROFILE, the Cloudera documentation about it is not really useful:

    http://www.cloudera.com/content/cloudera-content/cloudera-docs/Impala/latest/Installing-and-Using-Impala/ciiu_performance.html#perf_profile_unique_1

    Thx,
    Alex

    To unsubscribe from this group and stop receiving emails from it, send an
    email to impala-user...@cloudera.org <javascript:>.
    To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.
  • Alan Choi at Jan 20, 2014 at 5:56 pm
    Hi Alex,

    For the EXPLAIN output, please see our
    documentation<http://www.cloudera.com/content/cloudera-content/cloudera-docs/Impala/latest/Installing-and-Using-Impala/ciiu_performance.html>
    .

    Unfortunately, we don't have any tutorial for PROFILE. We know it's hard to
    read and we're trying to improve it!

    Thanks,
    Alan

    On Mon, Jan 20, 2014 at 1:31 AM, Alexander Schätzle wrote:

    Thx Chris for clarification.

    Is there a more detailed documentation about EXPLAIN and PROFILE
    somewhere? It is hard to understand without any kind of documentation and
    sometimes you may misinterpret the values.

    Thx,
    Alex

    Am Freitag, 17. Januar 2014 20:16:11 UTC+1 schrieb Chris Leroy:
    Alex,

    There is a bug in CM's computation of the duration. You are correct that
    it is simply reporting the total execution time and does not include other
    portions of the query lifecycle. A fix for this will be in the next version
    of CM (5.0 and 4.x).

    chris





    On Fri, Jan 17, 2014 at 8:22 AM, Alexander Schätzle <
    schaetzle...@gmail.com> wrote:
    Hi,

    this is a screenshot from the Impala Query Details page in Cloudera
    Manager (in German, sorry):


    <https://lh3.googleusercontent.com/-qb7uJf14Fy8/UtlUkhmjJoI/AAAAAAAAH5M/V8B-JvMVnlQ/s1600/Impala_Query_Result.jpg>

    We are confused about start time (Startzeit), end time (Endzeit) and
    duration (Dauer).
    If we compare start and end time we see that the query took *3min 26s* to
    complete but what does duration then mean? We expected duration to be the
    query runtime.
    The output in Cloudera Manager is just a formatted output of the query
    PROFILE (see the full output of PROFILE attached).
    The value of duration seems to correspond to the TotalTime in Execution
    Profile but the query actually took over 3min to complete.

    Can somebody help us to understand the output of Cloudera Manager or
    PROFILE, respectively?
    Is there somewhere a documentation how to interpret the output of
    PROFILE, the Cloudera documentation about it is not really useful:
    http://www.cloudera.com/content/cloudera-content/
    cloudera-docs/Impala/latest/Installing-and-Using-Impala/
    ciiu_performance.html#perf_profile_unique_1

    Thx,
    Alex

    To unsubscribe from this group and stop receiving emails from it, send
    an email to impala-user...@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to impala-user+unsubscribe@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.
  • Alexander Schätzle at Jan 21, 2014 at 8:50 am
    Hi Alan,

    thx but also EXPLAIN is also not really described in the documentation.
    Hopefully, there will be more details in the future :-)

    Best,
    Alex

    Am Montag, 20. Januar 2014 18:56:45 UTC+1 schrieb Alan:
    Hi Alex,

    For the EXPLAIN output, please see our documentation<http://www.cloudera.com/content/cloudera-content/cloudera-docs/Impala/latest/Installing-and-Using-Impala/ciiu_performance.html>
    .

    Unfortunately, we don't have any tutorial for PROFILE. We know it's hard
    to read and we're trying to improve it!

    Thanks,
    Alan


    On Mon, Jan 20, 2014 at 1:31 AM, Alexander Schätzle <
    schaetzle...@gmail.com <javascript:>> wrote:
    Thx Chris for clarification.

    Is there a more detailed documentation about EXPLAIN and PROFILE
    somewhere? It is hard to understand without any kind of documentation and
    sometimes you may misinterpret the values.

    Thx,
    Alex

    Am Freitag, 17. Januar 2014 20:16:11 UTC+1 schrieb Chris Leroy:
    Alex,

    There is a bug in CM's computation of the duration. You are correct that
    it is simply reporting the total execution time and does not include other
    portions of the query lifecycle. A fix for this will be in the next version
    of CM (5.0 and 4.x).

    chris





    On Fri, Jan 17, 2014 at 8:22 AM, Alexander Schätzle <
    schaetzle...@gmail.com> wrote:
    Hi,

    this is a screenshot from the Impala Query Details page in Cloudera
    Manager (in German, sorry):


    <https://lh3.googleusercontent.com/-qb7uJf14Fy8/UtlUkhmjJoI/AAAAAAAAH5M/V8B-JvMVnlQ/s1600/Impala_Query_Result.jpg>

    We are confused about start time (Startzeit), end time (Endzeit) and
    duration (Dauer).
    If we compare start and end time we see that the query took *3min 26s* to
    complete but what does duration then mean? We expected duration to be the
    query runtime.
    The output in Cloudera Manager is just a formatted output of the query
    PROFILE (see the full output of PROFILE attached).
    The value of duration seems to correspond to the TotalTime in Execution
    Profile but the query actually took over 3min to complete.

    Can somebody help us to understand the output of Cloudera Manager or
    PROFILE, respectively?
    Is there somewhere a documentation how to interpret the output of
    PROFILE, the Cloudera documentation about it is not really useful:
    http://www.cloudera.com/content/cloudera-content/
    cloudera-docs/Impala/latest/Installing-and-Using-Impala/
    ciiu_performance.html#perf_profile_unique_1

    Thx,
    Alex

    To unsubscribe from this group and stop receiving emails from it, send
    an email to impala-user...@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to impala-user...@cloudera.org <javascript:>.
    To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupimpala-user @
categorieshadoop
postedJan 17, '14 at 4:22p
activeJan 21, '14 at 8:50a
posts5
users3
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase