FAQ
Hello,

I have another query which stops due to memory issues:

SELECT b.dif, b.counts FROM
         (
           SELECT a.dif as dif, count(a.uuid) as counts FROM
           (
                 SELECT
                     uuid,
                     ceil((max(time) - min(time))/3600000) as dif,
                     count(*) AS events
                 FROM
                     log_par
                 GROUP BY
                     uuid
           )a
           WHERE
               a.events > 1
           GROUP BY
               a.dif
           LIMIT 500
         ) AS b
         ORDER BY
             b.dif
         LIMIT 500

This time I get the following error:

   java.sql.SQLException: Invalid query handle



The "Select" which results in "a" will have a quite large result (65m
rows). Is there a way to rewrite the query, or any other thing we can do to
fix those memory issues?

Cheers,

Klaus



On Tue, Oct 22, 2013 at 9:15 AM, Klausen Schaefersinho wrote:

Hi,

I am trying to execute a query that computes out the drop our rates from
our web server logs (140m row). For each log entry we have the browser,
browser version and uuid (cookie).


The query looks like this:

SELECT
c.browser, c.browserversionmayor, count(c.events) as counts
FROM
(
SELECT
count(*) AS events, browser AS browser, browserversionmayor AS
browserversionmayor
FROM log_par
GROUP BY uuid, browser, browserversionmayor
)
c
WHERE
-- User with one event left page
c.events = 1
GROUP BY
c.browser, c.browserversionmayor
ORDER BY
c.browser LIMIT 10000

The query basically computes for all browser / browserversion combinations
the number of one time visitors, by restricting the count of the events to
one. However, everytime I try to run the query on our cluster, I get the
following error message:

Backend 2:Memory limit exceeded


If I run a simple version without the browserversion, or with a fixed
browser the query performs fine.
To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupimpala-user @
categorieshadoop
postedOct 22, '13 at 2:29p
activeOct 22, '13 at 2:29p
posts1
users1
websitecloudera.com
irc#hadoop

1 user in discussion

Klausen Schaefersinho: 1 post

People

Translate

site design / logo © 2022 Grokbase