For the catalogd resource issue, can you do a "jstack <catalogd pid>" when
you see the cpu is running high?
We're working on 1.2.4 and will be released shortly.
Thanks,
Alan
On Tue, Jan 14, 2014 at 4:52 PM, Sammy Yu wrote:
Hi Alan,
Thanks so much for looking into the issue and determining the root
cause. I can see that the IMPALA-749 is assigned for 1.2.4.
Regarding the catalogd resource issue, is there anything that I can
provide in terms of log to confirm this is the issue I'm seeing? I
hate to ask this but regarding the roadmap I know it was mentioned in
another email that the next major release 1.3 will be available end of
Q1/early Q2, does this mean we will likely see a 1.2.4 release before
then and will it fix both of these issues?
Best,
Sammy
can't
and
my
general
send
To unsubscribe from this group and stop receiving emails from it, send
an
email to impala-user+unsubscribe@cloudera.org.To unsubscribe from this group and stop receiving emails from it, sendan
To unsubscribe from this group and stop receiving emails from it, send an
email to impala-user+unsubscribe@cloudera.org.To unsubscribe from this group and stop receiving emails from it, send an
email to impala-user+unsubscribe@cloudera.org.
To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.Hi Alan,
Thanks so much for looking into the issue and determining the root
cause. I can see that the IMPALA-749 is assigned for 1.2.4.
Regarding the catalogd resource issue, is there anything that I can
provide in terms of log to confirm this is the issue I'm seeing? I
hate to ask this but regarding the roadmap I know it was mentioned in
another email that the next major release 1.3 will be available end of
Q1/early Q2, does this mean we will likely see a 1.2.4 release before
then and will it fix both of these issues?
Best,
Sammy
On Mon, Jan 13, 2014 at 8:34 PM, Alan Choi wrote:
Hi Sammy,
Good catch. Thanks for reporting this issue. I've filed IMPALA-749 to track
it.
Thanks,
Alan
32,767..Hi Sammy,
Good catch. Thanks for reporting this issue. I've filed IMPALA-749 to track
it.
Thanks,
Alan
On Mon, Jan 13, 2014 at 7:52 PM, Sammy Yu wrote:
Hi Alan,
Thanks for the reply. The explain shows (32,767 partitions which
is less than the expected 38,687 partitions):
Hi Alan,
Thanks for the reply. The explain shows (32,767 partitions which
is less than the expected 38,687 partitions):
0:SCAN HDFS
table=default.raw_3p #partitions=32767/32767 size=130.69GB |
I ran "invalidate metadata", but the explain still came back withtable=default.raw_3p #partitions=32767/32767 size=130.69GB |
Are there any logs that I can provide? Is there a way to to dump out
what catalogd views of the partitions like I can see the partitions in
hive's metastore:
mysql> select count(1) from PARTITIONS where TBL_ID=6;
+----------+
1 row in set (0.00 sec)
Thanks,
Sammy
what catalogd views of the partitions like I can see the partitions in
hive's metastore:
mysql> select count(1) from PARTITIONS where TBL_ID=6;
+----------+
count(1) |
+----------+
38687 |
+----------++----------+
38687 |
1 row in set (0.00 sec)
Thanks,
Sammy
On Mon, Jan 13, 2014 at 7:32 PM, Alan Choi wrote:
Hi Sammy,
If you run "explain select count(*) from your_tbl", the plan will tell
you
the number of partitions being scanned. Is that number correct?
If it is correct, that probably means that some of the data files
Hi Sammy,
If you run "explain select count(*) from your_tbl", the plan will tell
you
the number of partitions being scanned. Is that number correct?
If it is correct, that probably means that some of the data files
be
read correctly by Impala.
If it's not correct, then maybe you can try running "invalidate
metadata"
(that's different from refresh; see this link for more details)?
Thanks,
Alan
read correctly by Impala.
If it's not correct, then maybe you can try running "invalidate
metadata"
(that's different from refresh; see this link for more details)?
Thanks,
Alan
On Fri, Jan 10, 2014 at 7:39 PM, Sammy Yu wrote:
Hi,
I'm running impala 1.2.3 on with a rcfile table with 38687
partitions
that was created from hive. Afterwards, I did a refresh metadata
Hi,
I'm running impala 1.2.3 on with a rcfile table with 38687
partitions
that was created from hive. Afterwards, I did a refresh metadata
compared the select count(1) results and noticed that the result
differed
(impala results was significantly smaller than hive). I did further
investigation and determined that impala was not considering some of
differed
(impala results was significantly smaller than hive). I did further
investigation and determined that impala was not considering some of
later partitions.
The hive show partition results came back as expected. I tried using
the
show table stats command in impala, but I'm getting an error:
[ip-10-124-195-6.ec2.internal:21000] > SHOW TABLE STATS rcfile_3p;
Query: show TABLE STATS rcfile_3p
ERROR: IllegalArgumentException: Comparison method violates its
The hive show partition results came back as expected. I tried using
the
show table stats command in impala, but I'm getting an error:
[ip-10-124-195-6.ec2.internal:21000] > SHOW TABLE STATS rcfile_3p;
Query: show TABLE STATS rcfile_3p
ERROR: IllegalArgumentException: Comparison method violates its
contract!
Thanks for your help.
Best,
Sammy
To unsubscribe from this group and stop receiving emails from it,
Thanks for your help.
Best,
Sammy
To unsubscribe from this group and stop receiving emails from it,
an
email to impala-user+unsubscribe@cloudera.org.
email to impala-user+unsubscribe@cloudera.org.
To unsubscribe from this group and stop receiving emails from it, send
an
email to impala-user+unsubscribe@cloudera.org.
email to impala-user+unsubscribe@cloudera.org.
To unsubscribe from this group and stop receiving emails from it, send an
email to impala-user+unsubscribe@cloudera.org.
email to impala-user+unsubscribe@cloudera.org.
