FAQ
Hi Sammy,

If you run "explain select count(*) from your_tbl", the plan will tell you
the number of partitions being scanned. Is that number correct?

If it is correct, that probably means that some of the data files can't be
read correctly by Impala.

If it's not correct, then maybe you can try running "invalidate metadata"
(that's different from refresh; see this
link<http://www.cloudera.com/content/cloudera-content/cloudera-docs/Impala/latest/Installing-and-Using-Impala/ciiu_langref_sql.html?scroll=invalidate_metadata_unique_1>for
more details)?

Thanks,
Alan

On Fri, Jan 10, 2014 at 7:39 PM, Sammy Yu wrote:

Hi,
I'm running impala 1.2.3 on with a rcfile table with 38687 partitions
that was created from hive. Afterwards, I did a refresh metadata and
compared the select count(1) results and noticed that the result differed
(impala results was significantly smaller than hive). I did further
investigation and determined that impala was not considering some of my
later partitions.

The hive show partition results came back as expected. I tried using the
show table stats command in impala, but I'm getting an error:
[ip-10-124-195-6.ec2.internal:21000] > SHOW TABLE STATS rcfile_3p;
Query: show TABLE STATS rcfile_3p
ERROR: IllegalArgumentException: Comparison method violates its general
contract!

Thanks for your help.

Best,
Sammy

To unsubscribe from this group and stop receiving emails from it, send an
email to impala-user+unsubscribe@cloudera.org.
To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 2 of 7 | next ›
Discussion Overview
groupimpala-user @
categorieshadoop
postedJan 11, '14 at 3:39a
activeJan 15, '14 at 6:15p
posts7
users2
websitecloudera.com
irc#hadoop

2 users in discussion

Alan Choi: 4 posts Sammy Yu: 3 posts

People

Translate

site design / logo © 2022 Grokbase