impala-user@cloudera.org discussions for August 2014

18	Re: impala hbase very slow planning time

See below. I0805 16:37:05.143970 25932 impala-beeswax-server.cc:167] query(): query=select * from network_quality_ipr where prit ='101_2014-08-03' and prit <= '101_2014-08-03' I0805 16:37:05.144059 ...

Evgeniy Shishkin
Aug 5, 2014 at 4:41 pm
Aug 9, 2014 at 12:12 am

17	communication between impala and an existing cluster

Hi, i am a starter with impala, i just setup an ambari cluster, now i would like to enable impala on it (without cloudera) First and follow that cloudera without impala guide ...

Donald fossouo
Aug 18, 2014 at 9:03 am
Aug 21, 2014 at 6:46 am

16	How to debug a slow "select count(*) from .."

Hi, We're puzzled about the performance of impala on one of our clusters. Most impala queries are way faster on our old hardware cluster (A) than our new cluster (B). (A) is 4 year old hardware, less ...

Laurens Bronwasser
Aug 25, 2014 at 3:29 pm
Sep 4, 2014 at 9:12 pm

11	Re: Impala queries running as root

Hi, Which version of Impala are you using (which rpms)? Regards, -Ippokratis. -- -Ippokratis. To unsubscribe from this group and stop receiving emails from it, send an email to ...

Ippokratis Pandis
Aug 19, 2014 at 11:14 pm
Sep 10, 2014 at 6:01 pm

9	Re: impala 1,4 and LZO compression

Hi RanG, Can you please try one more time by renaming a few files appending the .lzo suffix? Also, are those files sequence or plain text? Regards, -Ippokratis. -- -Ippokratis. To unsubscribe from ...

Ippokratis Pandis
Aug 20, 2014 at 11:08 pm
Sep 3, 2014 at 6:05 pm

9	Re: Invalid AVRO_VERSION_HEADER: '7b d a 20 '

Can you trying running "invalidate metadata impala_avro_table" from the shell and trying your query again? To unsubscribe from this group and stop receiving emails from it, send an email to ...

Skye Wanderman-Milne
Aug 11, 2014 at 7:54 pm
Aug 20, 2014 at 7:35 am

8	Re: How to merge small parquet files?

Hi Ippokratis, Thanks for the reply. I use default compression config, which is snappy. I have 9 nodes, I can see each node generate some(~11) parquet files(each ~250MB) guess it is because of this? ...

Binglin Chang
Aug 21, 2014 at 10:03 am
Sep 25, 2014 at 4:53 pm

8	Stuck Queries with high offset values

Hi, We've recently upgraded from CDH 4/Impala 1.3.1 to CDH 5.1.1/Impala 1.4.0. Everything went smoothly, but now we are encountering a very strange issue with queries with high offset counts getting ...

Sammy Yu
Aug 26, 2014 at 2:27 am
Sep 24, 2014 at 1:40 am

8	slow HDFS_SCAN_NODE time for simple query

The data is about 40gb snappy compressed parquet, with about 80m rows, 50 columns(5 columns of them are very big(1K per row)), other columns are all very small. query: select count(*) from ...

Binglin Chang
Aug 7, 2014 at 6:04 am
Aug 20, 2014 at 11:19 pm

7	INSERT INTO parquet table not honoring 'location' while storing data on hdfs

Hi, I have a partitioned parquet table in Impala as follows: CREATE EXTERNAL TABLE `MY_TABLE` ( ...) PARTITIONED BY (dt INT) row format delimited fields terminated by '\t' STORED AS PARQUET; ALTER ...

Hemalatha Chakrapani
Aug 27, 2014 at 3:40 pm
Aug 27, 2014 at 7:05 pm

6	Charset problem when SQL Server connect to Impala using OpenQuery and ODBC

Hello, I'm using SQL Server to query IMPALA using OpenQuery. Everything works well except the charset . The Query is : SELECT LogTime,AccountName,RoleID,*RoleName*,ServerID ...

Summer Nguyen
Aug 15, 2014 at 11:45 am
Sep 3, 2014 at 9:59 am

5	Trying to drop tables with computed stats throwing java.sql.BatchUpdateException

Any table that I compute stats on throws the following exception when I try and drop it. I have tried recalculating the stats in Hive as suggested here ...

Joshua Baxter
Aug 19, 2014 at 4:23 pm
Oct 29, 2014 at 2:52 pm

5	Optimize queries for joining on ordered columns

Hi there, We are trying to join tables based on timestamp. We have tons of events in a single day, and so joining events based on their timestamps seems to be a really heavy task for Impala. My guess ...

Julien L.
Aug 15, 2014 at 8:44 am
Aug 18, 2014 at 9:53 am

5	Is alter view atomic?

Are *Alter View* DDL commands atomic? For example, if I have foo_1 and foo_2, and a view "select * from foo_1", if I issue "Alter view select * from foo_2" where there be any period of time will ...

Keith Simmons
Aug 6, 2014 at 8:56 pm
Aug 7, 2014 at 11:27 pm

4	Re: NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.protobuf.ProtobufUtil

BTW, I am seeing this error when I query HBase table and not during start-up of impalad daemon. To unsubscribe from this group and stop receiving emails from it, send an email to ...

Gaurav Kumar
Aug 3, 2014 at 7:46 pm
Oct 11, 2014 at 10:08 am

4	load data into impala partitioned table

I have data in HDFS in following dir structure : /exported/2014/07/01/00/SEARCHES/part-m-*.bz2 /exported/2014/07/01/01/SEARCHES/part-m-*.bz2 /exported/2014/07/01/02/SEARCHES/part-m-*.bz2 . . . ...

Roy
Aug 21, 2014 at 9:48 pm
Aug 25, 2014 at 3:16 pm

4	impalad 1.2.4 webserver hangs

In our impala cluster, an impalad service webserver hangs, related gdb info bellow, looks like the lock query_locations_lock_ in( impala-server.cc:643) is held forever by some other thread forever. ...

Binglin Chang
Aug 22, 2014 at 4:25 am
Aug 22, 2014 at 6:48 am

4	Unable to start any impala service, getting error "CPU does not support the SSE3 instruction"

As per subject, when I start any impla service (impala-server, impala-state-store etc) on my debain machine I get following error message in /var/log/impala/impala-state-store.log WARNING: Logging ...

Gaurav Kumar
Aug 11, 2014 at 8:17 am
Aug 18, 2014 at 11:29 pm

3	Connection pool test query appears to return error

We're using a dbcp connection pool to connect our app to Impala via jdbc, and have specified "select 1" as the connection test query. This sql statement runs fine in impala-shell, and our connection ...

Mauricio Aristizabal
Aug 14, 2014 at 1:40 am
Oct 30, 2014 at 12:55 am

3	No luck with Impala 1.4.0 and Sentry

Hello I've updated our distribution to the latest CDH 5.1, which comes with Impala 1.4.0. I've got the Sentry Service set up and it's working fine with HiveServer2 (beeline). But Impala refuses to ...

Grisha Trubetskoy
Aug 14, 2014 at 2:47 pm
Sep 6, 2014 at 11:53 pm

3	Performance/profile help

Hi all, I'm trying to understand and improve the performance of a query. The full profile of the query is pasted below (and attached as a file for easier reading). I'm using a 5 machine cluster that ...

Edward Vogel
Aug 18, 2014 at 8:01 pm
Aug 20, 2014 at 12:20 am

3	Unable to read multiple parquet files FOLDER using Hive/Impala

I am trying to read large no. of "parquet files" folder using Hive external table. I could able to read single parquet file at a time using below Query. create external table ParqTable5 (name ...

Rafeeq s
Aug 18, 2014 at 9:58 am
Aug 19, 2014 at 5:34 pm

3	Imapla execution run in an unexpected pool

I'm using CDH 5.0.0 and executing Impala queries. In YARN's configuration tab, I set " *yarn.scheduler.fair.user-as-default-queue*" to "*true*", which means Impala execution should run in a pool ...

Tao Xiao
Aug 7, 2014 at 1:03 pm
Aug 8, 2014 at 6:39 pm

3	Cancel Impala query via JDBC

Hi I'm trying to cancel a long running query via JDBC threaded java program by calling cancel function. while running im getting the below exception Caused by: java.sql.SQLException ...

Impala explorer
Aug 5, 2014 at 6:22 am
Aug 6, 2014 at 1:51 am

2	How to merge small parquet files?

I am new to Parquet and using parquet format for storing spark stream data into hdfs. Is it possible to merge multiple small parquet files into one ? Please suggest an example. Thanks in Advance! To ...

Rafeeq Ece71
Aug 12, 2014 at 12:53 pm
Oct 21, 2014 at 5:52 am

2	Re: "ParquetScanner: Could not deserialize page header" when running Impala query

Hi Julia, How big is the schema (how many columns)? How big are these json strings you are storing (KB, MB, GB)? It seems like there are two likely issues: (1) The header is too large (this seems ...

Victor Bittorf
Aug 27, 2014 at 7:15 pm
Oct 6, 2014 at 1:57 pm

2	impala statestored crashed because network issue

Version: 1.4.0-cdh4 log: I0828 15:44:55.096390 25292 *statestore.cc:300*] Registering: *lg-hadoop-st34.bj:32402* I0828 15:44:55.096587 25292 *statestore.cc:331*] Subscriber ' ...

Binglin Chang
Aug 28, 2014 at 8:20 am
Sep 4, 2014 at 1:10 am

2	Impala table partitioned by Year / Month / Day - wow to specify query for date range between 2 month

I have a table partitioned by Year / Month / Day. E.g. Year=2014, Month=07, Day=15 Now I want to query date range between 2014/06/15 - 2014/07/04. How to specify that in a query? I tried with: select ...

Vladimir
Aug 29, 2014 at 1:28 pm
Aug 29, 2014 at 5:49 pm

2	Huge performance degradation when using to_date()

Hi, I'm seeing a huge performance degradation when running the following queries. I don't really get why the second query is taking so much longer to complete than the first one. We're running Impala ...

Vincent Kooijman
Aug 28, 2014 at 6:16 am
Aug 28, 2014 at 4:04 pm

2	impalad ERROR: short-circuit local reads is disabled

Fails to start the daemon impalad HDFS version CDH4 Release 4.6 and Impala version 1.4 Configured as described in the CDH Impala installation ...

Ramesh Gantyala
Aug 15, 2014 at 9:17 pm
Aug 17, 2014 at 3:02 pm

2	Re: Impala query - database locked!

To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.

Romain Rigaux
Aug 8, 2014 at 6:11 pm
Aug 15, 2014 at 10:09 pm

2	Re: UDF parallelism

Hi Jung-Yup, see responses inline. You are correct. The validator() UDF will indeed be evaluated at the coordinator by a single thread. This "lazy" evaluation strategy works well for many common use ...

Alex Behm
Aug 11, 2014 at 3:55 pm
Aug 12, 2014 at 12:56 am

1	Need your feedback about your use of this list

Hi all, We're interested in learning more about how you use this list and its value to you. So, please take a moment to complete this very short survey (not gated, no registration) ...

Justin Kestelyn
Aug 22, 2014 at 5:30 pm
Aug 29, 2014 at 3:49 pm

1	Error converting column: 0 TO TIMESTAMP

Hi, I have data in text files stored on HDFS with following directory structure : /exported/year/07/month/hour/SEARCHES/ /exported/2014/07/01/00/SEARCHES/part-m-00000 part-m-00001 ...

Roy
Aug 25, 2014 at 3:13 pm
Aug 27, 2014 at 9:29 pm

1	Re: Impala 1.4 Spiking above memory limit

What version of Impala are you running? To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.

Skye Wanderman-Milne
Aug 18, 2014 at 6:40 pm
Aug 18, 2014 at 6:59 pm

1	Add multi-level partition through java API

Hello guys I want to add multi-level partitions to a impala table through Java API. for example my table is partitioned by month and year. The hdfs directory is /hdfs/2013/01 /hdfs/2013/02 ... .. ...

Pengcheng Liu
Aug 12, 2014 at 5:29 pm
Aug 12, 2014 at 6:46 pm

1	Unknown query error when querying by PHP PDO

Hi Everyone, I am trying to use PHP to get data from Impala via Impala ODBC driver and the driver manager is unixODBC. In my environment, I can get data by PHP odbc_connect() and isql but it's failed ...

David Zhou
Aug 7, 2014 at 8:24 am
Aug 7, 2014 at 8:33 am

0	Impala error: column contains an unsupported column repetition type: 2

Impala gives below error while fetching *repeated* data. I am trying to write *repeated* data of same column in parquet file and Querying it in impala. Below is my schema: ("message basket { required ...

Rafeeq S
Aug 31, 2014 at 3:33 pm
Aug 31, 2014 at 3:33 pm

0	Re: JDBC driver does not use the database name provided in the connection url

Are you using an old version of the driver? Support for this was added just 9 months ago: https://issues.apache.org/jira/browse/HIVE-4256 To unsubscribe from this group and stop receiving emails from ...

Mauricio Aristizabal
Aug 29, 2014 at 5:27 pm
Aug 29, 2014 at 5:27 pm

0	Impala table partitioned by Year / Month / Day - how to specify date range between 2 months?

Hi, I have a table partitioned by Year / Month / Day. E.g. Year=2014, Month=07, Day=15 Now I want to query date range between 2014/06/15 - 2014/07/04. How to specify that in a query? I tried with ...

Vladimir
Aug 29, 2014 at 1:30 pm
Aug 29, 2014 at 1:30 pm

0	Which Writable class should I use to write string data into Parquet file?

I am trying to write string data into parquet file using parquet writer and There no specific writable class for string as Parquet data types doesn't support string. *Which Writable class should I ...

Rafeeq Ece71
Aug 28, 2014 at 1:16 pm
Aug 28, 2014 at 1:16 pm

0	Re: Is there a way to insert data into existing parquet file using spark ?

Hi Rafeeq, I would recommend seeing what tools Spark has for outputing parquet files. I think they describe them in this document. https://spark.apache.org/docs/latest/sql-programming-guide.html Once ...

Victor Bittorf
Aug 27, 2014 at 3:54 pm
Aug 27, 2014 at 3:54 pm

0	Does Impala 1.4.0, Can support nested objects in JSON ?

We are using *Impala 1.4.0* to access parquet files which are stored in HDFS. The parquet files contain JSON tweets. *Does Impala 1.4.0, Can support nested objects in JSON ? * sample Json ...

Rafeeq s
Aug 25, 2014 at 11:41 am
Aug 25, 2014 at 11:41 am

0	Impala - Unable to read data from hive External Table.

We are using Hive external table to load parquet files data from external location and Impala is used to query on loaded data. Below is the hive query to create external table and configuration ...

Rafeeq s
Aug 21, 2014 at 2:02 pm
Aug 21, 2014 at 2:02 pm

0	Re: UnicodeEncodeError in impala shell

Fixed! thanks. -Sean To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.

Sean O'Brien
Aug 14, 2014 at 9:54 pm
Aug 14, 2014 at 9:54 pm

0	UnicodeEncodeError in impala shell

Hi all, I know unicode support was added in impala 1.4, but it doesn't seem to be working for me. I'm getting stack traces like the following when I run queries that happened to hit some weird ...

Sean O'Brien
Aug 14, 2014 at 9:10 pm
Aug 14, 2014 at 9:10 pm

0	"Interactive SQL in Hadoop: A Closer Look at Impala” presentation

Hi, I was attending the "Interactive SQL in Hadoop: A Closer Look at Impala” presentation and wanted to find out if the slides that Justin Erickson presented could be shared? Many thanks, Jeff Jeff ...

Jeff Feng
Aug 14, 2014 at 5:59 pm
Aug 14, 2014 at 5:59 pm

0	Reading Parquet files using Hive and Execute in Impala

I have parquet files which are generated by *SPARK saveAsParquetFile() *and it is stored under external directory location. I want to read this Parquet files from *external location* using Hive ...

Rafeeq Ece71
Aug 14, 2014 at 8:26 am
Aug 14, 2014 at 8:26 am

0

subscribe

subscribe To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.

Tao Xiao
Aug 7, 2014 at 12:45 pm
Aug 7, 2014 at 12:45 pm

group	impala-user @
categories	hadoop
discussions	49
posts	217
users	55
website	cloudera.com
irc	#hadoop

summaries	on	Hotkey: s
users list	on	Hotkey: d
user style	avatars	Hotkey: a

Hadoop > impala-user > August 2014

Search Discussions

49 discussions - 217 posts

55 users for August 2014

Content

People

Support

Translate