FAQ
Hi Ran,

Impala doesn't support custom serdes. If your data fits into the "text"
file format, you don't need the CSVSerde.

Thanks,
Alan

On Sun, Aug 31, 2014 at 4:37 AM, Ran Grossman wrote:

I am doing the queries with impala latest version
can i manually add support for this SerDe? such as in hive "add
jar /serde/csv-serde-1.0.1-jar-with-dependencies.jar;"

_______________
Ran Grossman
+972-54-4842272

On Fri, Aug 29, 2014 at 4:09 AM, Ippokratis Pandis wrote:

Hi Ran,

Re second: You ran the same queries with Hive or Impala? Are you
running the latest version of Impala? I think the answer is that there
is no support for this SerDe. I would suggest again to reload the data
to say Parquet.
Btw LZO is not data type, but compression.

Re third: Great. You can connect to any node and the query will be
executed by all the other 19 nodes.

Regards,
-Ippokratis.



On Mon, Aug 25, 2014 at 5:33 AM, Ran Grossman <rang@liveperson.com>
wrote:
Hello,

First - Thanks for the INVALIDATE METADATA, it helped.

Second - i did the same queries on another hive server we have in
production, (tables are already created there for data type LZO)
and now i got the below error:

show create table test1;
Query: show create table test1
ERROR: AnalysisException: Failed to load metadata for table:
default.test1
CAUSED BY: TableLoadingException: Failed to load metadata for table: test1
CAUSED BY: InvalidStorageDescriptorException: Impala does not support tables
of this type. REASON: SerDe library
'com.liveperson.hive.serde.csv.CSVSerde'
is not supported.

is that mean that impala does not support this serde, or somthing i need to
change in my configuration?

Third - my plan is to put impala on 20 nodes hadoop cluster + one
management
node and i am following the cloudera recommendation for that as in here, so
i need to know how does impala works with all 20 nodes is there a network
and memory balance between them all? or more accurate, when i enter the
impala-shell from the management server i need to connect to one of the
hadoop datanodes (e.g. impala> connect datanode1;) when doing that is it
saying that all other impala-server over the other 19 nodes are working as
well? or do i need to configure something like VIP to connect to impala in a
balance way?

Thanks in advance for your answers.






_______________
Ran Grossman
+972-54-4842272


On Fri, Aug 22, 2014 at 9:12 AM, Ippokratis Pandis <
ipandis@cloudera.com>
wrote:
Hi Ran,

You are using 1.4, right? In upstream there is checked in code that
handles this issue. In the meantime, one workaround is to (re)load the
data to say Parquet tables (e.g.
http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Installation-Guide/cdh4ig_parquet.html
)
Re your second question: try executing INVALIDATE METADATA from the
Impala-shell. Other that show tables should work without having to
restart the impala service.

Regards,
-Ippokratis.

On Thu, Aug 21, 2014 at 4:46 AM, Ran Grossman <rang@liveperson.com>
wrote:
When doing that i can only create the table via hive since impala
cannot
create table using
And after creating it in hive i got this on impala:

select * from test4;
Query: select * from test4
ERROR: AnalysisException: Failed to load metadata for table:
default.test4
CAUSED BY: TableLoadingException: Failed to load metadata for table:
test4
CAUSED BY: RuntimeException: Compressed file not supported without
compression input format: ayelet.lzo

BTW, when creating a table with hive i cannot see it in impala (show
tables;) until i restart the impala service on the server, any idea
why?

_______________
Ran Grossman
+972-54-4842272


On Thu, Aug 21, 2014 at 8:40 AM, Ippokratis Pandis
<ipandis@cloudera.com>
wrote:
Hi,

Can you use the Impala-shell to create the table and use the "STORED
AS SEQUENCEFILE", on .lzo files?


http://www.cloudera.com/content/cloudera-content/cloudera-docs/Impala/latest/Installing-and-Using-Impala/ciiu_create_table.html
Regards,
-Ippokratis.

On Wed, Aug 20, 2014 at 10:09 PM, wrote:
Hi,
those files are sequence
and regarding the lzo suffix i already tried it before but still
got
empty
answer



On Thursday, August 21, 2014 2:08:38 AM UTC+3, Ippokratis Pandis
wrote:
Hi RanG,

Can you please try one more time by renaming a few files
appending
the
.lzo suffix? Also, are those files sequence or plain text?

Regards,
-Ippokratis.

On Wed, Aug 20, 2014 at 7:05 AM, wrote:

Hi,

I have installed impala 1.4 on my hadoop cluster (cdh4.3)
i have set all properties according to cloudera instructions in
this
link
our data is saved compressed on hadoop hdfs but without lzo
suffix
i created, using hive, the below table:

CREATE EXTERNAL TABLE test1(

header tinyint COMMENT 'from deserializer',

application_version string COMMENT 'from deserializer',

event_type string COMMENT 'from deserializer',

session_id string COMMENT 'from deserializer',

)

PARTITIONED BY (

year string,

month string,

day string)

ROW FORMAT DELIMITED

STORED AS INPUTFORMAT

'org.apache.hadoop.mapred.SequenceFileInputFormat'

OUTPUTFORMAT

'org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat'

LOCATION

'hdfs://mynode:542/bla/bla/bla';

also tried creating it with few more/different parameters such
as
and
few
more

ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'

STORED AS INPUTFORMAT
'com.hadoop.mapred.DeprecatedLzoTextInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'


my questions are:

in a few cases i succeeded getting output in hive for the
command:
select *
from test1; but doing the same on impala returs the below:

WARNINGS: This compression codec is currently unsupported:
com.hadoop.compression.lzo.LzoCodec

Backend 0:This compression codec is currently unsupported:
com.hadoop.compression.lzo.LzoCodec


2. in some other cases i got empty response on both hive
and
impala

Query: select * from test1

Returned 0 row(s) in 0.40s


Any suggestions on how to create the test1 table? or about the
errors?
will appreciate your quick response.

Thanks
RanG






This message may contain confidential and/or privileged
information.
If you are not the addressee or authorized to receive this on
behalf
of
the
addressee you must not use, copy, disclose or take action
based on
this
message or any information herein.
If you have received this message in error, please advise the
sender
immediately by reply email and delete this message. Thank you.

To unsubscribe from this group and stop receiving emails from
it,
send
an
email to impala-user...@cloudera.org.


--
-Ippokratis.

This message may contain confidential and/or privileged
information.
If you are not the addressee or authorized to receive this on
behalf
of
the
addressee you must not use, copy, disclose or take action based on
this
message or any information herein.
If you have received this message in error, please advise the
sender
immediately by reply email and delete this message. Thank you.

To unsubscribe from this group and stop receiving emails from it,
send
an
email to impala-user+unsubscribe@cloudera.org.


--
-Ippokratis.

To unsubscribe from this group and stop receiving emails from it,
send
an
email to impala-user+unsubscribe@cloudera.org.


This message may contain confidential and/or privileged information.
If you are not the addressee or authorized to receive this on behalf
of
the
addressee you must not use, copy, disclose or take action based on
this
message or any information herein.
If you have received this message in error, please advise the sender
immediately by reply email and delete this message. Thank you.

To unsubscribe from this group and stop receiving emails from it,
send
an
email to impala-user+unsubscribe@cloudera.org.


--
-Ippokratis.

To unsubscribe from this group and stop receiving emails from it, send
an
email to impala-user+unsubscribe@cloudera.org.


This message may contain confidential and/or privileged information.
If you are not the addressee or authorized to receive this on behalf of the
addressee you must not use, copy, disclose or take action based on this
message or any information herein.
If you have received this message in error, please advise the sender
immediately by reply email and delete this message. Thank you.

To unsubscribe from this group and stop receiving emails from it, send an
email to impala-user+unsubscribe@cloudera.org.


--
-Ippokratis.

To unsubscribe from this group and stop receiving emails from it, send an
email to impala-user+unsubscribe@cloudera.org.

This message may contain confidential and/or privileged information.
If you are not the addressee or authorized to receive this on behalf of
the addressee you must not use, copy, disclose or take action based on this
message or any information herein.
If you have received this message in error, please advise the sender
immediately by reply email and delete this message. Thank you.

To unsubscribe from this group and stop receiving emails from it, send an
email to impala-user+unsubscribe@cloudera.org.
To unsubscribe from this group and stop receiving emails from it, send an email to impala-user+unsubscribe@cloudera.org.

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 7 of 10 | next ›
Discussion Overview
groupimpala-user @
categorieshadoop
postedAug 20, '14 at 11:08p
activeSep 3, '14 at 6:05p
posts10
users5
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase