Hive vs. Impala counts

Marcel Kornacker

at Dec 31, 2012 at 7:26 pm

Can you give some more detail? What format was the data in? Also, we'd
need to see the info log.

On Sat, Dec 29, 2012 at 5:55 PM, Ram Krishnamurthy
wrote:

I loaded a file and ran a simple count in Impala and hive. I made sure
Impala catalog was refreshed.
The count(*) query yields different results. Hive has the correct result.
Any ideas?

Thanks, Ram

--

--

reply | permalink

Ram Krishnamurthy

at Dec 31, 2012 at 7:28 pm

⇧

it was a pipe delimited file.
Where would I get the info log from?

Ram

On Mon, Dec 31, 2012 at 2:26 PM, Marcel Kornacker wrote:

Can you give some more detail? What format was the data in? Also, we'd
need to see the info log.

On Sat, Dec 29, 2012 at 5:55 PM, Ram Krishnamurthy
wrote:

I loaded a file and ran a simple count in Impala and hive. I made sure
Impala catalog was refreshed.
The count(*) query yields different results. Hive has the correct result.
Any ideas?

Thanks, Ram

--

--

--
Thanks,
*Ram Krishnamurthy*
rkrishnamurthy@greenway-solutions.com
*Cell: 704-953-8125*

--

reply | permalink

Marcel Kornacker

at Dec 31, 2012 at 7:43 pm

⇧

On Mon, Dec 31, 2012 at 11:28 AM, Ram Krishnamurthy wrote:
it was a pipe delimited file.
Was it a single text file? Was it compressed?
Where would I get the info log from?
Typically it's /tmp/impalad.INFO
Ram

On Mon, Dec 31, 2012 at 2:26 PM, Marcel Kornacker wrote:

Can you give some more detail? What format was the data in? Also, we'd
need to see the info log.

On Sat, Dec 29, 2012 at 5:55 PM, Ram Krishnamurthy
wrote:

I loaded a file and ran a simple count in Impala and hive. I made sure
Impala catalog was refreshed.
The count(*) query yields different results. Hive has the correct
result.
Any ideas?

Thanks, Ram

--

--

--
Thanks,
Ram Krishnamurthy
rkrishnamurthy@greenway-solutions.com
Cell: 704-953-8125

--

--

reply | permalink

Ram Krishnamurthy

at Dec 31, 2012 at 7:47 pm

⇧

Single test file and non compressed.. i can provide the logs as well.

What email address or what part of the log...

Ram

On Mon, Dec 31, 2012 at 2:43 PM, Marcel Kornacker wrote:

On Mon, Dec 31, 2012 at 11:28 AM, Ram Krishnamurthy
wrote:

it was a pipe delimited file.
Was it a single text file? Was it compressed?
Where would I get the info log from?
Typically it's /tmp/impalad.INFO
Ram

On Mon, Dec 31, 2012 at 2:26 PM, Marcel Kornacker <marcel@cloudera.com>
wrote:

Can you give some more detail? What format was the data in? Also, we'd
need to see the info log.

On Sat, Dec 29, 2012 at 5:55 PM, Ram Krishnamurthy
wrote:

I loaded a file and ran a simple count in Impala and hive. I made sure
Impala catalog was refreshed.
The count(*) query yields different results. Hive has the correct
result.
Any ideas?

Thanks, Ram

--

--

--
Thanks,
Ram Krishnamurthy
rkrishnamurthy@greenway-solutions.com
Cell: 704-953-8125

--

--

--
Thanks,
*Ram Krishnamurthy*
rkrishnamurthy@greenway-solutions.com
*Cell: 704-953-8125*

--

reply | permalink

Nong

at Jan 1, 2013 at 10:00 pm

⇧

Ram,

You can email the log to nong@cloudera.com. Can you also include the
output
of running 'describe extended <table>' in hive?

What are the results from hive and impala on this query?

Thanks
Nong

On Monday, December 31, 2012 11:47:15 AM UTC-8, Ram Krishnamurthy wrote:

Single test file and non compressed.. i can provide the logs as well.

What email address or what part of the log...

Ram

On Mon, Dec 31, 2012 at 2:43 PM, Marcel Kornacker <mar...@cloudera.com<javascript:>

wrote:
On Mon, Dec 31, 2012 at 11:28 AM, Ram Krishnamurthy
<rkrishn...@greenway-solutions.com <javascript:>> wrote:

it was a pipe delimited file.
Was it a single text file? Was it compressed?
Where would I get the info log from?
Typically it's /tmp/impalad.INFO
Ram

On Mon, Dec 31, 2012 at 2:26 PM, Marcel Kornacker <mar...@cloudera.com<javascript:>

wrote:

Can you give some more detail? What format was the data in? Also, we'd
need to see the info log.

On Sat, Dec 29, 2012 at 5:55 PM, Ram Krishnamurthy
<rkrishn...@greenway-solutions.com <javascript:>> wrote:

I loaded a file and ran a simple count in Impala and hive. I made

sure

Impala catalog was refreshed.
The count(*) query yields different results. Hive has the correct
result.
Any ideas?

Thanks, Ram

--

--

--
Thanks,
Ram Krishnamurthy
rkrishn...@greenway-solutions.com <javascript:>
Cell: 704-953-8125

--

--

--
Thanks,
*Ram Krishnamurthy*
rkrishnamurthy@greenway-solutions.com <javascript:>
*Cell: 704-953-8125*

--

reply | permalink

Marcel Kornacker

at Jan 1, 2013 at 11:42 pm

⇧

I had already been in touch with him.

On Tue, Jan 1, 2013 at 2:00 PM, Nong wrote:
Ram,

You can email the log to nong@cloudera.com. Can you also include the output
of running 'describe extended <table>' in hive?

What are the results from hive and impala on this query?

Thanks
Nong

On Monday, December 31, 2012 11:47:15 AM UTC-8, Ram Krishnamurthy wrote:

Single test file and non compressed.. i can provide the logs as well.

What email address or what part of the log...

Ram

On Mon, Dec 31, 2012 at 2:43 PM, Marcel Kornacker <mar...@cloudera.com>
wrote:

On Mon, Dec 31, 2012 at 11:28 AM, Ram Krishnamurthy
wrote:

it was a pipe delimited file.
Was it a single text file? Was it compressed?
Where would I get the info log from?
Typically it's /tmp/impalad.INFO
Ram

On Mon, Dec 31, 2012 at 2:26 PM, Marcel Kornacker <mar...@cloudera.com>
wrote:

Can you give some more detail? What format was the data in? Also, we'd
need to see the info log.

On Sat, Dec 29, 2012 at 5:55 PM, Ram Krishnamurthy
wrote:

I loaded a file and ran a simple count in Impala and hive. I made
sure
Impala catalog was refreshed.
The count(*) query yields different results. Hive has the correct
result.
Any ideas?

Thanks, Ram

--

--

--
Thanks,
Ram Krishnamurthy
rkrishn...@greenway-solutions.com
Cell: 704-953-8125

--

--

--
Thanks,
Ram Krishnamurthy
rkrishnamurthy@greenway-solutions.com
Cell: 704-953-8125

--

reply | permalink

Ram Krishnamurthy

at Jan 2, 2013 at 12:07 am

⇧

Marcel & Nong,

I will send the details out tomorrow morning as requested.

Thanks, Ram

On Tue, Jan 1, 2013 at 6:42 PM, Marcel Kornacker wrote:

I had already been in touch with him.

On Tue, Jan 1, 2013 at 2:00 PM, Nong wrote:
Ram,

You can email the log to nong@cloudera.com. Can you also include the output
of running 'describe extended <table>' in hive?

What are the results from hive and impala on this query?

Thanks
Nong

On Monday, December 31, 2012 11:47:15 AM UTC-8, Ram Krishnamurthy wrote:

Single test file and non compressed.. i can provide the logs as well.

What email address or what part of the log...

Ram

On Mon, Dec 31, 2012 at 2:43 PM, Marcel Kornacker <mar...@cloudera.com>
wrote:

On Mon, Dec 31, 2012 at 11:28 AM, Ram Krishnamurthy
wrote:

it was a pipe delimited file.
Was it a single text file? Was it compressed?
Where would I get the info log from?
Typically it's /tmp/impalad.INFO
Ram

On Mon, Dec 31, 2012 at 2:26 PM, Marcel Kornacker <

mar...@cloudera.com>

wrote:

Can you give some more detail? What format was the data in? Also,

we'd

need to see the info log.

On Sat, Dec 29, 2012 at 5:55 PM, Ram Krishnamurthy
wrote:

I loaded a file and ran a simple count in Impala and hive. I made
sure
Impala catalog was refreshed.
The count(*) query yields different results. Hive has the correct
result.
Any ideas?

Thanks, Ram

--

--

--
Thanks,
Ram Krishnamurthy
rkrishn...@greenway-solutions.com
Cell: 704-953-8125

--

--

--
Thanks,
Ram Krishnamurthy
rkrishnamurthy@greenway-solutions.com
Cell: 704-953-8125

--

--
Thanks,
*Ram Krishnamurthy*
rkrishnamurthy@greenway-solutions.com
*Cell: 704-953-8125*

--

reply | permalink

Stefan

at Jun 13, 2013 at 8:20 pm

⇧

Are there any news on this topic?
I have the same issue. For every file that I add to a hive table I have 2
additional datasets in the impala count(*) result.
I have now added 3 files and the impala count result has 6 datasets too
many.

Best, Stefan

On Wednesday, January 2, 2013 1:07:38 AM UTC+1, Ram Krishnamurthy wrote:

Marcel & Nong,

I will send the details out tomorrow morning as requested.

Thanks, Ram

On Tue, Jan 1, 2013 at 6:42 PM, Marcel Kornacker <mar...@cloudera.com<javascript:>

wrote:
I had already been in touch with him.

On Tue, Jan 1, 2013 at 2:00 PM, Nong <no...@cloudera.com <javascript:>>
wrote:

Ram,

You can email the log to no...@cloudera.com <javascript:>. Can you
also include the output
of running 'describe extended <table>' in hive?

What are the results from hive and impala on this query?

Thanks
Nong

On Monday, December 31, 2012 11:47:15 AM UTC-8, Ram Krishnamurthy wrote:

Single test file and non compressed.. i can provide the logs as well.

What email address or what part of the log...

Ram

On Mon, Dec 31, 2012 at 2:43 PM, Marcel Kornacker <mar...@cloudera.com
wrote:

On Mon, Dec 31, 2012 at 11:28 AM, Ram Krishnamurthy
wrote:

it was a pipe delimited file.
Was it a single text file? Was it compressed?
Where would I get the info log from?
Typically it's /tmp/impalad.INFO
Ram

On Mon, Dec 31, 2012 at 2:26 PM, Marcel Kornacker <

mar...@cloudera.com>

wrote:

Can you give some more detail? What format was the data in? Also,

we'd

need to see the info log.

On Sat, Dec 29, 2012 at 5:55 PM, Ram Krishnamurthy
wrote:

I loaded a file and ran a simple count in Impala and hive. I made
sure
Impala catalog was refreshed.
The count(*) query yields different results. Hive has the correct
result.
Any ideas?

Thanks, Ram

--

--

--
Thanks,
Ram Krishnamurthy
rkrishn...@greenway-solutions.com
Cell: 704-953-8125

--

--

--
Thanks,
Ram Krishnamurthy
rkrishn...@greenway-solutions.com <javascript:>
Cell: 704-953-8125

--

--
Thanks,
*Ram Krishnamurthy*
rkrishnamurthy@greenway-solutions.com <javascript:>
*Cell: 704-953-8125*

reply | permalink

group	impala-user @
categories	hadoop
posted	Dec 30, '12 at 1:55a
active	Jun 13, '13 at 8:20p
posts	9
users	4
website	cloudera.com
irc	#hadoop

responses	expanded	Hotkey: s
font	variable	Hotkey: f
user style	avatars	Hotkey: a

Hive vs. Impala counts

Search Discussions

Related Discussions

4 users in discussion