Grokbase Groups Hive user July 2015
FAQ
Hi,
I've been experimenting with 'select *' and 'select * limit X' in beeline
and watching the hive-server2 log to understand when a M/R job is triggered
and when not. It seems like whenever I set a limit, the job is avoided,
but with no limit, it is run.

I found this param:
hive.limit.optimize.fetch.max

That defaults to 50,000 and as I understand it, whenever I set limit to
above that number, a job should be triggered. But I can set limit to
something very high (e.g. 10M) and no job runs.

If anyone has some insight into how this param is used or expected behavior
of the fetch optimization, would appreciate it.

This is on Hive 1.1 inside CDH5.4.

Thanks,
Adam

Search Discussions

Discussion Posts

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 1 of 6 | next ›
Discussion Overview
groupuser @
categorieshive, hadoop
postedJul 22, '15 at 1:37a
activeJul 22, '15 at 5:31p
posts6
users2
websitehive.apache.org

People

Translate

site design / logo © 2021 Grokbase