On 12 April 2013 06:05, Barry Becker wrote:
When Impala eventually supports writing to a table (like hive can do),
will it be necessary to run impala refresh to update the metadata?
We are wondering because we might want to write to a cached table, but we
do not want to have to so a lot of asynchronous refreshes. It would be nice
if impala could automatically update its own metadata without an explicit
refresh if it is the one doing the write. I realize this question is a bit
speculative since the write feature does not exist yet, but the answer
could influence things we are trying to design now.
-Barry
When Impala eventually supports writing to a table (like hive can do),
will it be necessary to run impala refresh to update the metadata?
We are wondering because we might want to write to a cached table, but we
do not want to have to so a lot of asynchronous refreshes. It would be nice
if impala could automatically update its own metadata without an explicit
refresh if it is the one doing the write. I realize this question is a bit
speculative since the write feature does not exist yet, but the answer
could influence things we are trying to design now.
-Barry
Impala already supports writing to a table via the INSERT statement. And
like you hope, if an Impala daemon runs an INSERT query, it will
automatically update its own metadata without need for a refresh.
Is that sufficient for your use case, or were you referring to another
mechanism that Hive has for writing data?
Best,
Henry
--
Henry Robinson
Software Engineer
Cloudera
415-994-6679
