Summary: metastore schema improvement for adding partition to Hive table
Issue Type: Bug
Reporter: Chu Tong
One of the performance bottlenecks for adding a partition in Hive table and the query takes most of the time in this process is:
SELECT A0.PART_NAME FROM PARTITIONS A0 LEFT OUTER JOIN TBLS B0 ON A0.TBL_ID = B0.TBL_ID LEFT OUTER JOIN DBS C0 ON B0.DB_ID = C0.DB_ID WHERE B0.TBL_NAME = @P0 AND C0."NAME" = @P1 AND A0.PART_NAME = @P2
This query joins partition table with table table and database table in Hive metastore and it becomes slow when these tables are big.
A viable way to optimize this is the de-normalize the partition table.
This message was sent by Atlassian JIRA