Grokbase Groups Hive user July 2010
FAQ
I read on the VLBD hive paper "Hive - A Warehousing Solution Over a
Map-Reduce Framework"
that Partitions could have different schemas : (section 3.1 MetaStore) "
Partition - Each partition can have its own columns
and SerDe and storage information. This can be used
in the future to support schema evolution in a Hive
warehouse.
"

but the API:
http://hadoop.apache.org/hive/docs/r0.5.0/api/

only lists getSchema() for Table, and Partition does not have a
separate getSchema().

is the schema evolution feature really there?

Thanks
Yang

Search Discussions

  • Paul Yang at Jul 1, 2010 at 11:58 pm
    There exists structures for supporting this in the metastore, but that feature isn't in Hive yet. For example, although the metadata for partitions include its own set of columns, parts of the code in the query processor still read from table level metadata.

    Some evolution can occur in the form of adding columns to a table.

    -----Original Message-----
    From: Yang
    Sent: Thursday, July 01, 2010 4:43 PM
    To: hive-user@hadoop.apache.org
    Subject: Schema evolution?

    I read on the VLBD hive paper "Hive - A Warehousing Solution Over a
    Map-Reduce Framework"
    that Partitions could have different schemas : (section 3.1 MetaStore) "
    Partition - Each partition can have its own columns
    and SerDe and storage information. This can be used
    in the future to support schema evolution in a Hive
    warehouse.
    "

    but the API:
    http://hadoop.apache.org/hive/docs/r0.5.0/api/

    only lists getSchema() for Table, and Partition does not have a
    separate getSchema().

    is the schema evolution feature really there?

    Thanks
    Yang
  • Yang at Jul 2, 2010 at 4:30 am
    Paul:

    thanks.
    currently I do not need this feature from Hive QL, just need it in metastore.

    you said "There exists structures for supporting this in the
    metastore", could you please give more details? I suppose the
    interface to metastore is basically classes like Table, Partition, but
    in the Partition API
    http://hadoop.apache.org/hive/docs/r0.5.0/api/org/apache/hadoop/hive/metastore/api/Partition.html
    I don't see any reference to schema,

    Yang
    On Thu, Jul 1, 2010 at 4:56 PM, Paul Yang wrote:
    There exists structures for supporting this in the metastore, but that feature isn't in Hive yet. For example, although the metadata for partitions include its own set of columns, parts of the code in the query processor still read from table level metadata.

    Some evolution can occur in the form of adding columns to a table.

    -----Original Message-----
    From: Yang
    Sent: Thursday, July 01, 2010 4:43 PM
    To: hive-user@hadoop.apache.org
    Subject: Schema evolution?

    I read on the VLBD hive paper "Hive - A Warehousing Solution Over a
    Map-Reduce    Framework"
    that Partitions could have different schemas : (section 3.1 MetaStore) "
    Partition - Each partition can have its own columns
    and SerDe and storage information. This can be used
    in the future to support schema evolution in a Hive
    warehouse.
    "

    but the API:
    http://hadoop.apache.org/hive/docs/r0.5.0/api/

    only lists getSchema() for Table, and Partition does not have a
    separate getSchema().

    is the schema evolution feature really there?

    Thanks
    Yang

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshive, hadoop
postedJul 1, '10 at 11:43p
activeJul 2, '10 at 4:30a
posts3
users2
websitehive.apache.org

2 users in discussion

Yang: 2 posts Paul Yang: 1 post

People

Translate

site design / logo © 2021 Grokbase