Grokbase Groups Hive user May 2011
FAQ
Here is more info about the issues I have been facing.



1. I rebuilt the hbase handler for both Hive 0.6.0 and 0.7.0 by
changing the appropriate settings below to generate the necessary
handler files;

2. None of them was working and both made the hbase master "out of
memory" (see attached HMaster log) after firing following hiveSQL
command ( from hbase hive integration site
http://wiki.apache.org/hadoop/Hive/GettingStarted ):



CREATE TABLE hbase_table_1(key int, value string) STORED BY
'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES
("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES
("hbase.table.name" = "xyz");



The 'OutOfMemoryError' caused the hamster service unavailable, which is
the final message we saw from the hive log file.



My question:

1. Is there any documented handler rebuild process anywhere to
integrate HBase 0.20.3/Zookeeper3.2.2/Hive0.6.0 (0.7.0)?

2. What is the best approach to set up Hive environment for my
environment? Is it better to install CDH3 (from documentation, the
initial release of CDH3 compatible with hadoop0.20.2 and hbase0.20.3)
without patch and copy the hive directory to my prod environment?



Thanks



Michael Sun



From: Sun, Michael
Sent: Friday, May 06, 2011 5:16 PM
To: 'user@hive.apache.org'
Subject: hive/hbase integration with rebuilt handler



Hi, can any of you help to solve the error in the attachment when I
tried to integrate Hive and HBase?



Environment:



Hive version: 0.7.0

HBase version: 0.20.3

Hadoop version: 0.20.2



Steps I did to rebuild hiver-hbase-handler.jar:



1. Checkout source from
http://svn.apache.org/repos/asf/hive/branches/branch-0.7 (and
branch-0.6)

2. Modify $checkout_root/build.properties to change the hadoop
version to 0.20.2

3. Modify $ checkout _root/ivy/library.properties and
$root/ivy/ivysetting.xml to use my local jars in hbase/hadoop
installation



The rebuild process generated a new handler called
hive-hbase-handler-0.7.1-SNAPSHOT.jar.



Thanks



Michael Sun



This message, including any attachments, is the property of Sears Holdings Corporation and/or one of its subsidiaries. It is confidential and may contain proprietary or legally privileged information. If you are not the intended recipient, please delete it without reading the contents. Thank you.

Search Discussions

  • Sun, Michael at May 9, 2011 at 5:55 am
    Here is more info about the issues I have been facing.



    1. I rebuilt the hbase handler for both Hive 0.6.0 and 0.7.0 by
    changing the appropriate settings below to generate the necessary
    handler files;

    2. None of them was working and both made the hbase master "out of
    memory" (see attached HMaster log) after firing following hiveSQL
    command ( from hbase hive integration site
    http://wiki.apache.org/hadoop/Hive/GettingStarted ):



    CREATE TABLE hbase_table_1(key int, value string) STORED BY
    'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES
    ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES
    ("hbase.table.name" = "xyz");



    The 'OutOfMemoryError' caused the hamster service unavailable, which is
    the final message we saw from the hive log file.



    My question:

    1. Is there any documented handler rebuild process anywhere to
    integrate HBase 0.20.3/Zookeeper3.2.2/Hive0.6.0 (0.7.0)?

    2. What is the best approach to set up Hive environment for my
    environment? Is it better to install CDH3 (from documentation, the
    initial release of CDH3 compatible with hadoop0.20.2 and hbase0.20.3)
    without patch and copy the hive directory to my prod environment?



    Thanks



    Michael Sun



    From: Sun, Michael
    Sent: Friday, May 06, 2011 5:16 PM
    To: 'user@hive.apache.org'
    Subject: hive/hbase integration with rebuilt handler



    Hi, can any of you help to solve the error in the attachment when I
    tried to integrate Hive and HBase?



    Environment:



    Hive version: 0.7.0

    HBase version: 0.20.3

    Hadoop version: 0.20.2



    Steps I did to rebuild hiver-hbase-handler.jar:



    1. Checkout source from
    http://svn.apache.org/repos/asf/hive/branches/branch-0.7 (and
    branch-0.6)

    2. Modify $checkout_root/build.properties to change the hadoop
    version to 0.20.2

    3. Modify $ checkout _root/ivy/library.properties and
    $root/ivy/ivysetting.xml to use my local jars in hbase/hadoop
    installation



    The rebuild process generated a new handler called
    hive-hbase-handler-0.7.1-SNAPSHOT.jar.



    Thanks



    Michael Sun

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshive, hadoop
postedMay 9, '11 at 5:47a
activeMay 9, '11 at 5:55a
posts2
users1
websitehive.apache.org

1 user in discussion

Sun, Michael: 2 posts

People

Translate

site design / logo © 2022 Grokbase