Grokbase Groups Hive user March 2010
FAQ
Hi,
I encountered Parse Error creating table in CLI:

[root@tyu-linux hive-0.5.0-bin]# bin/hive
Hive history file=/tmp/root/hive_job_log_root_201003240832_1752130264.txt
hive> SHOW TABLES;
OK
Time taken: 8.09 seconds
hive> CREATE EXTERNAL TABLE users(key string, state string, country string,
country_id int)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' >
WITH SERDEPROPERTIES ( >
"hbase.key.type" = "string", >
"hbase.columns.mapping" = "info:state,info:country,info:country_id", >
"hbase.table.name" = "users" >
);
FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table file
format specification

*Here is the tail of hive.log:*
2010-03-24 08:32:07,937 ERROR DataNucleus.Plugin
(Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires
"org.eclipse.text" but it cannot be resolved.
2010-03-24 08:34:18,337 ERROR ql.Driver (SessionState.java:printError(248))
- FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table
file format specification

org.apache.hadoop.hive.ql.parse.ParseException: line 2:0 cannot recognize
input 'STORED' in table file format specification

at
org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:357)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:267)
at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:320)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312)
at
org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123)
at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

If you know how to fix my query, please share.

Search Discussions

  • Carl Steinbach at Mar 25, 2010 at 6:26 pm
    Hi Ted,

    It looks like your copy of Hive does not have the changes that implemented
    support for integration with HBase. HBase support was committed to trunk on
    March 12th, and currently it is only available on trunk. In order to use it
    you need to checkout (or update) the source from the svn repository.
    Instructions describing how to do this are on the Hive wiki.

    Carl
    On Thu, Mar 25, 2010 at 9:29 AM, Ted Yu wrote:

    Hi,
    I encountered Parse Error creating table in CLI:

    [root@tyu-linux hive-0.5.0-bin]# bin/hive
    Hive history file=/tmp/root/hive_job_log_root_201003240832_1752130264.txt
    hive> SHOW TABLES;
    OK
    Time taken: 8.09 seconds
    hive> CREATE EXTERNAL TABLE users(key string, state string, country string,
    country_id int)
    STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'

    WITH SERDEPROPERTIES (

    "hbase.key.type" = "string",

    "hbase.columns.mapping" = "info:state,info:country,info:country_id",

    "hbase.table.name" = "users"

    );
    FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table file
    format specification

    *Here is the tail of hive.log:*
    2010-03-24 08:32:07,937 ERROR DataNucleus.Plugin
    (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires
    "org.eclipse.text" but it cannot be resolved.
    2010-03-24 08:34:18,337 ERROR ql.Driver (SessionState.java:printError(248))
    - FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table
    file format specification

    org.apache.hadoop.hive.ql.parse.ParseException: line 2:0 cannot recognize
    input 'STORED' in table file format specification

    at
    org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:357)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:267)
    at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:320)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312)
    at
    org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123)
    at
    org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at

    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at

    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

    If you know how to fix my query, please share.
  • Ted Yu at Mar 25, 2010 at 11:16 pm
    I got pass that error when I use CLI from trunk.

    I copied hbase-site.xml from hbase master machine to trunk/dist/conf.

    When I issue the following command in CLI:
    CREATE EXTERNAL TABLE ruletable(key string, exactmatch_cat string, lpm_cat
    int)
    STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
    WITH SERDEPROPERTIES (
    "hbase.key.type" = "string",
    "hbase.columns.mapping" = "exactmatch_1.0:category,lpm_1.0:category",
    "hbase.table.name" = "ruletable"
    );

    I get:

    2010-03-25 15:49:52,557 WARN client.HConnectionManager$TableServers
    (HConnectionManager.java:tableExists(411)) - Testing for table existence
    threw exception
    org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact
    region server 10.10.31.143:60020 for region .META.,,1, row '', but failed
    after 10 attempts.
    Exceptions:
    java.io.IOException: Call to /10.10.31.143:60020 failed on local exception:
    java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020 failed on local exception:
    java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020 failed on local exception:
    java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020 failed on local exception:
    java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020 failed on local exception:
    java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020 failed on local exception:
    java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020 failed on local exception:
    java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020 failed on local exception:
    java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020 failed on local exception:
    java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020 failed on local exception:
    java.io.EOFException

    at
    org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:1048)
    at
    org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:55)
    at
    org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:28)
    at
    org.apache.hadoop.hbase.client.HConnectionManager$TableServers.listTables(HConnectionManager.java:454)
    at
    org.apache.hadoop.hbase.client.HConnectionManager$TableServers.tableExists(HConnectionManager.java:404)
    at
    org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:113)
    at
    org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:100)
    at
    org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:149)
    at
    org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:280)

    However, I don't see any exception in hbase-hbaseadmin-regionserver.log on
    10.10.31.143 within the past 4 hours.
    No exception in hbase-hbaseadmin-master.log on hbase master machine either.

    Please comment.
    On Thu, Mar 25, 2010 at 11:22 AM, Carl Steinbach wrote:

    Hi Ted,

    It looks like your copy of Hive does not have the changes that implemented
    support for integration with HBase. HBase support was committed to trunk on
    March 12th, and currently it is only available on trunk. In order to use it
    you need to checkout (or update) the source from the svn repository.
    Instructions describing how to do this are on the Hive wiki.

    Carl

    On Thu, Mar 25, 2010 at 9:29 AM, Ted Yu wrote:

    Hi,
    I encountered Parse Error creating table in CLI:

    [root@tyu-linux hive-0.5.0-bin]# bin/hive
    Hive history file=/tmp/root/hive_job_log_root_201003240832_1752130264.txt
    hive> SHOW TABLES;
    OK
    Time taken: 8.09 seconds
    hive> CREATE EXTERNAL TABLE users(key string, state string, country
    string,
    country_id int)
    STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'

    WITH SERDEPROPERTIES (

    "hbase.key.type" = "string",

    "hbase.columns.mapping" = "info:state,info:country,info:country_id",

    "hbase.table.name" = "users"

    );
    FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table
    file
    format specification

    *Here is the tail of hive.log:*
    2010-03-24 08:32:07,937 ERROR DataNucleus.Plugin
    (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires
    "org.eclipse.text" but it cannot be resolved.
    2010-03-24 08:34:18,337 ERROR ql.Driver
    (SessionState.java:printError(248))
    - FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table
    file format specification

    org.apache.hadoop.hive.ql.parse.ParseException: line 2:0 cannot recognize
    input 'STORED' in table file format specification

    at
    org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:357)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:267)
    at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:320)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312)
    at
    org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123)
    at
    org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at

    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at

    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

    If you know how to fix my query, please share.
  • John Sichi at Mar 25, 2010 at 11:22 pm
    Don't copy hbase-site.xml; you don't want to set up a new HBase cluster. Just set your hive conf to point to your existing HBase master:

    hbase.master=hbase.yoyodyne.com:60000

    http://wiki.apache.org/hadoop/Hive/HBaseIntegration#Usage

    JVS

    On Mar 25, 2010, at 4:16 PM, Ted Yu wrote:

    I got pass that error when I use CLI from trunk.

    I copied hbase-site.xml from hbase master machine to trunk/dist/conf.

    When I issue the following command in CLI:
    CREATE EXTERNAL TABLE ruletable(key string, exactmatch_cat string, lpm_cat int)
    STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
    WITH SERDEPROPERTIES (
    "hbase.key.type" = "string",
    "hbase.columns.mapping" = "exactmatch_1.0:category,lpm_1.0:category",
    "hbase.table.name<http://hbase.table.name/>" = "ruletable"
    );

    I get:

    2010-03-25 15:49:52,557 WARN client.HConnectionManager$TableServers (HConnectionManager.java:tableExists(411)) - Testing for table existence threw exception
    org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact region server 10.10.31.143:60020<http://10.10.31.143:60020/> for region .META.,,1, row '', but failed after 10 attempts.
    Exceptions:
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException

    at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:1048)
    at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:55)
    at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:28)
    at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.listTables(HConnectionManager.java:454)
    at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.tableExists(HConnectionManager.java:404)
    at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:113)
    at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:100)
    at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:149)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:280)

    However, I don't see any exception in hbase-hbaseadmin-regionserver.log on 10.10.31.143 within the past 4 hours.
    No exception in hbase-hbaseadmin-master.log on hbase master machine either.

    Please comment.

    On Thu, Mar 25, 2010 at 11:22 AM, Carl Steinbach wrote:
    Hi Ted,

    It looks like your copy of Hive does not have the changes that implemented support for integration with HBase. HBase support was committed to trunk on March 12th, and currently it is only available on trunk. In order to use it you need to checkout (or update) the source from the svn repository. Instructions describing how to do this are on the Hive wiki.

    Carl


    On Thu, Mar 25, 2010 at 9:29 AM, Ted Yu wrote:
    Hi,
    I encountered Parse Error creating table in CLI:

    [root@tyu-linux hive-0.5.0-bin]# bin/hive
    Hive history file=/tmp/root/hive_job_log_root_201003240832_1752130264.txt
    hive> SHOW TABLES;
    OK
    Time taken: 8.09 seconds
    hive> CREATE EXTERNAL TABLE users(key string, state string, country string,
    country_id int)
    STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' >
    WITH SERDEPROPERTIES ( >
    "hbase.key.type" = "string", >
    "hbase.columns.mapping" = "info:state,info:country,info:country_id", >
    "hbase.table.name<http://hbase.table.name/>" = "users" >
    );
    FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table file
    format specification

    Here is the tail of hive.log:
    2010-03-24 08:32:07,937 ERROR DataNucleus.Plugin
    (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires
    "org.eclipse.text" but it cannot be resolved.
    2010-03-24 08:34:18,337 ERROR ql.Driver (SessionState.java:printError(248))
    - FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table
    file format specification

    org.apache.hadoop.hive.ql.parse.ParseException: line 2:0 cannot recognize
    input 'STORED' in table file format specification

    at
    org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:357)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:267)
    at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:320)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312)
    at
    org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123)
    at
    org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

    If you know how to fix my query, please share.
  • John Sichi at Mar 26, 2010 at 1:09 am
    Hmmm, what kind of HBase cluster are you running (standalone, pseudo-distributed, or distributed)? If distributed, you may need to pass zookeeper connection info instead of hbase.master; I haven't tested out distributed yet but will be soon.

    JVS

    On Mar 25, 2010, at 4:43 PM, Ted Yu wrote:

    I removed hbase-site.xml

    [root@tyu-linux dist]# bin/hive -hiveconf hbase.master=snv-it-lin-006:60000
    Hive history file=/tmp/root/hive_job_log_root_201003251614_1493474415.txt
    hive> CREATE EXTERNAL TABLE ruletable(key string, exactmatch_cat string, lpm_cat int)
    STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
    WITH SERDEPROPERTIES (
    "hbase.key.type" = "string",
    "hbase.columns.mapping" = "exactmatch_1.0:category,lpm_1.0:category",
    "hbase.table.name<http://hbase.table.name/>" = "ruletable"
    );
    FAILED: Error in metadata: MetaException(message:org.apache.hadoop.hbase.MasterNotRunningException
    at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getMaster(HConnectionManager.java:374)
    at org.apache.hadoop.hbase.client.HBaseAdmin.(HBaseStorageHandler.java:63)
    at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:149)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:280)
    at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:320)
    at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:1445)
    at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:124)
    at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
    at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:630)
    at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:504)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:382)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:138)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:197)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:303)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
    )

    But http://snv-it-lin-006:60010/master.jsp shows everything is normal.
    And hbase.master.port is 60000 in hbase-default.xml

    Some suggestions ?

    Thanks

    On Thu, Mar 25, 2010 at 4:21 PM, John Sichi wrote:
    Don't copy hbase-site.xml; you don't want to set up a new HBase cluster. Just set your hive conf to point to your existing HBase master:

    hbase.master=hbase.yoyodyne.com:60000<http://hbase.yoyodyne.com:60000/>

    http://wiki.apache.org/hadoop/Hive/HBaseIntegration#Usage

    JVS

    On Mar 25, 2010, at 4:16 PM, Ted Yu wrote:

    I got pass that error when I use CLI from trunk.

    I copied hbase-site.xml from hbase master machine to trunk/dist/conf.

    When I issue the following command in CLI:
    CREATE EXTERNAL TABLE ruletable(key string, exactmatch_cat string, lpm_cat int)
    STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
    WITH SERDEPROPERTIES (
    "hbase.key.type" = "string",
    "hbase.columns.mapping" = "exactmatch_1.0:category,lpm_1.0:category",
    "hbase.table.name<http://hbase.table.name/>" = "ruletable"
    );

    I get:

    2010-03-25 15:49:52,557 WARN client.HConnectionManager$TableServers (HConnectionManager.java:tableExists(411)) - Testing for table existence threw exception
    org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact region server 10.10.31.143:60020<http://10.10.31.143:60020/> for region .META.,,1, row '', but failed after 10 attempts.
    Exceptions:
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException

    at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:1048)
    at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:55)
    at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:28)
    at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.listTables(HConnectionManager.java:454)
    at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.tableExists(HConnectionManager.java:404)
    at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:113)
    at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:100)
    at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:149)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:280)

    However, I don't see any exception in hbase-hbaseadmin-regionserver.log on 10.10.31.143 within the past 4 hours.
    No exception in hbase-hbaseadmin-master.log on hbase master machine either.

    Please comment.

    On Thu, Mar 25, 2010 at 11:22 AM, Carl Steinbach wrote:
    Hi Ted,

    It looks like your copy of Hive does not have the changes that implemented support for integration with HBase. HBase support was committed to trunk on March 12th, and currently it is only available on trunk. In order to use it you need to checkout (or update) the source from the svn repository. Instructions describing how to do this are on the Hive wiki.

    Carl


    On Thu, Mar 25, 2010 at 9:29 AM, Ted Yu wrote:
    Hi,
    I encountered Parse Error creating table in CLI:

    [root@tyu-linux hive-0.5.0-bin]# bin/hive
    Hive history file=/tmp/root/hive_job_log_root_201003240832_1752130264.txt
    hive> SHOW TABLES;
    OK
    Time taken: 8.09 seconds
    hive> CREATE EXTERNAL TABLE users(key string, state string, country string,
    country_id int)
    STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' >
    WITH SERDEPROPERTIES ( >
    "hbase.key.type" = "string", >
    "hbase.columns.mapping" = "info:state,info:country,info:country_id", >
    "hbase.table.name<http://hbase.table.name/>" = "users" >
    );
    FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table file
    format specification

    Here is the tail of hive.log:
    2010-03-24 08:32:07,937 ERROR DataNucleus.Plugin
    (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires
    "org.eclipse.text" but it cannot be resolved.
    2010-03-24 08:34:18,337 ERROR ql.Driver (SessionState.java:printError(248))
    - FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table
    file format specification

    org.apache.hadoop.hive.ql.parse.ParseException: line 2:0 cannot recognize
    input 'STORED' in table file format specification

    at
    org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:357)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:267)
    at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:320)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312)
    at
    org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123)
    at
    org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

    If you know how to fix my query, please share.
  • John Sichi at Mar 26, 2010 at 4:06 am
    I'll update that page once I get our test cluster set up.

    In the meantime, if you figure it out, please post to this mailing list.

    Try hbase.zookeeper.quorum plus hbase.zookeeper.property.clientPort.

    Hbase mailing list says those are the minimum required.

    JVS

    On Mar 25, 2010, at 7:12 PM, "Ted Yu" wrote:

    We use distributed mode because the cluster would be deployed to production.

    I didn't find zookeeper commandline parameter on <http://wiki.apache.org/hadoop/Hive/HBaseIntegration#Usage> http://wiki.apache.org/hadoop/Hive/HBaseIntegration#Usage
    Do I need to use an entry in hbase-site.xml ?

    Our hbase cluster is based on 0.20.1
    But I used hbase client 0.20.3 against it before and didn't see RetriesExhaustedException.

    Thanks

    On Thu, Mar 25, 2010 at 6:07 PM, John Sichi wrote:
    Hmmm, what kind of HBase cluster are you running (standalone, pseudo-distributed, or distributed)? If distributed, you may need to pass zookeeper connection info instead of hbase.master; I haven't tested out distributed yet but will be soon.

    JVS

    On Mar 25, 2010, at 4:43 PM, Ted Yu wrote:

    I removed hbase-site.xml

    [root@tyu-linux dist]# bin/hive -hiveconf hbase.master=snv-it-lin-006:60000
    Hive history file=/tmp/root/hive_job_log_root_201003251614_1493474415.txt
    hive> CREATE EXTERNAL TABLE ruletable(key string, exactmatch_cat string, lpm_cat int)
    STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
    WITH SERDEPROPERTIES (
    "hbase.key.type" = "string",
    "hbase.columns.mapping" = "exactmatch_1.0:category,lpm_1.0:category",
    "<http://hbase.table.name/>hbase.table.name<http://hbase.table.name>" = "ruletable"
    );
    FAILED: Error in metadata: MetaException(message:org.apache.hadoop.hbase.MasterNotRunningException
    at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getMaster(HConnectionManager.java:374)
    at org.apache.hadoop.hbase.client.HBaseAdmin.(HBaseStorageHandler.java:63)
    at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:149)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:280)
    at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:320)
    at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:1445)
    at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:124)
    at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
    at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:630)
    at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:504)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:382)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:138)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:197)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:303)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
    )

    But <http://snv-it-lin-006:60010/master.jsp> http://snv-it-lin-006:60010/master.jsp shows everything is normal.
    And hbase.master.port is 60000 in hbase-default.xml

    Some suggestions ?

    Thanks

    On Thu, Mar 25, 2010 at 4:21 PM, John Sichi wrote:
    Don't copy hbase-site.xml; you don't want to set up a new HBase cluster. Just set your hive conf to point to your existing HBase master:

    hbase.master=hbase.yoyodyne.com:60000<http://hbase.yoyodyne.com:60000/>

    <http://wiki.apache.org/hadoop/Hive/HBaseIntegration#Usage>http://wiki.apache.org/hadoop/Hive/HBaseIntegration#Usage

    JVS

    On Mar 25, 2010, at 4:16 PM, Ted Yu wrote:

    I got pass that error when I use CLI from trunk.

    I copied hbase-site.xml from hbase master machine to trunk/dist/conf.

    When I issue the following command in CLI:
    CREATE EXTERNAL TABLE ruletable(key string, exactmatch_cat string, lpm_cat int)
    STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
    WITH SERDEPROPERTIES (
    "hbase.key.type" = "string",
    "hbase.columns.mapping" = "exactmatch_1.0:category,lpm_1.0:category",
    "<http://hbase.table.name/>hbase.table.name<http://hbase.table.name>" = "ruletable"
    );

    I get:

    2010-03-25 15:49:52,557 WARN client.HConnectionManager$TableServers (HConnectionManager.java:tableExists(411)) - Testing for table existence threw exception
    org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact region server 10.10.31.143:60020<http://10.10.31.143:60020/> for region .META.,,1, row '', but failed after 10 attempts.
    Exceptions:
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException

    at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:1048)
    at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:55)
    at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:28)
    at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.listTables(HConnectionManager.java:454)
    at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.tableExists(HConnectionManager.java:404)
    at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:113)
    at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:100)
    at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:149)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:280)

    However, I don't see any exception in hbase-hbaseadmin-regionserver.log on 10.10.31.143 within the past 4 hours.
    No exception in hbase-hbaseadmin-master.log on hbase master machine either.

    Please comment.

    On Thu, Mar 25, 2010 at 11:22 AM, Carl Steinbach wrote:
    Hi Ted,

    It looks like your copy of Hive does not have the changes that implemented support for integration with HBase. HBase support was committed to trunk on March 12th, and currently it is only available on trunk. In order to use it you need to checkout (or update) the source from the svn repository. Instructions describing how to do this are on the Hive wiki.

    Carl


    On Thu, Mar 25, 2010 at 9:29 AM, Ted Yu wrote:
    Hi,
    I encountered Parse Error creating table in CLI:

    [root@tyu-linux hive-0.5.0-bin]# bin/hive
    Hive history file=/tmp/root/hive_job_log_root_201003240832_1752130264.txt
    hive> SHOW TABLES;
    OK
    Time taken: 8.09 seconds
    hive> CREATE EXTERNAL TABLE users(key string, state string, country string,
    country_id int)
    STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' >
    WITH SERDEPROPERTIES ( >
    "hbase.key.type" = "string", >
    "hbase.columns.mapping" = "info:state,info:country,info:country_id", >
    "<http://hbase.table.name/>hbase.table.name<http://hbase.table.name>" = "users" >
    );
    FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table file
    format specification

    Here is the tail of hive.log:
    2010-03-24 08:32:07,937 ERROR DataNucleus.Plugin
    (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires
    "org.eclipse.text" but it cannot be resolved.
    2010-03-24 08:34:18,337 ERROR ql.Driver (SessionState.java:printError(248))
    - FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table
    file format specification

    org.apache.hadoop.hive.ql.parse.ParseException: line 2:0 cannot recognize
    input 'STORED' in table file format specification

    at
    org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:357)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:267)
    at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:320)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312)
    at
    org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123)
    at
    org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

    If you know how to fix my query, please share.
  • John Sichi at Mar 26, 2010 at 4:37 am
    Ted, when you reply, please reply-all to the list so that the conversation remains accessible to all.

    Parameters are passed implicitly via the configuration object, not explicitly by name.

    The unit test only specifies the master because that is all that is needed for a standalone server.

    JVS


    On Mar 25, 2010, at 9:18 PM, "Ted Yu" wrote:

    I searched source code tree under trunk and didn't see how hbase.master parameter is passed to HBaseStorageHandler.java

    Also it looks like the unit test only specifies hostname for hbase.master:
    "Z:\kindsight\hive\trunk\hbase-handler\src\test\org\apache\hadoop\hive\hbase\HBaseQTestUtil.java"(55,17): conf.set("hbase.master", "local");

    BTW I saw a lot of following exceptions:
    2010-03-25 13:24:27,056 WARN zookeeper.ClientCnxn (ClientCnxn.java:cleanup(1001)) - Ignoring exception during shutdown input
    java.nio.channels.ClosedChannelException
    at sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:638)
    at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
    at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:999)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)

    Thanks

    On Thu, Mar 25, 2010 at 7:12 PM, Ted Yu wrote:
    We use distributed mode because the cluster would be deployed to production.

    I didn't find zookeeper commandline parameter on <http://wiki.apache.org/hadoop/Hive/HBaseIntegration#Usage> http://wiki.apache.org/hadoop/Hive/HBaseIntegration#Usage
    Do I need to use an entry in hbase-site.xml ?

    Our hbase cluster is based on 0.20.1
    But I used hbase client 0.20.3 against it before and didn't see RetriesExhaustedException.

    Thanks


    On Thu, Mar 25, 2010 at 6:07 PM, John Sichi wrote:
    Hmmm, what kind of HBase cluster are you running (standalone, pseudo-distributed, or distributed)? If distributed, you may need to pass zookeeper connection info instead of hbase.master; I haven't tested out distributed yet but will be soon.

    JVS

    On Mar 25, 2010, at 4:43 PM, Ted Yu wrote:

    I removed hbase-site.xml

    [root@tyu-linux dist]# bin/hive -hiveconf hbase.master=snv-it-lin-006:60000
    Hive history file=/tmp/root/hive_job_log_root_201003251614_1493474415.txt
    hive> CREATE EXTERNAL TABLE ruletable(key string, exactmatch_cat string, lpm_cat int)
    STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
    WITH SERDEPROPERTIES (
    "hbase.key.type" = "string",
    "hbase.columns.mapping" = "exactmatch_1.0:category,lpm_1.0:category",
    "<http://hbase.table.name/>hbase.table.name<http://hbase.table.name>" = "ruletable"
    );
    FAILED: Error in metadata: MetaException(message:org.apache.hadoop.hbase.MasterNotRunningException
    at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getMaster(HConnectionManager.java:374)
    at org.apache.hadoop.hbase.client.HBaseAdmin.(HBaseStorageHandler.java:63)
    at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:149)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:280)
    at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:320)
    at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:1445)
    at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:124)
    at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
    at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:630)
    at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:504)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:382)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:138)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:197)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:303)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
    )

    But <http://snv-it-lin-006:60010/master.jsp> http://snv-it-lin-006:60010/master.jsp shows everything is normal.
    And hbase.master.port is 60000 in hbase-default.xml

    Some suggestions ?

    Thanks

    On Thu, Mar 25, 2010 at 4:21 PM, John Sichi wrote:
    Don't copy hbase-site.xml; you don't want to set up a new HBase cluster. Just set your hive conf to point to your existing HBase master:

    hbase.master=hbase.yoyodyne.com:60000<http://hbase.yoyodyne.com:60000/>

    <http://wiki.apache.org/hadoop/Hive/HBaseIntegration#Usage>http://wiki.apache.org/hadoop/Hive/HBaseIntegration#Usage

    JVS

    On Mar 25, 2010, at 4:16 PM, Ted Yu wrote:

    I got pass that error when I use CLI from trunk.

    I copied hbase-site.xml from hbase master machine to trunk/dist/conf.

    When I issue the following command in CLI:
    CREATE EXTERNAL TABLE ruletable(key string, exactmatch_cat string, lpm_cat int)
    STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
    WITH SERDEPROPERTIES (
    "hbase.key.type" = "string",
    "hbase.columns.mapping" = "exactmatch_1.0:category,lpm_1.0:category",
    "<http://hbase.table.name/>hbase.table.name<http://hbase.table.name>" = "ruletable"
    );

    I get:

    2010-03-25 15:49:52,557 WARN client.HConnectionManager$TableServers (HConnectionManager.java:tableExists(411)) - Testing for table existence threw exception
    org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact region server 10.10.31.143:60020<http://10.10.31.143:60020/> for region .META.,,1, row '', but failed after 10 attempts.
    Exceptions:
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException
    java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/> failed on local exception: java.io.EOFException

    at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:1048)
    at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:55)
    at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:28)
    at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.listTables(HConnectionManager.java:454)
    at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.tableExists(HConnectionManager.java:404)
    at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:113)
    at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:100)
    at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:149)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:280)

    However, I don't see any exception in hbase-hbaseadmin-regionserver.log on 10.10.31.143 within the past 4 hours.
    No exception in hbase-hbaseadmin-master.log on hbase master machine either.

    Please comment.

    On Thu, Mar 25, 2010 at 11:22 AM, Carl Steinbach wrote:
    Hi Ted,

    It looks like your copy of Hive does not have the changes that implemented support for integration with HBase. HBase support was committed to trunk on March 12th, and currently it is only available on trunk. In order to use it you need to checkout (or update) the source from the svn repository. Instructions describing how to do this are on the Hive wiki.

    Carl


    On Thu, Mar 25, 2010 at 9:29 AM, Ted Yu wrote:
    Hi,
    I encountered Parse Error creating table in CLI:

    [root@tyu-linux hive-0.5.0-bin]# bin/hive
    Hive history file=/tmp/root/hive_job_log_root_201003240832_1752130264.txt
    hive> SHOW TABLES;
    OK
    Time taken: 8.09 seconds
    hive> CREATE EXTERNAL TABLE users(key string, state string, country string,
    country_id int)
    STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' >
    WITH SERDEPROPERTIES ( >
    "hbase.key.type" = "string", >
    "hbase.columns.mapping" = "info:state,info:country,info:country_id", >
    "<http://hbase.table.name/>hbase.table.name<http://hbase.table.name>" = "users" >
    );
    FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table file
    format specification

    Here is the tail of hive.log:
    2010-03-24 08:32:07,937 ERROR DataNucleus.Plugin
    (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires
    "org.eclipse.text" but it cannot be resolved.
    2010-03-24 08:34:18,337 ERROR ql.Driver (SessionState.java:printError(248))
    - FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table
    file format specification

    org.apache.hadoop.hive.ql.parse.ParseException: line 2:0 cannot recognize
    input 'STORED' in table file format specification

    at
    org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:357)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:267)
    at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:320)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312)
    at
    org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123)
    at
    org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

    If you know how to fix my query, please share.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshive, hadoop
postedMar 25, '10 at 4:30p
activeMar 26, '10 at 4:37a
posts7
users3
websitehive.apache.org

People

Translate

site design / logo © 2022 Grokbase