[Hive-user] Hive create table

Hi Jinhang
I don't think hive supports multi character delimiters. The hassle free option here would be to preprocess the data using mapreduce to replace the multi character delimiter with another permissible one that suits your data.
Regards
Bejoy K S

-----Original Message-----
From: jinhang du <[email protected]>
Date: Wed, 25 May 2011 19:56:16
To: <[email protected]>
Reply-To: [email protected]
Subject: Hive create table

Hi all，

I want to custom the delimiter of the table in a row.
Like my data format is '1&&&&2&&&&4‘, and how could I create a table (int,
int, int)

Thanks.

you can use something like this:

CREATE EXTERNAL TABLE IF NOT EXISTS table1 (
str1 int,
str2 int,
str3 int
)
ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
With SERDEPROPERTIES (
"input.regex"="(\\d+)&&&&(\\d+)&&&&(\\d+)"
)
LOCATION '/path1;

2011/5/25 jinhang du <[email protected]>

Thanks for your information. It works.

在 2011年5月25日 下午9:57，valentina kroshilina <[email protected]>写道：

Are there any documents to help me understand the meaning and usage of
"input.regex"?

I create table as your introduction. And "select * from table1" returns the
right answer.
However, "select table1.str1 from table1" gains the following exception.

Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201104251139_2123, Tracking URL =
http://hadoop-12.apexlab.org:50030/jobdetails.jsp?jobid=job_201104251139_2123
Kill Command = /usr/lib/hadoop/bin/hadoop job
-Dmapred.job.tracker=hadoop-12:9001 -kill job_201104251139_2123
2011-05-27 20:04:41,618 Stage-1 map = 0%, reduce = 0%
2011-05-27 20:05:05,413 Stage-1 map = 100%, reduce = 100%
Ended Job = job_201104251139_2123 with errors
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.MapRedTask

Can you help me? thanks.



在 2011年5月25日 下午9:57，valentina kroshilina <[email protected]>写道：

Check the log file, you can find more information about the error. The log file could be found from the URL given below.


At 2011-05-27 20:07:17，"jinhang du" wrote:
Are there any documents to help me understand the meaning and usage of "input.regex"?


I create table as your introduction. And "select * from table1" returns the right answer.
However, "select table1.str1 from table1" gains the following exception.


Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201104251139_2123, Tracking URL =http://hadoop-12.apexlab.org:50030/jobdetails.jsp?jobid=job_201104251139_2123
Kill Command = /usr/lib/hadoop/bin/hadoop job -Dmapred.job.tracker=hadoop-12:9001 -kill job_201104251139_2123
2011-05-27 20:04:41,618 Stage-1 map = 0%, reduce = 0%
2011-05-27 20:05:05,413 Stage-1 map = 100%, reduce = 100%
Ended Job = job_201104251139_2123 with errors
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask

Can you help me? thanks.





在 2011年5月25日 下午9:57，valentina kroshilina<[email protected]>写道：
you can use something like this:

CREATE EXTERNAL TABLE IF NOT EXISTS table1 (
str1 int,
str2 int,
str3 int
)
ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
With SERDEPROPERTIES (
"input.regex"="(\\d+)&&&&(\\d+)&&&&(\\d+)"
)
LOCATION '/path1;



2011/5/25 jinhang du<[email protected]>
Hi all，


I want to custom the delimiter of the table in a row.
Like my data format is '1&&&&2&&&&4‘, and how could I create a table (int, int, int)


Thanks.

Thanks for all your help.
I fix my problem by editing the hive-site.xml.

<property>
<name>hive.aux.jars.path</name>
<value>file:///usr/lib/hive/lib/hive-contrib-0.7.0-CDH3B4.jar</value>
<description>These JAR file are available to all users for all
jobs</description>
</property>

Now, I want to understand the "input.regex" . Is this Java regular
expression?
Thanks.


在 2011年5月27日 下午11:21，bangbig <[email protected]>写道：