[Hive-user] Hive Loading Nulls When Using RegEx

On Jul 1, 2011, at 6:52 PM, Sal Scalisi wrote:

I'm new to hive and I'm having an issue loading a simple set of data via regex.

I have a data file called test.txt that contains the following:

TESTONE-1
TESTTWO-2
TESTTHREE-3
TESTFOUR-4
TESTFIVE-5

I have this hive script:

hive> CREATE TABLE test
Found class for org.apache.hadoop.hive.contrib.serde2.RegexSerDe
OK
Time taken: 0.064 seconds

hive> LOAD DATA LOCAL INPATH '/home/hadoop/test' OVERWRITE INTO TABLE test;
Copying data from file:/home/hadoop/test
Loading data to table test
OK
Time taken: 0.213 seconds

hive> SELECT * FROM test LIMIT 10;
OK
TESTONE-1
TESTTWO-2
TESTTHREE-3
TESTFOUR-4
TESTFIVE-5
Time taken: 0.153 seconds

Which produces the expected output.

When I alter the hive script to include two fields, I get all null values:

hive> CREATE TABLE test
Found class for org.apache.hadoop.hive.contrib.serde2.RegexSerDe
OK
Time taken: 0.025 seconds

hive> LOAD DATA LOCAL INPATH '/home/hadoop/test' OVERWRITE INTO TABLE test;
Copying data from file:/home/hadoop/test
Loading data to table test
OK
Time taken: 0.187 seconds

hive> SELECT * FROM test LIMIT 10;
OK
NULL NULL
NULL NULL
NULL NULL
NULL NULL
NULL NULL
Time taken: 0.162 seconds

I've checked the regular expression against http://regexpal.com/ and it seems to check out. I think there may be an issue with SerDe, but I don't know how to go about trouble shooting it.

I'm running this on Amazon's Elastic MapReduce

Any help is appreciated.

-Sal