I'm working on https://issues.apache.org/jira/browse/PIG-3297 and I ran
into something strange.
The issue is that Pig crashes with a Java Exception on a specific feature
in the Avro format (I attached a 225 byte file to the issue that shows the
problem).
I created a file with that feature and when I run the following lines it
correctly fails by throwing an Exception.
* REGISTER 'piggybank.jar'*
* in = LOAD '**
/home/nbasjes/workspace/pig/contrib/piggybank/java/src/test/java/org/apache/pig/piggybank/test/storage/avro/avro_test_files/test_stringType/test_record.avro
**'
*
* USING org.apache.pig.piggybank.storage.avro.AvroStorage ();*
* STORE in INTO 'output';*
So I added a jUnit test method to the project that I intended to be
identical that looks like this:
* @Test*
* public void testStringTypeOverride() throws IOException {*
* // Verify that load succeeds with a string field where the storage
class is not Utf8.*
* String testStringTypeOverrideFile =
getInputFile("test_stringType/test_record.avro");*
* String output = outbasedir + "testStringTypeOverride";*
* deleteDirectory(new File(output));*
* String [] queries = {*
* " in = LOAD '" + testStringTypeOverrideFile + "' USING
org.apache.pig.piggybank.storage.avro.AvroStorage ();",*
* " STORE in INTO '" + output + "';"*
* };*
* testAvroStorage( queries);*
* }*
Now I expect that since the current code has not been changed the same
Exception should occur.
But it doesn't.
When I run the unit tests then this test simply passes.
It says:
* Successfully read records from:
"file:///home/nbasjes/workspace/pig/contrib/piggybank/java/src/test/java/org/apache/pig/piggybank/test/storage/avro/avro_test_files/test_stringType/test_record.avro"
*
The only difference I see is that the script is using Grunt and the test is
using PigServer.
The crash is in the Avro reading part of Pig so I think both should use the
same code (and thus both should crash).
Does any one have a clue what I missed?
--
Best regards / Met vriendelijke groeten,
Niels Basjes