Grokbase Groups Pig user May 2013
FAQ
Hi,

I'm working on https://issues.apache.org/jira/browse/PIG-3297 and I ran
into something strange.
The issue is that Pig crashes with a Java Exception on a specific feature
in the Avro format (I attached a 225 byte file to the issue that shows the
problem).
I created a file with that feature and when I run the following lines it
correctly fails by throwing an Exception.

* REGISTER 'piggybank.jar'*
* in = LOAD '**
/home/nbasjes/workspace/pig/contrib/piggybank/java/src/test/java/org/apache/pig/piggybank/test/storage/avro/avro_test_files/test_stringType/test_record.avro
**'
*
* USING org.apache.pig.piggybank.storage.avro.AvroStorage ();*
* STORE in INTO 'output';*


So I added a jUnit test method to the project that I intended to be
identical that looks like this:

* @Test*
* public void testStringTypeOverride() throws IOException {*
* // Verify that load succeeds with a string field where the storage
class is not Utf8.*
* String testStringTypeOverrideFile =
getInputFile("test_stringType/test_record.avro");*
* String output = outbasedir + "testStringTypeOverride";*
* deleteDirectory(new File(output));*
* String [] queries = {*
* " in = LOAD '" + testStringTypeOverrideFile + "' USING
org.apache.pig.piggybank.storage.avro.AvroStorage ();",*
* " STORE in INTO '" + output + "';"*
* };*
* testAvroStorage( queries);*
* }*

Now I expect that since the current code has not been changed the same
Exception should occur.
But it doesn't.

When I run the unit tests then this test simply passes.

It says:
* Successfully read records from:

"file:///home/nbasjes/workspace/pig/contrib/piggybank/java/src/test/java/org/apache/pig/piggybank/test/storage/avro/avro_test_files/test_stringType/test_record.avro"
*

The only difference I see is that the script is using Grunt and the test is
using PigServer.
The crash is in the Avro reading part of Pig so I think both should use the
same code (and thus both should crash).

Does any one have a clue what I missed?

--
Best regards / Met vriendelijke groeten,

Niels Basjes

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedMay 8, '13 at 2:52p
activeMay 8, '13 at 2:52p
posts1
users1
websitepig.apache.org

1 user in discussion

Niels Basjes: 1 post

People

Translate

site design / logo © 2021 Grokbase