FAQ
Hi folks,

I am new to pig, and I am trying to get the basic pig + cassandra samples working.

I have created the PigTest Keyspace, and I am trying to run some of the command in test_storage.pig, but I get the following:


tsunami:pig schappetj$ bin/pig_cassandra -x local
Using /Library/pig-0.10.0/pig-0.10.0.jar.
2012-12-05 12:28:50,415 [main] INFO org.apache.pig.Main - Logging error messages to: /Library/apache-cassandra-1.1.7-src/examples/pig/pig_1354732130411.log
2012-12-05 12:28:50,616 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: file:///
2012-12-05 12:28:50.701 java[17814:1c03] Unable to load realm info from SCDynamicStore
grunt> U8 = load 'cassandra://PigTest/U8' using CassandraStorage();
2012-12-05 12:28:54,084 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2999: Unexpected internal error. null
Details at logfile: /Library/apache-cassandra-1.1.7-src/examples/pig/pig_1354732130411.log
grunt>





/Library/apache-cassandra-1.1.7-src/examples/pig/pig_1354732130411.log
Pig Stack Trace
---------------
ERROR 2999: Unexpected internal error. null

java.lang.NullPointerException
at org.apache.cassandra.utils.Hex.hexToBytes(Hex.java:51)
at org.apache.cassandra.hadoop.ConfigHelper.predicateFromString(ConfigHelper.java:206)
at org.apache.cassandra.hadoop.ConfigHelper.getInputSlicePredicate(ConfigHelper.java:176)
at org.apache.cassandra.hadoop.pig.CassandraStorage.setLocation(CassandraStorage.java:567)
at org.apache.cassandra.hadoop.pig.CassandraStorage.getSchema(CassandraStorage.java:600)
at org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:150)
at org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:109)
at org.apache.pig.newplan.logical.visitor.LineageFindRelVisitor.visit(LineageFindRelVisitor.java:100)
at org.apache.pig.newplan.logical.relational.LOLoad.accept(LOLoad.java:218)
at org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
at org.apache.pig.newplan.logical.visitor.CastLineageSetter.(PigServer.java:1679)
at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1610)
at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:942)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
at org.apache.pig.Main.run(Main.java:495)
at org.apache.pig.Main.main(Main.java:111)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)



PIG 0.10.0
Cassandra 1.1.7 w/source

ENV Osx Lion:
$ java -version
java version "1.7.0_04"
Java(TM) SE Runtime Environment (build 1.7.0_04-b21)
Java HotSpot(TM) 64-Bit Server VM (build 23.0-b21, mixed mode)

set | grep PIG
PIG_HOME=/Library/pig-0.10.0
PIG_INITIAL_ADDRESS=localhost
PIG_PARTITIONER=org.apache.cassandra.dht.RandomPartitioner
PIG_RPC_PORT=9160

Search Discussions

  • James Schappet at Dec 10, 2012 at 6:07 pm
    Any thoughts on how I can start diagnosing this problem?

    On 12/5/12 12:43 PM, "Schappet, James C" wrote:

    Hi folks,

    I am new to pig, and I am trying to get the basic pig + cassandra samples
    working.

    I have created the PigTest Keyspace, and I am trying to run some of the
    command in test_storage.pig, but I get the following:


    tsunami:pig schappetj$ bin/pig_cassandra -x local
    Using /Library/pig-0.10.0/pig-0.10.0.jar.
    2012-12-05 12:28:50,415 [main] INFO org.apache.pig.Main - Logging error
    messages to:
    /Library/apache-cassandra-1.1.7-src/examples/pig/pig_1354732130411.log
    2012-12-05 12:28:50,616 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
    Connecting to hadoop file system at: file:///
    2012-12-05 12:28:50.701 java[17814:1c03] Unable to load realm info from
    SCDynamicStore
    grunt> U8 = load 'cassandra://PigTest/U8' using CassandraStorage();
    2012-12-05 12:28:54,084 [main] ERROR org.apache.pig.tools.grunt.Grunt -
    ERROR 2999: Unexpected internal error. null
    Details at logfile:
    /Library/apache-cassandra-1.1.7-src/examples/pig/pig_1354732130411.log
    grunt>





    /Library/apache-cassandra-1.1.7-src/examples/pig/pig_1354732130411.log
    Pig Stack Trace
    ---------------
    ERROR 2999: Unexpected internal error. null

    java.lang.NullPointerException
    at org.apache.cassandra.utils.Hex.hexToBytes(Hex.java:51)
    at
    org.apache.cassandra.hadoop.ConfigHelper.predicateFromString(ConfigHelper.
    java:206)
    at
    org.apache.cassandra.hadoop.ConfigHelper.getInputSlicePredicate(ConfigHelp
    er.java:176)
    at
    org.apache.cassandra.hadoop.pig.CassandraStorage.setLocation(CassandraStor
    age.java:567)
    at
    org.apache.cassandra.hadoop.pig.CassandraStorage.getSchema(CassandraStorag
    e.java:600)
    at
    org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOL
    oad.java:150)
    at
    org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:109
    )
    at
    org.apache.pig.newplan.logical.visitor.LineageFindRelVisitor.visit(Lineage
    FindRelVisitor.java:100)
    at
    org.apache.pig.newplan.logical.relational.LOLoad.accept(LOLoad.java:218)
    at
    org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.ja
    va:75)
    at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
    at
    org.apache.pig.newplan.logical.visitor.CastLineageSetter.<init>(CastLineag
    eSetter.java:57)
    at org.apache.pig.PigServer$Graph.compile(PigServer.java:1679)
    at
    org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1610)
    at
    org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
    at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
    at
    org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:942)
    at
    org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParse
    r.java:386)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:1
    88)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:1
    64)
    at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
    at org.apache.pig.Main.run(Main.java:495)
    at org.apache.pig.Main.main(Main.java:111)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
    57)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm
    pl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)



    PIG 0.10.0
    Cassandra 1.1.7 w/source

    ENV Osx Lion:
    $ java -version
    java version "1.7.0_04"
    Java(TM) SE Runtime Environment (build 1.7.0_04-b21)
    Java HotSpot(TM) 64-Bit Server VM (build 23.0-b21, mixed mode)

    set | grep PIG
    PIG_HOME=/Library/pig-0.10.0
    PIG_INITIAL_ADDRESS=localhost
    PIG_PARTITIONER=org.apache.cassandra.dht.RandomPartitioner
    PIG_RPC_PORT=9160
  • Jonathan Coveney at Dec 11, 2012 at 6:00 pm
    If I were debugging this (note, I know nothing about cassandra), I would
    put a flag in my ide on cassandra storage and see what is going on in
    there, and why it is erroring out. Then I would follow that backwards into
    whatever in Pig was generating that issue. That's pretty vague but can't
    really say much else unless I knew a ton about CassandraStorage.


    2012/12/10 James Schappet <jschappet@gmail.com>
    Any thoughts on how I can start diagnosing this problem?

    On 12/5/12 12:43 PM, "Schappet, James C" wrote:

    Hi folks,

    I am new to pig, and I am trying to get the basic pig + cassandra samples
    working.

    I have created the PigTest Keyspace, and I am trying to run some of the
    command in test_storage.pig, but I get the following:


    tsunami:pig schappetj$ bin/pig_cassandra -x local
    Using /Library/pig-0.10.0/pig-0.10.0.jar.
    2012-12-05 12:28:50,415 [main] INFO org.apache.pig.Main - Logging error
    messages to:
    /Library/apache-cassandra-1.1.7-src/examples/pig/pig_1354732130411.log
    2012-12-05 12:28:50,616 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
    Connecting to hadoop file system at: file:///
    2012-12-05 12:28:50.701 java[17814:1c03] Unable to load realm info from
    SCDynamicStore
    grunt> U8 = load 'cassandra://PigTest/U8' using CassandraStorage();
    2012-12-05 12:28:54,084 [main] ERROR org.apache.pig.tools.grunt.Grunt -
    ERROR 2999: Unexpected internal error. null
    Details at logfile:
    /Library/apache-cassandra-1.1.7-src/examples/pig/pig_1354732130411.log
    grunt>





    /Library/apache-cassandra-1.1.7-src/examples/pig/pig_1354732130411.log
    Pig Stack Trace
    ---------------
    ERROR 2999: Unexpected internal error. null

    java.lang.NullPointerException
    at org.apache.cassandra.utils.Hex.hexToBytes(Hex.java:51)
    at
    org.apache.cassandra.hadoop.ConfigHelper.predicateFromString(ConfigHelper.
    java:206)
    at
    org.apache.cassandra.hadoop.ConfigHelper.getInputSlicePredicate(ConfigHelp
    er.java:176)
    at
    org.apache.cassandra.hadoop.pig.CassandraStorage.setLocation(CassandraStor
    age.java:567)
    at
    org.apache.cassandra.hadoop.pig.CassandraStorage.getSchema(CassandraStorag
    e.java:600)
    at
    org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOL
    oad.java:150)
    at
    org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:109
    )
    at
    org.apache.pig.newplan.logical.visitor.LineageFindRelVisitor.visit(Lineage
    FindRelVisitor.java:100)
    at
    org.apache.pig.newplan.logical.relational.LOLoad.accept(LOLoad.java:218)
    at
    org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.ja
    va:75)
    at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
    at
    org.apache.pig.newplan.logical.visitor.CastLineageSetter.<init>(CastLineag
    eSetter.java:57)
    at org.apache.pig.PigServer$Graph.compile(PigServer.java:1679)
    at
    org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1610)
    at
    org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
    at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
    at
    org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:942)
    at
    org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParse
    r.java:386)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:1
    88)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:1
    64)
    at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
    at org.apache.pig.Main.run(Main.java:495)
    at org.apache.pig.Main.main(Main.java:111)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
    57)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm
    pl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)



    PIG 0.10.0
    Cassandra 1.1.7 w/source

    ENV Osx Lion:
    $ java -version
    java version "1.7.0_04"
    Java(TM) SE Runtime Environment (build 1.7.0_04-b21)
    Java HotSpot(TM) 64-Bit Server VM (build 23.0-b21, mixed mode)

    set | grep PIG
    PIG_HOME=/Library/pig-0.10.0
    PIG_INITIAL_ADDRESS=localhost
    PIG_PARTITIONER=org.apache.cassandra.dht.RandomPartitioner
    PIG_RPC_PORT=9160
  • William Oberman at Dec 11, 2012 at 6:50 pm
    Your line numbers aren't matching up to the 1.1.7 release, which is weird.
    Based on the "stock" 1.1.7 source, there was a null check on str
    before predicateFromString(str),
    making your code path impossible...

    will

    On Tue, Dec 11, 2012 at 1:00 PM, Jonathan Coveney wrote:

    If I were debugging this (note, I know nothing about cassandra), I would
    put a flag in my ide on cassandra storage and see what is going on in
    there, and why it is erroring out. Then I would follow that backwards into
    whatever in Pig was generating that issue. That's pretty vague but can't
    really say much else unless I knew a ton about CassandraStorage.


    2012/12/10 James Schappet <jschappet@gmail.com>
    Any thoughts on how I can start diagnosing this problem?

    On 12/5/12 12:43 PM, "Schappet, James C" wrote:

    Hi folks,

    I am new to pig, and I am trying to get the basic pig + cassandra
    samples
    working.

    I have created the PigTest Keyspace, and I am trying to run some of the
    command in test_storage.pig, but I get the following:


    tsunami:pig schappetj$ bin/pig_cassandra -x local
    Using /Library/pig-0.10.0/pig-0.10.0.jar.
    2012-12-05 12:28:50,415 [main] INFO org.apache.pig.Main - Logging error
    messages to:
    /Library/apache-cassandra-1.1.7-src/examples/pig/pig_1354732130411.log
    2012-12-05 12:28:50,616 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
    Connecting to hadoop file system at: file:///
    2012-12-05 12:28:50.701 java[17814:1c03] Unable to load realm info from
    SCDynamicStore
    grunt> U8 = load 'cassandra://PigTest/U8' using CassandraStorage();
    2012-12-05 12:28:54,084 [main] ERROR org.apache.pig.tools.grunt.Grunt -
    ERROR 2999: Unexpected internal error. null
    Details at logfile:
    /Library/apache-cassandra-1.1.7-src/examples/pig/pig_1354732130411.log
    grunt>





    /Library/apache-cassandra-1.1.7-src/examples/pig/pig_1354732130411.log
    Pig Stack Trace
    ---------------
    ERROR 2999: Unexpected internal error. null

    java.lang.NullPointerException
    at org.apache.cassandra.utils.Hex.hexToBytes(Hex.java:51)
    at
    org.apache.cassandra.hadoop.ConfigHelper.predicateFromString(ConfigHelper.
    java:206)
    at
    org.apache.cassandra.hadoop.ConfigHelper.getInputSlicePredicate(ConfigHelp
    er.java:176)
    at
    org.apache.cassandra.hadoop.pig.CassandraStorage.setLocation(CassandraStor
    age.java:567)
    at
    org.apache.cassandra.hadoop.pig.CassandraStorage.getSchema(CassandraStorag
    e.java:600)
    at
    org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOL
    oad.java:150)
    at
    org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:109
    )
    at
    org.apache.pig.newplan.logical.visitor.LineageFindRelVisitor.visit(Lineage
    FindRelVisitor.java:100)
    at
    org.apache.pig.newplan.logical.relational.LOLoad.accept(LOLoad.java:218)
    at
    org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.ja
    va:75)
    at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
    at
    org.apache.pig.newplan.logical.visitor.CastLineageSetter.<init>(CastLineag
    eSetter.java:57)
    at org.apache.pig.PigServer$Graph.compile(PigServer.java:1679)
    at
    org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1610)
    at
    org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
    at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
    at
    org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:942)
    at
    org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParse
    r.java:386)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:1
    88)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:1
    64)
    at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
    at org.apache.pig.Main.run(Main.java:495)
    at org.apache.pig.Main.main(Main.java:111)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
    57)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm
    pl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)



    PIG 0.10.0
    Cassandra 1.1.7 w/source

    ENV Osx Lion:
    $ java -version
    java version "1.7.0_04"
    Java(TM) SE Runtime Environment (build 1.7.0_04-b21)
    Java HotSpot(TM) 64-Bit Server VM (build 23.0-b21, mixed mode)

    set | grep PIG
    PIG_HOME=/Library/pig-0.10.0
    PIG_INITIAL_ADDRESS=localhost
    PIG_PARTITIONER=org.apache.cassandra.dht.RandomPartitioner
    PIG_RPC_PORT=9160
  • Jonathan Coveney at Dec 11, 2012 at 7:11 pm
    Sounds like there could be a wrong version on the classpath then


    2012/12/11 William Oberman <oberman@civicscience.com>
    Your line numbers aren't matching up to the 1.1.7 release, which is weird.
    Based on the "stock" 1.1.7 source, there was a null check on str
    before predicateFromString(str),
    making your code path impossible...

    will


    On Tue, Dec 11, 2012 at 1:00 PM, Jonathan Coveney <jcoveney@gmail.com
    wrote:
    If I were debugging this (note, I know nothing about cassandra), I would
    put a flag in my ide on cassandra storage and see what is going on in
    there, and why it is erroring out. Then I would follow that backwards into
    whatever in Pig was generating that issue. That's pretty vague but can't
    really say much else unless I knew a ton about CassandraStorage.


    2012/12/10 James Schappet <jschappet@gmail.com>
    Any thoughts on how I can start diagnosing this problem?


    On 12/5/12 12:43 PM, "Schappet, James C" <james-schappet@uiowa.edu>
    wrote:
    Hi folks,

    I am new to pig, and I am trying to get the basic pig + cassandra
    samples
    working.

    I have created the PigTest Keyspace, and I am trying to run some of
    the
    command in test_storage.pig, but I get the following:


    tsunami:pig schappetj$ bin/pig_cassandra -x local
    Using /Library/pig-0.10.0/pig-0.10.0.jar.
    2012-12-05 12:28:50,415 [main] INFO org.apache.pig.Main - Logging
    error
    messages to:
    /Library/apache-cassandra-1.1.7-src/examples/pig/pig_1354732130411.log
    2012-12-05 12:28:50,616 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
    Connecting to hadoop file system at: file:///
    2012-12-05 12:28:50.701 java[17814:1c03] Unable to load realm info
    from
    SCDynamicStore
    grunt> U8 = load 'cassandra://PigTest/U8' using CassandraStorage();
    2012-12-05 12:28:54,084 [main] ERROR org.apache.pig.tools.grunt.Grunt
    -
    ERROR 2999: Unexpected internal error. null
    Details at logfile:
    /Library/apache-cassandra-1.1.7-src/examples/pig/pig_1354732130411.log
    grunt>




    /Library/apache-cassandra-1.1.7-src/examples/pig/pig_1354732130411.log
    Pig Stack Trace
    ---------------
    ERROR 2999: Unexpected internal error. null

    java.lang.NullPointerException
    at org.apache.cassandra.utils.Hex.hexToBytes(Hex.java:51)
    at
    org.apache.cassandra.hadoop.ConfigHelper.predicateFromString(ConfigHelper.
    java:206)
    at
    org.apache.cassandra.hadoop.ConfigHelper.getInputSlicePredicate(ConfigHelp
    er.java:176)
    at
    org.apache.cassandra.hadoop.pig.CassandraStorage.setLocation(CassandraStor
    age.java:567)
    at
    org.apache.cassandra.hadoop.pig.CassandraStorage.getSchema(CassandraStorag
    e.java:600)
    at
    org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOL
    oad.java:150)
    at
    org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:109
    )
    at
    org.apache.pig.newplan.logical.visitor.LineageFindRelVisitor.visit(Lineage
    FindRelVisitor.java:100)
    at
    org.apache.pig.newplan.logical.relational.LOLoad.accept(LOLoad.java:218)
    at
    org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.ja
    va:75)
    at
    org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
    at
    org.apache.pig.newplan.logical.visitor.CastLineageSetter.<init>(CastLineag
    eSetter.java:57)
    at org.apache.pig.PigServer$Graph.compile(PigServer.java:1679)
    at
    org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1610)
    at
    org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
    at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
    at
    org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:942)
    at
    org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParse
    r.java:386)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:1
    88)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:1
    64)
    at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
    at org.apache.pig.Main.run(Main.java:495)
    at org.apache.pig.Main.main(Main.java:111)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
    57)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm
    pl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)



    PIG 0.10.0
    Cassandra 1.1.7 w/source

    ENV Osx Lion:
    $ java -version
    java version "1.7.0_04"
    Java(TM) SE Runtime Environment (build 1.7.0_04-b21)
    Java HotSpot(TM) 64-Bit Server VM (build 23.0-b21, mixed mode)

    set | grep PIG
    PIG_HOME=/Library/pig-0.10.0
    PIG_INITIAL_ADDRESS=localhost
    PIG_PARTITIONER=org.apache.cassandra.dht.RandomPartitioner
    PIG_RPC_PORT=9160
  • James Schappet at Dec 11, 2012 at 7:14 pm
    This is what pig_cassandra with debug enabled outputs:




    tsunami:pig schappetj$ bin/pig_cassandra -x local rowcount.pig
    Using /Library/pig-0.10.0/pig-0.10.0.jar.
    Find hadoop at /Library/hadoop-1.0.2/bin/hadoop
    dry run:
    HADOOP_CLASSPATH:
    /Library/pig-0.10.0/bin/../conf:/Library/Java/JavaVirtualMachines/1.7.0.jdk
    /Contents/Home/lib/tools.jar:::/Library/apache-cassandra-1.1.7-src/lib/antl
    r-3.2.jar:/Library/apache-cassandra-1.1.7-src/lib/avro-1.4.0-fixes.jar:/Lib
    rary/apache-cassandra-1.1.7-src/lib/avro-1.4.0-sources-fixes.jar:/Library/a
    pache-cassandra-1.1.7-src/lib/commons-cli-1.1.jar:/Library/apache-cassandra
    -1.1.7-src/lib/commons-codec-1.2.jar:/Library/apache-cassandra-1.1.7-src/li
    b/commons-lang-2.4.jar:/Library/apache-cassandra-1.1.7-src/lib/compress-lzf
    -0.8.4.jar:/Library/apache-cassandra-1.1.7-src/lib/concurrentlinkedhashmap-
    lru-1.3.jar:/Library/apache-cassandra-1.1.7-src/lib/guava-r08.jar:/Library/
    apache-cassandra-1.1.7-src/lib/high-scale-lib-1.1.2.jar:/Library/apache-cas
    sandra-1.1.7-src/lib/jackson-core-asl-1.9.2.jar:/Library/apache-cassandra-1
    .1.7-src/lib/jackson-mapper-asl-1.9.2.jar:/Library/apache-cassandra-1.1.7-s
    rc/lib/jamm-0.2.5.jar:/Library/apache-cassandra-1.1.7-src/lib/jline-0.9.94.
    jar:/Library/apache-cassandra-1.1.7-src/lib/json-simple-1.1.jar:/Library/ap
    ache-cassandra-1.1.7-src/lib/libthrift-0.7.0.jar:/Library/apache-cassandra-
    1.1.7-src/lib/log4j-1.2.16.jar:/Library/apache-cassandra-1.1.7-src/lib/metr
    ics-core-2.0.3.jar:/Library/apache-cassandra-1.1.7-src/lib/servlet-api-2.5-
    20081211.jar:/Library/apache-cassandra-1.1.7-src/lib/slf4j-api-1.6.1.jar:/L
    ibrary/apache-cassandra-1.1.7-src/lib/slf4j-log4j12-1.6.1.jar:/Library/apac
    he-cassandra-1.1.7-src/lib/snakeyaml-1.6.jar:/Library/apache-cassandra-1.1.
    7-src/lib/snappy-java-1.0.4.1.jar:/Library/apache-cassandra-1.1.7-src/lib/s
    naptree-0.1.jar:/Library/apache-cassandra-1.1.7-src/build/lib/jars/ant-1.6.
    5.jar:/Library/apache-cassandra-1.1.7-src/build/lib/jars/apache-rat-0.6.jar
    :/Library/apache-cassandra-1.1.7-src/build/lib/jars/apache-rat-core-0.6.jar
    :/Library/apache-cassandra-1.1.7-src/build/lib/jars/apache-rat-tasks-0.6.ja
    r:/Library/apache-cassandra-1.1.7-src/build/lib/jars/asm-3.2.jar:/Library/a
    pache-cassandra-1.1.7-src/build/lib/jars/avro-1.3.2.jar:/Library/apache-cas
    sandra-1.1.7-src/build/lib/jars/commons-beanutils-1.7.0.jar:/Library/apache
    -cassandra-1.1.7-src/build/lib/jars/commons-beanutils-core-1.8.0.jar:/Libra
    ry/apache-cassandra-1.1.7-src/build/lib/jars/commons-cli-1.2.jar:/Library/a
    pache-cassandra-1.1.7-src/build/lib/jars/commons-codec-1.4.jar:/Library/apa
    che-cassandra-1.1.7-src/build/lib/jars/commons-collections-3.2.jar:/Library
    /apache-cassandra-1.1.7-src/build/lib/jars/commons-configuration-1.6.jar:/L
    ibrary/apache-cassandra-1.1.7-src/build/lib/jars/commons-digester-1.8.jar:/
    Library/apache-cassandra-1.1.7-src/build/lib/jars/commons-el-1.0.jar:/Libra
    ry/apache-cassandra-1.1.7-src/build/lib/jars/commons-httpclient-3.0.1.jar:/
    Library/apache-cassandra-1.1.7-src/build/lib/jars/commons-lang-2.4.jar:/Lib
    rary/apache-cassandra-1.1.7-src/build/lib/jars/commons-logging-1.1.1.jar:/L
    ibrary/apache-cassandra-1.1.7-src/build/lib/jars/commons-math-2.1.jar:/Libr
    ary/apache-cassandra-1.1.7-src/build/lib/jars/commons-net-1.4.1.jar:/Librar
    y/apache-cassandra-1.1.7-src/build/lib/jars/core-3.1.1.jar:/Library/apache-
    cassandra-1.1.7-src/build/lib/jars/hadoop-core-0.20.203.0.jar:/Library/apac
    he-cassandra-1.1.7-src/build/lib/jars/hsqldb-1.8.0.10.jar:/Library/apache-c
    assandra-1.1.7-src/build/lib/jars/jackson-core-asl-1.4.2.jar:/Library/apach
    e-cassandra-1.1.7-src/build/lib/jars/jackson-mapper-asl-1.4.2.jar:/Library/
    apache-cassandra-1.1.7-src/build/lib/jars/jasper-compiler-5.5.12.jar:/Libra
    ry/apache-cassandra-1.1.7-src/build/lib/jars/jasper-runtime-5.5.12.jar:/Lib
    rary/apache-cassandra-1.1.7-src/build/lib/jars/jets3t-0.7.1.jar:/Library/ap
    ache-cassandra-1.1.7-src/build/lib/jars/jetty-6.1.26.jar:/Library/apache-ca
    ssandra-1.1.7-src/build/lib/jars/jetty-util-6.1.26.jar:/Library/apache-cass
    andra-1.1.7-src/build/lib/jars/jna-3.2.7.jar:/Library/apache-cassandra-1.1.
    7-src/build/lib/jars/jopt-simple-3.2.jar:/Library/apache-cassandra-1.1.7-sr
    c/build/lib/jars/jsp-2.1-6.1.14.jar:/Library/apache-cassandra-1.1.7-src/bui
    ld/lib/jars/jsp-api-2.1-6.1.14.jar:/Library/apache-cassandra-1.1.7-src/buil
    d/lib/jars/junit-4.6.jar:/Library/apache-cassandra-1.1.7-src/build/lib/jars
    /kfs-0.3.jar:/Library/apache-cassandra-1.1.7-src/build/lib/jars/oro-2.0.8.j
    ar:/Library/apache-cassandra-1.1.7-src/build/lib/jars/paranamer-2.2.jar:/Li
    brary/apache-cassandra-1.1.7-src/build/lib/jars/paranamer-ant-2.1.jar:/Libr
    ary/apache-cassandra-1.1.7-src/build/lib/jars/paranamer-generator-2.1.jar:/
    Library/apache-cassandra-1.1.7-src/build/lib/jars/pig-0.9.2.jar:/Library/ap
    ache-cassandra-1.1.7-src/build/lib/jars/qdox-1.10.jar:/Library/apache-cassa
    ndra-1.1.7-src/build/lib/jars/servlet-api-2.5-20081211.jar:/Library/apache-
    cassandra-1.1.7-src/build/lib/jars/servlet-api-2.5-6.1.14.jar:/Library/apac
    he-cassandra-1.1.7-src/build/lib/jars/slf4j-api-1.5.11.jar:/Library/apache-
    cassandra-1.1.7-src/build/lib/jars/xmlenc-0.52.jar:/Library/apache-cassandr
    a-1.1.7-src/build/apache-cassandra-1.1.7-SNAPSHOT.jar:/Library/apache-cassa
    ndra-1.1.7-src/build/apache-cassandra-clientutil-1.1.7-SNAPSHOT.jar:/Librar
    y/apache-cassandra-1.1.7-src/build/apache-cassandra-thrift-1.1.7-SNAPSHOT.j
    ar:/Library/pig-0.10.0/pig-0.10.0.jar:/Library/pig-0.10.0/bin/../lib/automa
    ton.jar:/Library/pig-0.10.0/bin/../lib/jython-2.5.0.jar:/Library/pig-0.10.0
    /bin/../pig-0.10.0-withouthadoop.jar:
    HADOOP_OPTS: -Xmx1000m -Dudf.import.list=org.apache.cassandra.hadoop.pig
    -Dpig.log.dir=/Library/pig-0.10.0/bin/../logs -Dpig.log.file=pig.log
    -Dpig.home.dir=/Library/pig-0.10.0/bin/..
    /Library/hadoop-1.0.2/bin/hadoop jar
    /Library/pig-0.10.0/bin/../pig-0.10.0-withouthadoop.jar
    -Dudf.import.list=org.apache.cassandra.hadoop.pig -x local rowcount.pig






    On 12/11/12 1:10 PM, "Jonathan Coveney" wrote:

    Sounds like there could be a wrong version on the classpath then


    2012/12/11 William Oberman <oberman@civicscience.com>
    Your line numbers aren't matching up to the 1.1.7 release, which is
    weird.
    Based on the "stock" 1.1.7 source, there was a null check on str
    before predicateFromString(str),
    making your code path impossible...

    will


    On Tue, Dec 11, 2012 at 1:00 PM, Jonathan Coveney <jcoveney@gmail.com
    wrote:
    If I were debugging this (note, I know nothing about cassandra), I would
    put a flag in my ide on cassandra storage and see what is going on in
    there, and why it is erroring out. Then I would follow that backwards into
    whatever in Pig was generating that issue. That's pretty vague but can't
    really say much else unless I knew a ton about CassandraStorage.


    2012/12/10 James Schappet <jschappet@gmail.com>
    Any thoughts on how I can start diagnosing this problem?


    On 12/5/12 12:43 PM, "Schappet, James C" <james-schappet@uiowa.edu>
    wrote:
    Hi folks,

    I am new to pig, and I am trying to get the basic pig + cassandra
    samples
    working.

    I have created the PigTest Keyspace, and I am trying to run some of
    the
    command in test_storage.pig, but I get the following:


    tsunami:pig schappetj$ bin/pig_cassandra -x local
    Using /Library/pig-0.10.0/pig-0.10.0.jar.
    2012-12-05 12:28:50,415 [main] INFO org.apache.pig.Main - Logging
    error
    messages to:
    /Library/apache-cassandra-1.1.7-src/examples/pig/pig_1354732130411.log
    2012-12-05 12:28:50,616 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
    Connecting to hadoop file system at: file:///
    2012-12-05 12:28:50.701 java[17814:1c03] Unable to load realm info
    from
    SCDynamicStore
    grunt> U8 = load 'cassandra://PigTest/U8' using CassandraStorage();
    2012-12-05 12:28:54,084 [main] ERROR
    org.apache.pig.tools.grunt.Grunt
    -
    ERROR 2999: Unexpected internal error. null
    Details at logfile:
    /Library/apache-cassandra-1.1.7-src/examples/pig/pig_1354732130411.log
    grunt>




    /Library/apache-cassandra-1.1.7-src/examples/pig/pig_1354732130411.log
    Pig Stack Trace
    ---------------
    ERROR 2999: Unexpected internal error. null

    java.lang.NullPointerException
    at org.apache.cassandra.utils.Hex.hexToBytes(Hex.java:51)
    at
    org.apache.cassandra.hadoop.ConfigHelper.predicateFromString(ConfigHelpe
    r.
    java:206)
    at
    org.apache.cassandra.hadoop.ConfigHelper.getInputSlicePredicate(ConfigHe
    lp
    er.java:176)
    at
    org.apache.cassandra.hadoop.pig.CassandraStorage.setLocation(CassandraSt
    or
    age.java:567)
    at
    org.apache.cassandra.hadoop.pig.CassandraStorage.getSchema(CassandraStor
    ag
    e.java:600)
    at
    org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(L
    OL
    oad.java:150)
    at
    org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:1
    09
    )
    at
    org.apache.pig.newplan.logical.visitor.LineageFindRelVisitor.visit(Linea
    ge
    FindRelVisitor.java:100)
    at
    org.apache.pig.newplan.logical.relational.LOLoad.accept(LOLoad.java:218)
    at
    org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.
    ja
    va:75)
    at
    org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
    at
    org.apache.pig.newplan.logical.visitor.CastLineageSetter.<init>(CastLine
    ag
    eSetter.java:57)
    at
    org.apache.pig.PigServer$Graph.compile(PigServer.java:1679)
    at
    org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1610)
    at
    org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
    at
    org.apache.pig.PigServer.registerQuery(PigServer.java:584)
    at
    org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:942)
    at
    org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptPar
    se
    r.java:386)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java
    :1
    88)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java
    :1
    64)
    at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
    at org.apache.pig.Main.run(Main.java:495)
    at org.apache.pig.Main.main(Main.java:111)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
    Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
    a:
    57)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
    Im
    pl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)



    PIG 0.10.0
    Cassandra 1.1.7 w/source

    ENV Osx Lion:
    $ java -version
    java version "1.7.0_04"
    Java(TM) SE Runtime Environment (build 1.7.0_04-b21)
    Java HotSpot(TM) 64-Bit Server VM (build 23.0-b21, mixed mode)

    set | grep PIG
    PIG_HOME=/Library/pig-0.10.0
    PIG_INITIAL_ADDRESS=localhost
    PIG_PARTITIONER=org.apache.cassandra.dht.RandomPartitioner
    PIG_RPC_PORT=9160
  • William Oberman at Dec 11, 2012 at 7:21 pm
    For what it's worth, the error is on the cassandra side, so I'd post to
    that mailing list.

    On Tue, Dec 11, 2012 at 2:13 PM, James Schappet wrote:

    This is what pig_cassandra with debug enabled outputs:




    tsunami:pig schappetj$ bin/pig_cassandra -x local rowcount.pig
    Using /Library/pig-0.10.0/pig-0.10.0.jar.
    Find hadoop at /Library/hadoop-1.0.2/bin/hadoop
    dry run:
    HADOOP_CLASSPATH:
    /Library/pig-0.10.0/bin/../conf:/Library/Java/JavaVirtualMachines/1.7.0.jdk
    /Contents/Home/lib/tools.jar:::/Library/apache-cassandra-1.1.7-src/lib/antl
    r-3.2.jar:/Library/apache-cassandra-1.1.7-src/lib/avro-1.4.0-fixes.jar:/Lib
    rary/apache-cassandra-1.1.7-src/lib/avro-1.4.0-sources-fixes.jar:/Library/a
    pache-cassandra-1.1.7-src/lib/commons-cli-1.1.jar:/Library/apache-cassandra
    -1.1.7-src/lib/commons-codec-1.2.jar:/Library/apache-cassandra-1.1.7-src/li
    b/commons-lang-2.4.jar:/Library/apache-cassandra-1.1.7-src/lib/compress-lzf
    -0.8.4.jar:/Library/apache-cassandra-1.1.7-src/lib/concurrentlinkedhashmap-
    lru-1.3.jar:/Library/apache-cassandra-1.1.7-src/lib/guava-r08.jar:/Library/
    apache-cassandra-1.1.7-src/lib/high-scale-lib-1.1.2.jar:/Library/apache-cas
    sandra-1.1.7-src/lib/jackson-core-asl-1.9.2.jar:/Library/apache-cassandra-1
    .1.7-src/lib/jackson-mapper-asl-1.9.2.jar:/Library/apache-cassandra-1.1.7-s
    rc/lib/jamm-0.2.5.jar:/Library/apache-cassandra-1.1.7-src/lib/jline-0.9.94.
    jar:/Library/apache-cassandra-1.1.7-src/lib/json-simple-1.1.jar:/Library/ap
    ache-cassandra-1.1.7-src/lib/libthrift-0.7.0.jar:/Library/apache-cassandra-
    1.1.7-src/lib/log4j-1.2.16.jar:/Library/apache-cassandra-1.1.7-src/lib/metr
    ics-core-2.0.3.jar:/Library/apache-cassandra-1.1.7-src/lib/servlet-api-2.5-
    20081211.jar:/Library/apache-cassandra-1.1.7-src/lib/slf4j-api-1.6.1.jar:/L
    ibrary/apache-cassandra-1.1.7-src/lib/slf4j-log4j12-1.6.1.jar:/Library/apac
    he-cassandra-1.1.7-src/lib/snakeyaml-1.6.jar:/Library/apache-cassandra-1.1.
    7-src/lib/snappy-java-1.0.4.1.jar:/Library/apache-cassandra-1.1.7-src/lib/s
    naptree-0.1.jar:/Library/apache-cassandra-1.1.7-src/build/lib/jars/ant-1.6.
    5.jar:/Library/apache-cassandra-1.1.7-src/build/lib/jars/apache-rat-0.6.jar
    :/Library/apache-cassandra-1.1.7-src/build/lib/jars/apache-rat-core-0.6.jar
    :/Library/apache-cassandra-1.1.7-src/build/lib/jars/apache-rat-tasks-0.6.ja
    r:/Library/apache-cassandra-1.1.7-src/build/lib/jars/asm-3.2.jar:/Library/a
    pache-cassandra-1.1.7-src/build/lib/jars/avro-1.3.2.jar:/Library/apache-cas
    sandra-1.1.7-src/build/lib/jars/commons-beanutils-1.7.0.jar:/Library/apache
    -cassandra-1.1.7-src/build/lib/jars/commons-beanutils-core-1.8.0.jar:/Libra
    ry/apache-cassandra-1.1.7-src/build/lib/jars/commons-cli-1.2.jar:/Library/a
    pache-cassandra-1.1.7-src/build/lib/jars/commons-codec-1.4.jar:/Library/apa
    che-cassandra-1.1.7-src/build/lib/jars/commons-collections-3.2.jar:/Library
    /apache-cassandra-1.1.7-src/build/lib/jars/commons-configuration-1.6.jar:/L
    ibrary/apache-cassandra-1.1.7-src/build/lib/jars/commons-digester-1.8.jar:/
    Library/apache-cassandra-1.1.7-src/build/lib/jars/commons-el-1.0.jar:/Libra
    ry/apache-cassandra-1.1.7-src/build/lib/jars/commons-httpclient-3.0.1.jar:/
    Library/apache-cassandra-1.1.7-src/build/lib/jars/commons-lang-2.4.jar:/Lib
    rary/apache-cassandra-1.1.7-src/build/lib/jars/commons-logging-1.1.1.jar:/L
    ibrary/apache-cassandra-1.1.7-src/build/lib/jars/commons-math-2.1.jar:/Libr
    ary/apache-cassandra-1.1.7-src/build/lib/jars/commons-net-1.4.1.jar:/Librar
    y/apache-cassandra-1.1.7-src/build/lib/jars/core-3.1.1.jar:/Library/apache-
    cassandra-1.1.7-src/build/lib/jars/hadoop-core-0.20.203.0.jar:/Library/apac
    he-cassandra-1.1.7-src/build/lib/jars/hsqldb-1.8.0.10.jar:/Library/apache-c
    assandra-1.1.7-src/build/lib/jars/jackson-core-asl-1.4.2.jar:/Library/apach
    e-cassandra-1.1.7-src/build/lib/jars/jackson-mapper-asl-1.4.2.jar:/Library/
    apache-cassandra-1.1.7-src/build/lib/jars/jasper-compiler-5.5.12.jar:/Libra
    ry/apache-cassandra-1.1.7-src/build/lib/jars/jasper-runtime-5.5.12.jar:/Lib
    rary/apache-cassandra-1.1.7-src/build/lib/jars/jets3t-0.7.1.jar:/Library/ap
    ache-cassandra-1.1.7-src/build/lib/jars/jetty-6.1.26.jar:/Library/apache-ca
    ssandra-1.1.7-src/build/lib/jars/jetty-util-6.1.26.jar:/Library/apache-cass
    andra-1.1.7-src/build/lib/jars/jna-3.2.7.jar:/Library/apache-cassandra-1.1.
    7-src/build/lib/jars/jopt-simple-3.2.jar:/Library/apache-cassandra-1.1.7-sr
    c/build/lib/jars/jsp-2.1-6.1.14.jar:/Library/apache-cassandra-1.1.7-src/bui
    ld/lib/jars/jsp-api-2.1-6.1.14.jar:/Library/apache-cassandra-1.1.7-src/buil
    d/lib/jars/junit-4.6.jar:/Library/apache-cassandra-1.1.7-src/build/lib/jars
    /kfs-0.3.jar:/Library/apache-cassandra-1.1.7-src/build/lib/jars/oro-2.0.8.j
    ar:/Library/apache-cassandra-1.1.7-src/build/lib/jars/paranamer-2.2.jar:/Li
    brary/apache-cassandra-1.1.7-src/build/lib/jars/paranamer-ant-2.1.jar:/Libr
    ary/apache-cassandra-1.1.7-src/build/lib/jars/paranamer-generator-2.1.jar:/
    Library/apache-cassandra-1.1.7-src/build/lib/jars/pig-0.9.2.jar:/Library/ap
    ache-cassandra-1.1.7-src/build/lib/jars/qdox-1.10.jar:/Library/apache-cassa
    ndra-1.1.7-src/build/lib/jars/servlet-api-2.5-20081211.jar:/Library/apache-
    cassandra-1.1.7-src/build/lib/jars/servlet-api-2.5-6.1.14.jar:/Library/apac
    he-cassandra-1.1.7-src/build/lib/jars/slf4j-api-1.5.11.jar:/Library/apache-
    cassandra-1.1.7-src/build/lib/jars/xmlenc-0.52.jar:/Library/apache-cassandr
    a-1.1.7-src/build/apache-cassandra-1.1.7-SNAPSHOT.jar:/Library/apache-cassa
    ndra-1.1.7-src/build/apache-cassandra-clientutil-1.1.7-SNAPSHOT.jar:/Librar
    y/apache-cassandra-1.1.7-src/build/apache-cassandra-thrift-1.1.7-SNAPSHOT.j
    ar:/Library/pig-0.10.0/pig-0.10.0.jar:/Library/pig-0.10.0/bin/../lib/automa
    ton.jar:/Library/pig-0.10.0/bin/../lib/jython-2.5.0.jar:/Library/pig-0.10.0
    /bin/../pig-0.10.0-withouthadoop.jar:
    HADOOP_OPTS: -Xmx1000m -Dudf.import.list=org.apache.cassandra.hadoop.pig
    -Dpig.log.dir=/Library/pig-0.10.0/bin/../logs -Dpig.log.file=pig.log
    -Dpig.home.dir=/Library/pig-0.10.0/bin/..
    /Library/hadoop-1.0.2/bin/hadoop jar
    /Library/pig-0.10.0/bin/../pig-0.10.0-withouthadoop.jar
    -Dudf.import.list=org.apache.cassandra.hadoop.pig -x local rowcount.pig






    On 12/11/12 1:10 PM, "Jonathan Coveney" wrote:

    Sounds like there could be a wrong version on the classpath then


    2012/12/11 William Oberman <oberman@civicscience.com>
    Your line numbers aren't matching up to the 1.1.7 release, which is
    weird.
    Based on the "stock" 1.1.7 source, there was a null check on str
    before predicateFromString(str),
    making your code path impossible...

    will


    On Tue, Dec 11, 2012 at 1:00 PM, Jonathan Coveney <jcoveney@gmail.com
    wrote:
    If I were debugging this (note, I know nothing about cassandra), I would
    put a flag in my ide on cassandra storage and see what is going on in
    there, and why it is erroring out. Then I would follow that backwards into
    whatever in Pig was generating that issue. That's pretty vague but can't
    really say much else unless I knew a ton about CassandraStorage.


    2012/12/10 James Schappet <jschappet@gmail.com>
    Any thoughts on how I can start diagnosing this problem?


    On 12/5/12 12:43 PM, "Schappet, James C" <james-schappet@uiowa.edu>
    wrote:
    Hi folks,

    I am new to pig, and I am trying to get the basic pig + cassandra
    samples
    working.

    I have created the PigTest Keyspace, and I am trying to run some of
    the
    command in test_storage.pig, but I get the following:


    tsunami:pig schappetj$ bin/pig_cassandra -x local
    Using /Library/pig-0.10.0/pig-0.10.0.jar.
    2012-12-05 12:28:50,415 [main] INFO org.apache.pig.Main - Logging
    error
    messages to:
    /Library/apache-cassandra-1.1.7-src/examples/pig/pig_1354732130411.log
    2012-12-05 12:28:50,616 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
    Connecting to hadoop file system at: file:///
    2012-12-05 12:28:50.701 java[17814:1c03] Unable to load realm info
    from
    SCDynamicStore
    grunt> U8 = load 'cassandra://PigTest/U8' using CassandraStorage();
    2012-12-05 12:28:54,084 [main] ERROR
    org.apache.pig.tools.grunt.Grunt
    -
    ERROR 2999: Unexpected internal error. null
    Details at logfile:
    /Library/apache-cassandra-1.1.7-src/examples/pig/pig_1354732130411.log
    grunt>




    /Library/apache-cassandra-1.1.7-src/examples/pig/pig_1354732130411.log
    Pig Stack Trace
    ---------------
    ERROR 2999: Unexpected internal error. null

    java.lang.NullPointerException
    at org.apache.cassandra.utils.Hex.hexToBytes(Hex.java:51)
    at
    org.apache.cassandra.hadoop.ConfigHelper.predicateFromString(ConfigHelpe
    r.
    java:206)
    at
    org.apache.cassandra.hadoop.ConfigHelper.getInputSlicePredicate(ConfigHe
    lp
    er.java:176)
    at
    org.apache.cassandra.hadoop.pig.CassandraStorage.setLocation(CassandraSt
    or
    age.java:567)
    at
    org.apache.cassandra.hadoop.pig.CassandraStorage.getSchema(CassandraStor
    ag
    e.java:600)
    at
    org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(L
    OL
    oad.java:150)
    at
    org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:1
    09
    )
    at
    org.apache.pig.newplan.logical.visitor.LineageFindRelVisitor.visit(Linea
    ge
    FindRelVisitor.java:100)
    at
    org.apache.pig.newplan.logical.relational.LOLoad.accept(LOLoad.java:218)
    at
    org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.
    ja
    va:75)
    at
    org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
    at
    org.apache.pig.newplan.logical.visitor.CastLineageSetter.<init>(CastLine
    ag
    eSetter.java:57)
    at
    org.apache.pig.PigServer$Graph.compile(PigServer.java:1679)
    at
    org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1610)
    at
    org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
    at
    org.apache.pig.PigServer.registerQuery(PigServer.java:584)
    at
    org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:942)
    at
    org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptPar
    se
    r.java:386)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java
    :1
    88)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java
    :1
    64)
    at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
    at org.apache.pig.Main.run(Main.java:495)
    at org.apache.pig.Main.main(Main.java:111)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
    Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
    a:
    57)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
    Im
    pl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)



    PIG 0.10.0
    Cassandra 1.1.7 w/source

    ENV Osx Lion:
    $ java -version
    java version "1.7.0_04"
    Java(TM) SE Runtime Environment (build 1.7.0_04-b21)
    Java HotSpot(TM) 64-Bit Server VM (build 23.0-b21, mixed mode)

    set | grep PIG
    PIG_HOME=/Library/pig-0.10.0
    PIG_INITIAL_ADDRESS=localhost
    PIG_PARTITIONER=org.apache.cassandra.dht.RandomPartitioner
    PIG_RPC_PORT=9160
  • James Schappet at Dec 12, 2012 at 5:28 pm
    I was able to resolve this issue.


    Taking Hadoop out of my .profile, resolved the issue.
    PATH included: /Library/hadoop-1.0.2/bin, once removed I am able to
    connect Cassandra.

    Thanks for pointing me in the right direction.


    On 12/11/12 1:13 PM, "James Schappet" wrote:

    This is what pig_cassandra with debug enabled outputs:




    tsunami:pig schappetj$ bin/pig_cassandra -x local rowcount.pig
    Using /Library/pig-0.10.0/pig-0.10.0.jar.
    Find hadoop at /Library/hadoop-1.0.2/bin/hadoop
    dry run:
    HADOOP_CLASSPATH:
    /Library/pig-0.10.0/bin/../conf:/Library/Java/JavaVirtualMachines/1.7.0.jd
    k
    /Contents/Home/lib/tools.jar:::/Library/apache-cassandra-1.1.7-src/lib/ant
    l
    r-3.2.jar:/Library/apache-cassandra-1.1.7-src/lib/avro-1.4.0-fixes.jar:/Li
    b
    rary/apache-cassandra-1.1.7-src/lib/avro-1.4.0-sources-fixes.jar:/Library/
    a
    pache-cassandra-1.1.7-src/lib/commons-cli-1.1.jar:/Library/apache-cassandr
    a
    -1.1.7-src/lib/commons-codec-1.2.jar:/Library/apache-cassandra-1.1.7-src/l
    i
    b/commons-lang-2.4.jar:/Library/apache-cassandra-1.1.7-src/lib/compress-lz
    f
    -0.8.4.jar:/Library/apache-cassandra-1.1.7-src/lib/concurrentlinkedhashmap
    -
    lru-1.3.jar:/Library/apache-cassandra-1.1.7-src/lib/guava-r08.jar:/Library
    /
    apache-cassandra-1.1.7-src/lib/high-scale-lib-1.1.2.jar:/Library/apache-ca
    s
    sandra-1.1.7-src/lib/jackson-core-asl-1.9.2.jar:/Library/apache-cassandra-
    1
    .1.7-src/lib/jackson-mapper-asl-1.9.2.jar:/Library/apache-cassandra-1.1.7-
    s
    rc/lib/jamm-0.2.5.jar:/Library/apache-cassandra-1.1.7-src/lib/jline-0.9.94
    .
    jar:/Library/apache-cassandra-1.1.7-src/lib/json-simple-1.1.jar:/Library/a
    p
    ache-cassandra-1.1.7-src/lib/libthrift-0.7.0.jar:/Library/apache-cassandra
    -
    1.1.7-src/lib/log4j-1.2.16.jar:/Library/apache-cassandra-1.1.7-src/lib/met
    r
    ics-core-2.0.3.jar:/Library/apache-cassandra-1.1.7-src/lib/servlet-api-2.5
    -
    20081211.jar:/Library/apache-cassandra-1.1.7-src/lib/slf4j-api-1.6.1.jar:/
    L
    ibrary/apache-cassandra-1.1.7-src/lib/slf4j-log4j12-1.6.1.jar:/Library/apa
    c
    he-cassandra-1.1.7-src/lib/snakeyaml-1.6.jar:/Library/apache-cassandra-1.1
    .
    7-src/lib/snappy-java-1.0.4.1.jar:/Library/apache-cassandra-1.1.7-src/lib/
    s
    naptree-0.1.jar:/Library/apache-cassandra-1.1.7-src/build/lib/jars/ant-1.6
    .
    5.jar:/Library/apache-cassandra-1.1.7-src/build/lib/jars/apache-rat-0.6.ja
    r
    :/Library/apache-cassandra-1.1.7-src/build/lib/jars/apache-rat-core-0.6.ja
    r
    :/Library/apache-cassandra-1.1.7-src/build/lib/jars/apache-rat-tasks-0.6.j
    a
    r:/Library/apache-cassandra-1.1.7-src/build/lib/jars/asm-3.2.jar:/Library/
    a
    pache-cassandra-1.1.7-src/build/lib/jars/avro-1.3.2.jar:/Library/apache-ca
    s
    sandra-1.1.7-src/build/lib/jars/commons-beanutils-1.7.0.jar:/Library/apach
    e
    -cassandra-1.1.7-src/build/lib/jars/commons-beanutils-core-1.8.0.jar:/Libr
    a
    ry/apache-cassandra-1.1.7-src/build/lib/jars/commons-cli-1.2.jar:/Library/
    a
    pache-cassandra-1.1.7-src/build/lib/jars/commons-codec-1.4.jar:/Library/ap
    a
    che-cassandra-1.1.7-src/build/lib/jars/commons-collections-3.2.jar:/Librar
    y
    /apache-cassandra-1.1.7-src/build/lib/jars/commons-configuration-1.6.jar:/
    L
    ibrary/apache-cassandra-1.1.7-src/build/lib/jars/commons-digester-1.8.jar:
    /
    Library/apache-cassandra-1.1.7-src/build/lib/jars/commons-el-1.0.jar:/Libr
    a
    ry/apache-cassandra-1.1.7-src/build/lib/jars/commons-httpclient-3.0.1.jar:
    /
    Library/apache-cassandra-1.1.7-src/build/lib/jars/commons-lang-2.4.jar:/Li
    b
    rary/apache-cassandra-1.1.7-src/build/lib/jars/commons-logging-1.1.1.jar:/
    L
    ibrary/apache-cassandra-1.1.7-src/build/lib/jars/commons-math-2.1.jar:/Lib
    r
    ary/apache-cassandra-1.1.7-src/build/lib/jars/commons-net-1.4.1.jar:/Libra
    r
    y/apache-cassandra-1.1.7-src/build/lib/jars/core-3.1.1.jar:/Library/apache
    -
    cassandra-1.1.7-src/build/lib/jars/hadoop-core-0.20.203.0.jar:/Library/apa
    c
    he-cassandra-1.1.7-src/build/lib/jars/hsqldb-1.8.0.10.jar:/Library/apache-
    c
    assandra-1.1.7-src/build/lib/jars/jackson-core-asl-1.4.2.jar:/Library/apac
    h
    e-cassandra-1.1.7-src/build/lib/jars/jackson-mapper-asl-1.4.2.jar:/Library
    /
    apache-cassandra-1.1.7-src/build/lib/jars/jasper-compiler-5.5.12.jar:/Libr
    a
    ry/apache-cassandra-1.1.7-src/build/lib/jars/jasper-runtime-5.5.12.jar:/Li
    b
    rary/apache-cassandra-1.1.7-src/build/lib/jars/jets3t-0.7.1.jar:/Library/a
    p
    ache-cassandra-1.1.7-src/build/lib/jars/jetty-6.1.26.jar:/Library/apache-c
    a
    ssandra-1.1.7-src/build/lib/jars/jetty-util-6.1.26.jar:/Library/apache-cas
    s
    andra-1.1.7-src/build/lib/jars/jna-3.2.7.jar:/Library/apache-cassandra-1.1
    .
    7-src/build/lib/jars/jopt-simple-3.2.jar:/Library/apache-cassandra-1.1.7-s
    r
    c/build/lib/jars/jsp-2.1-6.1.14.jar:/Library/apache-cassandra-1.1.7-src/bu
    i
    ld/lib/jars/jsp-api-2.1-6.1.14.jar:/Library/apache-cassandra-1.1.7-src/bui
    l
    d/lib/jars/junit-4.6.jar:/Library/apache-cassandra-1.1.7-src/build/lib/jar
    s
    /kfs-0.3.jar:/Library/apache-cassandra-1.1.7-src/build/lib/jars/oro-2.0.8.
    j
    ar:/Library/apache-cassandra-1.1.7-src/build/lib/jars/paranamer-2.2.jar:/L
    i
    brary/apache-cassandra-1.1.7-src/build/lib/jars/paranamer-ant-2.1.jar:/Lib
    r
    ary/apache-cassandra-1.1.7-src/build/lib/jars/paranamer-generator-2.1.jar:
    /
    Library/apache-cassandra-1.1.7-src/build/lib/jars/pig-0.9.2.jar:/Library/a
    p
    ache-cassandra-1.1.7-src/build/lib/jars/qdox-1.10.jar:/Library/apache-cass
    a
    ndra-1.1.7-src/build/lib/jars/servlet-api-2.5-20081211.jar:/Library/apache
    -
    cassandra-1.1.7-src/build/lib/jars/servlet-api-2.5-6.1.14.jar:/Library/apa
    c
    he-cassandra-1.1.7-src/build/lib/jars/slf4j-api-1.5.11.jar:/Library/apache
    -
    cassandra-1.1.7-src/build/lib/jars/xmlenc-0.52.jar:/Library/apache-cassand
    r
    a-1.1.7-src/build/apache-cassandra-1.1.7-SNAPSHOT.jar:/Library/apache-cass
    a
    ndra-1.1.7-src/build/apache-cassandra-clientutil-1.1.7-SNAPSHOT.jar:/Libra
    r
    y/apache-cassandra-1.1.7-src/build/apache-cassandra-thrift-1.1.7-SNAPSHOT.
    j
    ar:/Library/pig-0.10.0/pig-0.10.0.jar:/Library/pig-0.10.0/bin/../lib/autom
    a
    ton.jar:/Library/pig-0.10.0/bin/../lib/jython-2.5.0.jar:/Library/pig-0.10.

    /bin/../pig-0.10.0-withouthadoop.jar:
    HADOOP_OPTS: -Xmx1000m -Dudf.import.list=org.apache.cassandra.hadoop.pig
    -Dpig.log.dir=/Library/pig-0.10.0/bin/../logs -Dpig.log.file=pig.log
    -Dpig.home.dir=/Library/pig-0.10.0/bin/..
    /Library/hadoop-1.0.2/bin/hadoop jar
    /Library/pig-0.10.0/bin/../pig-0.10.0-withouthadoop.jar
    -Dudf.import.list=org.apache.cassandra.hadoop.pig -x local rowcount.pig






    On 12/11/12 1:10 PM, "Jonathan Coveney" wrote:

    Sounds like there could be a wrong version on the classpath then


    2012/12/11 William Oberman <oberman@civicscience.com>
    Your line numbers aren't matching up to the 1.1.7 release, which is
    weird.
    Based on the "stock" 1.1.7 source, there was a null check on str
    before predicateFromString(str),
    making your code path impossible...

    will


    On Tue, Dec 11, 2012 at 1:00 PM, Jonathan Coveney <jcoveney@gmail.com
    wrote:
    If I were debugging this (note, I know nothing about cassandra), I would
    put a flag in my ide on cassandra storage and see what is going on in
    there, and why it is erroring out. Then I would follow that backwards into
    whatever in Pig was generating that issue. That's pretty vague but can't
    really say much else unless I knew a ton about CassandraStorage.


    2012/12/10 James Schappet <jschappet@gmail.com>
    Any thoughts on how I can start diagnosing this problem?


    On 12/5/12 12:43 PM, "Schappet, James C" <james-schappet@uiowa.edu>
    wrote:
    Hi folks,

    I am new to pig, and I am trying to get the basic pig + cassandra
    samples
    working.

    I have created the PigTest Keyspace, and I am trying to run some
    of
    the
    command in test_storage.pig, but I get the following:


    tsunami:pig schappetj$ bin/pig_cassandra -x local
    Using /Library/pig-0.10.0/pig-0.10.0.jar.
    2012-12-05 12:28:50,415 [main] INFO org.apache.pig.Main - Logging
    error
    messages to:
    /Library/apache-cassandra-1.1.7-src/examples/pig/pig_1354732130411.log
    2012-12-05 12:28:50,616 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
    Connecting to hadoop file system at: file:///
    2012-12-05 12:28:50.701 java[17814:1c03] Unable to load realm info
    from
    SCDynamicStore
    grunt> U8 = load 'cassandra://PigTest/U8' using
    CassandraStorage();
    2012-12-05 12:28:54,084 [main] ERROR
    org.apache.pig.tools.grunt.Grunt
    -
    ERROR 2999: Unexpected internal error. null
    Details at logfile:
    /Library/apache-cassandra-1.1.7-src/examples/pig/pig_1354732130411.log
    grunt>




    /Library/apache-cassandra-1.1.7-src/examples/pig/pig_1354732130411.log
    Pig Stack Trace
    ---------------
    ERROR 2999: Unexpected internal error. null

    java.lang.NullPointerException
    at org.apache.cassandra.utils.Hex.hexToBytes(Hex.java:51)
    at
    org.apache.cassandra.hadoop.ConfigHelper.predicateFromString(ConfigHelp
    e
    r.
    java:206)
    at
    org.apache.cassandra.hadoop.ConfigHelper.getInputSlicePredicate(ConfigH
    e
    lp
    er.java:176)
    at
    org.apache.cassandra.hadoop.pig.CassandraStorage.setLocation(CassandraS
    t
    or
    age.java:567)
    at
    org.apache.cassandra.hadoop.pig.CassandraStorage.getSchema(CassandraSto
    r
    ag
    e.java:600)
    at
    org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(
    L
    OL
    oad.java:150)
    at
    org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:
    1
    09
    )
    at
    org.apache.pig.newplan.logical.visitor.LineageFindRelVisitor.visit(Line
    a
    ge
    FindRelVisitor.java:100)
    at
    org.apache.pig.newplan.logical.relational.LOLoad.accept(LOLoad.java:218
    )
    at
    org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker
    .
    ja
    va:75)
    at
    org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
    at
    org.apache.pig.newplan.logical.visitor.CastLineageSetter.<init>(CastLin
    e
    ag
    eSetter.java:57)
    at
    org.apache.pig.PigServer$Graph.compile(PigServer.java:1679)
    at
    org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1610)
    at
    org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
    at
    org.apache.pig.PigServer.registerQuery(PigServer.java:584)
    at
    org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:942)
    at
    org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptPa
    r
    se
    r.java:386)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.jav
    a
    :1
    88)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.jav
    a
    :1
    64)
    at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
    at org.apache.pig.Main.run(Main.java:495)
    at org.apache.pig.Main.main(Main.java:111)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
    Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja
    v
    a:
    57)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso
    r
    Im
    pl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)



    PIG 0.10.0
    Cassandra 1.1.7 w/source

    ENV Osx Lion:
    $ java -version
    java version "1.7.0_04"
    Java(TM) SE Runtime Environment (build 1.7.0_04-b21)
    Java HotSpot(TM) 64-Bit Server VM (build 23.0-b21, mixed mode)

    set | grep PIG
    PIG_HOME=/Library/pig-0.10.0
    PIG_INITIAL_ADDRESS=localhost
    PIG_PARTITIONER=org.apache.cassandra.dht.RandomPartitioner
    PIG_RPC_PORT=9160
  • Jonathan Coveney at Dec 12, 2012 at 6:08 pm
    Glad you got to the bottom of it, and thanks for letting us know what fixed
    it!


    2012/12/12 James Schappet <jschappet@gmail.com>
    I was able to resolve this issue.


    Taking Hadoop out of my .profile, resolved the issue.
    PATH included: /Library/hadoop-1.0.2/bin, once removed I am able to
    connect Cassandra.

    Thanks for pointing me in the right direction.


    On 12/11/12 1:13 PM, "James Schappet" wrote:

    This is what pig_cassandra with debug enabled outputs:




    tsunami:pig schappetj$ bin/pig_cassandra -x local rowcount.pig
    Using /Library/pig-0.10.0/pig-0.10.0.jar.
    Find hadoop at /Library/hadoop-1.0.2/bin/hadoop
    dry run:
    HADOOP_CLASSPATH:
    /Library/pig-0.10.0/bin/../conf:/Library/Java/JavaVirtualMachines/1.7.0.jd
    k
    /Contents/Home/lib/tools.jar:::/Library/apache-cassandra-1.1.7-src/lib/ant
    l
    r-3.2.jar:/Library/apache-cassandra-1.1.7-src/lib/avro-1.4.0-fixes.jar:/Li
    b
    rary/apache-cassandra-1.1.7-src/lib/avro-1.4.0-sources-fixes.jar:/Library/
    a
    pache-cassandra-1.1.7-src/lib/commons-cli-1.1.jar:/Library/apache-cassandr
    a
    -1.1.7-src/lib/commons-codec-1.2.jar:/Library/apache-cassandra-1.1.7-src/l
    i
    b/commons-lang-2.4.jar:/Library/apache-cassandra-1.1.7-src/lib/compress-lz
    f
    -0.8.4.jar:/Library/apache-cassandra-1.1.7-src/lib/concurrentlinkedhashmap
    -
    lru-1.3.jar:/Library/apache-cassandra-1.1.7-src/lib/guava-r08.jar:/Library
    /
    apache-cassandra-1.1.7-src/lib/high-scale-lib-1.1.2.jar:/Library/apache-ca
    s
    sandra-1.1.7-src/lib/jackson-core-asl-1.9.2.jar:/Library/apache-cassandra-
    1
    .1.7-src/lib/jackson-mapper-asl-1.9.2.jar:/Library/apache-cassandra-1.1.7-
    s
    rc/lib/jamm-0.2.5.jar:/Library/apache-cassandra-1.1.7-src/lib/jline-0.9.94
    .
    jar:/Library/apache-cassandra-1.1.7-src/lib/json-simple-1.1.jar:/Library/a
    p
    ache-cassandra-1.1.7-src/lib/libthrift-0.7.0.jar:/Library/apache-cassandra
    -
    1.1.7-src/lib/log4j-1.2.16.jar:/Library/apache-cassandra-1.1.7-src/lib/met
    r
    ics-core-2.0.3.jar:/Library/apache-cassandra-1.1.7-src/lib/servlet-api-2.5
    -
    20081211.jar:/Library/apache-cassandra-1.1.7-src/lib/slf4j-api-1.6.1.jar:/
    L
    ibrary/apache-cassandra-1.1.7-src/lib/slf4j-log4j12-1.6.1.jar:/Library/apa
    c
    he-cassandra-1.1.7-src/lib/snakeyaml-1.6.jar:/Library/apache-cassandra-1.1
    .
    7-src/lib/snappy-java-1.0.4.1.jar:/Library/apache-cassandra-1.1.7-src/lib/
    s
    naptree-0.1.jar:/Library/apache-cassandra-1.1.7-src/build/lib/jars/ant-1.6
    .
    5.jar:/Library/apache-cassandra-1.1.7-src/build/lib/jars/apache-rat-0.6.ja
    r
    :/Library/apache-cassandra-1.1.7-src/build/lib/jars/apache-rat-core-0.6.ja
    r
    :/Library/apache-cassandra-1.1.7-src/build/lib/jars/apache-rat-tasks-0.6.j
    a
    r:/Library/apache-cassandra-1.1.7-src/build/lib/jars/asm-3.2.jar:/Library/
    a
    pache-cassandra-1.1.7-src/build/lib/jars/avro-1.3.2.jar:/Library/apache-ca
    s
    sandra-1.1.7-src/build/lib/jars/commons-beanutils-1.7.0.jar:/Library/apach
    e
    -cassandra-1.1.7-src/build/lib/jars/commons-beanutils-core-1.8.0.jar:/Libr
    a
    ry/apache-cassandra-1.1.7-src/build/lib/jars/commons-cli-1.2.jar:/Library/
    a
    pache-cassandra-1.1.7-src/build/lib/jars/commons-codec-1.4.jar:/Library/ap
    a
    che-cassandra-1.1.7-src/build/lib/jars/commons-collections-3.2.jar:/Librar
    y
    /apache-cassandra-1.1.7-src/build/lib/jars/commons-configuration-1.6.jar:/
    L
    ibrary/apache-cassandra-1.1.7-src/build/lib/jars/commons-digester-1.8.jar:
    /
    Library/apache-cassandra-1.1.7-src/build/lib/jars/commons-el-1.0.jar:/Libr
    a
    ry/apache-cassandra-1.1.7-src/build/lib/jars/commons-httpclient-3.0.1.jar:
    /
    Library/apache-cassandra-1.1.7-src/build/lib/jars/commons-lang-2.4.jar:/Li
    b
    rary/apache-cassandra-1.1.7-src/build/lib/jars/commons-logging-1.1.1.jar:/
    L
    ibrary/apache-cassandra-1.1.7-src/build/lib/jars/commons-math-2.1.jar:/Lib
    r
    ary/apache-cassandra-1.1.7-src/build/lib/jars/commons-net-1.4.1.jar:/Libra
    r
    y/apache-cassandra-1.1.7-src/build/lib/jars/core-3.1.1.jar:/Library/apache
    -
    cassandra-1.1.7-src/build/lib/jars/hadoop-core-0.20.203.0.jar:/Library/apa
    c
    he-cassandra-1.1.7-src/build/lib/jars/hsqldb-1.8.0.10.jar:/Library/apache-
    c
    assandra-1.1.7-src/build/lib/jars/jackson-core-asl-1.4.2.jar:/Library/apac
    h
    e-cassandra-1.1.7-src/build/lib/jars/jackson-mapper-asl-1.4.2.jar:/Library
    /
    apache-cassandra-1.1.7-src/build/lib/jars/jasper-compiler-5.5.12.jar:/Libr
    a
    ry/apache-cassandra-1.1.7-src/build/lib/jars/jasper-runtime-5.5.12.jar:/Li
    b
    rary/apache-cassandra-1.1.7-src/build/lib/jars/jets3t-0.7.1.jar:/Library/a
    p
    ache-cassandra-1.1.7-src/build/lib/jars/jetty-6.1.26.jar:/Library/apache-c
    a
    ssandra-1.1.7-src/build/lib/jars/jetty-util-6.1.26.jar:/Library/apache-cas
    s
    andra-1.1.7-src/build/lib/jars/jna-3.2.7.jar:/Library/apache-cassandra-1.1
    .
    7-src/build/lib/jars/jopt-simple-3.2.jar:/Library/apache-cassandra-1.1.7-s
    r
    c/build/lib/jars/jsp-2.1-6.1.14.jar:/Library/apache-cassandra-1.1.7-src/bu
    i
    ld/lib/jars/jsp-api-2.1-6.1.14.jar:/Library/apache-cassandra-1.1.7-src/bui
    l
    d/lib/jars/junit-4.6.jar:/Library/apache-cassandra-1.1.7-src/build/lib/jar
    s
    /kfs-0.3.jar:/Library/apache-cassandra-1.1.7-src/build/lib/jars/oro-2.0.8.
    j
    ar:/Library/apache-cassandra-1.1.7-src/build/lib/jars/paranamer-2.2.jar:/L
    i
    brary/apache-cassandra-1.1.7-src/build/lib/jars/paranamer-ant-2.1.jar:/Lib
    r
    ary/apache-cassandra-1.1.7-src/build/lib/jars/paranamer-generator-2.1.jar:
    /
    Library/apache-cassandra-1.1.7-src/build/lib/jars/pig-0.9.2.jar:/Library/a
    p
    ache-cassandra-1.1.7-src/build/lib/jars/qdox-1.10.jar:/Library/apache-cass
    a
    ndra-1.1.7-src/build/lib/jars/servlet-api-2.5-20081211.jar:/Library/apache
    -
    cassandra-1.1.7-src/build/lib/jars/servlet-api-2.5-6.1.14.jar:/Library/apa
    c
    he-cassandra-1.1.7-src/build/lib/jars/slf4j-api-1.5.11.jar:/Library/apache
    -
    cassandra-1.1.7-src/build/lib/jars/xmlenc-0.52.jar:/Library/apache-cassand
    r
    a-1.1.7-src/build/apache-cassandra-1.1.7-SNAPSHOT.jar:/Library/apache-cass
    a
    ndra-1.1.7-src/build/apache-cassandra-clientutil-1.1.7-SNAPSHOT.jar:/Libra
    r
    y/apache-cassandra-1.1.7-src/build/apache-cassandra-thrift-1.1.7-SNAPSHOT.
    j
    ar:/Library/pig-0.10.0/pig-0.10.0.jar:/Library/pig-0.10.0/bin/../lib/autom
    a
    ton.jar:/Library/pig-0.10.0/bin/../lib/jython-2.5.0.jar:/Library/pig-0.10.

    /bin/../pig-0.10.0-withouthadoop.jar:
    HADOOP_OPTS: -Xmx1000m -Dudf.import.list=org.apache.cassandra.hadoop.pig
    -Dpig.log.dir=/Library/pig-0.10.0/bin/../logs -Dpig.log.file=pig.log
    -Dpig.home.dir=/Library/pig-0.10.0/bin/..
    /Library/hadoop-1.0.2/bin/hadoop jar
    /Library/pig-0.10.0/bin/../pig-0.10.0-withouthadoop.jar
    -Dudf.import.list=org.apache.cassandra.hadoop.pig -x local rowcount.pig






    On 12/11/12 1:10 PM, "Jonathan Coveney" wrote:

    Sounds like there could be a wrong version on the classpath then


    2012/12/11 William Oberman <oberman@civicscience.com>
    Your line numbers aren't matching up to the 1.1.7 release, which is
    weird.
    Based on the "stock" 1.1.7 source, there was a null check on str
    before predicateFromString(str),
    making your code path impossible...

    will


    On Tue, Dec 11, 2012 at 1:00 PM, Jonathan Coveney <jcoveney@gmail.com
    wrote:
    If I were debugging this (note, I know nothing about cassandra), I would
    put a flag in my ide on cassandra storage and see what is going on in
    there, and why it is erroring out. Then I would follow that backwards into
    whatever in Pig was generating that issue. That's pretty vague but can't
    really say much else unless I knew a ton about CassandraStorage.


    2012/12/10 James Schappet <jschappet@gmail.com>
    Any thoughts on how I can start diagnosing this problem?


    On 12/5/12 12:43 PM, "Schappet, James C" <james-schappet@uiowa.edu
    wrote:
    Hi folks,

    I am new to pig, and I am trying to get the basic pig + cassandra
    samples
    working.

    I have created the PigTest Keyspace, and I am trying to run some
    of
    the
    command in test_storage.pig, but I get the following:


    tsunami:pig schappetj$ bin/pig_cassandra -x local
    Using /Library/pig-0.10.0/pig-0.10.0.jar.
    2012-12-05 12:28:50,415 [main] INFO org.apache.pig.Main - Logging
    error
    messages to:
    /Library/apache-cassandra-1.1.7-src/examples/pig/pig_1354732130411.log
    2012-12-05 12:28:50,616 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
    Connecting to hadoop file system at: file:///
    2012-12-05 12:28:50.701 java[17814:1c03] Unable to load realm info
    from
    SCDynamicStore
    grunt> U8 = load 'cassandra://PigTest/U8' using
    CassandraStorage();
    2012-12-05 12:28:54,084 [main] ERROR
    org.apache.pig.tools.grunt.Grunt
    -
    ERROR 2999: Unexpected internal error. null
    Details at logfile:
    /Library/apache-cassandra-1.1.7-src/examples/pig/pig_1354732130411.log
    grunt>




    /Library/apache-cassandra-1.1.7-src/examples/pig/pig_1354732130411.log
    Pig Stack Trace
    ---------------
    ERROR 2999: Unexpected internal error. null

    java.lang.NullPointerException
    at org.apache.cassandra.utils.Hex.hexToBytes(Hex.java:51)
    at
    org.apache.cassandra.hadoop.ConfigHelper.predicateFromString(ConfigHelp
    e
    r.
    java:206)
    at
    org.apache.cassandra.hadoop.ConfigHelper.getInputSlicePredicate(ConfigH
    e
    lp
    er.java:176)
    at
    org.apache.cassandra.hadoop.pig.CassandraStorage.setLocation(CassandraS
    t
    or
    age.java:567)
    at
    org.apache.cassandra.hadoop.pig.CassandraStorage.getSchema(CassandraSto
    r
    ag
    e.java:600)
    at
    org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(
    L
    OL
    oad.java:150)
    at
    org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:
    1
    09
    )
    at
    org.apache.pig.newplan.logical.visitor.LineageFindRelVisitor.visit(Line
    a
    ge
    FindRelVisitor.java:100)
    at
    org.apache.pig.newplan.logical.relational.LOLoad.accept(LOLoad.java:218
    )
    at
    org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker
    .
    ja
    va:75)
    at
    org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
    at
    org.apache.pig.newplan.logical.visitor.CastLineageSetter.<init>(CastLin
    e
    ag
    eSetter.java:57)
    at
    org.apache.pig.PigServer$Graph.compile(PigServer.java:1679)
    at
    org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1610)
    at
    org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
    at
    org.apache.pig.PigServer.registerQuery(PigServer.java:584)
    at
    org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:942)
    at
    org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptPa
    r
    se
    r.java:386)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.jav
    a
    :1
    88)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.jav
    a
    :1
    64)
    at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
    at org.apache.pig.Main.run(Main.java:495)
    at org.apache.pig.Main.main(Main.java:111)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
    Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja
    v
    a:
    57)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso
    r
    Im
    pl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)



    PIG 0.10.0
    Cassandra 1.1.7 w/source

    ENV Osx Lion:
    $ java -version
    java version "1.7.0_04"
    Java(TM) SE Runtime Environment (build 1.7.0_04-b21)
    Java HotSpot(TM) 64-Bit Server VM (build 23.0-b21, mixed mode)

    set | grep PIG
    PIG_HOME=/Library/pig-0.10.0
    PIG_INITIAL_ADDRESS=localhost
    PIG_PARTITIONER=org.apache.cassandra.dht.RandomPartitioner
    PIG_RPC_PORT=9160

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedDec 5, '12 at 8:06p
activeDec 12, '12 at 6:08p
posts9
users4
websitepig.apache.org

People

Translate

site design / logo © 2021 Grokbase