Grokbase Groups Pig user May 2013
FAQ
Hi,

I have 2 differents behaviour for the same Pig version (with hadoop 1.1.2) on different servers. If anyone can tell me why with the same versions and the same parameters I don't have the same behaviour. I'm trying to set the builtin import list but it's not used with one hadoop cluster whereas it's used when I use it on a single node cluster :

+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs'
+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log'
+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/..'
+ '[' '' == true ']'
+ '[' -n /usr/local/bin/hadoop ']'
+ '[' false == true ']'
+ '[' -f /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-withouthadoop.jar ']'
++ echo /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
+ PIG_JAR=/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
+ '[' -z /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
+ '[' -n /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
+ CLASSPATH='/etc/hadoop/conf:/usr/lib/jvm/java-7-oracle/lib/tools.jar:::/opt/apache-cassandra-1.2.3/lib/*:/usr/share/hadoop/lib/*:/opt/pig-0.11.1/pig-0.11.1-withouthadoop.jar:/etc/hadoop:/opt/pig-0.11.1/bin/../lib/jython-standalone-2.5.3.jar:/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar'
+ export 'HADOOP_CLASSPATH=/etc/hadoop/conf:/usr/lib/jvm/java-7-oracle/lib/tools.jar:::/opt/apache-cassandra-1.2.3/lib/*:/usr/share/hadoop/lib/*:/opt/pig-0.11.1/pig-0.11.1-withouthadoop.jar:/etc/hadoop:/opt/pig-0.11.1/bin/../lib/jython-standalone-2.5.3.jar:/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
+ HADOOP_CLASSPATH='/etc/hadoop/conf:/usr/lib/jvm/java-7-oracle/lib/tools.jar:::/opt/apache-cassandra-1.2.3/lib/*:/usr/share/hadoop/lib/*:/opt/pig-0.11.1/pig-0.11.1-withouthadoop.jar:/etc/hadoop:/opt/pig-0.11.1/bin/../lib/jython-standalone-2.5.3.jar:/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
+ export 'HADOOP_OPTS=-Xmx1000m -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/opt/apache-cassandra-1.2.3/lib/apache-cassandra-thrift-1.2.3.jar:/opt/apache-cassandra-1.2.3/lib/libthrift-0.7.0.jar -4 log4j.properties -Dpig.log.dir=/opt/pig-0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/opt/pig-0.11.1/bin/.. -Djava.net.preferIPv4Stack=true '
+ HADOOP_OPTS='-Xmx1000m -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/opt/apache-cassandra-1.2.3/lib/apache-cassandra-thrift-1.2.3.jar:/opt/apache-cassandra-1.2.3/lib/libthrift-0.7.0.jar -4 log4j.properties -Dpig.log.dir=/opt/pig-0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/opt/pig-0.11.1/bin/.. -Djava.net.preferIPv4Stack=true '
+ '[' false == true ']'
+ exec /usr/bin/hadoop jar /opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar ./hadoop-sum-value_1.pig
2013-05-03 07:57:55,140 [main] INFO org.apache.pig.Main - Apache Pig version 0.11.1 (r1459641) compiled Mar 22 2013, 02:13:53
2013-05-03 07:57:55,141 [main] INFO org.apache.pig.Main - Logging error messages to: /home/ubuntu/pig_1367567875107.log
2013-05-03 07:57:57,502 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/ubuntu/.pigbootup not found
2013-05-03 07:57:58,461 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://hadoop-cass-1:9100
2013-05-03 07:57:59,663 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: hadoop-cass-1:8021
2013-05-03 07:58:01,387 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve CassandraStorage2 using imports: [, org.apache.pig.builtin., org.apache.pig.impl.builtin.]

+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs'
+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log'
+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/..'
+ '[' '' == true ']'
+ '[' -n /usr/local/bin/hadoop ']'
+ '[' false == true ']'
+ '[' -f /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-withouthadoop.jar ']'
++ echo /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
+ PIG_JAR=/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
+ '[' -z /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
+ '[' -n /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
+ CLASSPATH='/usr/local/Cellar/hadoop/1.1.2/libexec/conf:/Library/Java/JavaVirtualMachines/1.6.0_32-b05-420.jdk/Contents/Home/lib/tools.jar:::/usr/local/Cellar/cassandra/1.2.4/*:/usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar:/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar'
+ export 'HADOOP_CLASSPATH=/usr/local/Cellar/hadoop/1.1.2/libexec/conf:/Library/Java/JavaVirtualMachines/1.6.0_32-b05-420.jdk/Contents/Home/lib/tools.jar:::/usr/local/Cellar/cassandra/1.2.4/*:/usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar:/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
+ HADOOP_CLASSPATH='/usr/local/Cellar/hadoop/1.1.2/libexec/conf:/Library/Java/JavaVirtualMachines/1.6.0_32-b05-420.jdk/Contents/Home/lib/tools.jar:::/usr/local/Cellar/cassandra/1.2.4/*:/usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar:/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
+ export 'HADOOP_OPTS=-Xmx1000m -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/.. '
+ HADOOP_OPTS='-Xmx1000m -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/.. '
+ '[' false == true ']'
+ exec /usr/local/bin/hadoop jar /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ./hadoop-sum-value_1.pig
2013-05-03 09:57:56,810 [main] INFO org.apache.pig.Main - Apache Pig version 0.11.1 (r1459641) compiled Mar 22 2013, 02:13:53
2013-05-03 09:57:56,811 [main] INFO org.apache.pig.Main - Logging error messages to: /private/tmp/pig_1367567876808.log
2013-05-03 09:57:56.980 java[81053:1203] Unable to load realm info from SCDynamicStore
2013-05-03 09:57:57,190 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /Users/cyril/.pigbootup not found
2013-05-03 09:57:57,333 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://localhost:9000
2013-05-03 09:57:57,513 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: localhost:9001
2013-05-03 09:57:57,944 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve CassandraStorage2 using imports: [org.apache.cassandra.hadoop.pig., , org.apache.pig.builtin., org.apache.pig.impl.builtin.]


I'm using an undefined function just to get the error which indicates the import list which is not the same as you can see even if I use the same options :
PIG_OPTS = ' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs'

If anyone has an explanation ?

thanks
--
Cyril SCETBON


_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

Search Discussions

  • Cscetbon Ext at May 6, 2013 at 8:08 am
    forget it ! It came from hadoop-env.sh provided by the package I installed on one cluster :(

    Regards
    --
    Cyril SCETBON

    On May 3, 2013, at 11:44 AM, cscetbon.ext@orange.com wrote:

    Hi,

    I have 2 differents behaviour for the same Pig version (with hadoop 1.1.2) on different servers. If anyone can tell me why with the same versions and the same parameters I don't have the same behaviour. I'm trying to set the builtin import list but it's not used with one hadoop cluster whereas it's used when I use it on a single node cluster :

    + PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs'
    + PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log'
    + PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/..'
    + '[' '' == true ']'
    + '[' -n /usr/local/bin/hadoop ']'
    + '[' false == true ']'
    + '[' -f /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-withouthadoop.jar ']'
    ++ echo /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
    + PIG_JAR=/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
    + '[' -z /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
    + '[' -n /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
    + CLASSPATH='/etc/hadoop/conf:/usr/lib/jvm/java-7-oracle/lib/tools.jar:::/opt/apache-cassandra-1.2.3/lib/*:/usr/share/hadoop/lib/*:/opt/pig-0.11.1/pig-0.11.1-withouthadoop.jar:/etc/hadoop:/opt/pig-0.11.1/bin/../lib/jython-standalone-2.5.3.jar:/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar'
    + export 'HADOOP_CLASSPATH=/etc/hadoop/conf:/usr/lib/jvm/java-7-oracle/lib/tools.jar:::/opt/apache-cassandra-1.2.3/lib/*:/usr/share/hadoop/lib/*:/opt/pig-0.11.1/pig-0.11.1-withouthadoop.jar:/etc/hadoop:/opt/pig-0.11.1/bin/../lib/jython-standalone-2.5.3.jar:/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
    + HADOOP_CLASSPATH='/etc/hadoop/conf:/usr/lib/jvm/java-7-oracle/lib/tools.jar:::/opt/apache-cassandra-1.2.3/lib/*:/usr/share/hadoop/lib/*:/opt/pig-0.11.1/pig-0.11.1-withouthadoop.jar:/etc/hadoop:/opt/pig-0.11.1/bin/../lib/jython-standalone-2.5.3.jar:/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
    + export 'HADOOP_OPTS=-Xmx1000m -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/opt/apache-cassandra-1.2.3/lib/apache-cassandra-thrift-1.2.3.jar:/opt/apache-cassandra-1.2.3/lib/libthrift-0.7.0.jar -4 log4j.properties -Dpig.log.dir=/opt/pig-0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/opt/pig-0.11.1/bin/.. -Djava.net.preferIPv4Stack=true '
    + HADOOP_OPTS='-Xmx1000m -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/opt/apache-cassandra-1.2.3/lib/apache-cassandra-thrift-1.2.3.jar:/opt/apache-cassandra-1.2.3/lib/libthrift-0.7.0.jar -4 log4j.properties -Dpig.log.dir=/opt/pig-0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/opt/pig-0.11.1/bin/.. -Djava.net.preferIPv4Stack=true '
    + '[' false == true ']'
    + exec /usr/bin/hadoop jar /opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar ./hadoop-sum-value_1.pig
    2013-05-03 07:57:55,140 [main] INFO org.apache.pig.Main - Apache Pig version 0.11.1 (r1459641) compiled Mar 22 2013, 02:13:53
    2013-05-03 07:57:55,141 [main] INFO org.apache.pig.Main - Logging error messages to: /home/ubuntu/pig_1367567875107.log
    2013-05-03 07:57:57,502 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/ubuntu/.pigbootup not found
    2013-05-03 07:57:58,461 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://hadoop-cass-1:9100
    2013-05-03 07:57:59,663 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: hadoop-cass-1:8021
    2013-05-03 07:58:01,387 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve CassandraStorage2 using imports: [, org.apache.pig.builtin., org.apache.pig.impl.builtin.]

    + PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs'
    + PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log'
    + PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/..'
    + '[' '' == true ']'
    + '[' -n /usr/local/bin/hadoop ']'
    + '[' false == true ']'
    + '[' -f /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-withouthadoop.jar ']'
    ++ echo /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
    + PIG_JAR=/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
    + '[' -z /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
    + '[' -n /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
    + CLASSPATH='/usr/local/Cellar/hadoop/1.1.2/libexec/conf:/Library/Java/JavaVirtualMachines/1.6.0_32-b05-420.jdk/Contents/Home/lib/tools.jar:::/usr/local/Cellar/cassandra/1.2.4/*:/usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar:/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar'
    + export 'HADOOP_CLASSPATH=/usr/local/Cellar/hadoop/1.1.2/libexec/conf:/Library/Java/JavaVirtualMachines/1.6.0_32-b05-420.jdk/Contents/Home/lib/tools.jar:::/usr/local/Cellar/cassandra/1.2.4/*:/usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar:/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
    + HADOOP_CLASSPATH='/usr/local/Cellar/hadoop/1.1.2/libexec/conf:/Library/Java/JavaVirtualMachines/1.6.0_32-b05-420.jdk/Contents/Home/lib/tools.jar:::/usr/local/Cellar/cassandra/1.2.4/*:/usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar:/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
    + export 'HADOOP_OPTS=-Xmx1000m -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/.. '
    + HADOOP_OPTS='-Xmx1000m -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/.. '
    + '[' false == true ']'
    + exec /usr/local/bin/hadoop jar /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ./hadoop-sum-value_1.pig
    2013-05-03 09:57:56,810 [main] INFO org.apache.pig.Main - Apache Pig version 0.11.1 (r1459641) compiled Mar 22 2013, 02:13:53
    2013-05-03 09:57:56,811 [main] INFO org.apache.pig.Main - Logging error messages to: /private/tmp/pig_1367567876808.log
    2013-05-03 09:57:56.980 java[81053:1203] Unable to load realm info from SCDynamicStore
    2013-05-03 09:57:57,190 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /Users/cyril/.pigbootup not found
    2013-05-03 09:57:57,333 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://localhost:9000
    2013-05-03 09:57:57,513 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: localhost:9001
    2013-05-03 09:57:57,944 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve CassandraStorage2 using imports: [org.apache.cassandra.hadoop.pig., , org.apache.pig.builtin., org.apache.pig.impl.builtin.]


    I'm using an undefined function just to get the error which indicates the import list which is not the same as you can see even if I use the same options :
    PIG_OPTS = ' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs'

    If anyone has an explanation ?

    thanks
    --
    Cyril SCETBON


    _________________________________________________________________________________________________________________________

    Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
    pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
    a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
    France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

    This message and its attachments may contain confidential or privileged information that may be protected by law;
    they should not be distributed, used or copied without authorisation.
    If you have received this email in error, please notify the sender and delete this message and its attachments.
    As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified.
    Thank you.



    _________________________________________________________________________________________________________________________

    Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
    pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
    a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
    France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

    This message and its attachments may contain confidential or privileged information that may be protected by law;
    they should not be distributed, used or copied without authorisation.
    If you have received this email in error, please notify the sender and delete this message and its attachments.
    As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified.
    Thank you.
  • Prashant Kommireddi at May 6, 2013 at 5:35 pm
    This blog post describes some of this behavior
    http://hadoopified.wordpress.com/2013/04/07/pig-startup-script-behavior/

    On Mon, May 6, 2013 at 1:07 AM, wrote:

    forget it ! It came from hadoop-env.sh provided by the package I installed
    on one cluster :(

    Regards
    --
    Cyril SCETBON

    On May 3, 2013, at 11:44 AM, cscetbon.ext@orange.com<mailto:
    cscetbon.ext@orange.com> wrote:

    Hi,

    I have 2 differents behaviour for the same Pig version (with hadoop 1.1.2)
    on different servers. If anyone can tell me why with the same versions and
    the same parameters I don't have the same behaviour. I'm trying to set the
    builtin import list but it's not used with one hadoop cluster whereas it's
    used when I use it on a single node cluster :

    + PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig
    -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
    -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs'
    + PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig
    -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
    -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs
    -Dpig.log.file=pig.log'
    + PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig
    -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
    -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs
    -Dpig.log.file=pig.log
    -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/..'
    + '[' '' == true ']'
    + '[' -n /usr/local/bin/hadoop ']'
    + '[' false == true ']'
    + '[' -f /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-withouthadoop.jar
    ']'
    ++ echo
    /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
    +
    PIG_JAR=/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
    + '[' -z
    /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
    + '[' -n
    /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
    +
    CLASSPATH='/etc/hadoop/conf:/usr/lib/jvm/java-7-oracle/lib/tools.jar:::/opt/apache-cassandra-1.2.3/lib/*:/usr/share/hadoop/lib/*:/opt/pig-0.11.1/pig-0.11.1-withouthadoop.jar:/etc/hadoop:/opt/pig-0.11.1/bin/../lib/jython-standalone-2.5.3.jar:/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar'
    + export
    'HADOOP_CLASSPATH=/etc/hadoop/conf:/usr/lib/jvm/java-7-oracle/lib/tools.jar:::/opt/apache-cassandra-1.2.3/lib/*:/usr/share/hadoop/lib/*:/opt/pig-0.11.1/pig-0.11.1-withouthadoop.jar:/etc/hadoop:/opt/pig-0.11.1/bin/../lib/jython-standalone-2.5.3.jar:/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
    +
    HADOOP_CLASSPATH='/etc/hadoop/conf:/usr/lib/jvm/java-7-oracle/lib/tools.jar:::/opt/apache-cassandra-1.2.3/lib/*:/usr/share/hadoop/lib/*:/opt/pig-0.11.1/pig-0.11.1-withouthadoop.jar:/etc/hadoop:/opt/pig-0.11.1/bin/../lib/jython-standalone-2.5.3.jar:/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
    + export 'HADOOP_OPTS=-Xmx1000m
    -Dudf.import.list=org.apache.cassandra.hadoop.pig
    -Dpig.additional.jars=/opt/apache-cassandra-1.2.3/lib/apache-cassandra-thrift-1.2.3.jar:/opt/apache-cassandra-1.2.3/lib/libthrift-0.7.0.jar
    -4 log4j.properties -Dpig.log.dir=/opt/pig-0.11.1/bin/../logs
    -Dpig.log.file=pig.log -Dpig.home.dir=/opt/pig-0.11.1/bin/..
    -Djava.net.preferIPv4Stack=true '
    + HADOOP_OPTS='-Xmx1000m
    -Dudf.import.list=org.apache.cassandra.hadoop.pig
    -Dpig.additional.jars=/opt/apache-cassandra-1.2.3/lib/apache-cassandra-thrift-1.2.3.jar:/opt/apache-cassandra-1.2.3/lib/libthrift-0.7.0.jar
    -4 log4j.properties -Dpig.log.dir=/opt/pig-0.11.1/bin/../logs
    -Dpig.log.file=pig.log -Dpig.home.dir=/opt/pig-0.11.1/bin/..
    -Djava.net.preferIPv4Stack=true '
    + '[' false == true ']'
    + exec /usr/bin/hadoop jar
    /opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar ./hadoop-sum-value_1.pig
    2013-05-03 07:57:55,140 [main] INFO org.apache.pig.Main - Apache Pig
    version 0.11.1 (r1459641) compiled Mar 22 2013, 02:13:53
    2013-05-03 07:57:55,141 [main] INFO org.apache.pig.Main - Logging error
    messages to: /home/ubuntu/pig_1367567875107.log
    2013-05-03 07:57:57,502 [main] INFO org.apache.pig.impl.util.Utils -
    Default bootup file /home/ubuntu/.pigbootup not found
    2013-05-03 07:57:58,461 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
    Connecting to hadoop file system at: hdfs://hadoop-cass-1:9100
    2013-05-03 07:57:59,663 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
    Connecting to map-reduce job tracker at: hadoop-cass-1:8021
    2013-05-03 07:58:01,387 [main] ERROR org.apache.pig.tools.grunt.Grunt -
    ERROR 1070: Could not resolve CassandraStorage2 using imports: [,
    org.apache.pig.builtin., org.apache.pig.impl.builtin.]

    + PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig
    -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
    -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs'
    + PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig
    -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
    -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs
    -Dpig.log.file=pig.log'
    + PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig
    -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
    -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs
    -Dpig.log.file=pig.log
    -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/..'
    + '[' '' == true ']'
    + '[' -n /usr/local/bin/hadoop ']'
    + '[' false == true ']'
    + '[' -f /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-withouthadoop.jar
    ']'
    ++ echo
    /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
    +
    PIG_JAR=/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
    + '[' -z
    /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
    + '[' -n
    /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
    +
    CLASSPATH='/usr/local/Cellar/hadoop/1.1.2/libexec/conf:/Library/Java/JavaVirtualMachines/1.6.0_32-b05-420.jdk/Contents/Home/lib/tools.jar:::/usr/local/Cellar/cassandra/1.2.4/*:/usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar:/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar'
    + export
    'HADOOP_CLASSPATH=/usr/local/Cellar/hadoop/1.1.2/libexec/conf:/Library/Java/JavaVirtualMachines/1.6.0_32-b05-420.jdk/Contents/Home/lib/tools.jar:::/usr/local/Cellar/cassandra/1.2.4/*:/usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar:/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
    +
    HADOOP_CLASSPATH='/usr/local/Cellar/hadoop/1.1.2/libexec/conf:/Library/Java/JavaVirtualMachines/1.6.0_32-b05-420.jdk/Contents/Home/lib/tools.jar:::/usr/local/Cellar/cassandra/1.2.4/*:/usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar:/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
    + export 'HADOOP_OPTS=-Xmx1000m
    -Dudf.import.list=org.apache.cassandra.hadoop.pig
    -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
    -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs
    -Dpig.log.file=pig.log
    -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/.. '
    + HADOOP_OPTS='-Xmx1000m
    -Dudf.import.list=org.apache.cassandra.hadoop.pig
    -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
    -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs
    -Dpig.log.file=pig.log
    -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/.. '
    + '[' false == true ']'
    + exec /usr/local/bin/hadoop jar
    /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
    ./hadoop-sum-value_1.pig
    2013-05-03 09:57:56,810 [main] INFO org.apache.pig.Main - Apache Pig
    version 0.11.1 (r1459641) compiled Mar 22 2013, 02:13:53
    2013-05-03 09:57:56,811 [main] INFO org.apache.pig.Main - Logging error
    messages to: /private/tmp/pig_1367567876808.log
    2013-05-03 09:57:56.980 java[81053:1203] Unable to load realm info from
    SCDynamicStore
    2013-05-03 09:57:57,190 [main] INFO org.apache.pig.impl.util.Utils -
    Default bootup file /Users/cyril/.pigbootup not found
    2013-05-03 09:57:57,333 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
    Connecting to hadoop file system at: hdfs://localhost:9000
    2013-05-03 09:57:57,513 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
    Connecting to map-reduce job tracker at: localhost:9001
    2013-05-03 09:57:57,944 [main] ERROR org.apache.pig.tools.grunt.Grunt -
    ERROR 1070: Could not resolve CassandraStorage2 using imports:
    [org.apache.cassandra.hadoop.pig., , org.apache.pig.builtin.,
    org.apache.pig.impl.builtin.]


    I'm using an undefined function just to get the error which indicates the
    import list which is not the same as you can see even if I use the same
    options :
    PIG_OPTS = ' -Dudf.import.list=org.apache.cassandra.hadoop.pig
    -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
    -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs'

    If anyone has an explanation ?

    thanks
    --
    Cyril SCETBON



    _________________________________________________________________________________________________________________________

    Ce message et ses pieces jointes peuvent contenir des informations
    confidentielles ou privilegiees et ne doivent donc
    pas etre diffuses, exploites ou copies sans autorisation. Si vous avez
    recu ce message par erreur, veuillez le signaler
    a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
    electroniques etant susceptibles d'alteration,
    France Telecom - Orange decline toute responsabilite si ce message a ete
    altere, deforme ou falsifie. Merci.

    This message and its attachments may contain confidential or privileged
    information that may be protected by law;
    they should not be distributed, used or copied without authorisation.
    If you have received this email in error, please notify the sender and
    delete this message and its attachments.
    As emails may be altered, France Telecom - Orange is not liable for
    messages that have been modified, changed or falsified.
    Thank you.




    _________________________________________________________________________________________________________________________

    Ce message et ses pieces jointes peuvent contenir des informations
    confidentielles ou privilegiees et ne doivent donc
    pas etre diffuses, exploites ou copies sans autorisation. Si vous avez
    recu ce message par erreur, veuillez le signaler
    a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
    electroniques etant susceptibles d'alteration,
    France Telecom - Orange decline toute responsabilite si ce message a ete
    altere, deforme ou falsifie. Merci.

    This message and its attachments may contain confidential or privileged
    information that may be protected by law;
    they should not be distributed, used or copied without authorisation.
    If you have received this email in error, please notify the sender and
    delete this message and its attachments.
    As emails may be altered, France Telecom - Orange is not liable for
    messages that have been modified, changed or falsified.
    Thank you.
  • Cscetbon Ext at May 7, 2013 at 3:06 pm
    I think it's a bug in the default hadoop-env.sh :

    # Extra Java runtime options. Empty by default.
    export HADOOP_OPTS="-Djava.net.preferIPv4Stack=true $HADOOP_CLIENT_OPTS"

    we should be able to overload this in the console without having to go into hadoop-env.sh to change it. So I replaced this line by :

    # Extra Java runtime options. Empty by default.
    export HADOOP_OPTS="-Djava.net.preferIPv4Stack=true $HADOOP_CLIENT_OPTS $HADOOP_OPTS"

    Regards
    --
    Cyril SCETBON

    On May 6, 2013, at 7:35 PM, Prashant Kommireddi wrote:

    This blog post describes some of this behavior
    http://hadoopified.wordpress.com/2013/04/07/pig-startup-script-behavior/


    On Mon, May 6, 2013 at 1:07 AM, wrote:

    forget it ! It came from hadoop-env.sh provided by the package I installed
    on one cluster :(

    Regards
    --
    Cyril SCETBON

    On May 3, 2013, at 11:44 AM, cscetbon.ext@orange.com<mailto:
    cscetbon.ext@orange.com> wrote:

    Hi,

    I have 2 differents behaviour for the same Pig version (with hadoop 1.1.2)
    on different servers. If anyone can tell me why with the same versions and
    the same parameters I don't have the same behaviour. I'm trying to set the
    builtin import list but it's not used with one hadoop cluster whereas it's
    used when I use it on a single node cluster :

    + PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig
    -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
    -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs'
    + PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig
    -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
    -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs
    -Dpig.log.file=pig.log'
    + PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig
    -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
    -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs
    -Dpig.log.file=pig.log
    -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/..'
    + '[' '' == true ']'
    + '[' -n /usr/local/bin/hadoop ']'
    + '[' false == true ']'
    + '[' -f /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-withouthadoop.jar
    ']'
    ++ echo
    /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
    +
    PIG_JAR=/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
    + '[' -z
    /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
    + '[' -n
    /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
    +
    CLASSPATH='/etc/hadoop/conf:/usr/lib/jvm/java-7-oracle/lib/tools.jar:::/opt/apache-cassandra-1.2.3/lib/*:/usr/share/hadoop/lib/*:/opt/pig-0.11.1/pig-0.11.1-withouthadoop.jar:/etc/hadoop:/opt/pig-0.11.1/bin/../lib/jython-standalone-2.5.3.jar:/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar'
    + export
    'HADOOP_CLASSPATH=/etc/hadoop/conf:/usr/lib/jvm/java-7-oracle/lib/tools.jar:::/opt/apache-cassandra-1.2.3/lib/*:/usr/share/hadoop/lib/*:/opt/pig-0.11.1/pig-0.11.1-withouthadoop.jar:/etc/hadoop:/opt/pig-0.11.1/bin/../lib/jython-standalone-2.5.3.jar:/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
    +
    HADOOP_CLASSPATH='/etc/hadoop/conf:/usr/lib/jvm/java-7-oracle/lib/tools.jar:::/opt/apache-cassandra-1.2.3/lib/*:/usr/share/hadoop/lib/*:/opt/pig-0.11.1/pig-0.11.1-withouthadoop.jar:/etc/hadoop:/opt/pig-0.11.1/bin/../lib/jython-standalone-2.5.3.jar:/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
    + export 'HADOOP_OPTS=-Xmx1000m
    -Dudf.import.list=org.apache.cassandra.hadoop.pig
    -Dpig.additional.jars=/opt/apache-cassandra-1.2.3/lib/apache-cassandra-thrift-1.2.3.jar:/opt/apache-cassandra-1.2.3/lib/libthrift-0.7.0.jar
    -4 log4j.properties -Dpig.log.dir=/opt/pig-0.11.1/bin/../logs
    -Dpig.log.file=pig.log -Dpig.home.dir=/opt/pig-0.11.1/bin/..
    -Djava.net.preferIPv4Stack=true '
    + HADOOP_OPTS='-Xmx1000m
    -Dudf.import.list=org.apache.cassandra.hadoop.pig
    -Dpig.additional.jars=/opt/apache-cassandra-1.2.3/lib/apache-cassandra-thrift-1.2.3.jar:/opt/apache-cassandra-1.2.3/lib/libthrift-0.7.0.jar
    -4 log4j.properties -Dpig.log.dir=/opt/pig-0.11.1/bin/../logs
    -Dpig.log.file=pig.log -Dpig.home.dir=/opt/pig-0.11.1/bin/..
    -Djava.net.preferIPv4Stack=true '
    + '[' false == true ']'
    + exec /usr/bin/hadoop jar
    /opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar ./hadoop-sum-value_1.pig
    2013-05-03 07:57:55,140 [main] INFO org.apache.pig.Main - Apache Pig
    version 0.11.1 (r1459641) compiled Mar 22 2013, 02:13:53
    2013-05-03 07:57:55,141 [main] INFO org.apache.pig.Main - Logging error
    messages to: /home/ubuntu/pig_1367567875107.log
    2013-05-03 07:57:57,502 [main] INFO org.apache.pig.impl.util.Utils -
    Default bootup file /home/ubuntu/.pigbootup not found
    2013-05-03 07:57:58,461 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
    Connecting to hadoop file system at: hdfs://hadoop-cass-1:9100
    2013-05-03 07:57:59,663 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
    Connecting to map-reduce job tracker at: hadoop-cass-1:8021
    2013-05-03 07:58:01,387 [main] ERROR org.apache.pig.tools.grunt.Grunt -
    ERROR 1070: Could not resolve CassandraStorage2 using imports: [,
    org.apache.pig.builtin., org.apache.pig.impl.builtin.]

    + PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig
    -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
    -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs'
    + PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig
    -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
    -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs
    -Dpig.log.file=pig.log'
    + PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig
    -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
    -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs
    -Dpig.log.file=pig.log
    -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/..'
    + '[' '' == true ']'
    + '[' -n /usr/local/bin/hadoop ']'
    + '[' false == true ']'
    + '[' -f /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-withouthadoop.jar
    ']'
    ++ echo
    /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
    +
    PIG_JAR=/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
    + '[' -z
    /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
    + '[' -n
    /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
    +
    CLASSPATH='/usr/local/Cellar/hadoop/1.1.2/libexec/conf:/Library/Java/JavaVirtualMachines/1.6.0_32-b05-420.jdk/Contents/Home/lib/tools.jar:::/usr/local/Cellar/cassandra/1.2.4/*:/usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar:/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar'
    + export
    'HADOOP_CLASSPATH=/usr/local/Cellar/hadoop/1.1.2/libexec/conf:/Library/Java/JavaVirtualMachines/1.6.0_32-b05-420.jdk/Contents/Home/lib/tools.jar:::/usr/local/Cellar/cassandra/1.2.4/*:/usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar:/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
    +
    HADOOP_CLASSPATH='/usr/local/Cellar/hadoop/1.1.2/libexec/conf:/Library/Java/JavaVirtualMachines/1.6.0_32-b05-420.jdk/Contents/Home/lib/tools.jar:::/usr/local/Cellar/cassandra/1.2.4/*:/usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar:/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
    + export 'HADOOP_OPTS=-Xmx1000m
    -Dudf.import.list=org.apache.cassandra.hadoop.pig
    -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
    -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs
    -Dpig.log.file=pig.log
    -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/.. '
    + HADOOP_OPTS='-Xmx1000m
    -Dudf.import.list=org.apache.cassandra.hadoop.pig
    -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
    -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs
    -Dpig.log.file=pig.log
    -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/.. '
    + '[' false == true ']'
    + exec /usr/local/bin/hadoop jar
    /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
    ./hadoop-sum-value_1.pig
    2013-05-03 09:57:56,810 [main] INFO org.apache.pig.Main - Apache Pig
    version 0.11.1 (r1459641) compiled Mar 22 2013, 02:13:53
    2013-05-03 09:57:56,811 [main] INFO org.apache.pig.Main - Logging error
    messages to: /private/tmp/pig_1367567876808.log
    2013-05-03 09:57:56.980 java[81053:1203] Unable to load realm info from
    SCDynamicStore
    2013-05-03 09:57:57,190 [main] INFO org.apache.pig.impl.util.Utils -
    Default bootup file /Users/cyril/.pigbootup not found
    2013-05-03 09:57:57,333 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
    Connecting to hadoop file system at: hdfs://localhost:9000
    2013-05-03 09:57:57,513 [main] INFO
    org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
    Connecting to map-reduce job tracker at: localhost:9001
    2013-05-03 09:57:57,944 [main] ERROR org.apache.pig.tools.grunt.Grunt -
    ERROR 1070: Could not resolve CassandraStorage2 using imports:
    [org.apache.cassandra.hadoop.pig., , org.apache.pig.builtin.,
    org.apache.pig.impl.builtin.]


    I'm using an undefined function just to get the error which indicates the
    import list which is not the same as you can see even if I use the same
    options :
    PIG_OPTS = ' -Dudf.import.list=org.apache.cassandra.hadoop.pig
    -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
    -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs'

    If anyone has an explanation ?

    thanks
    --
    Cyril SCETBON



    _________________________________________________________________________________________________________________________

    Ce message et ses pieces jointes peuvent contenir des informations
    confidentielles ou privilegiees et ne doivent donc
    pas etre diffuses, exploites ou copies sans autorisation. Si vous avez
    recu ce message par erreur, veuillez le signaler
    a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
    electroniques etant susceptibles d'alteration,
    France Telecom - Orange decline toute responsabilite si ce message a ete
    altere, deforme ou falsifie. Merci.

    This message and its attachments may contain confidential or privileged
    information that may be protected by law;
    they should not be distributed, used or copied without authorisation.
    If you have received this email in error, please notify the sender and
    delete this message and its attachments.
    As emails may be altered, France Telecom - Orange is not liable for
    messages that have been modified, changed or falsified.
    Thank you.




    _________________________________________________________________________________________________________________________

    Ce message et ses pieces jointes peuvent contenir des informations
    confidentielles ou privilegiees et ne doivent donc
    pas etre diffuses, exploites ou copies sans autorisation. Si vous avez
    recu ce message par erreur, veuillez le signaler
    a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
    electroniques etant susceptibles d'alteration,
    France Telecom - Orange decline toute responsabilite si ce message a ete
    altere, deforme ou falsifie. Merci.

    This message and its attachments may contain confidential or privileged
    information that may be protected by law;
    they should not be distributed, used or copied without authorisation.
    If you have received this email in error, please notify the sender and
    delete this message and its attachments.
    As emails may be altered, France Telecom - Orange is not liable for
    messages that have been modified, changed or falsified.
    Thank you.




    _________________________________________________________________________________________________________________________

    Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
    pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
    a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
    France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

    This message and its attachments may contain confidential or privileged information that may be protected by law;
    they should not be distributed, used or copied without authorisation.
    If you have received this email in error, please notify the sender and delete this message and its attachments.
    As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified.
    Thank you.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedMay 3, '13 at 9:45a
activeMay 7, '13 at 3:06p
posts4
users2
websitepig.apache.org

People

Translate

site design / logo © 2021 Grokbase