Grokbase Groups Pig user April 2011
FAQ
Currently using Pig 0.8 and Hadoop 0.20.2. I'm able to run things in local
mode as well as run fs -* commands from within Grunt running in MapReduce
mode. I can't seem to be able to execute any Pig Latin statement in
MapReduce mode. The fact that I can browse HDFS and access files leads me to
believe that the setup is correct (also verified by the correct namenode and
jobtracker addresses connected to when firing off any pig command). I don't
know why though I can't run anything else. A simple command like the
following:

data = LOAD 'test.csv' USING PigStorage(',');
DUMP data;

Produces the following stack trace:

Pig Stack Trace
---------------
ERROR 1066: Unable to open iterator for alias data

org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
open iterator for alias data
at org.apache.pig.PigServer.openIterator(PigServer.java:754)
at
org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:76)
at org.apache.pig.Main.run(Main.java:465)
at org.apache.pig.Main.main(Main.java:107)
Caused by: java.io.IOException: Job terminated with anomalous status FAILED
at org.apache.pig.PigServer.openIterator(PigServer.java:744)
... 7 more

Any other more elaborate collection of Pig statements will produce this:

Pig Stack Trace
---------------
ERROR 2244: Job failed, hadoop does not return any error message

org.apache.pig.backend.executionengine.ExecException: ERROR 2244: Job
failed, hadoop does not return any error message
at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:116)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:169)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
at org.apache.pig.Main.run(Main.java:510)
at org.apache.pig.Main.main(Main.java:107)

Where any input file is flagged as unreadable by the following error message
during execution:

Input(s):
Failed to read data from "test.csv"

All files are accessible in MapReduce mode via cat, ls etc. I might be
missing something very obvious here since this is the first time I've used
Pig so any help would be greatly appreciated.

- Dimitris

Search Discussions

  • Dmitriy Ryaboy at Apr 2, 2011 at 12:55 am
    Dimitris,
    You should be able to look at the jobs submitted to your jobtracker and find
    out what happened to the pig job (the message " Job terminated with
    anomalous status FAILED" leads me to believe there is some job setup
    problem). Do you see any pig job attempts on your jobtracker UI?

    D
    On Fri, Apr 1, 2011 at 2:16 PM, Dimitris Iliopoulos wrote:

    Currently using Pig 0.8 and Hadoop 0.20.2. I'm able to run things in local
    mode as well as run fs -* commands from within Grunt running in MapReduce
    mode. I can't seem to be able to execute any Pig Latin statement in
    MapReduce mode. The fact that I can browse HDFS and access files leads me
    to
    believe that the setup is correct (also verified by the correct namenode
    and
    jobtracker addresses connected to when firing off any pig command). I don't
    know why though I can't run anything else. A simple command like the
    following:

    data = LOAD 'test.csv' USING PigStorage(',');
    DUMP data;

    Produces the following stack trace:

    Pig Stack Trace
    ---------------
    ERROR 1066: Unable to open iterator for alias data

    org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
    open iterator for alias data
    at org.apache.pig.PigServer.openIterator(PigServer.java:754)
    at
    org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
    at

    org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
    at

    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
    at

    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
    at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:76)
    at org.apache.pig.Main.run(Main.java:465)
    at org.apache.pig.Main.main(Main.java:107)
    Caused by: java.io.IOException: Job terminated with anomalous status FAILED
    at org.apache.pig.PigServer.openIterator(PigServer.java:744)
    ... 7 more

    Any other more elaborate collection of Pig statements will produce this:

    Pig Stack Trace
    ---------------
    ERROR 2244: Job failed, hadoop does not return any error message

    org.apache.pig.backend.executionengine.ExecException: ERROR 2244: Job
    failed, hadoop does not return any error message
    at
    org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:116)
    at

    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:169)
    at

    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
    at org.apache.pig.Main.run(Main.java:510)
    at org.apache.pig.Main.main(Main.java:107)

    Where any input file is flagged as unreadable by the following error
    message
    during execution:

    Input(s):
    Failed to read data from "test.csv"

    All files are accessible in MapReduce mode via cat, ls etc. I might be
    missing something very obvious here since this is the first time I've used
    Pig so any help would be greatly appreciated.

    - Dimitris
  • Dimitris Iliopoulos at Apr 2, 2011 at 1:03 am
    Dmitriy,

    The jobtracker does indeed mark the job as failed but the problem is that
    there are no log files. All map and reduce tasks are flagged as killed and
    upon inspection produce no logs. The only logs are the ones I appended
    before produced by Pig.

    - Dimitris
    On Fri, Apr 1, 2011 at 5:55 PM, Dmitriy Ryaboy wrote:

    Dimitris,
    You should be able to look at the jobs submitted to your jobtracker and
    find
    out what happened to the pig job (the message " Job terminated with
    anomalous status FAILED" leads me to believe there is some job setup
    problem). Do you see any pig job attempts on your jobtracker UI?

    D

    On Fri, Apr 1, 2011 at 2:16 PM, Dimitris Iliopoulos <
    dimitris.iliopoulos@gmail.com> wrote:
    Currently using Pig 0.8 and Hadoop 0.20.2. I'm able to run things in local
    mode as well as run fs -* commands from within Grunt running in MapReduce
    mode. I can't seem to be able to execute any Pig Latin statement in
    MapReduce mode. The fact that I can browse HDFS and access files leads me
    to
    believe that the setup is correct (also verified by the correct namenode
    and
    jobtracker addresses connected to when firing off any pig command). I don't
    know why though I can't run anything else. A simple command like the
    following:

    data = LOAD 'test.csv' USING PigStorage(',');
    DUMP data;

    Produces the following stack trace:

    Pig Stack Trace
    ---------------
    ERROR 1066: Unable to open iterator for alias data

    org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
    open iterator for alias data
    at org.apache.pig.PigServer.openIterator(PigServer.java:754)
    at
    org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
    at

    org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
    at

    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
    at

    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
    at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:76)
    at org.apache.pig.Main.run(Main.java:465)
    at org.apache.pig.Main.main(Main.java:107)
    Caused by: java.io.IOException: Job terminated with anomalous status FAILED
    at org.apache.pig.PigServer.openIterator(PigServer.java:744)
    ... 7 more

    Any other more elaborate collection of Pig statements will produce this:

    Pig Stack Trace
    ---------------
    ERROR 2244: Job failed, hadoop does not return any error message

    org.apache.pig.backend.executionengine.ExecException: ERROR 2244: Job
    failed, hadoop does not return any error message
    at
    org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:116)
    at

    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:169)
    at

    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
    at org.apache.pig.Main.run(Main.java:510)
    at org.apache.pig.Main.main(Main.java:107)

    Where any input file is flagged as unreadable by the following error
    message
    during execution:

    Input(s):
    Failed to read data from "test.csv"

    All files are accessible in MapReduce mode via cat, ls etc. I might be
    missing something very obvious here since this is the first time I've used
    Pig so any help would be greatly appreciated.

    - Dimitris
  • Dmitriy Ryaboy at Apr 2, 2011 at 1:14 am
    Is there a line that says "Job Setup: Failed" at the top of the job page?
    Failed (or "Successful") should be a link, click through that to get the
    setup log.

    D
    On Fri, Apr 1, 2011 at 6:02 PM, Dimitris Iliopoulos wrote:

    Dmitriy,

    The jobtracker does indeed mark the job as failed but the problem is that
    there are no log files. All map and reduce tasks are flagged as killed and
    upon inspection produce no logs. The only logs are the ones I appended
    before produced by Pig.

    - Dimitris
    On Fri, Apr 1, 2011 at 5:55 PM, Dmitriy Ryaboy wrote:

    Dimitris,
    You should be able to look at the jobs submitted to your jobtracker and
    find
    out what happened to the pig job (the message " Job terminated with
    anomalous status FAILED" leads me to believe there is some job setup
    problem). Do you see any pig job attempts on your jobtracker UI?

    D

    On Fri, Apr 1, 2011 at 2:16 PM, Dimitris Iliopoulos <
    dimitris.iliopoulos@gmail.com> wrote:
    Currently using Pig 0.8 and Hadoop 0.20.2. I'm able to run things in local
    mode as well as run fs -* commands from within Grunt running in
    MapReduce
    mode. I can't seem to be able to execute any Pig Latin statement in
    MapReduce mode. The fact that I can browse HDFS and access files leads
    me
    to
    believe that the setup is correct (also verified by the correct
    namenode
    and
    jobtracker addresses connected to when firing off any pig command). I don't
    know why though I can't run anything else. A simple command like the
    following:

    data = LOAD 'test.csv' USING PigStorage(',');
    DUMP data;

    Produces the following stack trace:

    Pig Stack Trace
    ---------------
    ERROR 1066: Unable to open iterator for alias data

    org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable
    to
    open iterator for alias data
    at org.apache.pig.PigServer.openIterator(PigServer.java:754)
    at
    org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
    at
    org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
    at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:76)
    at org.apache.pig.Main.run(Main.java:465)
    at org.apache.pig.Main.main(Main.java:107)
    Caused by: java.io.IOException: Job terminated with anomalous status FAILED
    at org.apache.pig.PigServer.openIterator(PigServer.java:744)
    ... 7 more

    Any other more elaborate collection of Pig statements will produce
    this:
    Pig Stack Trace
    ---------------
    ERROR 2244: Job failed, hadoop does not return any error message

    org.apache.pig.backend.executionengine.ExecException: ERROR 2244: Job
    failed, hadoop does not return any error message
    at
    org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:116)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:169)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
    at org.apache.pig.Main.run(Main.java:510)
    at org.apache.pig.Main.main(Main.java:107)

    Where any input file is flagged as unreadable by the following error
    message
    during execution:

    Input(s):
    Failed to read data from "test.csv"

    All files are accessible in MapReduce mode via cat, ls etc. I might be
    missing something very obvious here since this is the first time I've used
    Pig so any help would be greatly appreciated.

    - Dimitris
  • Dimitris Iliopoulos at Apr 2, 2011 at 1:19 am
    There is a "Fail" link and the error trace that I get from there is as
    follows:

    java.io.IOException: Deserialization error:
    org.apache.commons.logging.impl.Log4JLogger;
    org.apache.commons.logging.impl.Log4JLogger; class invalid for
    deserialization

    at org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:55)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getStores(PigOutputFormat.java:217)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:244)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getOutputCommitter(PigOutputFormat.java:274)
    at org.apache.hadoop.mapred.Task.initialize(Task.java:415)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:354)
    at org.apache.hadoop.mapred.Child.main(Child.java:170)
    Caused by: java.io.InvalidClassException:
    org.apache.commons.logging.impl.Log4JLogger;
    org.apache.commons.logging.impl.Log4JLogger; class invalid for
    deserialization
    at java.io.ObjectStreamClass.checkDeserialize(ObjectStreamClass.java:713)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1732)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
    at java.util.LinkedList.readObject(LinkedList.java:964)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
    at org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:53)
    ... 6 more
    Caused by: java.io.InvalidClassException:
    org.apache.commons.logging.impl.Log4JLogger; class invalid for
    deserialization
    at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:587)
    at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1582)
    at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1495)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1731)
    ... 23 more

    On Fri, Apr 1, 2011 at 6:13 PM, Dmitriy Ryaboy wrote:

    Is there a line that says "Job Setup: Failed" at the top of the job page?
    Failed (or "Successful") should be a link, click through that to get the
    setup log.

    D

    On Fri, Apr 1, 2011 at 6:02 PM, Dimitris Iliopoulos <
    dimitris.iliopoulos@gmail.com> wrote:
    Dmitriy,

    The jobtracker does indeed mark the job as failed but the problem is that
    there are no log files. All map and reduce tasks are flagged as killed and
    upon inspection produce no logs. The only logs are the ones I appended
    before produced by Pig.

    - Dimitris
    On Fri, Apr 1, 2011 at 5:55 PM, Dmitriy Ryaboy wrote:

    Dimitris,
    You should be able to look at the jobs submitted to your jobtracker and
    find
    out what happened to the pig job (the message " Job terminated with
    anomalous status FAILED" leads me to believe there is some job setup
    problem). Do you see any pig job attempts on your jobtracker UI?

    D

    On Fri, Apr 1, 2011 at 2:16 PM, Dimitris Iliopoulos <
    dimitris.iliopoulos@gmail.com> wrote:
    Currently using Pig 0.8 and Hadoop 0.20.2. I'm able to run things in local
    mode as well as run fs -* commands from within Grunt running in
    MapReduce
    mode. I can't seem to be able to execute any Pig Latin statement in
    MapReduce mode. The fact that I can browse HDFS and access files
    leads
    me
    to
    believe that the setup is correct (also verified by the correct
    namenode
    and
    jobtracker addresses connected to when firing off any pig command). I don't
    know why though I can't run anything else. A simple command like the
    following:

    data = LOAD 'test.csv' USING PigStorage(',');
    DUMP data;

    Produces the following stack trace:

    Pig Stack Trace
    ---------------
    ERROR 1066: Unable to open iterator for alias data

    org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066:
    Unable
    to
    open iterator for alias data
    at org.apache.pig.PigServer.openIterator(PigServer.java:754)
    at
    org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
    at
    org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
    at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:76)
    at org.apache.pig.Main.run(Main.java:465)
    at org.apache.pig.Main.main(Main.java:107)
    Caused by: java.io.IOException: Job terminated with anomalous status FAILED
    at org.apache.pig.PigServer.openIterator(PigServer.java:744)
    ... 7 more

    Any other more elaborate collection of Pig statements will produce
    this:
    Pig Stack Trace
    ---------------
    ERROR 2244: Job failed, hadoop does not return any error message

    org.apache.pig.backend.executionengine.ExecException: ERROR 2244: Job
    failed, hadoop does not return any error message
    at
    org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:116)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:169)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
    at org.apache.pig.Main.run(Main.java:510)
    at org.apache.pig.Main.main(Main.java:107)

    Where any input file is flagged as unreadable by the following error
    message
    during execution:

    Input(s):
    Failed to read data from "test.csv"

    All files are accessible in MapReduce mode via cat, ls etc. I might
    be
    missing something very obvious here since this is the first time I've used
    Pig so any help would be greatly appreciated.

    - Dimitris
  • Dmitriy Ryaboy at Apr 2, 2011 at 1:27 am
    Are you running a released version of Pig or something from svn?
    Anything interesting about your log setup or classpath we should know about?

    D
    On Fri, Apr 1, 2011 at 6:19 PM, Dimitris Iliopoulos wrote:

    There is a "Fail" link and the error trace that I get from there is as
    follows:

    java.io.IOException: Deserialization error:
    org.apache.commons.logging.impl.Log4JLogger;
    org.apache.commons.logging.impl.Log4JLogger; class invalid for
    deserialization

    at
    org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:55)
    at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getStores(PigOutputFormat.java:217)
    at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:244)
    at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getOutputCommitter(PigOutputFormat.java:274)
    at org.apache.hadoop.mapred.Task.initialize(Task.java:415)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:354)
    at org.apache.hadoop.mapred.Child.main(Child.java:170)
    Caused by: java.io.InvalidClassException:
    org.apache.commons.logging.impl.Log4JLogger;
    org.apache.commons.logging.impl.Log4JLogger; class invalid for
    deserialization
    at
    java.io.ObjectStreamClass.checkDeserialize(ObjectStreamClass.java:713)
    at
    java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1732)
    at
    java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
    at
    java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946)
    at
    java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870)
    at
    java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
    at
    java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
    at java.util.LinkedList.readObject(LinkedList.java:964)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
    java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
    at
    java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
    at
    java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
    at
    java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
    at
    org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:53)
    ... 6 more
    Caused by: java.io.InvalidClassException:
    org.apache.commons.logging.impl.Log4JLogger; class invalid for
    deserialization
    at
    java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:587)
    at
    java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1582)
    at
    java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1495)
    at
    java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1731)
    ... 23 more

    On Fri, Apr 1, 2011 at 6:13 PM, Dmitriy Ryaboy wrote:

    Is there a line that says "Job Setup: Failed" at the top of the job page?
    Failed (or "Successful") should be a link, click through that to get the
    setup log.

    D

    On Fri, Apr 1, 2011 at 6:02 PM, Dimitris Iliopoulos <
    dimitris.iliopoulos@gmail.com> wrote:
    Dmitriy,

    The jobtracker does indeed mark the job as failed but the problem is
    that
    there are no log files. All map and reduce tasks are flagged as killed and
    upon inspection produce no logs. The only logs are the ones I appended
    before produced by Pig.

    - Dimitris

    On Fri, Apr 1, 2011 at 5:55 PM, Dmitriy Ryaboy <dvryaboy@gmail.com>
    wrote:
    Dimitris,
    You should be able to look at the jobs submitted to your jobtracker
    and
    find
    out what happened to the pig job (the message " Job terminated with
    anomalous status FAILED" leads me to believe there is some job setup
    problem). Do you see any pig job attempts on your jobtracker UI?

    D

    On Fri, Apr 1, 2011 at 2:16 PM, Dimitris Iliopoulos <
    dimitris.iliopoulos@gmail.com> wrote:
    Currently using Pig 0.8 and Hadoop 0.20.2. I'm able to run things
    in
    local
    mode as well as run fs -* commands from within Grunt running in
    MapReduce
    mode. I can't seem to be able to execute any Pig Latin statement in
    MapReduce mode. The fact that I can browse HDFS and access files
    leads
    me
    to
    believe that the setup is correct (also verified by the correct
    namenode
    and
    jobtracker addresses connected to when firing off any pig command).
    I
    don't
    know why though I can't run anything else. A simple command like
    the
    following:

    data = LOAD 'test.csv' USING PigStorage(',');
    DUMP data;

    Produces the following stack trace:

    Pig Stack Trace
    ---------------
    ERROR 1066: Unable to open iterator for alias data

    org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066:
    Unable
    to
    open iterator for alias data
    at org.apache.pig.PigServer.openIterator(PigServer.java:754)
    at
    org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
    at
    org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
    at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:76)
    at org.apache.pig.Main.run(Main.java:465)
    at org.apache.pig.Main.main(Main.java:107)
    Caused by: java.io.IOException: Job terminated with anomalous
    status
    FAILED
    at org.apache.pig.PigServer.openIterator(PigServer.java:744)
    ... 7 more

    Any other more elaborate collection of Pig statements will produce
    this:
    Pig Stack Trace
    ---------------
    ERROR 2244: Job failed, hadoop does not return any error message

    org.apache.pig.backend.executionengine.ExecException: ERROR 2244:
    Job
    failed, hadoop does not return any error message
    at
    org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:116)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:169)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
    at org.apache.pig.Main.run(Main.java:510)
    at org.apache.pig.Main.main(Main.java:107)

    Where any input file is flagged as unreadable by the following
    error
    message
    during execution:

    Input(s):
    Failed to read data from "test.csv"

    All files are accessible in MapReduce mode via cat, ls etc. I might
    be
    missing something very obvious here since this is the first time
    I've
    used
    Pig so any help would be greatly appreciated.

    - Dimitris
  • Dimitris Iliopoulos at Apr 2, 2011 at 1:52 am
    Running the 0.8.0 release with Hadoop 0.20.2 and the rest are pretty standard.
    Sent via BlackBerry from T-Mobile

    -----Original Message-----
    From: Dmitriy Ryaboy <dvryaboy@gmail.com>
    Date: Fri, 1 Apr 2011 18:26:34
    To: <user@pig.apache.org>
    Reply-To: user@pig.apache.org
    Subject: Re: Unable to run Pig in MapReduce mode

    Are you running a released version of Pig or something from svn?
    Anything interesting about your log setup or classpath we should know about?

    D
    On Fri, Apr 1, 2011 at 6:19 PM, Dimitris Iliopoulos wrote:

    There is a "Fail" link and the error trace that I get from there is as
    follows:

    java.io.IOException: Deserialization error:
    org.apache.commons.logging.impl.Log4JLogger;
    org.apache.commons.logging.impl.Log4JLogger; class invalid for
    deserialization

    at
    org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:55)
    at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getStores(PigOutputFormat.java:217)
    at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:244)
    at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getOutputCommitter(PigOutputFormat.java:274)
    at org.apache.hadoop.mapred.Task.initialize(Task.java:415)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:354)
    at org.apache.hadoop.mapred.Child.main(Child.java:170)
    Caused by: java.io.InvalidClassException:
    org.apache.commons.logging.impl.Log4JLogger;
    org.apache.commons.logging.impl.Log4JLogger; class invalid for
    deserialization
    at
    java.io.ObjectStreamClass.checkDeserialize(ObjectStreamClass.java:713)
    at
    java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1732)
    at
    java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
    at
    java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946)
    at
    java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870)
    at
    java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
    at
    java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
    at java.util.LinkedList.readObject(LinkedList.java:964)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
    java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
    at
    java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
    at
    java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
    at
    java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
    at
    org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:53)
    ... 6 more
    Caused by: java.io.InvalidClassException:
    org.apache.commons.logging.impl.Log4JLogger; class invalid for
    deserialization
    at
    java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:587)
    at
    java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1582)
    at
    java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1495)
    at
    java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1731)
    ... 23 more

    On Fri, Apr 1, 2011 at 6:13 PM, Dmitriy Ryaboy wrote:

    Is there a line that says "Job Setup: Failed" at the top of the job page?
    Failed (or "Successful") should be a link, click through that to get the
    setup log.

    D

    On Fri, Apr 1, 2011 at 6:02 PM, Dimitris Iliopoulos <
    dimitris.iliopoulos@gmail.com> wrote:
    Dmitriy,

    The jobtracker does indeed mark the job as failed but the problem is
    that
    there are no log files. All map and reduce tasks are flagged as killed and
    upon inspection produce no logs. The only logs are the ones I appended
    before produced by Pig.

    - Dimitris

    On Fri, Apr 1, 2011 at 5:55 PM, Dmitriy Ryaboy <dvryaboy@gmail.com>
    wrote:
    Dimitris,
    You should be able to look at the jobs submitted to your jobtracker
    and
    find
    out what happened to the pig job (the message " Job terminated with
    anomalous status FAILED" leads me to believe there is some job setup
    problem). Do you see any pig job attempts on your jobtracker UI?

    D

    On Fri, Apr 1, 2011 at 2:16 PM, Dimitris Iliopoulos <
    dimitris.iliopoulos@gmail.com> wrote:
    Currently using Pig 0.8 and Hadoop 0.20.2. I'm able to run things
    in
    local
    mode as well as run fs -* commands from within Grunt running in
    MapReduce
    mode. I can't seem to be able to execute any Pig Latin statement in
    MapReduce mode. The fact that I can browse HDFS and access files
    leads
    me
    to
    believe that the setup is correct (also verified by the correct
    namenode
    and
    jobtracker addresses connected to when firing off any pig command).
    I
    don't
    know why though I can't run anything else. A simple command like
    the
    following:

    data = LOAD 'test.csv' USING PigStorage(',');
    DUMP data;

    Produces the following stack trace:

    Pig Stack Trace
    ---------------
    ERROR 1066: Unable to open iterator for alias data

    org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066:
    Unable
    to
    open iterator for alias data
    at org.apache.pig.PigServer.openIterator(PigServer.java:754)
    at
    org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
    at
    org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
    at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:76)
    at org.apache.pig.Main.run(Main.java:465)
    at org.apache.pig.Main.main(Main.java:107)
    Caused by: java.io.IOException: Job terminated with anomalous
    status
    FAILED
    at org.apache.pig.PigServer.openIterator(PigServer.java:744)
    ... 7 more

    Any other more elaborate collection of Pig statements will produce
    this:
    Pig Stack Trace
    ---------------
    ERROR 2244: Job failed, hadoop does not return any error message

    org.apache.pig.backend.executionengine.ExecException: ERROR 2244:
    Job
    failed, hadoop does not return any error message
    at
    org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:116)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:169)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
    at org.apache.pig.Main.run(Main.java:510)
    at org.apache.pig.Main.main(Main.java:107)

    Where any input file is flagged as unreadable by the following
    error
    message
    during execution:

    Input(s):
    Failed to read data from "test.csv"

    All files are accessible in MapReduce mode via cat, ls etc. I might
    be
    missing something very obvious here since this is the first time
    I've
    used
    Pig so any help would be greatly appreciated.

    - Dimitris
  • Dimitris Iliopoulos at Apr 4, 2011 at 9:15 pm
    So in terms of my Pig setup as I said I am using 0.8.0 and have the
    following environment variables set:

    export PIG_INSTALL=/home/UNIXHOME/diliopoulos/Hadoop/lib/pig-0.8.0
    export PIG_CLASSPATH=$HADOOP_INSTALL/conf/
    export PATH=$PATH:$PIG_INSTALL/bin

    I have build Pig locally and have tried running the tutorial examples but I
    am running into the same error messages when I run them on hadoop versus
    locally. I have tried to move the above environment variables into
    hadoop_env.sh but the errors persist. Have also tried explicitly passing the
    classpath by using "hadoop -cp" but still no luck. I'm pretty sure I am
    missing something really obvious here but I can't seem to find it.

    - Dimitris
    On Fri, Apr 1, 2011 at 6:51 PM, wrote:

    Running the 0.8.0 release with Hadoop 0.20.2 and the rest are pretty
    standard.
    Sent via BlackBerry from T-Mobile

    -----Original Message-----
    From: Dmitriy Ryaboy <dvryaboy@gmail.com>
    Date: Fri, 1 Apr 2011 18:26:34
    To: <user@pig.apache.org>
    Reply-To: user@pig.apache.org
    Subject: Re: Unable to run Pig in MapReduce mode

    Are you running a released version of Pig or something from svn?
    Anything interesting about your log setup or classpath we should know
    about?

    D

    On Fri, Apr 1, 2011 at 6:19 PM, Dimitris Iliopoulos <
    dimitris.iliopoulos@gmail.com> wrote:
    There is a "Fail" link and the error trace that I get from there is as
    follows:

    java.io.IOException: Deserialization error:
    org.apache.commons.logging.impl.Log4JLogger;
    org.apache.commons.logging.impl.Log4JLogger; class invalid for
    deserialization

    at
    org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:55)
    at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getStores(PigOutputFormat.java:217)
    at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:244)
    at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getOutputCommitter(PigOutputFormat.java:274)
    at org.apache.hadoop.mapred.Task.initialize(Task.java:415)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:354)
    at org.apache.hadoop.mapred.Child.main(Child.java:170)
    Caused by: java.io.InvalidClassException:
    org.apache.commons.logging.impl.Log4JLogger;
    org.apache.commons.logging.impl.Log4JLogger; class invalid for
    deserialization
    at
    java.io.ObjectStreamClass.checkDeserialize(ObjectStreamClass.java:713)
    at
    java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1732)
    at
    java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
    at
    java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946)
    at
    java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870)
    at
    java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
    at
    java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
    at
    java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
    at java.util.LinkedList.readObject(LinkedList.java:964)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
    java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
    at
    java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
    at
    java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
    at
    java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
    at
    java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
    at
    org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:53)
    ... 6 more
    Caused by: java.io.InvalidClassException:
    org.apache.commons.logging.impl.Log4JLogger; class invalid for
    deserialization
    at
    java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:587)
    at
    java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1582)
    at
    java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1495)
    at
    java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1731)
    ... 23 more

    On Fri, Apr 1, 2011 at 6:13 PM, Dmitriy Ryaboy wrote:

    Is there a line that says "Job Setup: Failed" at the top of the job
    page?
    Failed (or "Successful") should be a link, click through that to get
    the
    setup log.

    D

    On Fri, Apr 1, 2011 at 6:02 PM, Dimitris Iliopoulos <
    dimitris.iliopoulos@gmail.com> wrote:
    Dmitriy,

    The jobtracker does indeed mark the job as failed but the problem is
    that
    there are no log files. All map and reduce tasks are flagged as
    killed
    and
    upon inspection produce no logs. The only logs are the ones I
    appended
    before produced by Pig.

    - Dimitris

    On Fri, Apr 1, 2011 at 5:55 PM, Dmitriy Ryaboy <dvryaboy@gmail.com>
    wrote:
    Dimitris,
    You should be able to look at the jobs submitted to your jobtracker
    and
    find
    out what happened to the pig job (the message " Job terminated with
    anomalous status FAILED" leads me to believe there is some job
    setup
    problem). Do you see any pig job attempts on your jobtracker UI?

    D

    On Fri, Apr 1, 2011 at 2:16 PM, Dimitris Iliopoulos <
    dimitris.iliopoulos@gmail.com> wrote:
    Currently using Pig 0.8 and Hadoop 0.20.2. I'm able to run things
    in
    local
    mode as well as run fs -* commands from within Grunt running in
    MapReduce
    mode. I can't seem to be able to execute any Pig Latin statement
    in
    MapReduce mode. The fact that I can browse HDFS and access files
    leads
    me
    to
    believe that the setup is correct (also verified by the correct
    namenode
    and
    jobtracker addresses connected to when firing off any pig
    command).
    I
    don't
    know why though I can't run anything else. A simple command like
    the
    following:

    data = LOAD 'test.csv' USING PigStorage(',');
    DUMP data;

    Produces the following stack trace:

    Pig Stack Trace
    ---------------
    ERROR 1066: Unable to open iterator for alias data

    org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066:
    Unable
    to
    open iterator for alias data
    at
    org.apache.pig.PigServer.openIterator(PigServer.java:754)
    at
    org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
    at
    org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
    at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:76)
    at org.apache.pig.Main.run(Main.java:465)
    at org.apache.pig.Main.main(Main.java:107)
    Caused by: java.io.IOException: Job terminated with anomalous
    status
    FAILED
    at
    org.apache.pig.PigServer.openIterator(PigServer.java:744)
    ... 7 more

    Any other more elaborate collection of Pig statements will
    produce
    this:
    Pig Stack Trace
    ---------------
    ERROR 2244: Job failed, hadoop does not return any error message

    org.apache.pig.backend.executionengine.ExecException: ERROR 2244:
    Job
    failed, hadoop does not return any error message
    at
    org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:116)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:169)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
    at org.apache.pig.Main.run(Main.java:510)
    at org.apache.pig.Main.main(Main.java:107)

    Where any input file is flagged as unreadable by the following
    error
    message
    during execution:

    Input(s):
    Failed to read data from "test.csv"

    All files are accessible in MapReduce mode via cat, ls etc. I
    might
    be
    missing something very obvious here since this is the first time
    I've
    used
    Pig so any help would be greatly appreciated.

    - Dimitris
  • Dmitriy Ryaboy at Apr 4, 2011 at 10:26 pm
    This kind of sounds like https://issues.apache.org/jira/browse/PIG-1955 but
    I don't understand why it doesn't happen for anyone else..
    On Mon, Apr 4, 2011 at 2:14 PM, Dimitris Iliopoulos wrote:

    So in terms of my Pig setup as I said I am using 0.8.0 and have the
    following environment variables set:

    export PIG_INSTALL=/home/UNIXHOME/diliopoulos/Hadoop/lib/pig-0.8.0
    export PIG_CLASSPATH=$HADOOP_INSTALL/conf/
    export PATH=$PATH:$PIG_INSTALL/bin

    I have build Pig locally and have tried running the tutorial examples but I
    am running into the same error messages when I run them on hadoop versus
    locally. I have tried to move the above environment variables into
    hadoop_env.sh but the errors persist. Have also tried explicitly passing
    the
    classpath by using "hadoop -cp" but still no luck. I'm pretty sure I am
    missing something really obvious here but I can't seem to find it.

    - Dimitris
    On Fri, Apr 1, 2011 at 6:51 PM, wrote:

    Running the 0.8.0 release with Hadoop 0.20.2 and the rest are pretty
    standard.
    Sent via BlackBerry from T-Mobile

    -----Original Message-----
    From: Dmitriy Ryaboy <dvryaboy@gmail.com>
    Date: Fri, 1 Apr 2011 18:26:34
    To: <user@pig.apache.org>
    Reply-To: user@pig.apache.org
    Subject: Re: Unable to run Pig in MapReduce mode

    Are you running a released version of Pig or something from svn?
    Anything interesting about your log setup or classpath we should know
    about?

    D

    On Fri, Apr 1, 2011 at 6:19 PM, Dimitris Iliopoulos <
    dimitris.iliopoulos@gmail.com> wrote:
    There is a "Fail" link and the error trace that I get from there is as
    follows:

    java.io.IOException: Deserialization error:
    org.apache.commons.logging.impl.Log4JLogger;
    org.apache.commons.logging.impl.Log4JLogger; class invalid for
    deserialization

    at
    org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:55)
    at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getStores(PigOutputFormat.java:217)
    at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:244)
    at
    org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getOutputCommitter(PigOutputFormat.java:274)
    at org.apache.hadoop.mapred.Task.initialize(Task.java:415)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:354)
    at org.apache.hadoop.mapred.Child.main(Child.java:170)
    Caused by: java.io.InvalidClassException:
    org.apache.commons.logging.impl.Log4JLogger;
    org.apache.commons.logging.impl.Log4JLogger; class invalid for
    deserialization
    at
    java.io.ObjectStreamClass.checkDeserialize(ObjectStreamClass.java:713)
    at
    java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1732)
    at
    java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
    at
    java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946)
    at
    java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870)
    at
    java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
    at
    java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
    at
    java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
    at java.util.LinkedList.readObject(LinkedList.java:964)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
    java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
    at
    java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
    at
    java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
    at
    java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
    at
    java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
    at
    org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:53)
    ... 6 more
    Caused by: java.io.InvalidClassException:
    org.apache.commons.logging.impl.Log4JLogger; class invalid for
    deserialization
    at
    java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:587)
    at
    java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1582)
    at
    java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1495)
    at
    java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1731)
    ... 23 more


    On Fri, Apr 1, 2011 at 6:13 PM, Dmitriy Ryaboy <dvryaboy@gmail.com>
    wrote:
    Is there a line that says "Job Setup: Failed" at the top of the job
    page?
    Failed (or "Successful") should be a link, click through that to get
    the
    setup log.

    D

    On Fri, Apr 1, 2011 at 6:02 PM, Dimitris Iliopoulos <
    dimitris.iliopoulos@gmail.com> wrote:
    Dmitriy,

    The jobtracker does indeed mark the job as failed but the problem
    is
    that
    there are no log files. All map and reduce tasks are flagged as
    killed
    and
    upon inspection produce no logs. The only logs are the ones I
    appended
    before produced by Pig.

    - Dimitris

    On Fri, Apr 1, 2011 at 5:55 PM, Dmitriy Ryaboy <dvryaboy@gmail.com
    wrote:
    Dimitris,
    You should be able to look at the jobs submitted to your
    jobtracker
    and
    find
    out what happened to the pig job (the message " Job terminated
    with
    anomalous status FAILED" leads me to believe there is some job
    setup
    problem). Do you see any pig job attempts on your jobtracker UI?

    D

    On Fri, Apr 1, 2011 at 2:16 PM, Dimitris Iliopoulos <
    dimitris.iliopoulos@gmail.com> wrote:
    Currently using Pig 0.8 and Hadoop 0.20.2. I'm able to run
    things
    in
    local
    mode as well as run fs -* commands from within Grunt running in
    MapReduce
    mode. I can't seem to be able to execute any Pig Latin
    statement
    in
    MapReduce mode. The fact that I can browse HDFS and access
    files
    leads
    me
    to
    believe that the setup is correct (also verified by the correct
    namenode
    and
    jobtracker addresses connected to when firing off any pig
    command).
    I
    don't
    know why though I can't run anything else. A simple command
    like
    the
    following:

    data = LOAD 'test.csv' USING PigStorage(',');
    DUMP data;

    Produces the following stack trace:

    Pig Stack Trace
    ---------------
    ERROR 1066: Unable to open iterator for alias data

    org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066:
    Unable
    to
    open iterator for alias data
    at
    org.apache.pig.PigServer.openIterator(PigServer.java:754)
    at
    org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
    at
    org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
    at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:76)
    at org.apache.pig.Main.run(Main.java:465)
    at org.apache.pig.Main.main(Main.java:107)
    Caused by: java.io.IOException: Job terminated with anomalous
    status
    FAILED
    at
    org.apache.pig.PigServer.openIterator(PigServer.java:744)
    ... 7 more

    Any other more elaborate collection of Pig statements will
    produce
    this:
    Pig Stack Trace
    ---------------
    ERROR 2244: Job failed, hadoop does not return any error
    message
    org.apache.pig.backend.executionengine.ExecException: ERROR
    2244:
    Job
    failed, hadoop does not return any error message
    at
    org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:116)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:169)
    at
    org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
    at org.apache.pig.Main.run(Main.java:510)
    at org.apache.pig.Main.main(Main.java:107)

    Where any input file is flagged as unreadable by the following
    error
    message
    during execution:

    Input(s):
    Failed to read data from "test.csv"

    All files are accessible in MapReduce mode via cat, ls etc. I
    might
    be
    missing something very obvious here since this is the first
    time
    I've
    used
    Pig so any help would be greatly appreciated.

    - Dimitris

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedApr 1, '11 at 9:17p
activeApr 4, '11 at 10:26p
posts9
users2
websitepig.apache.org

People

Translate

site design / logo © 2022 Grokbase