FAQ
Hi,

While trying to run Pig in local mode using "pig -x local" I get the
following error:
10/11/04 07:29:18 INFO pig.Main: Logging error messages to:
/Users/some-name/Software/cloudera/hadoop-0.20.2+737/pig_1288880958619.log
2010-11-04 07:29:18,807 [main] WARN
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Passing
Hadoop Site Configurations in classpath is not recommended for Local Mode
2010-11-04 07:29:18,971 [main] ERROR org.apache.pig.Main - ERROR 0:
mapred.system.dir:
/Users/some-name/Software/cloudera/datadir/hadoop-some-name/mapred/system
mentioned in the configuration does not exist

I found the following patch that talks about this issue and about a patch
that generates this error message:
https://issues.apache.org/jira/browse/PIG-1154

What I'm not clear about is how to address this issue.

I'm using:
pig-0.7.0+16
hadoop-0.20.2+737

Thank you

Search Discussions

  • Rekha Joshi at Nov 8, 2010 at 5:45 am
    Hi,

    AFAIK, the characteristic of the parameter mapred.system.dir is not conducive to be modified per job.It must be modified on mapred-default.xml, conf/mapred-site.xml.
    So why local, I don't recall passing it on non-local mode either, and if passed I don't think it should work, as its final.
    The parameters passed on command line usually pertain to the job performance/memory needs of the specific job.Makes sense?

    Thanks & Regards,
    /Rekha.

    On 11/4/10 8:41 PM, "Michael Sundell" wrote:


    org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Passing
    Hadoop Site Configurations in classpath is not recommended for Local Mode
    2010-11-04 07:29:18,971 [main] ERROR org.apache.pig.Main - ERROR 0:
    mapred.system.dir:
    /Users/some-name/Software/cloudera/datadir/hadoop-some-name/mapred/system
    mentioned in the configuration does not exist

    I found the following patch that talks about this issue and about
  • Thejas M Nair at Nov 8, 2010 at 1:58 pm
    Looks like one of the environment variables in the pig shell script (`which pig`) is pointing to the hadoop cluster configuration. Check if environment variables PIG_CLASSPATH or PIG_CONF_DIR points to the hadoop configuration dir/file .
    -Thejas



    On 11/4/10 8:11 AM, "Michael Sundell" wrote:

    Hi,

    While trying to run Pig in local mode using "pig -x local" I get the
    following error:
    10/11/04 07:29:18 INFO pig.Main: Logging error messages to:
    /Users/some-name/Software/cloudera/hadoop-0.20.2+737/pig_1288880958619.log
    2010-11-04 07:29:18,807 [main] WARN
    org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Passing
    Hadoop Site Configurations in classpath is not recommended for Local Mode
    2010-11-04 07:29:18,971 [main] ERROR org.apache.pig.Main - ERROR 0:
    mapred.system.dir:
    /Users/some-name/Software/cloudera/datadir/hadoop-some-name/mapred/system
    mentioned in the configuration does not exist

    I found the following patch that talks about this issue and about a patch
    that generates this error message:
    https://issues.apache.org/jira/browse/PIG-1154

    What I'm not clear about is how to address this issue.

    I'm using:
    pig-0.7.0+16
    hadoop-0.20.2+737

    Thank you
  • Michael Sundell at Nov 17, 2010 at 4:11 pm
    It turns out that Pig calls $HADOOP_HOME/bin/hadoop-config.sh
    Inside this script this is set by default (among other things) which
    causes the warning:

    HADOOP_CONF_DIR="${HADOOP_CONF_DIR:-$HADOOP_HOME/conf}"

    Since this is a default setting for a Hadoop installation I'm assuming
    that this warning can be ignored.

    /Michael
    Looks like one of the environment variables in the pig shell script (`which pig`) is pointing to the hadoop cluster configuration. Check if environment variables PIG_CLASSPATH or PIG_CONF_DIR points to the hadoop configuration dir/file .
    -Thejas

    Hi,

    While trying to run Pig in local mode using "pig -x local" I get the
    following error:
    10/11/04 07:29:18 INFO pig.Main: Logging error messages to:
    /Users/some-name/Software/cloudera/hadoop-0.20.2+737/pig_1288880958619.log
    2010-11-04 07:29:18,807 [main] WARN
    org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Passing
    Hadoop Site Configurations in classpath is not recommended for Local Mode
    2010-11-04 07:29:18,971 [main] ERROR org.apache.pig.Main - ERROR 0:
    mapred.system.dir:
    /Users/some-name/Software/cloudera/datadir/hadoop-some-name/mapred/system
    mentioned in the configuration does not exist

    I found the following patch that talks about this issue and about a patch
    that generates this error message:
    https://issues.apache.org/jira/browse/PIG-1154

    What I'm not clear about is how to address this issue.

    I'm using:
    pig-0.7.0+16
    hadoop-0.20.2+737

    Thank you
  • Daniel Dai at Nov 17, 2010 at 7:44 pm
    This is a defect in Pig 0.7. Pig 0.8 will automatically exclude hadoop
    config file in local mode (https://issues.apache.org/jira/browse/PIG-1338)

    Daniel

    Michael Sundell wrote:
    It turns out that Pig calls $HADOOP_HOME/bin/hadoop-config.sh
    Inside this script this is set by default (among other things) which
    causes the warning:

    HADOOP_CONF_DIR="${HADOOP_CONF_DIR:-$HADOOP_HOME/conf}"

    Since this is a default setting for a Hadoop installation I'm assuming
    that this warning can be ignored.

    /Michael

    Looks like one of the environment variables in the pig shell script (`which pig`) is pointing to the hadoop cluster configuration. Check if environment variables PIG_CLASSPATH or PIG_CONF_DIR points to the hadoop configuration dir/file .
    -Thejas

    Hi,

    While trying to run Pig in local mode using "pig -x local" I get the
    following error:
    10/11/04 07:29:18 INFO pig.Main: Logging error messages to:
    /Users/some-name/Software/cloudera/hadoop-0.20.2+737/pig_1288880958619.log
    2010-11-04 07:29:18,807 [main] WARN
    org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Passing
    Hadoop Site Configurations in classpath is not recommended for Local Mode
    2010-11-04 07:29:18,971 [main] ERROR org.apache.pig.Main - ERROR 0:
    mapred.system.dir:
    /Users/some-name/Software/cloudera/datadir/hadoop-some-name/mapred/system
    mentioned in the configuration does not exist

    I found the following patch that talks about this issue and about a patch
    that generates this error message:
    https://issues.apache.org/jira/browse/PIG-1154

    What I'm not clear about is how to address this issue.

    I'm using:
    pig-0.7.0+16
    hadoop-0.20.2+737

    Thank you

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedNov 4, '10 at 3:11p
activeNov 17, '10 at 7:44p
posts5
users4
websitepig.apache.org

People

Translate

site design / logo © 2022 Grokbase