FAQ
Any idea why I get this following error?

When I run pig script in local mode it works but when i run it in
hadoop/mapreduce mode it throws the following error:
ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2100:
hdfs://domU-12-31-38-00-40-D1.compute-1.internal:9000/tmp/temp1351891394/tmp1417921370
does not exist.

Command used to run in local mode:
pig -x local -p OUTPUT=file:///home/hadoop/output/unique_users_sorted8
file:///home/hadoop/pigscripts/unique_users_sorted.pig

Command used to run in hadoop/mapreduce mode:
pig -p OUTPUT=file:///home/hadoop/output/unique_users_sorted8
file:///home/hadoop/pigscripts/unique_users_sorted.pig

I am not accessing anything under tmp directory anywhere in my pig script.
Is it trying to use tmp internally for some intermediate process?

Any help will be much appreciated. If anyone wants to look at the pig script
I can post it here.

Thanks in advance.

-Sesha

Search Discussions

  • Zjffdu at Sep 11, 2009 at 4:30 pm
    In mapreduce mode, One pig script maybe consist several hadoop jobs, So
    intermediate will been put in tmp directory.

    I guess maybe some jobs of your script failed which caused the intermediate
    data is empty.

    PS, it looks like you run your pig in EC2 machine. Maybe the problem is
    related with the environment.
    So could you run it in local machine's mapreduce mode ? It will facilitate
    the debugging.

    And if you could provide your pig script and sample data, it will make
    others better understand your problems.


    -----Original Message-----
    From: seshadri bashyam
    Sent: 2009年9月10日 0:39
    To: pig-user@hadoop.apache.org
    Subject: RE: Jobs failing during map phase due to low memory

    Any idea why I get this following error?

    When I run pig script in local mode it works but when i run it in
    hadoop/mapreduce mode it throws the following error:
    ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2100:
    hdfs://domU-12-31-38-00-40-D1.compute-1.internal:9000/tmp/temp1351891394/tmp
    1417921370
    does not exist.

    Command used to run in local mode:
    pig -x local -p OUTPUT=file:///home/hadoop/output/unique_users_sorted8
    file:///home/hadoop/pigscripts/unique_users_sorted.pig

    Command used to run in hadoop/mapreduce mode:
    pig -p OUTPUT=file:///home/hadoop/output/unique_users_sorted8
    file:///home/hadoop/pigscripts/unique_users_sorted.pig

    I am not accessing anything under tmp directory anywhere in my pig script.
    Is it trying to use tmp internally for some intermediate process?

    Any help will be much appreciated. If anyone wants to look at the pig script
    I can post it here.

    Thanks in advance.

    -Sesha

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedSep 10, '09 at 4:22p
activeSep 11, '09 at 4:30p
posts2
users2
websitepig.apache.org

2 users in discussion

Seshadri bashyam: 1 post Zjffdu: 1 post

People

Translate

site design / logo © 2021 Grokbase