[Pig-user] RE: Jobs failing during map phase due to low memory

In mapreduce mode, One pig script maybe consist several hadoop jobs, So
intermediate will been put in tmp directory.

I guess maybe some jobs of your script failed which caused the intermediate
data is empty.

PS, it looks like you run your pig in EC2 machine. Maybe the problem is
related with the environment.
So could you run it in local machine's mapreduce mode ? It will facilitate
the debugging.

And if you could provide your pig script and sample data, it will make
others better understand your problems.


-----Original Message-----
From: seshadri bashyam
Sent: 2009年9月10日 0:39
To: pig-user@hadoop.apache.org
Subject: RE: Jobs failing during map phase due to low memory

Any idea why I get this following error?

When I run pig script in local mode it works but when i run it in
hadoop/mapreduce mode it throws the following error:
ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2100:
hdfs://domU-12-31-38-00-40-D1.compute-1.internal:9000/tmp/temp1351891394/tmp
1417921370
does not exist.

Command used to run in local mode:
pig -x local -p OUTPUT=file:///home/hadoop/output/unique_users_sorted8
file:///home/hadoop/pigscripts/unique_users_sorted.pig

Command used to run in hadoop/mapreduce mode:
pig -p OUTPUT=file:///home/hadoop/output/unique_users_sorted8
file:///home/hadoop/pigscripts/unique_users_sorted.pig

I am not accessing anything under tmp directory anywhere in my pig script.
Is it trying to use tmp internally for some intermediate process?

Any help will be much appreciated. If anyone wants to look at the pig script
I can post it here.

Thanks in advance.

-Sesha