FAQ

On Wed, Jul 21, 2010 at 3:01 PM, Allen Wittenauer wrote:

On Jul 21, 2010, at 12:45 PM, Travis Crawford wrote:
Any tips on how to avoid getting into this situation?
What we started to do was two-fold:

a) During every maintenance, we'd blow away the mapred temp dirs. The
TaskTracker does a very bad job of cleaning up after jobs and there is
usually a lot of cruft. If you have a 'flat' disk/fs structure such that MR
temp and HDFS is shared, this is a huge problem.
I setup a cron job to delete files older than 5 days in mapred.local.dir.
I've also found that sometimes userlogs aren't cleaned up correctly, so
setting up a cron job to delete old files in userlogs is also a good idea.

Good luck, Travis!

Alex

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 3 of 7 | next ›
Discussion Overview
grouphdfs-user @
categorieshadoop
postedJul 21, '10 at 7:47p
activeJul 21, '10 at 11:49p
posts7
users3
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase