Grokbase › Groups › Hadoop › common-user › March 2011
FAQ
I have a Hadoop job that creates a temporary sequence file at a path it gets
from FileOutputFormat.getPathForWorkFile(). On successful completion of the
job, I'd like to delete the file, but I can't figure out how.

Is there an HDFS temporary file API with automatic cleanup? In failing that
is there a way I can programmatically delete the file when I'm done with it?

These seem like basic questions, but I haven't been able to find the answers
in the O'Reilly book or online.

Search Discussions

  • Harsh J at Mar 28, 2011 at 8:29 pm ⇧
    Generally you can delete files on any hadoop-used FS using a suitable
    FileSystem.delete(Path) call. Would this work for you?

    I'm not aware of any auto temporary-create-and-delete functionality
    though. Would be good to know if this too exists inside Hadoop's libs.
    On Tue, Mar 29, 2011 at 1:47 AM, W.P. McNeill wrote:
    These seem like basic questions, but I haven't been able to find the answers
    in the O'Reilly book or online.
    Tom White's book covers the deleting part at least. Check it out:
    Chapter 3 (HDFS) -> Java Interface -> Deleting Data.
  • W.P. McNeill at Mar 28, 2011 at 9:41 pm ⇧
    FileSystem.delete(Path) appears to be deprecated in the new API. Is there a
    non-deprecated way to do this.
    On Mon, Mar 28, 2011 at 1:28 PM, Harsh J wrote:

    Generally you can delete files on any hadoop-used FS using a suitable
    FileSystem.delete(Path) call. Would this work for you?

    I'm not aware of any auto temporary-create-and-delete functionality
    though. Would be good to know if this too exists inside Hadoop's libs.
    On Tue, Mar 29, 2011 at 1:47 AM, W.P. McNeill wrote:
    These seem like basic questions, but I haven't been able to find the answers
    in the O'Reilly book or online.
    Tom White's book covers the deleting part at least. Check it out:
    Chapter 3 (HDFS) -> Java Interface -> Deleting Data.

    --
    Harsh J
    http://harshj.com
  • W.P. McNeill at Mar 28, 2011 at 9:47 pm ⇧
    Never mind. I overlooked the link that says the non-deprecated version is
    FileSystem.delete(Path, BOOLEAN).

    I'm still curious if there is a an automatically self-cleaning temporary
    file system.
    On Mon, Mar 28, 2011 at 2:41 PM, W.P. McNeill wrote:

    FileSystem.delete(Path) appears to be deprecated in the new API. Is there
    a non-deprecated way to do this.

    On Mon, Mar 28, 2011 at 1:28 PM, Harsh J wrote:

    Generally you can delete files on any hadoop-used FS using a suitable
    FileSystem.delete(Path) call. Would this work for you?

    I'm not aware of any auto temporary-create-and-delete functionality
    though. Would be good to know if this too exists inside Hadoop's libs.
    On Tue, Mar 29, 2011 at 1:47 AM, W.P. McNeill wrote:
    These seem like basic questions, but I haven't been able to find the answers
    in the O'Reilly book or online.
    Tom White's book covers the deleting part at least. Check it out:
    Chapter 3 (HDFS) -> Java Interface -> Deleting Data.

    --
    Harsh J
    http://harshj.com

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedMar 28, '11 at 8:18p
activeMar 28, '11 at 9:47p
posts4
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

W.P. McNeill: 3 posts Harsh J: 1 post

Content

People

Support

Translate

site design / logo © 2022 Grokbase