Grokbase › Groups › Hadoop › mapreduce-user › December 2011
FAQ
Hi,

We're sometimes seeing this exception if a map task already failed before due
to, for example, an OOM error. Any ideas on how to address this issue?

org.apache.hadoop.io.SecureIOUtils$AlreadyExistsException: File
/opt/hadoop/hadoop-0.20.205.0/libexec/../logs/userlogs/job_201112261420_0003/attempt_201112261420_0003_m_000029_1/log.tmp
already exists
at
org.apache.hadoop.io.SecureIOUtils.insecureCreateForWrite(SecureIOUtils.java:130)
at
org.apache.hadoop.io.SecureIOUtils.createForWrite(SecureIOUtils.java:157)
at org.apache.hadoop.mapred.TaskLog.writeToIndexFile(TaskLog.java:296)
at org.apache.hadoop.mapred.TaskLog.syncLogs(TaskLog.java:369)
at org.apache.hadoop.mapred.Child$4.run(Child.java:257)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
at org.apache.hadoop.mapred.Child.main(Child.java:249)


Thanks

Search Discussions

  • Arun C Murthy at Dec 27, 2011 at 7:33 am ⇧
    Is this with jvm reuse turned on?
    On Dec 26, 2011, at 9:38 AM, Markus Jelsma wrote:

    Hi,

    We're sometimes seeing this exception if a map task already failed before due
    to, for example, an OOM error. Any ideas on how to address this issue?

    org.apache.hadoop.io.SecureIOUtils$AlreadyExistsException: File
    /opt/hadoop/hadoop-0.20.205.0/libexec/../logs/userlogs/job_201112261420_0003/attempt_201112261420_0003_m_000029_1/log.tmp
    already exists
    at
    org.apache.hadoop.io.SecureIOUtils.insecureCreateForWrite(SecureIOUtils.java:130)
    at
    org.apache.hadoop.io.SecureIOUtils.createForWrite(SecureIOUtils.java:157)
    at org.apache.hadoop.mapred.TaskLog.writeToIndexFile(TaskLog.java:296)
    at org.apache.hadoop.mapred.TaskLog.syncLogs(TaskLog.java:369)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:257)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
    at org.apache.hadoop.mapred.Child.main(Child.java:249)


    Thanks
  • Markus Jelsma at Dec 27, 2011 at 11:17 am ⇧
    No, although i didn't say it in the other thread we had it disabled already on
    20.205 as we suspected a leak somewhere.
    Is this with jvm reuse turned on?
    On Dec 26, 2011, at 9:38 AM, Markus Jelsma wrote:
    Hi,

    We're sometimes seeing this exception if a map task already failed before
    due to, for example, an OOM error. Any ideas on how to address this
    issue?

    org.apache.hadoop.io.SecureIOUtils$AlreadyExistsException: File
    /opt/hadoop/hadoop-0.20.205.0/libexec/../logs/userlogs/job_201112261420_0
    003/attempt_201112261420_0003_m_000029_1/log.tmp already exists

    at

    org.apache.hadoop.io.SecureIOUtils.insecureCreateForWrite(SecureIOUtils.j
    ava:130)

    at

    org.apache.hadoop.io.SecureIOUtils.createForWrite(SecureIOUtils.java:157)

    at org.apache.hadoop.mapred.TaskLog.writeToIndexFile(TaskLog.java:296)
    at org.apache.hadoop.mapred.TaskLog.syncLogs(TaskLog.java:369)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:257)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at

    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation
    .java:1059)

    at org.apache.hadoop.mapred.Child.main(Child.java:249)

    Thanks

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupmapreduce-user @
categorieshadoop
postedDec 26, '11 at 5:39p
activeDec 27, '11 at 11:17a
posts3
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Markus Jelsma: 2 posts Arun C Murthy: 1 post

Content

People

Support

Translate

site design / logo © 2022 Grokbase