FAQ
Sometimes large number of internal errors keeps happening on Amazon S3 which
does not let put command to go through, and it causes failure in
Jets3tFileSystemStore.put(Jets3tFileSystemStore.java:203). I think it would
be great to have an exponential back-off and retry mechanism in put() method
when we get these errors. Here is a sample of the exception that s3 failed
put() causes:


org.apache.hadoop.fs.s3.S3Exception: org.jets3t.service.S3ServiceException:
S3 PUT failed. XML Error Message: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InternalError</Code><Message>We encountered an internal error.
Please try
again.(
Jets3tFileSystemStore.java:203)
at org.apache.hadoop.fs.s3.Jets3tFileSystemStore.storeBlock(
Jets3tFileSystemStore.java:212)
at org.apache.hadoop.fs.s3.S3OutputStream.endBlock(
S3OutputStream.java:151)
at org.apache.hadoop.fs.s3.S3OutputStream.close(S3OutputStream.java
:188)
at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
at org.apache.hadoop.fs.FSDataOutputStream$Summer.close(
FSDataOutputStream.java:99)
at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:326)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:371)

Thanks, Mike

Search Discussions

Discussion Posts

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 1 of 2 | next ›
Discussion Overview
groupcommon-dev @
categorieshadoop
postedJan 17, '07 at 11:38p
activeJan 18, '07 at 9:54a
posts2
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Mike Smith: 1 post Tom White: 1 post

People

Translate

site design / logo © 2022 Grokbase