FAQ
Thanks for reporting this. I've had a few of these exceptions too.
There's a Jira issue here:
https://issues.apache.org/jira/browse/HADOOP-882.

Tom
On 17/01/07, Mike Smith wrote:
Sometimes large number of internal errors keeps happening on Amazon S3 which
does not let put command to go through, and it causes failure in
Jets3tFileSystemStore.put(Jets3tFileSystemStore.java:203). I think it would
be great to have an exponential back-off and retry mechanism in put() method
when we get these errors. Here is a sample of the exception that s3 failed
put() causes:


org.apache.hadoop.fs.s3.S3Exception: org.jets3t.service.S3ServiceException:
S3 PUT failed. XML Error Message: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InternalError</Code><Message>We encountered an internal error.
Please try
again.</Message><RequestId>201316FD</RequestId><HostId>sc5AtWptF6nqAkBzaaTo19oGtz9O</HostId></Error>

at org.apache.hadoop.fs.s3.Jets3tFileSystemStore.put(
Jets3tFileSystemStore.java:203)
at org.apache.hadoop.fs.s3.Jets3tFileSystemStore.storeBlock(
Jets3tFileSystemStore.java:212)
at org.apache.hadoop.fs.s3.S3OutputStream.endBlock(
S3OutputStream.java:151)
at org.apache.hadoop.fs.s3.S3OutputStream.close(S3OutputStream.java
:188)
at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
at org.apache.hadoop.fs.FSDataOutputStream$Summer.close(
FSDataOutputStream.java:99)
at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
at java.io.FilterOutputStream.close(FilterOutputStream.java:143)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:326)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:371)

Thanks, Mike

Search Discussions

Discussion Posts

Previous

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 2 of 2 | next ›
Discussion Overview
groupcommon-dev @
categorieshadoop
postedJan 17, '07 at 11:38p
activeJan 18, '07 at 9:54a
posts2
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Mike Smith: 1 post Tom White: 1 post

People

Translate

site design / logo © 2022 Grokbase