Grokbase Groups Hive user July 2009
FAQ
Below is the output of Hive for an INSERT-SELECT from one 'EXTERNAL' table
to another. This is running in EC2 and the external tables have partitions
registered as path-keys in S3. The final upload of data to S3 fails. This
does not always happen, yet when it does the only option seems to be to
re-run the job.

Is there another way? Perhaps a way to tell hive to retry data-uploads to
S3 if a failure occurs? - Neal


[snip]
map = 100%, reduce =52%
map = 100%, reduce =88%
map = 100%, reduce =93%
map = 100%, reduce =100%
map = 100%, reduce =49%
map = 100%, reduce =100%
map = 100%, reduce =45%
map = 100%, reduce =86%
map = 100%, reduce =100%
map = 100%, reduce =67%
map = 100%, reduce =12%
map = 100%, reduce =14%
map = 100%, reduce =0%
map = 100%, reduce =17%
map = 100%, reduce =0%
map = 100%, reduce =45%
Ended Job = job_200907281406_0315
Job Commit failed with exception
'org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.hadoop.fs.s3.S3Exception:
org.jets3t.service.S3ServiceException: Encountered too many S3 Internal
Server errors (6), aborting request.)'
FAILED: Execution Error, return code 3 from
org.apache.hadoop.hive.ql.exec.ExecDriver

Search Discussions

  • Neal Richter at Aug 6, 2009 at 2:37 pm
    Any response on this? We had some hadoop output to S3 failures last week as
    well. Should I post to hive-dev list?

    Thanks! - Neal
    On Thu, Jul 30, 2009 at 7:11 PM, Neal Richter wrote:

    Below is the output of Hive for an INSERT-SELECT from one 'EXTERNAL' table
    to another. This is running in EC2 and the external tables have partitions
    registered as path-keys in S3. The final upload of data to S3 fails. This
    does not always happen, yet when it does the only option seems to be to
    re-run the job.

    Is there another way? Perhaps a way to tell hive to retry data-uploads to
    S3 if a failure occurs? - Neal


    [snip]
    map = 100%, reduce =52%
    map = 100%, reduce =88%
    map = 100%, reduce =93%
    map = 100%, reduce =100%
    map = 100%, reduce =49%
    map = 100%, reduce =100%
    map = 100%, reduce =45%
    map = 100%, reduce =86%
    map = 100%, reduce =100%
    map = 100%, reduce =67%
    map = 100%, reduce =12%
    map = 100%, reduce =14%
    map = 100%, reduce =0%
    map = 100%, reduce =17%
    map = 100%, reduce =0%
    map = 100%, reduce =45%
    Ended Job = job_200907281406_0315
    Job Commit failed with exception
    'org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.hadoop.fs.s3.S3Exception:
    org.jets3t.service.S3ServiceException: Encountered too many S3 Internal
    Server errors (6), aborting request.)'
    FAILED: Execution Error, return code 3 from
    org.apache.hadoop.hive.ql.exec.ExecDriver
  • Zheng Shao at Aug 6, 2009 at 10:04 pm
    Hi Neal,

    It seems like that the exception is thrown at:
    org.apache.hadoop.fs.s3.S3Exception:>> org.jets3t.service.S3ServiceException

    Is there any parameter to adjust the s3 file system?

    Zheng

    On Thu, Aug 6, 2009 at 7:32 AM, Neal Richterwrote:
    Any response on this?  We had some hadoop output to S3 failures last week as
    well.  Should I post to hive-dev list?

    Thanks! - Neal
    On Thu, Jul 30, 2009 at 7:11 PM, Neal Richter wrote:

    Below is the output of Hive for an INSERT-SELECT from one 'EXTERNAL' table
    to another.  This is running in EC2 and the external tables have partitions
    registered as path-keys in S3.   The final upload of data to S3 fails.  This
    does not always happen, yet when it does the only option seems to be to
    re-run the job.

    Is there another way?  Perhaps a way to tell hive to retry data-uploads to
    S3 if a failure occurs? - Neal


    [snip]
    map = 100%,  reduce =52%
    map = 100%,  reduce =88%
    map = 100%,  reduce =93%
    map = 100%,  reduce =100%
    map = 100%,  reduce =49%
    map = 100%,  reduce =100%
    map = 100%,  reduce =45%
    map = 100%,  reduce =86%
    map = 100%,  reduce =100%
    map = 100%,  reduce =67%
    map = 100%,  reduce =12%
    map = 100%,  reduce =14%
    map = 100%,  reduce =0%
    map = 100%,  reduce =17%
    map = 100%,  reduce =0%
    map = 100%,  reduce =45%
    Ended Job = job_200907281406_0315
    Job Commit failed with exception
    'org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.hadoop.fs.s3.S3Exception:
    org.jets3t.service.S3ServiceException: Encountered too many S3 Internal
    Server errors (6), aborting request.)'
    FAILED: Execution Error, return code 3 from
    org.apache.hadoop.hive.ql.exec.ExecDriver


    --
    Yours,
    Zheng
  • Neal Richter at Aug 8, 2009 at 5:39 am
    Not sure, I will look into it.

    My prediction is that we go with HDFS rather than S3 linked destination
    tables soon anyway. In the mean time we're going to instrument the code to
    just retry failed queries.

    - Neal
    On Thu, Aug 6, 2009 at 4:03 PM, Zheng Shao wrote:

    Hi Neal,

    It seems like that the exception is thrown at:
    org.apache.hadoop.fs.s3.S3Exception:>>
    org.jets3t.service.S3ServiceException

    Is there any parameter to adjust the s3 file system?

    Zheng

    On Thu, Aug 6, 2009 at 7:32 AM, Neal Richterwrote:
    Any response on this? We had some hadoop output to S3 failures last week as
    well. Should I post to hive-dev list?

    Thanks! - Neal
    On Thu, Jul 30, 2009 at 7:11 PM, Neal Richter wrote:

    Below is the output of Hive for an INSERT-SELECT from one 'EXTERNAL'
    table
    to another. This is running in EC2 and the external tables have
    partitions
    registered as path-keys in S3. The final upload of data to S3 fails.
    This
    does not always happen, yet when it does the only option seems to be to
    re-run the job.

    Is there another way? Perhaps a way to tell hive to retry data-uploads
    to
    S3 if a failure occurs? - Neal


    [snip]
    map = 100%, reduce =52%
    map = 100%, reduce =88%
    map = 100%, reduce =93%
    map = 100%, reduce =100%
    map = 100%, reduce =49%
    map = 100%, reduce =100%
    map = 100%, reduce =45%
    map = 100%, reduce =86%
    map = 100%, reduce =100%
    map = 100%, reduce =67%
    map = 100%, reduce =12%
    map = 100%, reduce =14%
    map = 100%, reduce =0%
    map = 100%, reduce =17%
    map = 100%, reduce =0%
    map = 100%, reduce =45%
    Ended Job = job_200907281406_0315
    Job Commit failed with exception
    'org.apache.hadoop.hive.ql.metadata.HiveException(org.apache.hadoop.fs.s3.S3Exception:
    org.jets3t.service.S3ServiceException: Encountered too many S3 Internal
    Server errors (6), aborting request.)'
    FAILED: Execution Error, return code 3 from
    org.apache.hadoop.hive.ql.exec.ExecDriver


    --
    Yours,
    Zheng

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshive, hadoop
postedJul 31, '09 at 1:12a
activeAug 8, '09 at 5:39a
posts4
users2
websitehive.apache.org

2 users in discussion

Neal Richter: 3 posts Zheng Shao: 1 post

People

Translate

site design / logo © 2021 Grokbase