Grokbase Groups Hive user March 2011
FAQ
Hi all,

I am testing RCFile on S3.
I could execute queries which don't specify columns such as "select * from
table".
But, I could not execute queries which specify columns such as "select id
from table".

This job progress to near the end of a map task, but cannot finish the task
as the below log message.

2011-03-22 17:12:04,325 INFO
org.apache.hadoop.fs.s3native.NativeS3FileSystem: Opening key
'user/hive/warehouse/rcfile_logs/dt=20110312/controller=recipe/000001_0' for
reading at position '50000365'
2011-03-22 17:12:04,362 INFO
org.apache.hadoop.fs.s3native.NativeS3FileSystem: Opening key
'user/hive/warehouse/rcfile_logs/dt=20110312/controller=recipe/000001_0' for
reading at position '50458664'
2011-03-22 17:12:04,444 INFO
org.apache.hadoop.fs.s3native.NativeS3FileSystem: Opening key
'user/hive/warehouse/rcfile_logs/dt=20110312/controller=recipe/000001_0' for
reading at position '50603753'
2011-03-22 17:12:04,509 INFO
org.apache.hadoop.fs.s3native.NativeS3FileSystem: Opening key
'user/hive/warehouse/rcfile_logs/dt=20110312/controller=recipe/000001_0' for
reading at position '50651845'
2011-03-22 17:12:04,536 INFO
org.apache.hadoop.fs.s3native.NativeS3FileSystem: Opening key
'user/hive/warehouse/rcfile_logs/dt=20110312/controller=recipe/000001_0' for
reading at position '50735249'
2011-03-22 17:12:04,570 INFO
org.apache.hadoop.fs.s3native.NativeS3FileSystem: Opening key
'user/hive/warehouse/rcfile_logs/dt=20110312/controller=recipe/000001_0' for
reading at position '50956751'
2011-03-22 17:12:04,600 INFO
org.apache.hadoop.fs.s3native.NativeS3FileSystem: Opening key
'user/hive/warehouse/rcfile_logs/dt=20110312/controller=recipe/000001_0' for
reading at position '51025754'
2011-03-22 17:12:04,633 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 9
finished. closing...
...
2011-03-22 17:12:05,167 WARN org.apache.hadoop.mapred.Child: Error running
child org.apache.hadoop.fs.s3.S3Exception:
org.jets3t.service.S3ServiceException: S3 GET failed for
'/user%2Fhive%2Fwarehouse%2Frcfile_logs%2Fdt%3D20110312%2Fcontroller%3Drecipe%2F000001_0'
XML Error Message: <?xml version="1.0"
encoding="UTF-8"?><Error><Code>InvalidRange</Code><Message>The requested
range is not
satisfiable(Jets3tNativeFileSystemStore.java:229)
at
org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.handleServiceException(Jets3tNativeFileSystemStore.java:220)
at
org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.retrieve(Jets3tNativeFileSystemStore.java:133)
at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source) at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597) at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
at org.apache.hadoop.fs.s3native.$Proxy1.retrieve(Unknown Source) at
org.apache.hadoop.fs.s3native.NativeS3FileSystem$NativeS3FsInputStream.seek(NativeS3FileSystem.java:150)
at
org.apache.hadoop.fs.BufferedFSInputStream.seek(BufferedFSInputStream.java:76)
at
org.apache.hadoop.fs.BufferedFSInputStream.skip(BufferedFSInputStream.java:56)
at java.io.DataInputStream.skipBytes(DataInputStream.java:203) at
org.apache.hadoop.hive.ql.io.RCFile$ValueBuffer.readFields(RCFile.java:443)
at
org.apache.hadoop.hive.ql.io.RCFile$Reader.currentValueBuffer(RCFile.java:1304)
at
org.apache.hadoop.hive.ql.io.RCFile$Reader.getCurrentRow(RCFile.java:1425)
at
org.apache.hadoop.hive.ql.io.RCFileRecordReader.next(RCFileRecordReader.java:88)
at
org.apache.hadoop.hive.ql.io.RCFileRecordReader.next(RCFileRecordReader.java:39)
at
org.apache.hadoop.hive.ql.io.HiveRecordReader.doNext(HiveRecordReader.java:66)
at
org.apache.hadoop.hive.ql.io.HiveRecordReader.doNext(HiveRecordReader.java:32)
at
org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.next(HiveContextAwareRecordReader.java:67)
at
org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:208)
at
org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:193)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48) at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:390) at
org.apache.hadoop.mapred.MapTask.run(MapTask.java:324) at
org.apache.hadoop.mapred.Child$4.run(Child.java:240) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:396) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
at org.apache.hadoop.mapred.Child.main(Child.java:234) Caused by:
org.jets3t.service.S3ServiceException: S3 GET failed for
'/user%2Fhive%2Fwarehouse%2Frcfile_logs%2Fdt%3D20110312%2Fcontroller%3Drecipe%2F000001_0'
XML Error Message: <?xml version="1.0"
encoding="UTF-8"?><Error><Code>InvalidRange</Code><Message>The requested
range is not
satisfiable(RestS3Service.java:424)
at
org.jets3t.service.impl.rest.httpclient.RestS3Service.performRestGet(RestS3Service.java:686)
at
org.jets3t.service.impl.rest.httpclient.RestS3Service.getObjectImpl(RestS3Service.java:1558)
at
org.jets3t.service.impl.rest.httpclient.RestS3Service.getObjectImpl(RestS3Service.java:1501)
at org.jets3t.service.S3Service.getObject(S3Service.java:1876) at
org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.retrieve(Jets3tNativeFileSystemStore.java:129)
... 28 more 2011-03-22 17:12:05,170 INFO org.apache.hadoop.mapred.Task:
Runnning cleanup for the task


The client seems to request to the invalid range like this error code shows.

S3 GET failed for
'/user%2Fhive%2Fwarehouse%2Frcfile_logs%2Fdt%3D20110312%2Fcontroller%3Drecipe%2F000001_0'
XML Error Message: <?xml version="1.0" encoding="UTF-8"?><Error>
<Code>InvalidRange</Code>
<Message>The requested range is not satisfiable</Message>
<ActualObjectSize>51025754</ActualObjectSize>
<RequestId>***</RequestId>
<HostId>***</HostId>
<RangeRequested>bytes=51025754-</RangeRequested></Error>

This error did not occur on HDFS, so I guess this is a bug.
Or is there a person was able to run queries using RCFile on S3?

Thanks,
--
Shusuke Mikami
shun0102@gmail.com

Search Discussions

  • Yongqiang he at Mar 22, 2011 at 6:45 pm
    Don't know the reason, but can you try with "set hive.optimize.cp=false;"
    The problem seems it reports error when trying to skip some data.
    On Tue, Mar 22, 2011 at 7:27 AM, Shunsuke Mikami wrote:
    Hi all,
    I am testing RCFile on S3.
    I could execute queries which don't specify columns such as "select * from
    table".
    But, I could not execute queries which specify columns such as "select id
    from table".
    This job progress to near the end of a map task, but cannot finish the task
    as the below log message.
    2011-03-22 17:12:04,325 INFO
    org.apache.hadoop.fs.s3native.NativeS3FileSystem: Opening key
    'user/hive/warehouse/rcfile_logs/dt=20110312/controller=recipe/000001_0' for
    reading at position '50000365'
    2011-03-22 17:12:04,362 INFO
    org.apache.hadoop.fs.s3native.NativeS3FileSystem: Opening key
    'user/hive/warehouse/rcfile_logs/dt=20110312/controller=recipe/000001_0' for
    reading at position '50458664'
    2011-03-22 17:12:04,444 INFO
    org.apache.hadoop.fs.s3native.NativeS3FileSystem: Opening key
    'user/hive/warehouse/rcfile_logs/dt=20110312/controller=recipe/000001_0' for
    reading at position '50603753'
    2011-03-22 17:12:04,509 INFO
    org.apache.hadoop.fs.s3native.NativeS3FileSystem: Opening key
    'user/hive/warehouse/rcfile_logs/dt=20110312/controller=recipe/000001_0' for
    reading at position '50651845'
    2011-03-22 17:12:04,536 INFO
    org.apache.hadoop.fs.s3native.NativeS3FileSystem: Opening key
    'user/hive/warehouse/rcfile_logs/dt=20110312/controller=recipe/000001_0' for
    reading at position '50735249'
    2011-03-22 17:12:04,570 INFO
    org.apache.hadoop.fs.s3native.NativeS3FileSystem: Opening key
    'user/hive/warehouse/rcfile_logs/dt=20110312/controller=recipe/000001_0' for
    reading at position '50956751'
    2011-03-22 17:12:04,600 INFO
    org.apache.hadoop.fs.s3native.NativeS3FileSystem: Opening key
    'user/hive/warehouse/rcfile_logs/dt=20110312/controller=recipe/000001_0' for
    reading at position '51025754'
    2011-03-22 17:12:04,633 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 9
    finished. closing...
    ...
    2011-03-22 17:12:05,167 WARN org.apache.hadoop.mapred.Child: Error running
    child org.apache.hadoop.fs.s3.S3Exception:
    org.jets3t.service.S3ServiceException: S3 GET failed for
    '/user%2Fhive%2Fwarehouse%2Frcfile_logs%2Fdt%3D20110312%2Fcontroller%3Drecipe%2F000001_0'
    XML Error Message: <?xml version="1.0"
    encoding="UTF-8"?><Error><Code>InvalidRange</Code><Message>The requested
    range is not
    satisfiable</Message><ActualObjectSize>51025754</ActualObjectSize><RequestId>***</RequestId><HostId>***</HostId><RangeRequested>bytes=51025754-</RangeRequested></Error>
    at
    org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.handleServiceException(Jets3tNativeFileSystemStore.java:229)
    at
    org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.handleServiceException(Jets3tNativeFileSystemStore.java:220)
    at
    org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.retrieve(Jets3tNativeFileSystemStore.java:133)
    at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source) at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597) at
    org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
    at
    org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
    at org.apache.hadoop.fs.s3native.$Proxy1.retrieve(Unknown Source) at
    org.apache.hadoop.fs.s3native.NativeS3FileSystem$NativeS3FsInputStream.seek(NativeS3FileSystem.java:150)
    at
    org.apache.hadoop.fs.BufferedFSInputStream.seek(BufferedFSInputStream.java:76)
    at
    org.apache.hadoop.fs.BufferedFSInputStream.skip(BufferedFSInputStream.java:56)
    at java.io.DataInputStream.skipBytes(DataInputStream.java:203) at
    org.apache.hadoop.hive.ql.io.RCFile$ValueBuffer.readFields(RCFile.java:443)
    at
    org.apache.hadoop.hive.ql.io.RCFile$Reader.currentValueBuffer(RCFile.java:1304)
    at
    org.apache.hadoop.hive.ql.io.RCFile$Reader.getCurrentRow(RCFile.java:1425)
    at
    org.apache.hadoop.hive.ql.io.RCFileRecordReader.next(RCFileRecordReader.java:88)
    at
    org.apache.hadoop.hive.ql.io.RCFileRecordReader.next(RCFileRecordReader.java:39)
    at
    org.apache.hadoop.hive.ql.io.HiveRecordReader.doNext(HiveRecordReader.java:66)
    at
    org.apache.hadoop.hive.ql.io.HiveRecordReader.doNext(HiveRecordReader.java:32)
    at
    org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.next(HiveContextAwareRecordReader.java:67)
    at
    org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:208)
    at
    org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:193)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48) at
    org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:390) at
    org.apache.hadoop.mapred.MapTask.run(MapTask.java:324) at
    org.apache.hadoop.mapred.Child$4.run(Child.java:240) at
    java.security.AccessController.doPrivileged(Native Method) at
    javax.security.auth.Subject.doAs(Subject.java:396) at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
    at org.apache.hadoop.mapred.Child.main(Child.java:234) Caused by:
    org.jets3t.service.S3ServiceException: S3 GET failed for
    '/user%2Fhive%2Fwarehouse%2Frcfile_logs%2Fdt%3D20110312%2Fcontroller%3Drecipe%2F000001_0'
    XML Error Message: <?xml version="1.0"
    encoding="UTF-8"?><Error><Code>InvalidRange</Code><Message>The requested
    range is not
    satisfiable</Message><ActualObjectSize>51025754</ActualObjectSize><RequestId>4E5BD7E6D94DBA1B</RequestId><HostId>l+oM6yDUt+MbQgDB4pzcGckUQ1E7pbaUGy26yuTqNE4Gn+FdiJIA6u4VvsQl2+aR</HostId><RangeRequested>bytes=51025754-</RangeRequested></Error>
    at
    org.jets3t.service.impl.rest.httpclient.RestS3Service.performRequest(RestS3Service.java:424)
    at
    org.jets3t.service.impl.rest.httpclient.RestS3Service.performRestGet(RestS3Service.java:686)
    at
    org.jets3t.service.impl.rest.httpclient.RestS3Service.getObjectImpl(RestS3Service.java:1558)
    at
    org.jets3t.service.impl.rest.httpclient.RestS3Service.getObjectImpl(RestS3Service.java:1501)
    at org.jets3t.service.S3Service.getObject(S3Service.java:1876) at
    org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.retrieve(Jets3tNativeFileSystemStore.java:129)
    ... 28 more 2011-03-22 17:12:05,170 INFO org.apache.hadoop.mapred.Task:
    Runnning cleanup for the task

    The client seems to request to the invalid range like this error code shows.
    S3 GET failed for
    '/user%2Fhive%2Fwarehouse%2Frcfile_logs%2Fdt%3D20110312%2Fcontroller%3Drecipe%2F000001_0'
    XML Error Message: <?xml version="1.0" encoding="UTF-8"?><Error>
    <Code>InvalidRange</Code>
    <Message>The requested range is not satisfiable</Message>
    <ActualObjectSize>51025754</ActualObjectSize>
    <RequestId>***</RequestId>
    <HostId>***</HostId>
    <RangeRequested>bytes=51025754-</RangeRequested></Error>
    This error did not occur on HDFS, so I guess this is a bug.
    Or is there a person was able to run queries using RCFile on S3?
    Thanks,
    --
    Shusuke Mikami
    shun0102@gmail.com
  • Shunsuke Mikami at Mar 23, 2011 at 12:47 am
    Thank you!

    I tried with "set hive.optimize.cp=false;", then it works!
    However, it reduce the merit of RCFile which skip unnecessary data.
    It may be better to use *SequenceFile in the present state of things?*

    --
    Shunsuke Mikami

    2011/3/23 yongqiang he <heyongqiangict@gmail.com>
    Don't know the reason, but can you try with "set hive.optimize.cp=false;"
    The problem seems it reports error when trying to skip some data.
    On Tue, Mar 22, 2011 at 7:27 AM, Shunsuke Mikami wrote:
    Hi all,
    I am testing RCFile on S3.
    I could execute queries which don't specify columns such as "select * from
    table".
    But, I could not execute queries which specify columns such as "select id
    from table".
    This job progress to near the end of a map task, but cannot finish the task
    as the below log message.
    2011-03-22 17:12:04,325 INFO
    org.apache.hadoop.fs.s3native.NativeS3FileSystem: Opening key
    'user/hive/warehouse/rcfile_logs/dt=20110312/controller=recipe/000001_0' for
    reading at position '50000365'
    2011-03-22 17:12:04,362 INFO
    org.apache.hadoop.fs.s3native.NativeS3FileSystem: Opening key
    'user/hive/warehouse/rcfile_logs/dt=20110312/controller=recipe/000001_0' for
    reading at position '50458664'
    2011-03-22 17:12:04,444 INFO
    org.apache.hadoop.fs.s3native.NativeS3FileSystem: Opening key
    'user/hive/warehouse/rcfile_logs/dt=20110312/controller=recipe/000001_0' for
    reading at position '50603753'
    2011-03-22 17:12:04,509 INFO
    org.apache.hadoop.fs.s3native.NativeS3FileSystem: Opening key
    'user/hive/warehouse/rcfile_logs/dt=20110312/controller=recipe/000001_0' for
    reading at position '50651845'
    2011-03-22 17:12:04,536 INFO
    org.apache.hadoop.fs.s3native.NativeS3FileSystem: Opening key
    'user/hive/warehouse/rcfile_logs/dt=20110312/controller=recipe/000001_0' for
    reading at position '50735249'
    2011-03-22 17:12:04,570 INFO
    org.apache.hadoop.fs.s3native.NativeS3FileSystem: Opening key
    'user/hive/warehouse/rcfile_logs/dt=20110312/controller=recipe/000001_0' for
    reading at position '50956751'
    2011-03-22 17:12:04,600 INFO
    org.apache.hadoop.fs.s3native.NativeS3FileSystem: Opening key
    'user/hive/warehouse/rcfile_logs/dt=20110312/controller=recipe/000001_0' for
    reading at position '51025754'
    2011-03-22 17:12:04,633 INFO org.apache.hadoop.hive.ql.exec.MapOperator: 9
    finished. closing...
    ...
    2011-03-22 17:12:05,167 WARN org.apache.hadoop.mapred.Child: Error running
    child org.apache.hadoop.fs.s3.S3Exception:
    org.jets3t.service.S3ServiceException: S3 GET failed for
    '/user%2Fhive%2Fwarehouse%2Frcfile_logs%2Fdt%3D20110312%2Fcontroller%3Drecipe%2F000001_0'
    XML Error Message: <?xml version="1.0"
    encoding="UTF-8"?><Error><Code>InvalidRange</Code><Message>The requested
    range is not
    satisfiable</Message><ActualObjectSize>51025754</ActualObjectSize><RequestId>***</RequestId><HostId>***</HostId><RangeRequested>bytes=51025754-</RangeRequested></Error>
    at
    org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.handleServiceException(Jets3tNativeFileSystemStore.java:229)
    at
    org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.handleServiceException(Jets3tNativeFileSystemStore.java:220)
    at
    org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.retrieve(Jets3tNativeFileSystemStore.java:133)
    at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source) at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597) at
    org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
    at
    org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
    at org.apache.hadoop.fs.s3native.$Proxy1.retrieve(Unknown Source) at
    org.apache.hadoop.fs.s3native.NativeS3FileSystem$NativeS3FsInputStream.seek(NativeS3FileSystem.java:150)
    at
    org.apache.hadoop.fs.BufferedFSInputStream.seek(BufferedFSInputStream.java:76)
    at
    org.apache.hadoop.fs.BufferedFSInputStream.skip(BufferedFSInputStream.java:56)
    at java.io.DataInputStream.skipBytes(DataInputStream.java:203) at
    org.apache.hadoop.hive.ql.io.RCFile$ValueBuffer.readFields(RCFile.java:443)
    at
    org.apache.hadoop.hive.ql.io.RCFile$Reader.currentValueBuffer(RCFile.java:1304)
    at
    org.apache.hadoop.hive.ql.io.RCFile$Reader.getCurrentRow(RCFile.java:1425)
    at
    org.apache.hadoop.hive.ql.io.RCFileRecordReader.next(RCFileRecordReader.java:88)
    at
    org.apache.hadoop.hive.ql.io.RCFileRecordReader.next(RCFileRecordReader.java:39)
    at
    org.apache.hadoop.hive.ql.io.HiveRecordReader.doNext(HiveRecordReader.java:66)
    at
    org.apache.hadoop.hive.ql.io.HiveRecordReader.doNext(HiveRecordReader.java:32)
    at
    org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.next(HiveContextAwareRecordReader.java:67)
    at
    org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:208)
    at
    org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:193)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48) at
    org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:390) at
    org.apache.hadoop.mapred.MapTask.run(MapTask.java:324) at
    org.apache.hadoop.mapred.Child$4.run(Child.java:240) at
    java.security.AccessController.doPrivileged(Native Method) at
    javax.security.auth.Subject.doAs(Subject.java:396) at
    org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
    at org.apache.hadoop.mapred.Child.main(Child.java:234) Caused by:
    org.jets3t.service.S3ServiceException: S3 GET failed for
    '/user%2Fhive%2Fwarehouse%2Frcfile_logs%2Fdt%3D20110312%2Fcontroller%3Drecipe%2F000001_0'
    XML Error Message: <?xml version="1.0"
    encoding="UTF-8"?><Error><Code>InvalidRange</Code><Message>The requested
    range is not
    satisfiable</Message><ActualObjectSize>51025754</ActualObjectSize><RequestId>4E5BD7E6D94DBA1B</RequestId><HostId>l+oM6yDUt+MbQgDB4pzcGckUQ1E7pbaUGy26yuTqNE4Gn+FdiJIA6u4VvsQl2+aR</HostId><RangeRequested>bytes=51025754-</RangeRequested></Error>
    at
    org.jets3t.service.impl.rest.httpclient.RestS3Service.performRequest(RestS3Service.java:424)
    at
    org.jets3t.service.impl.rest.httpclient.RestS3Service.performRestGet(RestS3Service.java:686)
    at
    org.jets3t.service.impl.rest.httpclient.RestS3Service.getObjectImpl(RestS3Service.java:1558)
    at
    org.jets3t.service.impl.rest.httpclient.RestS3Service.getObjectImpl(RestS3Service.java:1501)
    at org.jets3t.service.S3Service.getObject(S3Service.java:1876) at
    org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.retrieve(Jets3tNativeFileSystemStore.java:129)
    ... 28 more 2011-03-22 17:12:05,170 INFO org.apache.hadoop.mapred.Task:
    Runnning cleanup for the task

    The client seems to request to the invalid range like this error code shows.
    S3 GET failed for
    '/user%2Fhive%2Fwarehouse%2Frcfile_logs%2Fdt%3D20110312%2Fcontroller%3Drecipe%2F000001_0'
    XML Error Message: <?xml version="1.0" encoding="UTF-8"?><Error>
    <Code>InvalidRange</Code>
    <Message>The requested range is not satisfiable</Message>
    <ActualObjectSize>51025754</ActualObjectSize>
    <RequestId>***</RequestId>
    <HostId>***</HostId>
    <RangeRequested>bytes=51025754-</RangeRequested></Error>
    This error did not occur on HDFS, so I guess this is a bug.
    Or is there a person was able to run queries using RCFile on S3?
    Thanks,
    --
    Shusuke Mikami
    shun0102@gmail.com

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshive, hadoop
postedMar 22, '11 at 2:28p
activeMar 23, '11 at 12:47a
posts3
users2
websitehive.apache.org

2 users in discussion

Shunsuke Mikami: 2 posts Yongqiang he: 1 post

People

Translate

site design / logo © 2021 Grokbase