Grokbase Groups Hive user August 2009
FAQ

We run the following query:

create table test_map (other_properties map<string, string>);
insert overwrite table test_map
select other_properties from log_table where dateint = 20090825;
The other_properties column in log_table is also of the same type. We are
using such a query to merge small files in our tables.

We get the following array index out of bounds exceptions from all the
reducers of the 2nd map-reduce job and the query failed.

java.io.IOException: java.lang.ArrayIndexOutOfBoundsException: 308
at org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:202)
at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:465)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:413)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 308
at org.apache.hadoop.hive.serde2.lazy.LazyMap.parse(LazyMap.java:172)
at org.apache.hadoop.hive.serde2.lazy.LazyMap.getMap(LazyMap.java:293)
at
org.apache.hadoop.hive.serde2.lazy.objectinspector.LazyMapObjectInspector.ge
tMap(LazyMapObjectInspector.java:100)
at
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serialize(LazySimpleSerDe
.java:453)
at
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serialize(LazySimpleSerDe
.java:385)
at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.jav
a:151)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:464)
at
org.apache.hadoop.hive.ql.exec.ExtractOperator.process(ExtractOperator.java:
42)
at org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:193)
... 3 more


Is this a known issue? Our hive version is trunk version 795559 running on
hadoop 0.20.

Thanks a lot!
Eva.

Search Discussions

  • Zheng Shao at Aug 31, 2009 at 6:37 am
    Yes, see https://issues.apache.org/jira/browse/HIVE-719

    Zheng
    On Sun, Aug 30, 2009 at 9:03 PM, Eva Tse wrote:

    We run the following query:

    create table test_map (other_properties map<string, string>);
    insert overwrite table test_map
    select other_properties from log_table where dateint = 20090825;


    The other_properties column in log_table is also of the same type. We are
    using such a query to merge small files in our tables.

    We get the following array index out of bounds exceptions from all the
    reducers of the 2nd map-reduce job and the query failed.

    java.io.IOException: java.lang.ArrayIndexOutOfBoundsException: 308
    at org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:202)
    at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:465)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:413)
    at org.apache.hadoop.mapred.Child.main(Child.java:170)
    Caused by: java.lang.ArrayIndexOutOfBoundsException: 308
    at org.apache.hadoop.hive.serde2.lazy.LazyMap.parse(LazyMap.java:172)
    at org.apache.hadoop.hive.serde2.lazy.LazyMap.getMap(LazyMap.java:293)
    at
    org.apache.hadoop.hive.serde2.lazy.objectinspector.LazyMapObjectInspector.getMap(LazyMapObjectInspector.java:100)
    at
    org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serialize(LazySimpleSerDe.java:453)
    at
    org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serialize(LazySimpleSerDe.java:385)
    at
    org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:151)
    at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:464)
    at
    org.apache.hadoop.hive.ql.exec.ExtractOperator.process(ExtractOperator.java:42)
    at org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:193)
    ... 3 more


    Is this a known issue? Our hive version is trunk version *795559 *running
    on hadoop 0.20.

    Thanks a lot!
    Eva.


    --
    Yours,
    Zheng
  • Eva Tse at Aug 31, 2009 at 9:58 pm
    Thanks for pointing that out.

    On 8/30/09 11:37 PM, "Zheng Shao" wrote:

    Yes, see https://issues.apache.org/jira/browse/HIVE-719

    Zheng
    On Sun, Aug 30, 2009 at 9:03 PM, Eva Tse wrote:
    We run the following query:
    create table test_map (other_properties map<string, string>);
    insert overwrite table test_map
    select other_properties from log_table where dateint = 20090825;
    The other_properties column in log_table is also of the same type. We are
    using such a query to merge small files in our tables.

    We get the following array index out of bounds exceptions from all the
    reducers of the 2nd map-reduce job and the query failed.

    java.io.IOException: java.lang.ArrayIndexOutOfBoundsException: 308
    at org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:202)
    at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:465)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:413)
    at org.apache.hadoop.mapred.Child.main(Child.java:170)
    Caused by: java.lang.ArrayIndexOutOfBoundsException: 308
    at org.apache.hadoop.hive.serde2.lazy.LazyMap.parse(LazyMap.java:172)
    at org.apache.hadoop.hive.serde2.lazy.LazyMap.getMap(LazyMap.java:293)
    at
    org.apache.hadoop.hive.serde2.lazy.objectinspector.LazyMapObjectInspector.get
    Map(LazyMapObjectInspector.java:100)
    at
    org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serialize(LazySimpleSerDe.
    java:453)
    at
    org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serialize(LazySimpleSerDe.
    java:385)
    at
    org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java
    :151)
    at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:464)
    at
    org.apache.hadoop.hive.ql.exec.ExtractOperator.process(ExtractOperator.java:4
    2)
    at org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:193)
    ... 3 more


    Is this a known issue? Our hive version is trunk version 795559 running on
    hadoop 0.20.

    Thanks a lot!
    Eva.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshive, hadoop
postedAug 31, '09 at 4:07a
activeAug 31, '09 at 9:58p
posts3
users2
websitehive.apache.org

2 users in discussion

Eva Tse: 2 posts Zheng Shao: 1 post

People

Translate

site design / logo © 2022 Grokbase