FAQ
Hello all,

My hive queries are returning back successfully, however if I did in the
map/reduce job that's running as a result of running the query, I see the
following errors in a lot of map tasks that get killed:

java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:110)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:83)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:83)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:328)
at org.apache.hadoop.mapred.Child.main(Child.java:155)
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:207)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:316)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:353)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:338)
at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:59)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:316)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:353)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:338)
at org.apache.hadoop.hive.ql.exec.Operator.initializeOp(Operator.java:324)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:316)
at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:301)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:316)
at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:82)

... 7 more


The reducers also fail at times with similar exceptions/errors.

Is this expected, or do I have something misconfigured? I don't have the
hive distribution on all of my hadoop nodes, just the single node where I
run the queries. I am using 0.4-rc2.

Thanks,
Ryan

Search Discussions

  • Ashish Thusoo at Oct 12, 2009 at 8:45 pm
    This is not expected. Can you send the query and the explain plan for the query.

    Ashish

    ________________________________
    From: Ryan LeCompte
    Sent: Sunday, October 11, 2009 4:02 PM
    To: hive-user@hadoop.apache.org
    Subject: Failures during map/reduce when running a hive query

    Hello all,

    My hive queries are returning back successfully, however if I did in the map/reduce job that's running as a result of running the query, I see the following errors in a lot of map tasks that get killed:


    java.lang.RuntimeException: Map operator initialization failed
    at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:110)
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)

    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:83)
    at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)

    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:83)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:328)
    at org.apache.hadoop.mapred.Child.main(Child.java:155)
    Caused by: java.lang.NullPointerException

    at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:207)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:316)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:353)

    at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:338)
    at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:59)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:316)

    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:353)
    at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:338)
    at org.apache.hadoop.hive.ql.exec.Operator.initializeOp(Operator.java:324)

    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:316)
    at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:301)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:316)

    at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:82)

    ... 7 more

    The reducers also fail at times with similar exceptions/errors.

    Is this expected, or do I have something misconfigured? I don't have the hive distribution on all of my hadoop nodes, just the single node where I run the queries. I am using 0.4-rc2.

    Thanks,
    Ryan
  • Ryan LeCompte at Oct 12, 2009 at 9:04 pm
    Hi Ashish,

    I recently upgraded to Hadoop 0.20.1 and also am now feeding log data in
    compressed gzip form to a hive sequence table, and I haven't seen the errors
    any more.

    Thanks!

    Ryan

    On Mon, Oct 12, 2009 at 4:44 PM, Ashish Thusoo wrote:

    This is not expected. Can you send the query and the explain plan for the
    query.

    Ashish

    ------------------------------
    *From:* Ryan LeCompte
    *Sent:* Sunday, October 11, 2009 4:02 PM
    *To:* hive-user@hadoop.apache.org
    *Subject:* Failures during map/reduce when running a hive query

    Hello all,

    My hive queries are returning back successfully, however if I did in the
    map/reduce job that's running as a result of running the query, I see the
    following errors in a lot of map tasks that get killed:

    java.lang.RuntimeException: Map operator initialization failed
    at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:110)
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)

    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:83)
    at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)

    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:83)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:328)
    at org.apache.hadoop.mapred.Child.main(Child.java:155)
    Caused by: java.lang.NullPointerException

    at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:207)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:316)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:353)

    at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:338)
    at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:59)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:316)

    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:353)
    at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:338)
    at org.apache.hadoop.hive.ql.exec.Operator.initializeOp(Operator.java:324)

    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:316)
    at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:301)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:316)

    at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:82)

    ... 7 more


    The reducers also fail at times with similar exceptions/errors.

    Is this expected, or do I have something misconfigured? I don't have the
    hive distribution on all of my hadoop nodes, just the single node where I
    run the queries. I am using 0.4-rc2.

    Thanks,
    Ryan

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshive, hadoop
postedOct 11, '09 at 11:02p
activeOct 12, '09 at 9:04p
posts3
users2
websitehive.apache.org

2 users in discussion

Ryan LeCompte: 2 posts Ashish Thusoo: 1 post

People

Translate

site design / logo © 2022 Grokbase