FAQ
Hello,

I am trying to run a "insert overwrite into ... select * from" query. The hadoop job submitted by Hive finishes successfully, but there seems to be an error while collecting results. Please see below for exception stack.

The interesting point is that the error is NoSuchMethodError. The environment has Hive 0.7.1 and Hadoop 0.20.2+320 (CDH3) along with Sun Java 6. It has been running fine, including executing other INSERT OVERWRITE INTO queries. Hive seems to be unable to find the method provided in the runtime.


Any idea?

Thanks,

Juwon




11/09/30 16:39:38 INFO io.CombineHiveInputFormat: CombineHiveInputSplit creating pool for hdfs://coral-m01:54310/tmp/hive-analytics/hive_2011-09-30_16-38-36_053_6654126894793555908/-ext-10002; using filter path hdfs://coral-m01:54310/tmp/hive-analytics/hive_2011-09-30_16-38-36_053_6654126894793555908/-ext-10002
11/09/30 16:39:38 DEBUG ipc.Client: IPC Client (47) connection to coral-m01/10.100.81.1:54310 from analytics sending #1273
11/09/30 16:39:38 DEBUG ipc.Client: IPC Client (47) connection to coral-m01/10.100.81.1:54310 from analytics got value #1273
11/09/30 16:39:38 DEBUG ipc.RPC: Call: getFileInfo 1
11/09/30 16:39:38 DEBUG ipc.Client: IPC Client (47) connection to coral-m01/10.100.81.1:54310 from analytics sending #1274
11/09/30 16:39:38 DEBUG ipc.Client: IPC Client (47) connection to coral-m01/10.100.81.1:54310 from analytics got value #1274
11/09/30 16:39:38 DEBUG ipc.RPC: Call: delete 1
java.lang.NoSuchMethodError: org.apache.hadoop.mapred.lib.CombineFileInputFormat.createPool(Lorg/apache/hadoop/mapred/JobConf;[Lorg/apache/hadoop/fs/PathFilter;)V
at org.apache.hadoop.hive.shims.Hadoop20Shims$CombineFileInputFormatShim.createPool(Hadoop20Shims.java:325)
at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:365)
at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:851)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:822)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:771)
at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:657)
at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:123)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:130)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1063)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:900)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:748)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
FAILED: Execution Error, return code -101 from org.apache.hadoop.hive.ql.exec.MapRedTask
11/09/30 16:39:38 ERROR ql.Driver: FAILED: Execution Error, return code -101 from org.apache.hadoop.hive.ql.exec.MapRedTask
11/09/30 16:39:38 DEBUG ipc.Client: IPC Client (47) connection to coral-m01/10.100.81.1:54310 from analytics sending #1275
11/09/30 16:39:38 DEBUG ipc.Client: IPC Client (47) connection to coral-m01/10.100.81.1:54310 from analytics got value #1275
11/09/30 16:39:38 DEBUG ipc.RPC: Call: delete 35
hive> 11/09/30 16:39:47 DEBUG ipc.Client: IPC Client (47) connection to coral-m01/10.100.81.1:9001 from analytics: closed

Search Discussions

  • Juwon Lee at Oct 3, 2011 at 9:17 pm
    More information on this. There are 2 different kinds of data having the same schema. One of them is failing with the exception below while the other works fine. The only difference between the two data sets is volume but I guess it is not causing the issue.

    Thanks,

    Juwon

    From: Juwon Lee
    Sent: Friday, September 30, 2011 5:58 PM
    To: user@hive.apache.org
    Subject: Hive NoSuchMethodError

    Hello,

    I am trying to run a "insert overwrite into ... select * from" query. The hadoop job submitted by Hive finishes successfully, but there seems to be an error while collecting results. Please see below for exception stack.

    The interesting point is that the error is NoSuchMethodError. The environment has Hive 0.7.1 and Hadoop 0.20.2+320 (CDH3) along with Sun Java 6. It has been running fine, including executing other INSERT OVERWRITE INTO queries. Hive seems to be unable to find the method provided in the runtime.


    Any idea?

    Thanks,

    Juwon




    11/09/30 16:39:38 INFO io.CombineHiveInputFormat: CombineHiveInputSplit creating pool for hdfs://coral-m01:54310/tmp/hive-analytics/hive_2011-09-30_16-38-36_053_6654126894793555908/-ext-10002; using filter path hdfs://coral-m01:54310/tmp/hive-analytics/hive_2011-09-30_16-38-36_053_6654126894793555908/-ext-10002
    11/09/30 16:39:38 DEBUG ipc.Client: IPC Client (47) connection to coral-m01/10.100.81.1:54310 from analytics sending #1273
    11/09/30 16:39:38 DEBUG ipc.Client: IPC Client (47) connection to coral-m01/10.100.81.1:54310 from analytics got value #1273
    11/09/30 16:39:38 DEBUG ipc.RPC: Call: getFileInfo 1
    11/09/30 16:39:38 DEBUG ipc.Client: IPC Client (47) connection to coral-m01/10.100.81.1:54310 from analytics sending #1274
    11/09/30 16:39:38 DEBUG ipc.Client: IPC Client (47) connection to coral-m01/10.100.81.1:54310 from analytics got value #1274
    11/09/30 16:39:38 DEBUG ipc.RPC: Call: delete 1
    java.lang.NoSuchMethodError: org.apache.hadoop.mapred.lib.CombineFileInputFormat.createPool(Lorg/apache/hadoop/mapred/JobConf;[Lorg/apache/hadoop/fs/PathFilter;)V
    at org.apache.hadoop.hive.shims.Hadoop20Shims$CombineFileInputFormatShim.createPool(Hadoop20Shims.java:325)
    at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:365)
    at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:851)
    at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:822)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:771)
    at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:657)
    at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:123)
    at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:130)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
    at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1063)
    at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:900)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:748)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
    FAILED: Execution Error, return code -101 from org.apache.hadoop.hive.ql.exec.MapRedTask
    11/09/30 16:39:38 ERROR ql.Driver: FAILED: Execution Error, return code -101 from org.apache.hadoop.hive.ql.exec.MapRedTask
    11/09/30 16:39:38 DEBUG ipc.Client: IPC Client (47) connection to coral-m01/10.100.81.1:54310 from analytics sending #1275
    11/09/30 16:39:38 DEBUG ipc.Client: IPC Client (47) connection to coral-m01/10.100.81.1:54310 from analytics got value #1275
    11/09/30 16:39:38 DEBUG ipc.RPC: Call: delete 35
    hive> 11/09/30 16:39:47 DEBUG ipc.Client: IPC Client (47) connection to coral-m01/10.100.81.1:9001 from analytics: closed

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshive, hadoop
postedSep 30, '11 at 10:58p
activeOct 3, '11 at 9:17p
posts2
users1
websitehive.apache.org

1 user in discussion

Juwon Lee: 2 posts

People

Translate

site design / logo © 2021 Grokbase