FAQ
Hi,

I'm running Hadoop 0.20.1 and Hive (checked out revision 824063).
Direct MapReduce task succeeds, but Map task created by Hive fails:

hive> select * from pokes where foo>100;
Total MapReduce jobs = 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_200910111626_0001, Tracking URL = http://itstl0016.iil.intel.com:50030/jobdetails.jsp?jobid=job_200910111626_0001
Kill Command = /nfs/iil/disks/rep_tests_gtouret01/hadoop/bin/hadoop job -Dmapred.job.tracker=itstl0016.iil.intel.com:9001 -kill job_200910111626_0001
2009-10-11 04:26:57,844 map = 100%, reduce = 100%
Ended Job = job_200910111626_0001 with errors
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.ExecDriver
From the logs/hadoop-UUUU-jobtracker-XXXX.iil.intel.com.log:
2009-10-11 16:26:56,829 INFO org.apache.hadoop.mapred.JobInProgress: Initializing job_200910111626_0001
2009-10-11 16:26:57,091 INFO org.apache.hadoop.mapred.JobInProgress: Input size for job job_200910111626_0001 = 13. Number of splits = 1
2009-10-11 16:26:57,225 ERROR org.apache.hadoop.mapred.JobTracker: Job initialization failed:
java.lang.IllegalArgumentException: Network location name contains /: /IDC1-DC201/WE/34 (I've had the same issue with the /default_rack)
at org.apache.hadoop.net.NodeBase.set(NodeBase.java:75)
at org.apache.hadoop.net.NodeBase.(JobTracker.java:2390)
at org.apache.hadoop.mapred.JobTracker.resolveAndAddToTopology(JobTracker.java:2384)
at org.apache.hadoop.mapred.JobInProgress.createCache(JobInProgress.java:349)
at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:450)
at org.apache.hadoop.mapred.JobTracker.initJob(JobTracker.java:3147)
at org.apache.hadoop.mapred.EagerTaskInitializationListener$InitJob.run(EagerTaskInitializationListener.java:79)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)

2009-10-11 16:26:57,225 INFO org.apache.hadoop.mapred.JobTracker: Failing job job_200910111626_0001
2009-10-11 16:26:57,866 INFO org.apache.hadoop.mapred.JobTracker: Killing job job_200910111626_0001

Any suggestion?
I saw patches in https://issues.apache.org/jira/browse/HADOOP-5759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712524#action_12712524, but I can't apply all of them cleanly to my Hadoop sources...

Thanks,
Gregory
---------------------------------------------------------------------
Intel Israel (74) Limited

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Search Discussions

  • Touretsky, Gregory at Oct 12, 2009 at 10:03 am
    Ok, the patch below actually works. Re-built Hadoop cluster and everything works now.
    Now I have to understand how to force Hive to run >1 mapper for complicated query on the large table...

    From: Touretsky, Gregory
    Sent: Sunday, October 11, 2009 4:39 PM
    To: common-user@hadoop.apache.org
    Cc: Touretsky, Gregory
    Subject: Hive and MapReduce

    Hi,

    I'm running Hadoop 0.20.1 and Hive (checked out revision 824063).
    Direct MapReduce task succeeds, but Map task created by Hive fails:

    hive> select * from pokes where foo>100;
    Total MapReduce jobs = 1
    Number of reduce tasks is set to 0 since there's no reduce operator
    Starting Job = job_200910111626_0001, Tracking URL = http://itstl0016.iil.intel.com:50030/jobdetails.jsp?jobid=job_200910111626_0001
    Kill Command = /nfs/iil/disks/rep_tests_gtouret01/hadoop/bin/hadoop job -Dmapred.job.tracker=itstl0016.iil.intel.com:9001 -kill job_200910111626_0001
    2009-10-11 04:26:57,844 map = 100%, reduce = 100%
    Ended Job = job_200910111626_0001 with errors
    FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.ExecDriver
    From the logs/hadoop-UUUU-jobtracker-XXXX.iil.intel.com.log:
    2009-10-11 16:26:56,829 INFO org.apache.hadoop.mapred.JobInProgress: Initializing job_200910111626_0001
    2009-10-11 16:26:57,091 INFO org.apache.hadoop.mapred.JobInProgress: Input size for job job_200910111626_0001 = 13. Number of splits = 1
    2009-10-11 16:26:57,225 ERROR org.apache.hadoop.mapred.JobTracker: Job initialization failed:
    java.lang.IllegalArgumentException: Network location name contains /: /IDC1-DC201/WE/34 (I've had the same issue with the /default_rack)
    at org.apache.hadoop.net.NodeBase.set(NodeBase.java:75)
    at org.apache.hadoop.net.NodeBase.(JobTracker.java:2390)
    at org.apache.hadoop.mapred.JobTracker.resolveAndAddToTopology(JobTracker.java:2384)
    at org.apache.hadoop.mapred.JobInProgress.createCache(JobInProgress.java:349)
    at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:450)
    at org.apache.hadoop.mapred.JobTracker.initJob(JobTracker.java:3147)
    at org.apache.hadoop.mapred.EagerTaskInitializationListener$InitJob.run(EagerTaskInitializationListener.java:79)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)

    2009-10-11 16:26:57,225 INFO org.apache.hadoop.mapred.JobTracker: Failing job job_200910111626_0001
    2009-10-11 16:26:57,866 INFO org.apache.hadoop.mapred.JobTracker: Killing job job_200910111626_0001

    Any suggestion?
    I saw patches in https://issues.apache.org/jira/browse/HADOOP-5759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712524#action_12712524, but I can't apply all of them cleanly to my Hadoop sources...

    Thanks,
    Gregory
    ---------------------------------------------------------------------
    Intel Israel (74) Limited

    This e-mail and any attachments may contain confidential material for
    the sole use of the intended recipient(s). Any review or distribution
    by others is strictly prohibited. If you are not the intended
    recipient, please contact the sender and delete all copies.
  • Ashish Thusoo at Oct 12, 2009 at 6:04 pm
    adding hive-user and hive-dev lists.
    And removing the common mailing list..

    Can you elaborate a bit on the datasize. By default Hive should just be relying on hadoop to give you the number of mappers depending on the number of splits you have in your data.

    Ashish

    -----Original Message-----
    From: Touretsky, Gregory
    Sent: Monday, October 12, 2009 3:02 AM
    To: Touretsky, Gregory; common-user@hadoop.apache.org
    Subject: RE: Hive and MapReduce

    Ok, the patch below actually works. Re-built Hadoop cluster and everything works now.
    Now I have to understand how to force Hive to run >1 mapper for complicated query on the large table...

    From: Touretsky, Gregory
    Sent: Sunday, October 11, 2009 4:39 PM
    To: common-user@hadoop.apache.org
    Cc: Touretsky, Gregory
    Subject: Hive and MapReduce

    Hi,

    I'm running Hadoop 0.20.1 and Hive (checked out revision 824063).
    Direct MapReduce task succeeds, but Map task created by Hive fails:

    hive> select * from pokes where foo>100;
    Total MapReduce jobs = 1
    Number of reduce tasks is set to 0 since there's no reduce operator Starting Job = job_200910111626_0001, Tracking URL = http://itstl0016.iil.intel.com:50030/jobdetails.jsp?jobid=job_200910111626_0001
    Kill Command = /nfs/iil/disks/rep_tests_gtouret01/hadoop/bin/hadoop job -Dmapred.job.tracker=itstl0016.iil.intel.com:9001 -kill job_200910111626_0001
    2009-10-11 04:26:57,844 map = 100%, reduce = 100% Ended Job = job_200910111626_0001 with errors
    FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.ExecDriver
    From the logs/hadoop-UUUU-jobtracker-XXXX.iil.intel.com.log:
    2009-10-11 16:26:56,829 INFO org.apache.hadoop.mapred.JobInProgress: Initializing job_200910111626_0001
    2009-10-11 16:26:57,091 INFO org.apache.hadoop.mapred.JobInProgress: Input size for job job_200910111626_0001 = 13. Number of splits = 1
    2009-10-11 16:26:57,225 ERROR org.apache.hadoop.mapred.JobTracker: Job initialization failed:
    java.lang.IllegalArgumentException: Network location name contains /: /IDC1-DC201/WE/34 (I've had the same issue with the /default_rack)
    at org.apache.hadoop.net.NodeBase.set(NodeBase.java:75)
    at org.apache.hadoop.net.NodeBase.(JobTracker.java:2390)
    at org.apache.hadoop.mapred.JobTracker.resolveAndAddToTopology(JobTracker.java:2384)
    at org.apache.hadoop.mapred.JobInProgress.createCache(JobInProgress.java:349)
    at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:450)
    at org.apache.hadoop.mapred.JobTracker.initJob(JobTracker.java:3147)
    at org.apache.hadoop.mapred.EagerTaskInitializationListener$InitJob.run(EagerTaskInitializationListener.java:79)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)

    2009-10-11 16:26:57,225 INFO org.apache.hadoop.mapred.JobTracker: Failing job job_200910111626_0001
    2009-10-11 16:26:57,866 INFO org.apache.hadoop.mapred.JobTracker: Killing job job_200910111626_0001

    Any suggestion?
    I saw patches in https://issues.apache.org/jira/browse/HADOOP-5759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712524#action_12712524, but I can't apply all of them cleanly to my Hadoop sources...

    Thanks,
    Gregory
    ---------------------------------------------------------------------
    Intel Israel (74) Limited

    This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedOct 11, '09 at 2:41p
activeOct 12, '09 at 6:04p
posts3
users2
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase