Does it get stuck before the creating a Hadoop job or after creating a Hadoop job.

In case it is stuck before creating a hadoop job you can look at Hive.log (wherever you are directing it) for what is taking a long time to setup the job.
In case the Hadoop job has already started you can look at the task attempt logs.

Sometimes if you have a lot of small files or lot of partitions Hive can take long to setup and start map reduce jobs.

-----Original Message-----
From: Thulasi Ram Naidu Peddineni
Sent: Tuesday, September 27, 2011 10:59 AM
To: user@hive.apache.org
Subject: Reg: Map joins in hive

Thulasi Ram P

---------- Forwarded message ----------
From: Thulasi Ram Naidu Peddineni <thulasiram333@gmail.com>
Date: Tue, Sep 27, 2011 at 11:21 PM
Subject: Reg: Map joins in hive
To: dev@hive.apache.org, user@hive.apache.org

I have a huge table x (~150M records and ~5GB) with one partition and another table (~200 records and <10KB). I want to join both these tables and thought MapJoin is perfect optimization for this. However, my job log says..

Total MapReduce jobs = 2

Mapred Local Task Succeeded . Convert the Join into MapJoin

Launching Job 1 out of 2

Number of reduce tasks is set to 0 since there's no reduce operator

and then it is stuck at this point of time for a long time. Can you some explain what could be happening here ?

Thulasi Ram P

Search Discussions

Discussion Posts


Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 2 of 2 | next ›
Discussion Overview
groupuser @
categorieshive, hadoop
postedSep 27, '11 at 5:59p
activeSep 27, '11 at 9:57p



site design / logo © 2021 Grokbase