FAQ
-----
Regards,
Thulasi Ram P




---------- Forwarded message ----------
From: Thulasi Ram Naidu Peddineni <thulasiram333@gmail.com>
Date: Tue, Sep 27, 2011 at 11:21 PM
Subject: Reg: Map joins in hive
To: dev@hive.apache.org, user@hive.apache.org


Hi,
I have a huge table x (~150M records and ~5GB) with one partition
and another table (~200 records and <10KB). I want to join both these
tables and thought MapJoin is perfect optimization for this. However,
my job log says..

Total MapReduce jobs = 2
...
...

Mapred Local Task Succeeded . Convert the Join into MapJoin

Launching Job 1 out of 2

Number of reduce tasks is set to 0 since there's no reduce operator

and then it is stuck at this point of time for a long time. Can you
some explain what could be happening here ?


-----
Thanks,
Thulasi Ram P

Search Discussions

  • Aggarwal, Vaibhav at Sep 27, 2011 at 9:57 pm
    Does it get stuck before the creating a Hadoop job or after creating a Hadoop job.

    In case it is stuck before creating a hadoop job you can look at Hive.log (wherever you are directing it) for what is taking a long time to setup the job.
    In case the Hadoop job has already started you can look at the task attempt logs.

    Sometimes if you have a lot of small files or lot of partitions Hive can take long to setup and start map reduce jobs.

    -----Original Message-----
    From: Thulasi Ram Naidu Peddineni
    Sent: Tuesday, September 27, 2011 10:59 AM
    To: user@hive.apache.org
    Subject: Reg: Map joins in hive

    -----
    Regards,
    Thulasi Ram P




    ---------- Forwarded message ----------
    From: Thulasi Ram Naidu Peddineni <thulasiram333@gmail.com>
    Date: Tue, Sep 27, 2011 at 11:21 PM
    Subject: Reg: Map joins in hive
    To: dev@hive.apache.org, user@hive.apache.org


    Hi,
    I have a huge table x (~150M records and ~5GB) with one partition and another table (~200 records and <10KB). I want to join both these tables and thought MapJoin is perfect optimization for this. However, my job log says..

    Total MapReduce jobs = 2
    ...
    ...

    Mapred Local Task Succeeded . Convert the Join into MapJoin

    Launching Job 1 out of 2

    Number of reduce tasks is set to 0 since there's no reduce operator

    and then it is stuck at this point of time for a long time. Can you some explain what could be happening here ?


    -----
    Thanks,
    Thulasi Ram P

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshive, hadoop
postedSep 27, '11 at 5:59p
activeSep 27, '11 at 9:57p
posts2
users2
websitehive.apache.org

People

Translate

site design / logo © 2021 Grokbase