FAQ
Hi,
I am using CDH3U4 release versions of hadoop, hue, hive, sqoop.
I have three node cluster. All nodes show up in "live nodes" section.

In hue query editor, when i execute a query the job runs successfully
all the time.
The only doubt(problem) i have is, all map and reduce runs on a single node
only.
To be clear, one job runs everything(map and reduce tasks) on node1 and
after that the next job
runs on either the same node1 or node2(but all map and reduce tasks runs on
single node if started).

What i need to know is whether i can choose which nodes to run map and
reduce tasks.
i.e., map tasks on one node and reduce tasks on the other node.

Can anyone give an explanation or a solution to my problem.

Thanks in advance!!!

Search Discussions

  • bc Wong at Jul 19, 2012 at 6:23 am

    On Wed, Jul 18, 2012 at 10:46 PM, Rajesh wrote:

    Hi,
    I am using CDH3U4 release versions of hadoop, hue, hive, sqoop.
    I have three node cluster. All nodes show up in "live nodes" section.

    In hue query editor, when i execute a query the job runs successfully
    all the time.
    The only doubt(problem) i have is, all map and reduce runs on a single
    node only.
    To be clear, one job runs everything(map and reduce tasks) on node1 and
    after that the next job
    runs on either the same node1 or node2(but all map and reduce tasks runs
    on single node if started).

    What i need to know is whether i can choose which nodes to run map and
    reduce tasks.
    i.e., map tasks on one node and reduce tasks on the other node.
    No you can't. MR will assign the tasks according to a combo of data
    locality and slot availability.

    Can anyone give an explanation or a solution to my problem.
    Now many map and reduce tasks did that job generate?

    Cheers,
    bc
  • Rajesh at Jul 19, 2012 at 9:44 am
    Thanks for your reply.

    As you have mentioned, the input split is done based on processing load.
    So, i tried to submit a job with more maps and reduces tasks and it run on
    all slave nodes.

    Thanks again.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouphue-user @
categorieshadoop
postedJul 19, '12 at 5:46a
activeJul 19, '12 at 9:44a
posts3
users2
websitecloudera.com
irc#hadoop

2 users in discussion

Rajesh: 2 posts bc Wong: 1 post

People

Translate

site design / logo © 2023 Grokbase