FAQ
Hi All,

I've just finished setting up a small cluster consisting of the following:

     1 single server for the Cloudera manager
     2 servers for the primary and secondary namenodes
     3 servers for the datanodes.

I used the default templates and allowed CM to determine what each node
would be. For the most part it looks like CM chose the correct items per
member. My question is how to tell if the primary namenode has all the
correct services running? This question also applies to the other nodes as
well. Part of my concern is that the Manager is also a datanode is this
correct? Can I have the Clodera manage be just a manager and not a datanode?

+++++++++++++++++++++++++++++++++++++++++++++
The primary namenode has the the following services running:

master (usherlxsnn1)
regionserver (usherlxsnn1)
datanode (usherlxsnn1)
namenode (usherlxsnn1)
gateway (usherlxsnn1)
hivemetastore (usherlxsnn1)
beeswax_server (usherlxsnn1)
hue_server (usherlxsnn1)
impalad (usherlxsnn1)
statestore (usherlxsnn1)
jobtracker (usherlxsnn1)
tasktracker (usherlxsnn1)
activitymonitor (usherlxsnn1)
alertpublisher (usherlxsnn1)
eventserver (usherlxsnn1)
hostmonitor (usherlxsnn1)
servicemonitor (usherlxsnn1)
oozie_server (usherlxsnn1)
sqoop_server (usherlxsnn1)
server (usherlxsnn1)

The secondary namenode has these services:

regionserver (usherlxsnn2)
datanode (usherlxsnn2)
secondarynamenode (usherlxsnn2)
gateway (usherlxsnn2)
impalad (usherlxsnn2)
tasktracker (usherlxsnn2)

The all the datanodes have the following services:

regionserver (usherlxsdn1)
datanode (usherlxsdn1)
gateway (usherlxsdn1)
impalad (usherlxsdn1)
tasktracker (usherlxsdn1)

The manager has the following services:

regionserver (usherlxsmgr1)
datanode (usherlxsmgr1)
gateway (usherlxsmgr1)
impalad (usherlxsmgr1)
tasktracker (usherlxsmgr1)
+++++++++++++++++++++++++++++++++

Regards
Tim Washburn

Search Discussions

  • Darren Lo at Jul 16, 2013 at 8:18 pm
    (adding back scm-users)

    On Tue, Jul 16, 2013 at 1:16 PM, Darren Lo wrote:

    Hi Tim,

    If you don't have anything running on the cluster yet, then it's probably
    easiest to just stop your cluster, delete everything in the data
    directories, and delete your cluster, then go through the add cluster
    wizard again to set things up how you want.

    If you want to move roles around instead, then for each role that supports
    decommissioning (such as datanode and regionserver), you can select the
    Decommission action one by one. Navigate to the page for that role instance
    and select the Decommission command from the Actions menu. If you don't see
    the action, then the role can't / doesn't need to be Decommissioned. You
    may need to read other documentation for the right way to move roles like
    this, such as the NameNode. It's potentially fairly complex, which is why I
    recommend setting up your cluster from scratch with the right host
    assignments if you can.

    If you want to decommission the entire host (which will not affect
    Cloudera Manager), you can select the host from the Hosts page and select
    the action to decommission everything on the host.

    Gateways don't take any resources. They just help keep track of which
    hosts should get client configuration when you run the Deploy Client
    Configuration command.

    Thanks,
    Darren

    On Tue, Jul 16, 2013 at 12:56 PM, Tim Washburn wrote:

    Hi Darren,

    Thanks for the reply. If I elect to decomm the manager from being a
    datanode, regionserver, gateway etc will still operate as my manager? I ask
    as the particular machine was selected as the manager based on disk size
    and memory. Thinking that the manager's requirements weren't that intensive
    when compared to the Namenodes and datanodes. Yes it is small and a POC to
    wet the appetite of the business - it may grow up to become a real cluster
    some day :--)

    Regards
    Tim Washburn
    Global IT Security
    Bio-Rad Laboratories, Inc
    (510)741-6888



    From: Darren Lo <dlo@cloudera.com>
    To: Tim Washburn <tim_washburn@bio-rad.com>
    Date: 07/16/2013 11:35 AM
    Subject: Re: Cluster Setup with Cloudera Manger on with samll
    cluster - determining what services/roles for wach member of the cluster.
    ------------------------------



    Hi Tim,

    During initial installation you can pick which hosts have which roles by
    clicking Inspect Role Assignments. CM will automatically suggest
    assignments, which is what happened on your cluster.

    If you want to move roles around, you can use the decommission command to
    accomplish this, but there's probably nothing wrong with your current
    cluster setup. If you find that you don't have enough RAM on your namenode
    host to run your roles well, then you can consider moving some roles off of
    that host. Note that I'm assuming this is not a production cluster since
    there are so few nodes.

    Thanks,
    Darren


    On Mon, Jul 15, 2013 at 3:49 PM, Tim Washburn <*tim_washburn@bio-rad.com*<tim_washburn@bio-rad.com>>
    wrote:

    Hi All,

    I've just finished setting up a small cluster consisting of the following:

    1 single server for the Cloudera manager
    2 servers for the primary and secondary namenodes
    3 servers for the datanodes.

    I used the default templates and allowed CM to determine what each node
    would be. For the most part it looks like CM chose the correct items per
    member. My question is how to tell if the primary namenode has all the
    correct services running? This question also applies to the other nodes as
    well. Part of my concern is that the Manager is also a datanode is this
    correct? Can I have the Clodera manage be just a manager and not a datanode?

    +++++++++++++++++++++++++++++++++++++++++++++
    The primary namenode has the the following services running:

    master (usherlxsnn1)
    regionserver (usherlxsnn1)
    datanode (usherlxsnn1)
    namenode (usherlxsnn1)
    gateway (usherlxsnn1)
    hivemetastore (usherlxsnn1)
    beeswax_server (usherlxsnn1)
    hue_server (usherlxsnn1)
    impalad (usherlxsnn1)
    statestore (usherlxsnn1)
    jobtracker (usherlxsnn1)
    tasktracker (usherlxsnn1)
    activitymonitor (usherlxsnn1)
    alertpublisher (usherlxsnn1)
    eventserver (usherlxsnn1)
    hostmonitor (usherlxsnn1)
    servicemonitor (usherlxsnn1)
    oozie_server (usherlxsnn1)
    sqoop_server (usherlxsnn1)
    server (usherlxsnn1)

    The secondary namenode has these services:

    regionserver (usherlxsnn2)
    datanode (usherlxsnn2)
    secondarynamenode (usherlxsnn2)
    gateway (usherlxsnn2)
    impalad (usherlxsnn2)
    tasktracker (usherlxsnn2)

    The all the datanodes have the following services:

    regionserver (usherlxsdn1)
    datanode (usherlxsdn1)
    gateway (usherlxsdn1)
    impalad (usherlxsdn1)
    tasktracker (usherlxsdn1)

    The manager has the following services:

    regionserver (usherlxsmgr1)
    datanode (usherlxsmgr1)
    gateway (usherlxsmgr1)
    impalad (usherlxsmgr1)
    tasktracker (usherlxsmgr1)
    +++++++++++++++++++++++++++++++++

    Regards
    Tim Washburn



    --
    Thanks,
    Darren


    --
    Thanks,
    Darren


    --
    Thanks,
    Darren

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedJul 15, '13 at 10:49p
activeJul 16, '13 at 8:18p
posts2
users2
websitecloudera.com
irc#hadoop

2 users in discussion

Darren Lo: 1 post Tim Washburn: 1 post

People

Translate

site design / logo © 2022 Grokbase