FAQ
Dear all,
I have problem about one node was failed that I would like to remove dead
node from Cloudera Manager 4.6. But I can not delete it from them. Could
you please help me to advise that how to Remove dead node from Cloudera
Manager 4.6.?
Regards,
Voravit

Search Discussions

  • Adam Smieszny at Jul 9, 2013 at 1:53 pm
    If the node is dead, it will not respond to commands such as
    "decommission," which does present something of a challenge.
    However, you should be able to go through each service, for the dead host,
    and "stop" and "delete" the instances of the service from that host.
    Then, once all services that were running on that host have been removed
    (and it should show 0 roles under the "Hosts" tab), you should be able to
    delete the host from the hosts tab.

    Thanks,
    Adam

    On Mon, Jul 8, 2013 at 11:25 PM, ชาวนา ชาวไร่ wrote:

    Dear all,
    I have problem about one node was failed that I would like to remove dead
    node from Cloudera Manager 4.6. But I can not delete it from them. Could
    you please help me to advise that how to Remove dead node from Cloudera
    Manager 4.6.?
    Regards,
    Voravit


    --
    Adam Smieszny
    Cloudera | Systems Engineer | http://www.linkedin.com/in/adamsmieszny
    917.830.4156
  • Omar Alrubaiyan at Jul 9, 2013 at 3:46 pm
    Hi,

    You should be able to do this in one step by going to the hosts page,
    selecting the node and clicking the "Remove from Cluster" action. In the
    confirmation dialogue, deselect "Decommission Roles" as that won't work if
    the node is dead.

    On Tue, Jul 9, 2013 at 6:52 AM, Adam Smieszny wrote:

    If the node is dead, it will not respond to commands such as
    "decommission," which does present something of a challenge.
    However, you should be able to go through each service, for the dead host,
    and "stop" and "delete" the instances of the service from that host.
    Then, once all services that were running on that host have been removed
    (and it should show 0 roles under the "Hosts" tab), you should be able to
    delete the host from the hosts tab.

    Thanks,
    Adam

    On Mon, Jul 8, 2013 at 11:25 PM, ชาวนา ชาวไร่ wrote:

    Dear all,
    I have problem about one node was failed that I would like to remove dead
    node from Cloudera Manager 4.6. But I can not delete it from them. Could
    you please help me to advise that how to Remove dead node from Cloudera
    Manager 4.6.?
    Regards,
    Voravit


    --
    Adam Smieszny
    Cloudera | Systems Engineer | http://www.linkedin.com/in/adamsmieszny
    917.830.4156
  • PM at Jul 9, 2013 at 8:09 pm
    Hi All,

    I just would like give a bit more detail on the issue Voravit and me are
    facing:
    We have a Cloudera Manager 4.6 running CDH4. We are in the testing process.
    We have 20 datanodes and 2 namenodes HA and 2 jobtrackers HA.
    During the deployment of HDFS, the 20 datanodes where deployed properly
    except one node stay with the state "Starting" (either when we placed the
    node offline, the console still sees the service as "Starting").
    When we try to stop the service on that node, it answers:

    Selected roles are already stopped.

    When we try to start the service on that node, it answers:

    Service is already started

    When we try to decommission that node in HDFS, it answers:

    Decommissioning requires the NameNode roles of service HDFS to be running and the DataNode roles to be running or stopped. Additionally, decommissioning may not be performed while other DataNode roles are being decommissioned or recommissioned.

    When we try to delete that node in HDFS, it answers:

    *The following role(s) need to be stopped before they can be deleted.*

        - datanode (name_of_the_node): Starting DataNode on name_of_the_node.fqdn

    We reached now a state where we would like either, make it work
    properly, delete the node from HDFS or delete it from the cluster and
    re-apply it after.
    We found the following documentation to force delete of a "Dead Node":
    https://ccp.cloudera.com/display/FREE373/Known+Issues+and+Work+Arounds+in+Cloudera+Manager+Free+Edition+3.7
    But it applies to older version. We tried it anyway (secition "— If a host
    has become disconnected from the Cloudera Server, it cannot be deleted
    through the Cloudera Manager Admin Console.") but when trying the command
    below, we have the following error:
    $ sudo /etc/init.d/cloudera-scm-server-db stop
    waiting for server to shut
    down............................................................... failed
    pg_ctl: server does not shut down

    So we can't continue the process to delete that node.
    We are now reaching a state where we feel this is linked to a bug of
    Cloudera Manager and we would like be able to solve this in order to bring
    this system in production.

    Any help will be welcome in order to solve this issue.

    Thank you


    PM


    Le mardi 9 juillet 2013 10:25:43 UTC+7, ชาวนา ชาวไร่ a écrit :
    Dear all,
    I have problem about one node was failed that I would like to remove dead
    node from Cloudera Manager 4.6. But I can not delete it from them. Could
    you please help me to advise that how to Remove dead node from Cloudera
    Manager 4.6.?
    Regards,
    Voravit

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedJul 9, '13 at 3:25a
activeJul 9, '13 at 8:09p
posts4
users4
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase