FAQ
Hello Cloudera folks & users,



I have just used the Cloudera Manager to install the Standard version of
CDH4 on a new Linux cluster of 18 nodes (a namenode, a jobtracker node, and
16 data nodes).



I got a success msg at the end of “Installation Path A – Automated
Installation by Cloudera Manager”.



That is, I got this:



%%%

Starting your cluster services.

Completed 6 of 6 steps.



Waiting for ZooKeeper Service to initialize

Finished waiting



Checking if the name directories of the NameNode are empty. Formatting HDFS
only if empty.

Successfully formatted NameNode.



Creating Hive Metastore Database

Created Hive Metastore Database.



Creating Hive Metastore Database Tables

Created Hive Metastore Database Tables successfully.



Creating Oozie database

Oozie database created successfully.



Installing Oozie ShareLib in HDFS

Command Install Oozie ShareLib is not currently available for execution.

%%



And then I clicked “Continue” on that install web page, and got a final web
page with this msg:



“Congratulations! The Hadoop services are installed, configured, and
running on your cluster.”



%%


However, when I then access the Cloudera Manager Admin Console at port
7180 on the gateway/head node on the cluster (where I ran
Cloudera-manager-installer.bin, but which was left out of the list of
Hadoop-configured nodes), I am told that Host Monitor is not running, and
I see no charts. I tried to do a “Start” on the cluster in the left panel,
thinking that for some reason the cluster services (and the Host Monitor)
did not start automatically, But that failed. In the popup box I was told
that mapreduce1 service could not be started, and that I did not have
enough job trackers. In the install process I had specified one job tracker
node for the cluster. That choice appeared to be accepted during the
install process - and should have been enough, of course.

The precise msg was

Service did not start successfully; not all of the required roles started:
Service doesn't have sufficient running JobTrackers.

I tried another "Start" command after the first failed, and got this
different error msg:

Command 'ZkStartPreservingDatastore' failed for service 'zookeeper1'



Unfortunately, I don't know what is causing the problem(s), and I need to
some help to finish this install. I have the Cloudera Manager Installation
Guide, but so far I have not found any answer in that. I would very much
appreciate some guidance.



    - Ron Taylor



PS – yep, still working right now (all the scientists here are contractors,
not direct fed employees, and we haven’t been furloughed yet)



Ronald Taylor, Ph.D.

Computational Biology & Bioinformatics Group

Pacific Northwest National Laboratory (U.S. Dept of Energy/Battelle)

Richland, WA 99352

phone: (509) 372-6568

email: ronald.taylor@pnnl.gov

web page: http://www.sysbio.org/resources/staff/taylor.stm



To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.

Search Discussions

  • Vikram Srivastava at Oct 9, 2013 at 12:35 am
    Can you goto the host with ZooKeeper Server(s) and paste the logs from
    /var/log/zookeeper on them?

    On Tue, Oct 8, 2013 at 4:15 PM, wrote:

    Hello Cloudera folks & users,******

    ** **

    I have just used the Cloudera Manager to install the Standard version of
    CDH4 on a new Linux cluster of 18 nodes (a namenode, a jobtracker node, and
    16 data nodes). ****

    ** **

    I got a success msg at the end of “Installation Path A – Automated
    Installation by Cloudera Manager”.****

    ** **

    That is, I got this:****

    ** **

    %%%****

    Starting your cluster services.****

    Completed 6 of 6 steps.****

    ** **

    Waiting for ZooKeeper Service to initialize****

    Finished waiting****

    ** **

    Checking if the name directories of the NameNode are empty. Formatting
    HDFS only if empty.****

    Successfully formatted NameNode.****

    ** **

    Creating Hive Metastore Database****

    Created Hive Metastore Database.****

    ****

    Creating Hive Metastore Database Tables****

    Created Hive Metastore Database Tables successfully.****

    ****

    Creating Oozie database****

    Oozie database created successfully.****

    ****

    Installing Oozie ShareLib in HDFS****

    Command Install Oozie ShareLib is not currently available for execution.**
    **

    %%****

    ** **

    And then I clicked “Continue” on that install web page, and got a final
    web page with this msg:****

    ** **

    “Congratulations! The Hadoop services are installed, configured, and
    running on your cluster.”****

    ** **

    %%****

    ** **
    However, when I then access the Cloudera Manager Admin Console at port
    7180 on the gateway/head node on the cluster (where I ran
    Cloudera-manager-installer.bin, but which was left out of the list of
    Hadoop-configured nodes), I am told that Host Monitor is not running,
    and I see no charts. I tried to do a “Start” on the cluster in the left
    panel, thinking that for some reason the cluster services (and the Host
    Monitor) did not start automatically, But that failed. In the popup box I
    was told that mapreduce1 service could not be started, and that I did not
    have enough job trackers. In the install process I had specified one job
    tracker node for the cluster. That choice appeared to be accepted during
    the install process - and should have been enough, of course.

    The precise msg was

    Service did not start successfully; not all of the required roles started:
    Service doesn't have sufficient running JobTrackers.

    I tried another "Start" command after the first failed, and got this
    different error msg:

    Command 'ZkStartPreservingDatastore' failed for service 'zookeeper1'

    ** **

    Unfortunately, I don't know what is causing the problem(s), and I need to
    some help to finish this install. I have the Cloudera Manager
    Installation Guide, but so far I have not found any answer in that. I would
    very much appreciate some guidance.****

    ****

    - Ron Taylor****

    ** **

    PS – yep, still working right now (all the scientists here are
    contractors, not direct fed employees, and we haven’t been furloughed yet)
    ****

    ** **

    Ronald Taylor, Ph.D.****

    Computational Biology & Bioinformatics Group****

    Pacific Northwest National Laboratory (U.S. Dept of Energy/Battelle)****

    Richland, WA 99352****

    phone: (509) 372-6568****

    email: ronald.taylor@pnnl.gov****

    web page: http://www.sysbio.org/resources/staff/taylor.stm****

    ** **

    To unsubscribe from this group and stop receiving emails from it, send an
    email to scm-users+unsubscribe@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.
  • Ronald Taylor at Oct 9, 2013 at 12:49 am
    Hi Vikram,

    The entire file is 48K, so I don't want to paste the entire thing. I did a
    head and tail below. If you can tell me what to search for, I can do that.
    Or happy to send the entire file as an attachment, if that is OK.

    Anyway, here are the start and end of the one zookeeper log file. Does that
    provide any hints as to the problem?

    Ron

    [rtaylor@bigdatann zookeeper]$ pwd
    /var/log/zookeeper

    [rtaylor@bigdatann zookeeper]$ ls -l
    total 52
    -rw-r--r-- 1 zookeeper 390 48815 Oct 8 14:38
    zookeeper-cmf-zookeeper1-SERVER-bigdatann.local.log
    [rtaylor@bigdatann zookeeper]$


    [rtaylor@bigdatann zookeeper]$ head
    zookeeper-cmf-zookeeper1-SERVER-bigdatann.local.log
    2013-10-07 15:47:07,686 INFO
    org.apache.zookeeper.server.quorum.QuorumPeerConfig: Reading configuration
    from: /var/run/cloudera-scm-agent/process/26-zookeeper-server/zoo.cfg
    2013-10-07 15:47:07,701 ERROR
    org.apache.zookeeper.server.quorum.QuorumPeerConfig: Invalid configuration,
    only one server specified (ignoring)
    2013-10-07 15:47:07,702 INFO
    org.apache.zookeeper.server.DatadirCleanupManager:
    autopurge.snapRetainCount set to 5
    2013-10-07 15:47:07,703 INFO
    org.apache.zookeeper.server.DatadirCleanupManager: autopurge.purgeInterval
    set to 24
    2013-10-07 15:47:07,703 WARN
    org.apache.zookeeper.server.quorum.QuorumPeerMain: Either no config or no
    quorum defined in config, running in standalone mode
    2013-10-07 15:47:07,703 INFO
    org.apache.zookeeper.server.DatadirCleanupManager: Purge task started.
    2013-10-07 15:47:07,714 INFO
    org.apache.zookeeper.server.DatadirCleanupManager: Purge task completed.
    2013-10-07 15:47:07,721 INFO
    org.apache.zookeeper.server.quorum.QuorumPeerConfig: Reading configuration
    from: /var/run/cloudera-scm-agent/process/26-zookeeper-server/zoo.cfg
    2013-10-07 15:47:07,722 ERROR
    org.apache.zookeeper.server.quorum.QuorumPeerConfig: Invalid configuration,
    only one server specified (ignoring)
    2013-10-07 15:47:07,722 INFO
    org.apache.zookeeper.server.ZooKeeperServerMain: Starting server
    [rtaylor@bigdatann zookeeper]$


    [rtaylor@bigdatann zookeeper]$ tail
    zookeeper-cmf-zookeeper1-SERVER-bigdatann.local.log
    2013-10-08 14:37:08,909 INFO org.apache.zookeeper.server.ZooKeeperServer:
    Client attempting to establish new session at /172.16.115.14:39187
    2013-10-08 14:37:08,910 INFO org.apache.zookeeper.server.ZooKeeperServer:
    Established session 0x141951a411d000e with negotiated timeout 60000 for
    client /172.16.115.14:39187
    2013-10-08 14:37:08,918 INFO
    org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted socket
    connection from /172.16.115.11:33231
    2013-10-08 14:37:08,921 INFO org.apache.zookeeper.server.ZooKeeperServer:
    Client attempting to establish new session at /172.16.115.11:33231
    2013-10-08 14:37:08,922 INFO org.apache.zookeeper.server.ZooKeeperServer:
    Established session 0x141951a411d000f with negotiated timeout 60000 for
    client /172.16.115.11:33231
    2013-10-08 14:37:08,961 INFO
    org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted socket
    connection from /172.16.115.16:44441
    2013-10-08 14:37:08,964 INFO org.apache.zookeeper.server.ZooKeeperServer:
    Client attempting to establish new session at /172.16.115.16:44441
    2013-10-08 14:37:08,965 INFO org.apache.zookeeper.server.ZooKeeperServer:
    Established session 0x141951a411d0010 with negotiated timeout 60000 for
    client /172.16.115.16:44441
    2013-10-08 14:38:04,736 INFO
    org.apache.zookeeper.server.PrepRequestProcessor: Processed session
    termination for sessionid: 0x141951a411d0003
    2013-10-08 14:38:04,737 INFO org.apache.zookeeper.server.NIOServerCnxn:
    Closed socket connection for client /172.16.115.18:55763 which had
    sessionid 0x141951a411d0003
    [rtaylor@bigdatann zookeeper]$
    [rtaylor@bigdatann zookeeper]$
    [rtaylor@bigdatann zookeeper]$





    On Tue, Oct 8, 2013 at 5:35 PM, Vikram Srivastava wrote:

    Can you goto the host with ZooKeeper Server(s) and paste the logs from
    /var/log/zookeeper on them?

    On Tue, Oct 8, 2013 at 4:15 PM, wrote:

    Hello Cloudera folks & users,******

    ** **

    I have just used the Cloudera Manager to install the Standard version of
    CDH4 on a new Linux cluster of 18 nodes (a namenode, a jobtracker node, and
    16 data nodes). ****

    ** **

    I got a success msg at the end of “Installation Path A – Automated
    Installation by Cloudera Manager”.****

    ** **

    That is, I got this:****

    ** **

    %%%****

    Starting your cluster services.****

    Completed 6 of 6 steps.****

    ** **

    Waiting for ZooKeeper Service to initialize****

    Finished waiting****

    ** **

    Checking if the name directories of the NameNode are empty. Formatting
    HDFS only if empty.****

    Successfully formatted NameNode.****

    ** **

    Creating Hive Metastore Database****

    Created Hive Metastore Database.****

    ****

    Creating Hive Metastore Database Tables****

    Created Hive Metastore Database Tables successfully.****

    ****

    Creating Oozie database****

    Oozie database created successfully.****

    ****

    Installing Oozie ShareLib in HDFS****

    Command Install Oozie ShareLib is not currently available for execution.*
    ***

    %%****

    ** **

    And then I clicked “Continue” on that install web page, and got a final
    web page with this msg:****

    ** **

    “Congratulations! The Hadoop services are installed, configured, and
    running on your cluster.”****

    ** **

    %%****

    ** **
    However, when I then access the Cloudera Manager Admin Console at port
    7180 on the gateway/head node on the cluster (where I ran
    Cloudera-manager-installer.bin, but which was left out of the list of
    Hadoop-configured nodes), I am told that Host Monitor is not running,
    and I see no charts. I tried to do a “Start” on the cluster in the left
    panel, thinking that for some reason the cluster services (and the Host
    Monitor) did not start automatically, But that failed. In the popup box I
    was told that mapreduce1 service could not be started, and that I did not
    have enough job trackers. In the install process I had specified one job
    tracker node for the cluster. That choice appeared to be accepted during
    the install process - and should have been enough, of course.

    The precise msg was

    Service did not start successfully; not all of the required roles
    started: Service doesn't have sufficient running JobTrackers.

    I tried another "Start" command after the first failed, and got this
    different error msg:

    Command 'ZkStartPreservingDatastore' failed for service 'zookeeper1'

    ** **

    Unfortunately, I don't know what is causing the problem(s), and I need to
    some help to finish this install. I have the Cloudera Manager
    Installation Guide, but so far I have not found any answer in that. I would
    very much appreciate some guidance.****

    ****

    - Ron Taylor****

    ** **

    PS – yep, still working right now (all the scientists here are
    contractors, not direct fed employees, and we haven’t been furloughed yet)
    ****

    ** **

    Ronald Taylor, Ph.D.****

    Computational Biology & Bioinformatics Group****

    Pacific Northwest National Laboratory (U.S. Dept of Energy/Battelle)****

    Richland, WA 99352****

    phone: (509) 372-6568****

    email: ronald.taylor@pnnl.gov****

    web page: http://www.sysbio.org/resources/staff/taylor.stm****

    ** **

    To unsubscribe from this group and stop receiving emails from it, send an
    email to scm-users+unsubscribe@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.
  • Vikram Srivastava at Oct 9, 2013 at 12:55 am
    From the tail, it looks like ZooKeeper service is running, since the head
    and tail are about 1 day apart. How did you run into the situation where
    ZooKeeper "Start" is failing? Did you create a new ZooKeeper server and
    deleted the previous one?

    On Tue, Oct 8, 2013 at 5:49 PM, Ronald Taylor wrote:

    Hi Vikram,

    The entire file is 48K, so I don't want to paste the entire thing. I did a
    head and tail below. If you can tell me what to search for, I can do that.
    Or happy to send the entire file as an attachment, if that is OK.

    Anyway, here are the start and end of the one zookeeper log file. Does
    that provide any hints as to the problem?

    Ron

    [rtaylor@bigdatann zookeeper]$ pwd
    /var/log/zookeeper

    [rtaylor@bigdatann zookeeper]$ ls -l
    total 52
    -rw-r--r-- 1 zookeeper 390 48815 Oct 8 14:38
    zookeeper-cmf-zookeeper1-SERVER-bigdatann.local.log
    [rtaylor@bigdatann zookeeper]$


    [rtaylor@bigdatann zookeeper]$ head
    zookeeper-cmf-zookeeper1-SERVER-bigdatann.local.log
    2013-10-07 15:47:07,686 INFO
    org.apache.zookeeper.server.quorum.QuorumPeerConfig: Reading configuration
    from: /var/run/cloudera-scm-agent/process/26-zookeeper-server/zoo.cfg
    2013-10-07 15:47:07,701 ERROR
    org.apache.zookeeper.server.quorum.QuorumPeerConfig: Invalid configuration,
    only one server specified (ignoring)
    2013-10-07 15:47:07,702 INFO
    org.apache.zookeeper.server.DatadirCleanupManager:
    autopurge.snapRetainCount set to 5
    2013-10-07 15:47:07,703 INFO
    org.apache.zookeeper.server.DatadirCleanupManager: autopurge.purgeInterval
    set to 24
    2013-10-07 15:47:07,703 WARN
    org.apache.zookeeper.server.quorum.QuorumPeerMain: Either no config or no
    quorum defined in config, running in standalone mode
    2013-10-07 15:47:07,703 INFO
    org.apache.zookeeper.server.DatadirCleanupManager: Purge task started.
    2013-10-07 15:47:07,714 INFO
    org.apache.zookeeper.server.DatadirCleanupManager: Purge task completed.
    2013-10-07 15:47:07,721 INFO
    org.apache.zookeeper.server.quorum.QuorumPeerConfig: Reading configuration
    from: /var/run/cloudera-scm-agent/process/26-zookeeper-server/zoo.cfg
    2013-10-07 15:47:07,722 ERROR
    org.apache.zookeeper.server.quorum.QuorumPeerConfig: Invalid configuration,
    only one server specified (ignoring)
    2013-10-07 15:47:07,722 INFO
    org.apache.zookeeper.server.ZooKeeperServerMain: Starting server
    [rtaylor@bigdatann zookeeper]$


    [rtaylor@bigdatann zookeeper]$ tail
    zookeeper-cmf-zookeeper1-SERVER-bigdatann.local.log
    2013-10-08 14:37:08,909 INFO org.apache.zookeeper.server.ZooKeeperServer:
    Client attempting to establish new session at /172.16.115.14:39187
    2013-10-08 14:37:08,910 INFO org.apache.zookeeper.server.ZooKeeperServer:
    Established session 0x141951a411d000e with negotiated timeout 60000 for
    client /172.16.115.14:39187
    2013-10-08 14:37:08,918 INFO
    org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted socket
    connection from /172.16.115.11:33231
    2013-10-08 14:37:08,921 INFO org.apache.zookeeper.server.ZooKeeperServer:
    Client attempting to establish new session at /172.16.115.11:33231
    2013-10-08 14:37:08,922 INFO org.apache.zookeeper.server.ZooKeeperServer:
    Established session 0x141951a411d000f with negotiated timeout 60000 for
    client /172.16.115.11:33231
    2013-10-08 14:37:08,961 INFO
    org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted socket
    connection from /172.16.115.16:44441
    2013-10-08 14:37:08,964 INFO org.apache.zookeeper.server.ZooKeeperServer:
    Client attempting to establish new session at /172.16.115.16:44441
    2013-10-08 14:37:08,965 INFO org.apache.zookeeper.server.ZooKeeperServer:
    Established session 0x141951a411d0010 with negotiated timeout 60000 for
    client /172.16.115.16:44441
    2013-10-08 14:38:04,736 INFO
    org.apache.zookeeper.server.PrepRequestProcessor: Processed session
    termination for sessionid: 0x141951a411d0003
    2013-10-08 14:38:04,737 INFO org.apache.zookeeper.server.NIOServerCnxn:
    Closed socket connection for client /172.16.115.18:55763 which had
    sessionid 0x141951a411d0003
    [rtaylor@bigdatann zookeeper]$
    [rtaylor@bigdatann zookeeper]$
    [rtaylor@bigdatann zookeeper]$





    On Tue, Oct 8, 2013 at 5:35 PM, Vikram Srivastava wrote:

    Can you goto the host with ZooKeeper Server(s) and paste the logs from
    /var/log/zookeeper on them?

    On Tue, Oct 8, 2013 at 4:15 PM, wrote:

    Hello Cloudera folks & users,******

    ** **

    I have just used the Cloudera Manager to install the Standard version of
    CDH4 on a new Linux cluster of 18 nodes (a namenode, a jobtracker node, and
    16 data nodes). ****

    ** **

    I got a success msg at the end of “Installation Path A – Automated
    Installation by Cloudera Manager”.****

    ** **

    That is, I got this:****

    ** **

    %%%****

    Starting your cluster services.****

    Completed 6 of 6 steps.****

    ** **

    Waiting for ZooKeeper Service to initialize****

    Finished waiting****

    ** **

    Checking if the name directories of the NameNode are empty. Formatting
    HDFS only if empty.****

    Successfully formatted NameNode.****

    ** **

    Creating Hive Metastore Database****

    Created Hive Metastore Database.****

    ****

    Creating Hive Metastore Database Tables****

    Created Hive Metastore Database Tables successfully.****

    ****

    Creating Oozie database****

    Oozie database created successfully.****

    ****

    Installing Oozie ShareLib in HDFS****

    Command Install Oozie ShareLib is not currently available for execution.
    ****

    %%****

    ** **

    And then I clicked “Continue” on that install web page, and got a final
    web page with this msg:****

    ** **

    “Congratulations! The Hadoop services are installed, configured, and
    running on your cluster.”****

    ** **

    %%****

    ** **
    However, when I then access the Cloudera Manager Admin Console at port
    7180 on the gateway/head node on the cluster (where I ran
    Cloudera-manager-installer.bin, but which was left out of the list of
    Hadoop-configured nodes), I am told that Host Monitor is not running,
    and I see no charts. I tried to do a “Start” on the cluster in the left
    panel, thinking that for some reason the cluster services (and the Host
    Monitor) did not start automatically, But that failed. In the popup box I
    was told that mapreduce1 service could not be started, and that I did not
    have enough job trackers. In the install process I had specified one job
    tracker node for the cluster. That choice appeared to be accepted during
    the install process - and should have been enough, of course.

    The precise msg was

    Service did not start successfully; not all of the required roles
    started: Service doesn't have sufficient running JobTrackers.

    I tried another "Start" command after the first failed, and got this
    different error msg:

    Command 'ZkStartPreservingDatastore' failed for service 'zookeeper1'

    ** **

    Unfortunately, I don't know what is causing the problem(s), and I need
    to some help to finish this install. I have the Cloudera Manager
    Installation Guide, but so far I have not found any answer in that. I would
    very much appreciate some guidance.****

    ****

    - Ron Taylor****

    ** **

    PS – yep, still working right now (all the scientists here are
    contractors, not direct fed employees, and we haven’t been furloughed yet)
    ****

    ** **

    Ronald Taylor, Ph.D.****

    Computational Biology & Bioinformatics Group****

    Pacific Northwest National Laboratory (U.S. Dept of Energy/Battelle)****

    Richland, WA 99352****

    phone: (509) 372-6568****

    email: ronald.taylor@pnnl.gov****

    web page: http://www.sysbio.org/resources/staff/taylor.stm****

    ** **

    To unsubscribe from this group and stop receiving emails from it, send
    an email to scm-users+unsubscribe@cloudera.org.
    To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedOct 8, '13 at 11:15p
activeOct 9, '13 at 12:55a
posts4
users2
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase