FAQ
Hi Cloudera support,

Initially, I setup a cluster of 3 nodes(Ubuntu 12.04 LTS, medium) in AWS
EC2 using CM, everything is good like it should be. I updated Parcels to
the latest version of CDH & Impala. After that, both running fine.

when running out of space, I tried to add 3 more hosts with exactly the
same machines I had, using CM "Add new hosts to Cluster" button. I went
through the whole process OK, but got a warning message: "There are
mismatched versions across the system. See details below for details on
which hosts are running what versions of components. " at the end, although
I did not see anything different.

I tried two things after that, but none of them worked.
(1) Tried "Re-run Host Upgrade Wizard", but it seems CM used wrong hosts
name, so upgrades failed. (Instead of either using AWS public or private
DNS names, CM used AWS instance name)
(2) Tried to ignore version difference, so I created a datanode template
and applied to the 3 new hosts. Everything seems fine. The CM "Hosts" shows
6 hosts without any problem. However, when I loaded more data in to Hive on
the original datanodes, the new hosts hdfs are not updated. So I concluded,
the new hosts are not working with the existing cluster at all.

Is that CM issue or mine?

Thanks,

Aaron A

Search Discussions

  • Darren Lo at May 23, 2013 at 9:48 pm
    What does the Host Inspector say about your versions? You can run it from
    the Hosts tab.

    After adding the hosts, you need to assign roles to your new hosts. Did you
    do that and then restart the cluster?

    On Thu, May 23, 2013 at 2:46 PM, Aaron A wrote:

    Hi Cloudera support,

    Initially, I setup a cluster of 3 nodes(Ubuntu 12.04 LTS, medium) in AWS
    EC2 using CM, everything is good like it should be. I updated Parcels to
    the latest version of CDH & Impala. After that, both running fine.

    when running out of space, I tried to add 3 more hosts with exactly the
    same machines I had, using CM "Add new hosts to Cluster" button. I went
    through the whole process OK, but got a warning message: "There are
    mismatched versions across the system. See details below for details on
    which hosts are running what versions of components. " at the end, although
    I did not see anything different.

    I tried two things after that, but none of them worked.
    (1) Tried "Re-run Host Upgrade Wizard", but it seems CM used wrong hosts
    name, so upgrades failed. (Instead of either using AWS public or private
    DNS names, CM used AWS instance name)
    (2) Tried to ignore version difference, so I created a datanode template
    and applied to the 3 new hosts. Everything seems fine. The CM "Hosts" shows
    6 hosts without any problem. However, when I loaded more data in to Hive on
    the original datanodes, the new hosts hdfs are not updated. So I concluded,
    the new hosts are not working with the existing cluster at all.

    Is that CM issue or mine?

    Thanks,

    Aaron A


    --
    Thanks,
    Darren
  • Aaron A at May 23, 2013 at 9:54 pm
    Darren,

    The followings are the Inspector Results & I did restart all services.

    Thanks,

    Aaron A

    =============================================

    Inspector Results Validations

      Inspector ran on all 6 hosts.
      Individual hosts resolved their own hostnames correctly.
      No errors were found while looking for conflicting init scripts.
      No errors were found while checking /etc/hosts.
      All hosts resolved localhost to 127.0.0.1.
      All hosts checked resolved each other's hostnames correctly.
      Host clocks are approximately in sync (within ten minutes).
      Host time zones are consistent across the cluster.
      No users or groups are missing.
      No kernel versions that are known to be bad are running.
      0 hosts are running CDH3 and 6 hosts are running CDH4.
      There are mismatched versions across the system. See details below for
    details on which hosts are running what versions of components.
      All checked Cloudera Management Daemons versions are consistent with the
    server.
      All checked Cloudera Management Agents versions are consistent with the
    server.
    Version Summary
        *Group 1 (CDH4)* Hosts ip-10-147-191-160.ec2.internal,
    ip-10-165-22-189.ec2.internal, ip-10-165-7-41.ec2.internal *Component*
    Version CDH Version Impala 1.0 Not applicable HDFS (CDH4 only) 2.0.0+960
    CDH4 Hue Plugins 2.2.0+194 CDH4 MapReduce 2 (CDH4 only) 2.0.0+960 CDH4
    HBase 0.94.2+218 CDH4 Oozie 3.3.0+79 CDH4 Yarn (CDH4 only) 2.0.0+960 CDH4
    Zookeeper 3.4.5+16 CDH4 Hue 2.2.0+194 CDH4 MapReduce 1 (CDH4 only)
    0.20.2+1359 CDH4 Pig 0.10.0+510 CDH4 HttpFS (CDH4 only) 2.0.0+960 CDH4
    Hadoop 2.0.0+960 CDH4 Hive 0.10.0+78 CDH4 Flume NG 1.3.0+96 CDH4 Cloudera
    Manager Management Daemons 4.5.2 Not applicable Cloudera Manager Agent
    4.5.2 Not applicable
       *Group 2 (CDH4)* Hosts ip-10-119-1-147.ec2.internal,
    ip-10-38-138-103.ec2.internal, ip-10-80-222-80.ec2.internal *Component*
    Version CDH Version Flume NG 1.3.0+96 CDH4 MapReduce 1 (CDH4 only)
    0.20.2+1359 CDH4 Hadoop 2.0.0+960 CDH4 HDFS (CDH4 only) 2.0.0+960 CDH4 HttpFS
    (CDH4 only) 2.0.0+960 CDH4 MapReduce 2 (CDH4 only) 2.0.0+960 CDH4 Yarn
    (CDH4 only) 2.0.0+960 CDH4 HBase 0.94.2+218 CDH4
    label.cdhVersionTable.hcatalog 0.4.0+218 CDH4 Hive 0.10.0+78 CDH4 Mahout
    0.7+15 CDH4 Oozie 3.3.0+79 CDH4 Pig 0.10.0+510 CDH4 Sqoop2 (CDH4 only)
    1.99.1+33 CDH4 Sqoop 1.4.2+60 CDH4 Whirr 0.8.0+26 CDH4 Zookeeper 3.4.5+16
    CDH4 Impala 1.0 Not applicable Cloudera Manager Management Daemons 4.5.2 Not
    applicable Cloudera Manager Agent 4.5.2 Not applicable
  • Darren Lo at May 23, 2013 at 11:48 pm
    Hi Aaron,

    I asked around some more on this, and found that there is currently a bug
    where CM does not correctly add hosts to a cluster on AWS. Sorry for the
    trouble. We're currently working on a fix and hope to have it in a release
    soon.

    Thanks,
    Darren

    On Thu, May 23, 2013 at 2:54 PM, Aaron A wrote:

    Darren,

    The followings are the Inspector Results & I did restart all services.

    Thanks,

    Aaron A

    =============================================

    Inspector Results Validations

    Inspector ran on all 6 hosts.
    Individual hosts resolved their own hostnames correctly.
    No errors were found while looking for conflicting init scripts.
    No errors were found while checking /etc/hosts.
    All hosts resolved localhost to 127.0.0.1.
    All hosts checked resolved each other's hostnames correctly.
    Host clocks are approximately in sync (within ten minutes).
    Host time zones are consistent across the cluster.
    No users or groups are missing.
    No kernel versions that are known to be bad are running.
    0 hosts are running CDH3 and 6 hosts are running CDH4.
    There are mismatched versions across the system. See details below for
    details on which hosts are running what versions of components.
    All checked Cloudera Management Daemons versions are consistent with the
    server.
    All checked Cloudera Management Agents versions are consistent with the
    server.
    Version Summary
    *Group 1 (CDH4)* Hosts ip-10-147-191-160.ec2.internal,
    ip-10-165-22-189.ec2.internal, ip-10-165-7-41.ec2.internal *Component*
    Version CDH Version Impala 1.0 Not applicable HDFS (CDH4 only) 2.0.0+960
    CDH4 Hue Plugins 2.2.0+194 CDH4 MapReduce 2 (CDH4 only) 2.0.0+960 CDH4
    HBase 0.94.2+218 CDH4 Oozie 3.3.0+79 CDH4 Yarn (CDH4 only) 2.0.0+960
    CDH4 Zookeeper 3.4.5+16 CDH4 Hue 2.2.0+194 CDH4 MapReduce 1 (CDH4 only)
    0.20.2+1359 CDH4 Pig 0.10.0+510 CDH4 HttpFS (CDH4 only) 2.0.0+960 CDH4
    Hadoop 2.0.0+960 CDH4 Hive 0.10.0+78 CDH4 Flume NG 1.3.0+96 CDH4 Cloudera
    Manager Management Daemons 4.5.2 Not applicable Cloudera Manager Agent
    4.5.2 Not applicable
    *Group 2 (CDH4)* Hosts ip-10-119-1-147.ec2.internal,
    ip-10-38-138-103.ec2.internal, ip-10-80-222-80.ec2.internal *Component*
    Version CDH Version Flume NG 1.3.0+96 CDH4 MapReduce 1 (CDH4 only)
    0.20.2+1359 CDH4 Hadoop 2.0.0+960 CDH4 HDFS (CDH4 only) 2.0.0+960 CDH4 HttpFS
    (CDH4 only) 2.0.0+960 CDH4 MapReduce 2 (CDH4 only) 2.0.0+960 CDH4 Yarn
    (CDH4 only) 2.0.0+960 CDH4 HBase 0.94.2+218 CDH4
    label.cdhVersionTable.hcatalog 0.4.0+218 CDH4 Hive 0.10.0+78 CDH4 Mahout
    0.7+15 CDH4 Oozie 3.3.0+79 CDH4 Pig 0.10.0+510 CDH4 Sqoop2 (CDH4 only)
    1.99.1+33 CDH4 Sqoop 1.4.2+60 CDH4 Whirr 0.8.0+26 CDH4 Zookeeper
    3.4.5+16 CDH4 Impala 1.0 Not applicable Cloudera Manager Management
    Daemons 4.5.2 Not applicable Cloudera Manager Agent 4.5.2 Not applicable

    --
    Thanks,
    Darren

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedMay 23, '13 at 9:46p
activeMay 23, '13 at 11:48p
posts4
users2
websitecloudera.com
irc#hadoop

2 users in discussion

Darren Lo: 2 posts Aaron A: 2 posts

People

Translate

site design / logo © 2022 Grokbase