FAQ
On behalf of the Cloudera Manager team, I'm excited to announce the Cloudera
Manager (Free Edition) 4.5
Beta<https://ccp.cloudera.com/display/SUPPORT/Cloudera+Manager+Downloads>.
It is the preview of our upcoming feature-packed 4.5 release. I'll
highlight the following:

- *Easy CDH upgrade*. You can now manage and upgrade the CDH binaries on
the cluster using Cloudera Manager, which supports a new packaging format
called "parcel". Rather than manually upgrading the CDH packages across
your cluster, you can now download, distribute and activate a new CDH
version all from within Cloudera Manager.

Parcels are available for CDH 4.1.2 and onwards. Cloudera Manager
continues to work with RPM and Debian packages.

- *Improved configuration management for role instances*. You can now
group an arbitrary set of roles instances (of the same type, e.g.
DataNodes) together and manage their configuration as a unit. Called "Role
Configuration Groups", this feature simplifies configuring groups of roles
that need to be configured differently, for example when running on nodes
with heterogeneous hardware.

- *Improved role to host assignment with Host Templates*. A host
template defines a set of role configurations that can be applied to new
hosts. This simplifies the process of adding hosts and instantiating the
roles that should run on those hosts.

- *Support for Hive*. Hive is now represented as its own service. Other
services that depend on the Hive metastore (such as Impala and Hue) will be
converted to depend on the corresponding Hive service(s), and will have
their Hive related configuration automatically migrated over.

- *Support for HBase REST and Thrift servers*. These can be added as
role instances for HBase services.

- *Automatic creation of Oozie ShareLib* when adding an Oozie service.

Download:
https://ccp.cloudera.com/display/SUPPORT/Cloudera+Manager+Downloads
Installation Guide:
https://ccp.cloudera.com/display/FREE45BETADOC/Cloudera+Manager+Free+Edition+Installation+Guide
Feedback and support: scm-users@cloudera.org <
https://groups.google.com/a/cloudera.org/forum/?hl=en&fromgroups#!forum/scm-users
>
Jira: https://issues.cloudera.org/browse/cm

We encourage you to try it out, and look forward to your feedback.

(Note that this beta release is available only for the Free Edition of
Cloudera Manager.)

Cheers,
bc

Search Discussions

  • Benjamin Kim at Dec 18, 2012 at 5:07 pm
    Hi Wong,

    I have few questions.

    Can I upgrade my current CM installation to this? How long is the Beta
    until release? And is your team thinking about adding database setup
    too? For instance, creating the database, applying permissions, and running
    any initialization scripts for table creation and data population, if
    necessary.

    Thanks,
    Ben
    On Monday, December 17, 2012 11:40:54 AM UTC-8, bc Wong wrote:

    On behalf of the Cloudera Manager team, I'm excited to announce the Cloudera
    Manager (Free Edition) 4.5 Beta<https://ccp.cloudera.com/display/SUPPORT/Cloudera+Manager+Downloads>.
    It is the preview of our upcoming feature-packed 4.5 release. I'll
    highlight the following:

    - *Easy CDH upgrade*. You can now manage and upgrade the CDH binaries
    on the cluster using Cloudera Manager, which supports a new packaging
    format called "parcel". Rather than manually upgrading the CDH packages
    across your cluster, you can now download, distribute and activate a new
    CDH version all from within Cloudera Manager.

    Parcels are available for CDH 4.1.2 and onwards. Cloudera Manager
    continues to work with RPM and Debian packages.

    - *Improved configuration management for role instances*. You can now
    group an arbitrary set of roles instances (of the same type, e.g.
    DataNodes) together and manage their configuration as a unit. Called "Role
    Configuration Groups", this feature simplifies configuring groups of roles
    that need to be configured differently, for example when running on nodes
    with heterogeneous hardware.

    - *Improved role to host assignment with Host Templates*. A host
    template defines a set of role configurations that can be applied to new
    hosts. This simplifies the process of adding hosts and instantiating the
    roles that should run on those hosts.

    - *Support for Hive*. Hive is now represented as its own service.
    Other services that depend on the Hive metastore (such as Impala and Hue)
    will be converted to depend on the corresponding Hive service(s), and will
    have their Hive related configuration automatically migrated over.

    - *Support for HBase REST and Thrift servers*. These can be added as
    role instances for HBase services.

    - *Automatic creation of Oozie ShareLib* when adding an Oozie service.

    Download:
    https://ccp.cloudera.com/display/SUPPORT/Cloudera+Manager+Downloads
    Installation Guide:
    https://ccp.cloudera.com/display/FREE45BETADOC/Cloudera+Manager+Free+Edition+Installation+Guide
    Feedback and support: scm-...@cloudera.org <javascript:> <
    https://groups.google.com/a/cloudera.org/forum/?hl=en&fromgroups#!forum/scm-users
    Jira: https://issues.cloudera.org/browse/cm

    We encourage you to try it out, and look forward to your feedback.

    (Note that this beta release is available only for the Free Edition of
    Cloudera Manager.)

    Cheers,
    bc
  • Benjamin Kim at Dec 18, 2012 at 5:11 pm
    Wong,

    I forgot to add. Is Hive Server2 included in this service? Can a server be
    a Gateway node for Hive? I want to know if Hive configurations can be
    pushed to a Gateway/Client node.

    Thanks,
    Ben
    On Monday, December 17, 2012 11:40:54 AM UTC-8, bc Wong wrote:

    On behalf of the Cloudera Manager team, I'm excited to announce the Cloudera
    Manager (Free Edition) 4.5 Beta<https://ccp.cloudera.com/display/SUPPORT/Cloudera+Manager+Downloads>.
    It is the preview of our upcoming feature-packed 4.5 release. I'll
    highlight the following:

    - *Easy CDH upgrade*. You can now manage and upgrade the CDH binaries
    on the cluster using Cloudera Manager, which supports a new packaging
    format called "parcel". Rather than manually upgrading the CDH packages
    across your cluster, you can now download, distribute and activate a new
    CDH version all from within Cloudera Manager.

    Parcels are available for CDH 4.1.2 and onwards. Cloudera Manager
    continues to work with RPM and Debian packages.

    - *Improved configuration management for role instances*. You can now
    group an arbitrary set of roles instances (of the same type, e.g.
    DataNodes) together and manage their configuration as a unit. Called "Role
    Configuration Groups", this feature simplifies configuring groups of roles
    that need to be configured differently, for example when running on nodes
    with heterogeneous hardware.

    - *Improved role to host assignment with Host Templates*. A host
    template defines a set of role configurations that can be applied to new
    hosts. This simplifies the process of adding hosts and instantiating the
    roles that should run on those hosts.

    - *Support for Hive*. Hive is now represented as its own service.
    Other services that depend on the Hive metastore (such as Impala and Hue)
    will be converted to depend on the corresponding Hive service(s), and will
    have their Hive related configuration automatically migrated over.

    - *Support for HBase REST and Thrift servers*. These can be added as
    role instances for HBase services.

    - *Automatic creation of Oozie ShareLib* when adding an Oozie service.

    Download:
    https://ccp.cloudera.com/display/SUPPORT/Cloudera+Manager+Downloads
    Installation Guide:
    https://ccp.cloudera.com/display/FREE45BETADOC/Cloudera+Manager+Free+Edition+Installation+Guide
    Feedback and support: scm-...@cloudera.org <javascript:> <
    https://groups.google.com/a/cloudera.org/forum/?hl=en&fromgroups#!forum/scm-users
    Jira: https://issues.cloudera.org/browse/cm

    We encourage you to try it out, and look forward to your feedback.

    (Note that this beta release is available only for the Free Edition of
    Cloudera Manager.)

    Cheers,
    bc
  • Darren Lo at Dec 18, 2012 at 5:41 pm
    Hi Ben,

    For Hive, the Beta has a Gateway role that can be assigned to any host.
    When you deploy Hive client configs, it will update the /etc/hive/conf on
    all hosts that have a Hive role. For GA (but not in beta), we will also
    support HiveServer2. Deploy client configs works on hosts with any kind of
    role, so it will work on both hosts that have a Gateway Role or a
    HiveServer2 Role.

    Hive support in Beta is very basic. Hive schema management is part of what
    we're looking to improve in the GA timeframe, and if you have suggestions
    for features in that arena, please let us know!

    Thanks,
    Darren

    On Tue, Dec 18, 2012 at 9:11 AM, Benjamin Kim wrote:

    Wong,

    I forgot to add. Is Hive Server2 included in this service? Can a server be
    a Gateway node for Hive? I want to know if Hive configurations can be
    pushed to a Gateway/Client node.

    Thanks,
    Ben
    On Monday, December 17, 2012 11:40:54 AM UTC-8, bc Wong wrote:

    On behalf of the Cloudera Manager team, I'm excited to announce the Cloudera
    Manager (Free Edition) 4.5 Beta<https://ccp.cloudera.com/display/SUPPORT/Cloudera+Manager+Downloads>.
    It is the preview of our upcoming feature-packed 4.5 release. I'll
    highlight the following:

    - *Easy CDH upgrade*. You can now manage and upgrade the CDH binaries
    on the cluster using Cloudera Manager, which supports a new packaging
    format called "parcel". Rather than manually upgrading the CDH packages
    across your cluster, you can now download, distribute and activate a new
    CDH version all from within Cloudera Manager.

    Parcels are available for CDH 4.1.2 and onwards. Cloudera Manager
    continues to work with RPM and Debian packages.

    - *Improved configuration management for role instances*. You can now
    group an arbitrary set of roles instances (of the same type, e.g.
    DataNodes) together and manage their configuration as a unit. Called "Role
    Configuration Groups", this feature simplifies configuring groups of roles
    that need to be configured differently, for example when running on nodes
    with heterogeneous hardware.

    - *Improved role to host assignment with Host Templates*. A host
    template defines a set of role configurations that can be applied to new
    hosts. This simplifies the process of adding hosts and instantiating the
    roles that should run on those hosts.

    - *Support for Hive*. Hive is now represented as its own service.
    Other services that depend on the Hive metastore (such as Impala and Hue)
    will be converted to depend on the corresponding Hive service(s), and will
    have their Hive related configuration automatically migrated over.

    - *Support for HBase REST and Thrift servers*. These can be added as
    role instances for HBase services.

    - *Automatic creation of Oozie ShareLib* when adding an Oozie service.

    Download: https://ccp.**cloudera.com/display/SUPPORT/**
    Cloudera+Manager+Downloads<https://ccp.cloudera.com/display/SUPPORT/Cloudera+Manager+Downloads>
    Installation Guide: https://ccp.cloudera.**com/display/FREE45BETADOC/**
    Cloudera+Manager+Free+Edition+**Installation+Guide<https://ccp.cloudera.com/display/FREE45BETADOC/Cloudera+Manager+Free+Edition+Installation+Guide>
    Feedback and support: scm-...@cloudera.**org <https://groups.google.**
    com/a/cloudera.org/forum/?hl=**en&fromgroups#!forum/scm-users<https://groups.google.com/a/cloudera.org/forum/?hl=en&fromgroups#!forum/scm-users>
    **>
    Jira: https://issues.cloudera.**org/browse/cm<https://issues.cloudera.org/browse/cm>

    We encourage you to try it out, and look forward to your feedback.

    (Note that this beta release is available only for the Free Edition of
    Cloudera Manager.)

    Cheers,
    bc

    --
    Thanks,
    Darren
  • Benjamin Kim at Dec 18, 2012 at 6:19 pm
    Hi Darren,

    Are you guys thinking about database setup too so we don't have to create
    databases and users for Hue, Hive, Oozie?

    What is the timeframe for GA? Also, I don't see a way to upgrade my current
    CM installation to 4.5 beta. Is there a way?

    Thanks,
    Ben

    On Tuesday, December 18, 2012 9:41:02 AM UTC-8, Darren Lo wrote:

    Hi Ben,

    For Hive, the Beta has a Gateway role that can be assigned to any host.
    When you deploy Hive client configs, it will update the /etc/hive/conf on
    all hosts that have a Hive role. For GA (but not in beta), we will also
    support HiveServer2. Deploy client configs works on hosts with any kind of
    role, so it will work on both hosts that have a Gateway Role or a
    HiveServer2 Role.

    Hive support in Beta is very basic. Hive schema management is part of what
    we're looking to improve in the GA timeframe, and if you have suggestions
    for features in that arena, please let us know!

    Thanks,
    Darren


    On Tue, Dec 18, 2012 at 9:11 AM, Benjamin Kim <bbui...@gmail.com<javascript:>
    wrote:
    Wong,

    I forgot to add. Is Hive Server2 included in this service? Can a server
    be a Gateway node for Hive? I want to know if Hive configurations can be
    pushed to a Gateway/Client node.

    Thanks,
    Ben
    On Monday, December 17, 2012 11:40:54 AM UTC-8, bc Wong wrote:

    On behalf of the Cloudera Manager team, I'm excited to announce the Cloudera
    Manager (Free Edition) 4.5 Beta<https://ccp.cloudera.com/display/SUPPORT/Cloudera+Manager+Downloads>.
    It is the preview of our upcoming feature-packed 4.5 release. I'll
    highlight the following:

    - *Easy CDH upgrade*. You can now manage and upgrade the CDH
    binaries on the cluster using Cloudera Manager, which supports a new
    packaging format called "parcel". Rather than manually upgrading the CDH
    packages across your cluster, you can now download, distribute and activate
    a new CDH version all from within Cloudera Manager.

    Parcels are available for CDH 4.1.2 and onwards. Cloudera Manager
    continues to work with RPM and Debian packages.

    - *Improved configuration management for role instances*. You can
    now group an arbitrary set of roles instances (of the same type, e.g.
    DataNodes) together and manage their configuration as a unit. Called "Role
    Configuration Groups", this feature simplifies configuring groups of roles
    that need to be configured differently, for example when running on nodes
    with heterogeneous hardware.

    - *Improved role to host assignment with Host Templates*. A host
    template defines a set of role configurations that can be applied to new
    hosts. This simplifies the process of adding hosts and instantiating the
    roles that should run on those hosts.

    - *Support for Hive*. Hive is now represented as its own service.
    Other services that depend on the Hive metastore (such as Impala and Hue)
    will be converted to depend on the corresponding Hive service(s), and will
    have their Hive related configuration automatically migrated over.

    - *Support for HBase REST and Thrift servers*. These can be added as
    role instances for HBase services.

    - *Automatic creation of Oozie ShareLib* when adding an Oozie
    service.

    Download: https://ccp.**cloudera.com/display/SUPPORT/**
    Cloudera+Manager+Downloads<https://ccp.cloudera.com/display/SUPPORT/Cloudera+Manager+Downloads>
    Installation Guide: https://ccp.cloudera.**com/display/FREE45BETADOC/**
    Cloudera+Manager+Free+Edition+**Installation+Guide<https://ccp.cloudera.com/display/FREE45BETADOC/Cloudera+Manager+Free+Edition+Installation+Guide>
    Feedback and support: scm-...@cloudera.**org <https://groups.google.**
    com/a/cloudera.org/forum/?hl=**en&fromgroups#!forum/scm-users<https://groups.google.com/a/cloudera.org/forum/?hl=en&fromgroups#!forum/scm-users>
    **>
    Jira: https://issues.cloudera.**org/browse/cm<https://issues.cloudera.org/browse/cm>

    We encourage you to try it out, and look forward to your feedback.

    (Note that this beta release is available only for the Free Edition of
    Cloudera Manager.)

    Cheers,
    bc

    --
    Thanks,
    Darren
  • bc Wong at Dec 18, 2012 at 7:24 pm
    [bcc: cdh-user]
    On Tue, Dec 18, 2012 at 10:19 AM, Benjamin Kim wrote:

    Hi Darren,

    Are you guys thinking about database setup too so we don't have to create
    databases and users for Hue, Hive, Oozie?
    We're planning to do that for Hive first, because it's more distributed
    than Hue & Oozie. But yes, it's a highly desirable feature.

    What is the timeframe for GA? Also, I don't see a way to upgrade my
    current CM installation to 4.5 beta. Is there a way?
    To upgrade, you can follow this doc:
    https://ccp.cloudera.com/display/ENT4DOC/Upgrade+from+Cloudera+Manager+4+to+the+Latest+Cloudera+Manager+4.0
    Note that the CM45 beta bits are in a different directory. You'll need to
    pick the right distro from http://archive.cloudera.com/cm45beta/ and update
    your repo file accordingly.

    I can't give a concrete date for GA. It'll happen when we're satisfied with
    the quality of the product. Early next year I hope.

    Cheers,
    bc

    On Tuesday, December 18, 2012 9:41:02 AM UTC-8, Darren Lo wrote:

    Hi Ben,

    For Hive, the Beta has a Gateway role that can be assigned to any host.
    When you deploy Hive client configs, it will update the /etc/hive/conf on
    all hosts that have a Hive role. For GA (but not in beta), we will also
    support HiveServer2. Deploy client configs works on hosts with any kind of
    role, so it will work on both hosts that have a Gateway Role or a
    HiveServer2 Role.

    Hive support in Beta is very basic. Hive schema management is part of
    what we're looking to improve in the GA timeframe, and if you have
    suggestions for features in that arena, please let us know!

    Thanks,
    Darren

    On Tue, Dec 18, 2012 at 9:11 AM, Benjamin Kim wrote:

    Wong,

    I forgot to add. Is Hive Server2 included in this service? Can a server
    be a Gateway node for Hive? I want to know if Hive configurations can be
    pushed to a Gateway/Client node.

    Thanks,
    Ben
    On Monday, December 17, 2012 11:40:54 AM UTC-8, bc Wong wrote:

    On behalf of the Cloudera Manager team, I'm excited to announce the Cloudera
    Manager (Free Edition) 4.5 Beta<https://ccp.cloudera.com/display/SUPPORT/Cloudera+Manager+Downloads>.
    It is the preview of our upcoming feature-packed 4.5 release. I'll
    highlight the following:

    - *Easy CDH upgrade*. You can now manage and upgrade the CDH
    binaries on the cluster using Cloudera Manager, which supports a new
    packaging format called "parcel". Rather than manually upgrading the CDH
    packages across your cluster, you can now download, distribute and activate
    a new CDH version all from within Cloudera Manager.

    Parcels are available for CDH 4.1.2 and onwards. Cloudera Manager
    continues to work with RPM and Debian packages.

    - *Improved configuration management for role instances*. You can
    now group an arbitrary set of roles instances (of the same type, e.g.
    DataNodes) together and manage their configuration as a unit. Called "Role
    Configuration Groups", this feature simplifies configuring groups of roles
    that need to be configured differently, for example when running on nodes
    with heterogeneous hardware.

    - *Improved role to host assignment with Host Templates*. A host
    template defines a set of role configurations that can be applied to new
    hosts. This simplifies the process of adding hosts and instantiating the
    roles that should run on those hosts.

    - *Support for Hive*. Hive is now represented as its own service.
    Other services that depend on the Hive metastore (such as Impala and Hue)
    will be converted to depend on the corresponding Hive service(s), and will
    have their Hive related configuration automatically migrated over.

    - *Support for HBase REST and Thrift servers*. These can be added
    as role instances for HBase services.

    - *Automatic creation of Oozie ShareLib* when adding an Oozie
    service.

    Download: https://ccp.**cloudera**.com/display/SUPPORT/**Cloudera+**
    Manager+Downloads<https://ccp.cloudera.com/display/SUPPORT/Cloudera+Manager+Downloads>
    Installation Guide: https://ccp.cloudera.**co**
    m/display/FREE45BETADOC/**Cloude**ra+Manager+Free+Edition+**Instal**
    lation+Guide<https://ccp.cloudera.com/display/FREE45BETADOC/Cloudera+Manager+Free+Edition+Installation+Guide>
    Feedback and support: scm-...@cloudera.**org **<https://groups.google.*
    *com/a/**cloudera.org/forum/?hl=**en&**fromgroups#!forum/scm-users<https://groups.google.com/a/cloudera.org/forum/?hl=en&fromgroups#!forum/scm-users>
    **>
    Jira: https://issues.cloudera.****org/browse/cm<https://issues.cloudera.org/browse/cm>

    We encourage you to try it out, and look forward to your feedback.

    (Note that this beta release is available only for the Free Edition of
    Cloudera Manager.)

    Cheers,
    bc

    --
    Thanks,
    Darren
  • Benjamin Kim at Dec 18, 2012 at 11:02 pm
    Wong,

    I tried to upgrade, but the agent installs on the cluster had errors. The
    error message said that the heartbeat timed out so it reverted. Have the
    ports changed for the agent? If I can't resolve this, then I will have to
    uninstall. Is there documentation for that?

    Thanks,
    Ben
    On Tuesday, December 18, 2012 11:24:06 AM UTC-8, bc Wong wrote:

    [bcc: cdh-user]

    On Tue, Dec 18, 2012 at 10:19 AM, Benjamin Kim <bbui...@gmail.com<javascript:>
    wrote:
    Hi Darren,

    Are you guys thinking about database setup too so we don't have to create
    databases and users for Hue, Hive, Oozie?
    We're planning to do that for Hive first, because it's more distributed
    than Hue & Oozie. But yes, it's a highly desirable feature.

    What is the timeframe for GA? Also, I don't see a way to upgrade my
    current CM installation to 4.5 beta. Is there a way?
    To upgrade, you can follow this doc:
    https://ccp.cloudera.com/display/ENT4DOC/Upgrade+from+Cloudera+Manager+4+to+the+Latest+Cloudera+Manager+4.0
    Note that the CM45 beta bits are in a different directory. You'll need to
    pick the right distro from http://archive.cloudera.com/cm45beta/ and
    update your repo file accordingly.

    I can't give a concrete date for GA. It'll happen when we're satisfied
    with the quality of the product. Early next year I hope.

    Cheers,
    bc

    On Tuesday, December 18, 2012 9:41:02 AM UTC-8, Darren Lo wrote:

    Hi Ben,

    For Hive, the Beta has a Gateway role that can be assigned to any host.
    When you deploy Hive client configs, it will update the /etc/hive/conf on
    all hosts that have a Hive role. For GA (but not in beta), we will also
    support HiveServer2. Deploy client configs works on hosts with any kind of
    role, so it will work on both hosts that have a Gateway Role or a
    HiveServer2 Role.

    Hive support in Beta is very basic. Hive schema management is part of
    what we're looking to improve in the GA timeframe, and if you have
    suggestions for features in that arena, please let us know!

    Thanks,
    Darren

    On Tue, Dec 18, 2012 at 9:11 AM, Benjamin Kim wrote:

    Wong,

    I forgot to add. Is Hive Server2 included in this service? Can a server
    be a Gateway node for Hive? I want to know if Hive configurations can be
    pushed to a Gateway/Client node.

    Thanks,
    Ben
    On Monday, December 17, 2012 11:40:54 AM UTC-8, bc Wong wrote:

    On behalf of the Cloudera Manager team, I'm excited to announce the Cloudera
    Manager (Free Edition) 4.5 Beta<https://ccp.cloudera.com/display/SUPPORT/Cloudera+Manager+Downloads>.
    It is the preview of our upcoming feature-packed 4.5 release. I'll
    highlight the following:

    - *Easy CDH upgrade*. You can now manage and upgrade the CDH
    binaries on the cluster using Cloudera Manager, which supports a new
    packaging format called "parcel". Rather than manually upgrading the CDH
    packages across your cluster, you can now download, distribute and activate
    a new CDH version all from within Cloudera Manager.

    Parcels are available for CDH 4.1.2 and onwards. Cloudera Manager
    continues to work with RPM and Debian packages.

    - *Improved configuration management for role instances*. You can
    now group an arbitrary set of roles instances (of the same type, e.g.
    DataNodes) together and manage their configuration as a unit. Called "Role
    Configuration Groups", this feature simplifies configuring groups of roles
    that need to be configured differently, for example when running on nodes
    with heterogeneous hardware.

    - *Improved role to host assignment with Host Templates*. A host
    template defines a set of role configurations that can be applied to new
    hosts. This simplifies the process of adding hosts and instantiating the
    roles that should run on those hosts.

    - *Support for Hive*. Hive is now represented as its own service.
    Other services that depend on the Hive metastore (such as Impala and Hue)
    will be converted to depend on the corresponding Hive service(s), and will
    have their Hive related configuration automatically migrated over.

    - *Support for HBase REST and Thrift servers*. These can be added
    as role instances for HBase services.

    - *Automatic creation of Oozie ShareLib* when adding an Oozie
    service.

    Download: https://ccp.**cloudera**.com/display/SUPPORT/**Cloudera+**
    Manager+Downloads<https://ccp.cloudera.com/display/SUPPORT/Cloudera+Manager+Downloads>
    Installation Guide: https://ccp.cloudera.**co**
    m/display/FREE45BETADOC/**Cloude**ra+Manager+Free+Edition+**Instal**
    lation+Guide<https://ccp.cloudera.com/display/FREE45BETADOC/Cloudera+Manager+Free+Edition+Installation+Guide>
    Feedback and support: scm-...@cloudera.**org **<https://groups.google.
    **com/a/**cloudera.org/forum/?hl=**en&**fromgroups#!forum/scm-users<https://groups.google.com/a/cloudera.org/forum/?hl=en&fromgroups#!forum/scm-users>
    **>
    Jira: https://issues.cloudera.****org/browse/cm<https://issues.cloudera.org/browse/cm>

    We encourage you to try it out, and look forward to your feedback.

    (Note that this beta release is available only for the Free Edition of
    Cloudera Manager.)

    Cheers,
    bc

    --
    Thanks,
    Darren
  • Herman Chen at Dec 18, 2012 at 11:07 pm
    Benjamin,

    You should be able to uninstall directly from the wizard -- there are
    buttons to Uninstall Failed Hosts and Abort. No the agent ports have not
    changed. To debug why your agent failed, before you uninstall them could
    you check for any errors in their log files? You can find them at
    /var/log/cloudera-scm-agent/cloudera-scm-agent.log. Thanks.

    Herman

    On Tue, Dec 18, 2012 at 3:02 PM, Benjamin Kim wrote:

    Wong,

    I tried to upgrade, but the agent installs on the cluster had errors. The
    error message said that the heartbeat timed out so it reverted. Have the
    ports changed for the agent? If I can't resolve this, then I will have to
    uninstall. Is there documentation for that?

    Thanks,
    Ben

    On Tuesday, December 18, 2012 11:24:06 AM UTC-8, bc Wong wrote:

    [bcc: cdh-user]
    On Tue, Dec 18, 2012 at 10:19 AM, Benjamin Kim wrote:

    Hi Darren,

    Are you guys thinking about database setup too so we don't have to
    create databases and users for Hue, Hive, Oozie?
    We're planning to do that for Hive first, because it's more distributed
    than Hue & Oozie. But yes, it's a highly desirable feature.

    What is the timeframe for GA? Also, I don't see a way to upgrade my
    current CM installation to 4.5 beta. Is there a way?
    To upgrade, you can follow this doc: https://ccp.cloudera.com/**
    display/ENT4DOC/Upgrade+from+**Cloudera+Manager+4+to+the+**
    Latest+Cloudera+Manager+4.0<https://ccp.cloudera.com/display/ENT4DOC/Upgrade+from+Cloudera+Manager+4+to+the+Latest+Cloudera+Manager+4.0>
    Note that the CM45 beta bits are in a different directory. You'll need to
    pick the right distro from http://archive.cloudera.**com/cm45beta/<http://archive.cloudera.com/cm45beta/> and
    update your repo file accordingly.

    I can't give a concrete date for GA. It'll happen when we're satisfied
    with the quality of the product. Early next year I hope.

    Cheers,
    bc

    On Tuesday, December 18, 2012 9:41:02 AM UTC-8, Darren Lo wrote:

    Hi Ben,

    For Hive, the Beta has a Gateway role that can be assigned to any host.
    When you deploy Hive client configs, it will update the /etc/hive/conf on
    all hosts that have a Hive role. For GA (but not in beta), we will also
    support HiveServer2. Deploy client configs works on hosts with any kind of
    role, so it will work on both hosts that have a Gateway Role or a
    HiveServer2 Role.

    Hive support in Beta is very basic. Hive schema management is part of
    what we're looking to improve in the GA timeframe, and if you have
    suggestions for features in that arena, please let us know!

    Thanks,
    Darren

    On Tue, Dec 18, 2012 at 9:11 AM, Benjamin Kim wrote:

    Wong,

    I forgot to add. Is Hive Server2 included in this service? Can a
    server be a Gateway node for Hive? I want to know if Hive configurations
    can be pushed to a Gateway/Client node.

    Thanks,
    Ben
    On Monday, December 17, 2012 11:40:54 AM UTC-8, bc Wong wrote:

    On behalf of the Cloudera Manager team, I'm excited to announce the Cloudera
    Manager (Free Edition) 4.5 Beta<https://ccp.cloudera.com/display/SUPPORT/Cloudera+Manager+Downloads>.
    It is the preview of our upcoming feature-packed 4.5 release. I'll
    highlight the following:

    - *Easy CDH upgrade*. You can now manage and upgrade the CDH
    binaries on the cluster using Cloudera Manager, which supports a new
    packaging format called "parcel". Rather than manually upgrading the CDH
    packages across your cluster, you can now download, distribute and activate
    a new CDH version all from within Cloudera Manager.

    Parcels are available for CDH 4.1.2 and onwards. Cloudera Manager
    continues to work with RPM and Debian packages.

    - *Improved configuration management for role instances*. You can
    now group an arbitrary set of roles instances (of the same type, e.g.
    DataNodes) together and manage their configuration as a unit. Called "Role
    Configuration Groups", this feature simplifies configuring groups of roles
    that need to be configured differently, for example when running on nodes
    with heterogeneous hardware.

    - *Improved role to host assignment with Host Templates*. A host
    template defines a set of role configurations that can be applied to new
    hosts. This simplifies the process of adding hosts and instantiating the
    roles that should run on those hosts.

    - *Support for Hive*. Hive is now represented as its own service.
    Other services that depend on the Hive metastore (such as Impala and Hue)
    will be converted to depend on the corresponding Hive service(s), and will
    have their Hive related configuration automatically migrated over.

    - *Support for HBase REST and Thrift servers*. These can be added
    as role instances for HBase services.

    - *Automatic creation of Oozie ShareLib* when adding an Oozie
    service.

    Download: https://ccp.**cloudera****.com/display/SUPPORT/**Cloudera+*
    ***Manager+Downloads<https://ccp.cloudera.com/display/SUPPORT/Cloudera+Manager+Downloads>
    Installation Guide: https://ccp.cloudera.**co****
    m/display/FREE45BETADOC/**Cloude****ra+Manager+Free+Edition+**Instal*
    ***lation+Guide<https://ccp.cloudera.com/display/FREE45BETADOC/Cloudera+Manager+Free+Edition+Installation+Guide>
    Feedback and support: scm-...@cloudera.**org ****<
    https://groups.google.**com/a/**c**loudera.org/forum/?hl=**en&**fromg
    **roups#!forum/scm-users<https://groups.google.com/a/cloudera.org/forum/?hl=en&fromgroups#!forum/scm-users>
    **>
    Jira: https://issues.cloudera.******org/browse/cm<https://issues.cloudera.org/browse/cm>

    We encourage you to try it out, and look forward to your feedback.

    (Note that this beta release is available only for the Free Edition
    of Cloudera Manager.)

    Cheers,
    bc

    --
    Thanks,
    Darren
  • Benjamin Kim at Dec 18, 2012 at 11:20 pm
    Herman,

    Here is the tail end of the log in question.

    18/Dec/2012 22:09:07 +0000] 5766 MainThread throttling_logger ERROR
    Failed to collect disk stats for device 'md'
    Traceback (most recent call last):
    File "/usr/lib64/cmf/agent/src/cmf/monitor/host/disk_device.py", line 54,
    in collect_update
    stat_file = open(self._stat_path)
    IOError: [Errno 2] No such file or directory: '/sys/block/md/stat'
    [18/Dec/2012 22:09:07 +0000] 5766 MainThread disk_devices WARNING Failed
    to add DiskDevice for disk='md'
    [18/Dec/2012 22:09:07 +0000] 5766 MainThread disk_devices WARNING Failed
    to add DiskDevice for partition='md0'
    [18/Dec/2012 22:09:07 +0000] 5766 MainThread __init__ INFO
    Importing metric schema from file
    /usr/lib64/cmf/agent/src/cmf/monitor/schema.json
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Supervised processes will inherit the following environment:
    {'CMF_CONF_DIR': '/etc/cloudera-scm-agent', 'CMF_AGENT_HIVE_DEFAULT_XML':
    '/usr/lib/hive/conf/hive-default.xml', 'CMF_AGENT_MGMT_HOME':
    '/usr/share/cmf', 'CMF_AGENT_CDH_HTTPFS_HOME': '/usr/lib/hadoop-httpfs',
    'PATH': '/sbin:/usr/sbin:/bin:/usr/bin', 'CMF_AGENT_CDH_FLUME_HOME':
    '/usr/lib/flume-ng', 'CMF_AGENT_CDH_HBASE_HOME': '/usr/lib/hbase', 'LANG':
    'en_US.UTF-8', 'TERM': 'vt100', 'CMF_AGENT_CDH_HUE_HOME': '/usr/share/hue',
    'SHLVL': '1', 'CMF_AGENT_TOMCAT_HOME': '/usr/lib/bigtop-tomcat',
    'CMF_AGENT_CDH_PIG_HOME': '/usr/lib/pig', 'CMF_AGENT_CDH_HIVE_HOME':
    '/usr/lib/hive', 'CLOUDERA_MYSQL_CONNECTOR_JAR':
    '/usr/share/java/mysql-connector-java.jar', 'CMF_AGENT_CDH_YARN_HOME':
    '/usr/lib/hadoop-yarn', 'CMF_AGENT_CDH_HADOOP_BIN': '/usr/bin/hadoop',
    'CMF_AGENT_CDH_OOZIE_HOME': '/usr/lib/oozie',
    'CMF_AGENT_CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper',
    'CMF_AGENT_CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CMF_AGENT_JSVC_HOME':
    '/usr/libexec/bigtop-utils', 'CMF_AGENT_CDH_HDFS_HOME':
    '/usr/lib/hadoop-hdfs', 'CMF_AGENT_CDH_MR1_HOME':
    '/usr/lib/hadoop-0.20-mapreduce', 'CMF_AGENT_CDH_MR2_HOME':
    '/usr/lib/hadoop-mapreduce', 'PWD': '/', 'CMF_AGENT_CDH_HADOOP_HOME':
    '/usr/lib/hadoop', 'CMF_AGENT_CDH_IMPALA_HOME': '/usr/lib/impala'}
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Supervised processes will add the following their environment environment:
    {'CDH_MR2_HOME': '/usr/lib/hadoop-mapreduce', 'CMF_PACKAGE_DIR':
    '/usr/lib64/cmf/service', 'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME':
    '/usr/share/cmf', 'CDH_IMPALA_HOME': '/usr/lib/impala', 'CDH_YARN_HOME':
    '/usr/lib/hadoop-yarn', 'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs',
    'CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CM_STATUS_CODES': u'STATUS_NONE
    HDFS_DFS_DIR_NOT_EMPTY', 'CDH_HIVE_HOME': '/usr/lib/hive', 'JSVC_HOME':
    '/usr/libexec/bigtop-utils', 'HIVE_DEFAULT_XML':
    '/usr/lib/hive/conf/hive-default.xml', 'CDH_HBASE_HOME': '/usr/lib/hbase',
    'CDH_OOZIE_HOME': '/usr/lib/oozie', 'CDH_ZOOKEEPER_HOME':
    '/usr/lib/zookeeper', 'CDH_HUE_HOME': '/usr/share/hue',
    'CLOUDERA_MYSQL_CONNECTOR_JAR': '/usr/share/java/mysql-connector-java.jar',
    'CDH_MR1_HOME': '/usr/lib/hadoop-0.20-mapreduce', 'CDH_PIG_HOME':
    '/usr/lib/pig', 'CDH_HTTPFS_HOME': '/usr/lib/hadoop-httpfs',
    'CDH_HADOOP_HOME': '/usr/lib/hadoop', 'TOMCAT_HOME':
    '/usr/lib/bigtop-tomcat', 'CDH_FLUME_HOME': '/usr/lib/flume-ng'}
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO Re-using
    pre-existing directory: /var/run/cloudera-scm-agent/process
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO Re-using
    pre-existing directory: /var/run/cloudera-scm-agent/supervisor
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO Re-using
    pre-existing directory: /var/run/cloudera-scm-agent/supervisor/include
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO Failed
    to connect to previous supervisor.
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO Deleting
    stale supervisor include
    /var/run/cloudera-scm-agent/supervisor/include/519-host-inspector.conf.
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO Deleting
    stale supervisor include
    /var/run/cloudera-scm-agent/supervisor/include/738-impala-IMPALAD.conf.
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO Deleting
    stale supervisor include
    /var/run/cloudera-scm-agent/supervisor/include/783-deploy-client-config.conf.
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO Deleting
    stale supervisor include
    /var/run/cloudera-scm-agent/supervisor/include/351-hbase-REGIONSERVER.conf.
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO Deleting
    stale supervisor include
    /var/run/cloudera-scm-agent/supervisor/include/806-hbase-REGIONSERVER.conf.
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO Deleting
    stale supervisor include
    /var/run/cloudera-scm-agent/supervisor/include/726-hdfs-DATANODE.conf.
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO Deleting
    stale supervisor include
    /var/run/cloudera-scm-agent/supervisor/include/754-mapreduce-TASKTRACKER.conf.
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO Deleting
    stale supervisor include
    /var/run/cloudera-scm-agent/supervisor/include/374-mapreduce-TASKTRACKER.conf.
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO Deleting
    stale supervisor include
    /var/run/cloudera-scm-agent/supervisor/include/469-impala-IMPALAD.conf.
    [18/Dec/2012 22:09:09 +0000] 5766 MainThread agent INFO Trying
    to connect to newly launched supervisor (Attempt 1)
    [18/Dec/2012 22:09:09 +0000] 5766 MainThread agent ERROR Failed!
    trying again in 1 second(s)
    Traceback (most recent call last):
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1056, in
    connect_to_new_supervisor
    self.get_supervisor_process_info()
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1078, in
    get_supervisor_process_info
    self.identifier = self.supervisor_client.supervisor.getIdentification()
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
    File
    "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0a12-py2.6.egg/supervisor/xmlrpc.py",
    line 461, in request
    self.connection.request('POST', handler, request_body, self.headers)
    File "/usr/lib64/python2.6/httplib.py", line 914, in request
    self._send_request(method, url, body, headers)
    File "/usr/lib64/python2.6/httplib.py", line 951, in _send_request
    self.endheaders()
    File "/usr/lib64/python2.6/httplib.py", line 908, in endheaders
    self._send_output()
    File "/usr/lib64/python2.6/httplib.py", line 780, in _send_output
    self.send(msg)
    File "/usr/lib64/python2.6/httplib.py", line 739, in send
    self.connect()
    File "/usr/lib64/python2.6/httplib.py", line 720, in connect
    self.timeout)
    File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused
    [18/Dec/2012 22:09:10 +0000] 5766 MainThread agent INFO Trying
    to connect to newly launched supervisor (Attempt 2)
    [18/Dec/2012 22:09:10 +0000] 5766 MainThread agent ERROR Failed!
    trying again in 1 second(s)
    Traceback (most recent call last):
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1056, in
    connect_to_new_supervisor
    self.get_supervisor_process_info()
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1078, in
    get_supervisor_process_info
    self.identifier = self.supervisor_client.supervisor.getIdentification()
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
    File
    "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0a12-py2.6.egg/supervisor/xmlrpc.py",
    line 461, in request
    self.connection.request('POST', handler, request_body, self.headers)
    File "/usr/lib64/python2.6/httplib.py", line 914, in request
    self._send_request(method, url, body, headers)
    File "/usr/lib64/python2.6/httplib.py", line 951, in _send_request
    self.endheaders()
    File "/usr/lib64/python2.6/httplib.py", line 908, in endheaders
    self._send_output()
    File "/usr/lib64/python2.6/httplib.py", line 780, in _send_output
    self.send(msg)
    File "/usr/lib64/python2.6/httplib.py", line 739, in send
    self.connect()
    File "/usr/lib64/python2.6/httplib.py", line 720, in connect
    self.timeout)
    File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused
    [18/Dec/2012 22:09:11 +0000] 5766 MainThread agent INFO Trying
    to connect to newly launched supervisor (Attempt 3)
    [18/Dec/2012 22:09:11 +0000] 5766 MainThread agent ERROR Failed!
    trying again in 1 second(s)
    Traceback (most recent call last):
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1056, in
    connect_to_new_supervisor
    self.get_supervisor_process_info()
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1078, in
    get_supervisor_process_info
    self.identifier = self.supervisor_client.supervisor.getIdentification()
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
    File
    "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0a12-py2.6.egg/supervisor/xmlrpc.py",
    line 461, in request
    self.connection.request('POST', handler, request_body, self.headers)
    File "/usr/lib64/python2.6/httplib.py", line 914, in request
    self._send_request(method, url, body, headers)
    File "/usr/lib64/python2.6/httplib.py", line 951, in _send_request
    self.endheaders()
    File "/usr/lib64/python2.6/httplib.py", line 908, in endheaders
    self._send_output()
    File "/usr/lib64/python2.6/httplib.py", line 780, in _send_output
    self.send(msg)
    File "/usr/lib64/python2.6/httplib.py", line 739, in send
    self.connect()
    File "/usr/lib64/python2.6/httplib.py", line 720, in connect
    self.timeout)
    File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused
    [18/Dec/2012 22:09:12 +0000] 5766 MainThread agent INFO Trying
    to connect to newly launched supervisor (Attempt 4)
    [18/Dec/2012 22:09:12 +0000] 5766 MainThread agent ERROR Failed!
    trying again in 1 second(s)
    Traceback (most recent call last):
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1056, in
    connect_to_new_supervisor
    self.get_supervisor_process_info()
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1078, in
    get_supervisor_process_info
    self.identifier = self.supervisor_client.supervisor.getIdentification()
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
    File
    "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0a12-py2.6.egg/supervisor/xmlrpc.py",
    line 461, in request
    self.connection.request('POST', handler, request_body, self.headers)
    File "/usr/lib64/python2.6/httplib.py", line 914, in request
    self._send_request(method, url, body, headers)
    File "/usr/lib64/python2.6/httplib.py", line 951, in _send_request
    self.endheaders()
    File "/usr/lib64/python2.6/httplib.py", line 908, in endheaders
    self._send_output()
    File "/usr/lib64/python2.6/httplib.py", line 780, in _send_output
    self.send(msg)
    File "/usr/lib64/python2.6/httplib.py", line 739, in send
    self.connect()
    File "/usr/lib64/python2.6/httplib.py", line 720, in connect
    self.timeout)
    File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused
    [18/Dec/2012 22:09:13 +0000] 5766 MainThread agent INFO Trying
    to connect to newly launched supervisor (Attempt 5)
    [18/Dec/2012 22:09:13 +0000] 5766 MainThread agent ERROR Failed!
    trying again in 1 second(s)
    Traceback (most recent call last):
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1056, in
    connect_to_new_supervisor
    self.get_supervisor_process_info()
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1078, in
    get_supervisor_process_info
    self.identifier = self.supervisor_client.supervisor.getIdentification()
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
    File
    "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0a12-py2.6.egg/supervisor/xmlrpc.py",
    line 461, in request
    self.connection.request('POST', handler, request_body, self.headers)
    File "/usr/lib64/python2.6/httplib.py", line 914, in request
    self._send_request(method, url, body, headers)
    File "/usr/lib64/python2.6/httplib.py", line 951, in _send_request
    self.endheaders()
    File "/usr/lib64/python2.6/httplib.py", line 908, in endheaders
    self._send_output()
    File "/usr/lib64/python2.6/httplib.py", line 780, in _send_output
    self.send(msg)
    File "/usr/lib64/python2.6/httplib.py", line 739, in send
    self.connect()
    File "/usr/lib64/python2.6/httplib.py", line 720, in connect
    self.timeout)
    File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused
    [18/Dec/2012 22:09:13 +0000] 5766 MainThread agent ERROR Failed
    to connect to newly launched supervisor. Agent will exit
    [18/Dec/2012 22:09:13 +0000] 5766 MainThread agent INFO Stopping
    agent...

    Thanks for any help.

    Cheers,
    Ben
    On Tuesday, December 18, 2012 3:07:03 PM UTC-8, Herman Chen wrote:

    Benjamin,

    You should be able to uninstall directly from the wizard -- there are
    buttons to Uninstall Failed Hosts and Abort. No the agent ports have not
    changed. To debug why your agent failed, before you uninstall them could
    you check for any errors in their log files? You can find them at
    /var/log/cloudera-scm-agent/cloudera-scm-agent.log. Thanks.

    Herman


    On Tue, Dec 18, 2012 at 3:02 PM, Benjamin Kim <bbui...@gmail.com<javascript:>
    wrote:
    Wong,

    I tried to upgrade, but the agent installs on the cluster had errors. The
    error message said that the heartbeat timed out so it reverted. Have the
    ports changed for the agent? If I can't resolve this, then I will have to
    uninstall. Is there documentation for that?

    Thanks,
    Ben

    On Tuesday, December 18, 2012 11:24:06 AM UTC-8, bc Wong wrote:

    [bcc: cdh-user]
    On Tue, Dec 18, 2012 at 10:19 AM, Benjamin Kim wrote:

    Hi Darren,

    Are you guys thinking about database setup too so we don't have to
    create databases and users for Hue, Hive, Oozie?
    We're planning to do that for Hive first, because it's more distributed
    than Hue & Oozie. But yes, it's a highly desirable feature.

    What is the timeframe for GA? Also, I don't see a way to upgrade my
    current CM installation to 4.5 beta. Is there a way?
    To upgrade, you can follow this doc: https://ccp.cloudera.com/**
    display/ENT4DOC/Upgrade+from+**Cloudera+Manager+4+to+the+**
    Latest+Cloudera+Manager+4.0<https://ccp.cloudera.com/display/ENT4DOC/Upgrade+from+Cloudera+Manager+4+to+the+Latest+Cloudera+Manager+4.0>
    Note that the CM45 beta bits are in a different directory. You'll need
    to pick the right distro from http://archive.cloudera.**com/cm45beta/<http://archive.cloudera.com/cm45beta/> and
    update your repo file accordingly.

    I can't give a concrete date for GA. It'll happen when we're satisfied
    with the quality of the product. Early next year I hope.

    Cheers,
    bc

    On Tuesday, December 18, 2012 9:41:02 AM UTC-8, Darren Lo wrote:

    Hi Ben,

    For Hive, the Beta has a Gateway role that can be assigned to any
    host. When you deploy Hive client configs, it will update the
    /etc/hive/conf on all hosts that have a Hive role. For GA (but not in
    beta), we will also support HiveServer2. Deploy client configs works on
    hosts with any kind of role, so it will work on both hosts that have a
    Gateway Role or a HiveServer2 Role.

    Hive support in Beta is very basic. Hive schema management is part of
    what we're looking to improve in the GA timeframe, and if you have
    suggestions for features in that arena, please let us know!

    Thanks,
    Darren

    On Tue, Dec 18, 2012 at 9:11 AM, Benjamin Kim wrote:

    Wong,

    I forgot to add. Is Hive Server2 included in this service? Can a
    server be a Gateway node for Hive? I want to know if Hive configurations
    can be pushed to a Gateway/Client node.

    Thanks,
    Ben
    On Monday, December 17, 2012 11:40:54 AM UTC-8, bc Wong wrote:

    On behalf of the Cloudera Manager team, I'm excited to announce the Cloudera
    Manager (Free Edition) 4.5 Beta<https://ccp.cloudera.com/display/SUPPORT/Cloudera+Manager+Downloads>.
    It is the preview of our upcoming feature-packed 4.5 release. I'll
    highlight the following:

    - *Easy CDH upgrade*. You can now manage and upgrade the CDH
    binaries on the cluster using Cloudera Manager, which supports a new
    packaging format called "parcel". Rather than manually upgrading the CDH
    packages across your cluster, you can now download, distribute and activate
    a new CDH version all from within Cloudera Manager.

    Parcels are available for CDH 4.1.2 and onwards. Cloudera
    Manager continues to work with RPM and Debian packages.

    - *Improved configuration management for role instances*. You
    can now group an arbitrary set of roles instances (of the same type, e.g.
    DataNodes) together and manage their configuration as a unit. Called "Role
    Configuration Groups", this feature simplifies configuring groups of roles
    that need to be configured differently, for example when running on nodes
    with heterogeneous hardware.

    - *Improved role to host assignment with Host Templates*. A host
    template defines a set of role configurations that can be applied to new
    hosts. This simplifies the process of adding hosts and instantiating the
    roles that should run on those hosts.

    - *Support for Hive*. Hive is now represented as its own
    service. Other services that depend on the Hive metastore (such as Impala
    and Hue) will be converted to depend on the corresponding Hive service(s),
    and will have their Hive related configuration automatically migrated over.

    - *Support for HBase REST and Thrift servers*. These can be
    added as role instances for HBase services.

    - *Automatic creation of Oozie ShareLib* when adding an Oozie
    service.

    Download: https://ccp.**cloudera****.com/display/SUPPORT/**Cloudera+
    ****Manager+Downloads<https://ccp.cloudera.com/display/SUPPORT/Cloudera+Manager+Downloads>
    Installation Guide: https://ccp.cloudera.**co****
    m/display/FREE45BETADOC/**Cloude****ra+Manager+Free+Edition+**Instal
    ****lation+Guide<https://ccp.cloudera.com/display/FREE45BETADOC/Cloudera+Manager+Free+Edition+Installation+Guide>
    Feedback and support: scm-...@cloudera.**org ****<
    https://groups.google.**com/a/**c**loudera.org/forum/?hl=**en&**
    fromg**roups#!forum/scm-users<https://groups.google.com/a/cloudera.org/forum/?hl=en&fromgroups#!forum/scm-users>
    **>
    Jira: https://issues.cloudera.******org/browse/cm<https://issues.cloudera.org/browse/cm>

    We encourage you to try it out, and look forward to your feedback.

    (Note that this beta release is available only for the Free Edition
    of Cloudera Manager.)

    Cheers,
    bc

    --
    Thanks,
    Darren
  • bc Wong at Dec 19, 2012 at 12:56 am
    The supervisor seems to have gone awry. That means your agents won't run,
    and hence the wizard failure. Try `/etc/init.d/cloudera-scm-agent
    hard_restart'. That resets supervisor, but note that all the running roles
    on the host will also restart.

    Cheers,
    bc
    On Tue, Dec 18, 2012 at 3:20 PM, Benjamin Kim wrote:

    Herman,

    Here is the tail end of the log in question.

    18/Dec/2012 22:09:07 +0000] 5766 MainThread throttling_logger ERROR
    Failed to collect disk stats for device 'md'
    Traceback (most recent call last):
    File "/usr/lib64/cmf/agent/src/cmf/monitor/host/disk_device.py", line
    54, in collect_update
    stat_file = open(self._stat_path)
    IOError: [Errno 2] No such file or directory: '/sys/block/md/stat'
    [18/Dec/2012 22:09:07 +0000] 5766 MainThread disk_devices WARNING Failed
    to add DiskDevice for disk='md'
    [18/Dec/2012 22:09:07 +0000] 5766 MainThread disk_devices WARNING Failed
    to add DiskDevice for partition='md0'
    [18/Dec/2012 22:09:07 +0000] 5766 MainThread __init__ INFO
    Importing metric schema from file
    /usr/lib64/cmf/agent/src/cmf/monitor/schema.json
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Supervised processes will inherit the following environment:
    {'CMF_CONF_DIR': '/etc/cloudera-scm-agent', 'CMF_AGENT_HIVE_DEFAULT_XML':
    '/usr/lib/hive/conf/hive-default.xml', 'CMF_AGENT_MGMT_HOME':
    '/usr/share/cmf', 'CMF_AGENT_CDH_HTTPFS_HOME': '/usr/lib/hadoop-httpfs',
    'PATH': '/sbin:/usr/sbin:/bin:/usr/bin', 'CMF_AGENT_CDH_FLUME_HOME':
    '/usr/lib/flume-ng', 'CMF_AGENT_CDH_HBASE_HOME': '/usr/lib/hbase', 'LANG':
    'en_US.UTF-8', 'TERM': 'vt100', 'CMF_AGENT_CDH_HUE_HOME': '/usr/share/hue',
    'SHLVL': '1', 'CMF_AGENT_TOMCAT_HOME': '/usr/lib/bigtop-tomcat',
    'CMF_AGENT_CDH_PIG_HOME': '/usr/lib/pig', 'CMF_AGENT_CDH_HIVE_HOME':
    '/usr/lib/hive', 'CLOUDERA_MYSQL_CONNECTOR_JAR':
    '/usr/share/java/mysql-connector-java.jar', 'CMF_AGENT_CDH_YARN_HOME':
    '/usr/lib/hadoop-yarn', 'CMF_AGENT_CDH_HADOOP_BIN': '/usr/bin/hadoop',
    'CMF_AGENT_CDH_OOZIE_HOME': '/usr/lib/oozie',
    'CMF_AGENT_CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper',
    'CMF_AGENT_CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CMF_AGENT_JSVC_HOME':
    '/usr/libexec/bigtop-utils', 'CMF_AGENT_CDH_HDFS_HOME':
    '/usr/lib/hadoop-hdfs', 'CMF_AGENT_CDH_MR1_HOME':
    '/usr/lib/hadoop-0.20-mapreduce', 'CMF_AGENT_CDH_MR2_HOME':
    '/usr/lib/hadoop-mapreduce', 'PWD': '/', 'CMF_AGENT_CDH_HADOOP_HOME':
    '/usr/lib/hadoop', 'CMF_AGENT_CDH_IMPALA_HOME': '/usr/lib/impala'}
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Supervised processes will add the following their environment environment:
    {'CDH_MR2_HOME': '/usr/lib/hadoop-mapreduce', 'CMF_PACKAGE_DIR':
    '/usr/lib64/cmf/service', 'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME':
    '/usr/share/cmf', 'CDH_IMPALA_HOME': '/usr/lib/impala', 'CDH_YARN_HOME':
    '/usr/lib/hadoop-yarn', 'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs',
    'CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CM_STATUS_CODES': u'STATUS_NONE
    HDFS_DFS_DIR_NOT_EMPTY', 'CDH_HIVE_HOME': '/usr/lib/hive', 'JSVC_HOME':
    '/usr/libexec/bigtop-utils', 'HIVE_DEFAULT_XML':
    '/usr/lib/hive/conf/hive-default.xml', 'CDH_HBASE_HOME': '/usr/lib/hbase',
    'CDH_OOZIE_HOME': '/usr/lib/oozie', 'CDH_ZOOKEEPER_HOME':
    '/usr/lib/zookeeper', 'CDH_HUE_HOME': '/usr/share/hue',
    'CLOUDERA_MYSQL_CONNECTOR_JAR': '/usr/share/java/mysql-connector-java.jar',
    'CDH_MR1_HOME': '/usr/lib/hadoop-0.20-mapreduce', 'CDH_PIG_HOME':
    '/usr/lib/pig', 'CDH_HTTPFS_HOME': '/usr/lib/hadoop-httpfs',
    'CDH_HADOOP_HOME': '/usr/lib/hadoop', 'TOMCAT_HOME':
    '/usr/lib/bigtop-tomcat', 'CDH_FLUME_HOME': '/usr/lib/flume-ng'}
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Re-using pre-existing directory: /var/run/cloudera-scm-agent/process
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Re-using pre-existing directory: /var/run/cloudera-scm-agent/supervisor
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Re-using pre-existing directory:
    /var/run/cloudera-scm-agent/supervisor/include
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO Failed
    to connect to previous supervisor.
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Deleting stale supervisor include
    /var/run/cloudera-scm-agent/supervisor/include/519-host-inspector.conf.
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Deleting stale supervisor include
    /var/run/cloudera-scm-agent/supervisor/include/738-impala-IMPALAD.conf.
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Deleting stale supervisor include
    /var/run/cloudera-scm-agent/supervisor/include/783-deploy-client-config.conf.
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Deleting stale supervisor include
    /var/run/cloudera-scm-agent/supervisor/include/351-hbase-REGIONSERVER.conf.
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Deleting stale supervisor include
    /var/run/cloudera-scm-agent/supervisor/include/806-hbase-REGIONSERVER.conf.
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Deleting stale supervisor include
    /var/run/cloudera-scm-agent/supervisor/include/726-hdfs-DATANODE.conf.
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Deleting stale supervisor include
    /var/run/cloudera-scm-agent/supervisor/include/754-mapreduce-TASKTRACKER.conf.
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Deleting stale supervisor include
    /var/run/cloudera-scm-agent/supervisor/include/374-mapreduce-TASKTRACKER.conf.
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Deleting stale supervisor include
    /var/run/cloudera-scm-agent/supervisor/include/469-impala-IMPALAD.conf.
    [18/Dec/2012 22:09:09 +0000] 5766 MainThread agent INFO Trying
    to connect to newly launched supervisor (Attempt 1)
    [18/Dec/2012 22:09:09 +0000] 5766 MainThread agent ERROR Failed!
    trying again in 1 second(s)
    Traceback (most recent call last):
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1056, in
    connect_to_new_supervisor
    self.get_supervisor_process_info()
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1078, in
    get_supervisor_process_info
    self.identifier = self.supervisor_client.supervisor.getIdentification()
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
    File
    "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0a12-py2.6.egg/supervisor/xmlrpc.py",
    line 461, in request
    self.connection.request('POST', handler, request_body, self.headers)
    File "/usr/lib64/python2.6/httplib.py", line 914, in request
    self._send_request(method, url, body, headers)
    File "/usr/lib64/python2.6/httplib.py", line 951, in _send_request
    self.endheaders()
    File "/usr/lib64/python2.6/httplib.py", line 908, in endheaders
    self._send_output()
    File "/usr/lib64/python2.6/httplib.py", line 780, in _send_output
    self.send(msg)
    File "/usr/lib64/python2.6/httplib.py", line 739, in send
    self.connect()
    File "/usr/lib64/python2.6/httplib.py", line 720, in connect
    self.timeout)
    File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused
    [18/Dec/2012 22:09:10 +0000] 5766 MainThread agent INFO Trying
    to connect to newly launched supervisor (Attempt 2)
    [18/Dec/2012 22:09:10 +0000] 5766 MainThread agent ERROR Failed!
    trying again in 1 second(s)
    Traceback (most recent call last):
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1056, in
    connect_to_new_supervisor
    self.get_supervisor_process_info()
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1078, in
    get_supervisor_process_info
    self.identifier = self.supervisor_client.supervisor.getIdentification()
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
    File
    "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0a12-py2.6.egg/supervisor/xmlrpc.py",
    line 461, in request
    self.connection.request('POST', handler, request_body, self.headers)
    File "/usr/lib64/python2.6/httplib.py", line 914, in request
    self._send_request(method, url, body, headers)
    File "/usr/lib64/python2.6/httplib.py", line 951, in _send_request
    self.endheaders()
    File "/usr/lib64/python2.6/httplib.py", line 908, in endheaders
    self._send_output()
    File "/usr/lib64/python2.6/httplib.py", line 780, in _send_output
    self.send(msg)
    File "/usr/lib64/python2.6/httplib.py", line 739, in send
    self.connect()
    File "/usr/lib64/python2.6/httplib.py", line 720, in connect
    self.timeout)
    File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused
    [18/Dec/2012 22:09:11 +0000] 5766 MainThread agent INFO Trying
    to connect to newly launched supervisor (Attempt 3)
    [18/Dec/2012 22:09:11 +0000] 5766 MainThread agent ERROR Failed!
    trying again in 1 second(s)
    Traceback (most recent call last):
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1056, in
    connect_to_new_supervisor
    self.get_supervisor_process_info()
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1078, in
    get_supervisor_process_info
    self.identifier = self.supervisor_client.supervisor.getIdentification()
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
    File
    "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0a12-py2.6.egg/supervisor/xmlrpc.py",
    line 461, in request
    self.connection.request('POST', handler, request_body, self.headers)
    File "/usr/lib64/python2.6/httplib.py", line 914, in request
    self._send_request(method, url, body, headers)
    File "/usr/lib64/python2.6/httplib.py", line 951, in _send_request
    self.endheaders()
    File "/usr/lib64/python2.6/httplib.py", line 908, in endheaders
    self._send_output()
    File "/usr/lib64/python2.6/httplib.py", line 780, in _send_output
    self.send(msg)
    File "/usr/lib64/python2.6/httplib.py", line 739, in send
    self.connect()
    File "/usr/lib64/python2.6/httplib.py", line 720, in connect
    self.timeout)
    File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused
    [18/Dec/2012 22:09:12 +0000] 5766 MainThread agent INFO Trying
    to connect to newly launched supervisor (Attempt 4)
    [18/Dec/2012 22:09:12 +0000] 5766 MainThread agent ERROR Failed!
    trying again in 1 second(s)
    Traceback (most recent call last):
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1056, in
    connect_to_new_supervisor
    self.get_supervisor_process_info()
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1078, in
    get_supervisor_process_info
    self.identifier = self.supervisor_client.supervisor.getIdentification()
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
    File
    "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0a12-py2.6.egg/supervisor/xmlrpc.py",
    line 461, in request
    self.connection.request('POST', handler, request_body, self.headers)
    File "/usr/lib64/python2.6/httplib.py", line 914, in request
    self._send_request(method, url, body, headers)
    File "/usr/lib64/python2.6/httplib.py", line 951, in _send_request
    self.endheaders()
    File "/usr/lib64/python2.6/httplib.py", line 908, in endheaders
    self._send_output()
    File "/usr/lib64/python2.6/httplib.py", line 780, in _send_output
    self.send(msg)
    File "/usr/lib64/python2.6/httplib.py", line 739, in send
    self.connect()
    File "/usr/lib64/python2.6/httplib.py", line 720, in connect
    self.timeout)
    File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused
    [18/Dec/2012 22:09:13 +0000] 5766 MainThread agent INFO Trying
    to connect to newly launched supervisor (Attempt 5)
    [18/Dec/2012 22:09:13 +0000] 5766 MainThread agent ERROR Failed!
    trying again in 1 second(s)
    Traceback (most recent call last):
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1056, in
    connect_to_new_supervisor
    self.get_supervisor_process_info()
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1078, in
    get_supervisor_process_info
    self.identifier = self.supervisor_client.supervisor.getIdentification()
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
    File
    "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0a12-py2.6.egg/supervisor/xmlrpc.py",
    line 461, in request
    self.connection.request('POST', handler, request_body, self.headers)
    File "/usr/lib64/python2.6/httplib.py", line 914, in request
    self._send_request(method, url, body, headers)
    File "/usr/lib64/python2.6/httplib.py", line 951, in _send_request
    self.endheaders()
    File "/usr/lib64/python2.6/httplib.py", line 908, in endheaders
    self._send_output()
    File "/usr/lib64/python2.6/httplib.py", line 780, in _send_output
    self.send(msg)
    File "/usr/lib64/python2.6/httplib.py", line 739, in send
    self.connect()
    File "/usr/lib64/python2.6/httplib.py", line 720, in connect
    self.timeout)
    File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused
    [18/Dec/2012 22:09:13 +0000] 5766 MainThread agent ERROR Failed
    to connect to newly launched supervisor. Agent will exit
    [18/Dec/2012 22:09:13 +0000] 5766 MainThread agent INFO
    Stopping agent...

    Thanks for any help.

    Cheers,
    Ben
    On Tuesday, December 18, 2012 3:07:03 PM UTC-8, Herman Chen wrote:

    Benjamin,

    You should be able to uninstall directly from the wizard -- there are
    buttons to Uninstall Failed Hosts and Abort. No the agent ports have not
    changed. To debug why your agent failed, before you uninstall them could
    you check for any errors in their log files? You can find them at
    /var/log/cloudera-scm-agent/**cloudera-scm-agent.log. Thanks.

    Herman

    On Tue, Dec 18, 2012 at 3:02 PM, Benjamin Kim wrote:

    Wong,

    I tried to upgrade, but the agent installs on the cluster had errors.
    The error message said that the heartbeat timed out so it reverted. Have
    the ports changed for the agent? If I can't resolve this, then I will have
    to uninstall. Is there documentation for that?

    Thanks,
    Ben

    On Tuesday, December 18, 2012 11:24:06 AM UTC-8, bc Wong wrote:

    [bcc: cdh-user]
    On Tue, Dec 18, 2012 at 10:19 AM, Benjamin Kim wrote:

    Hi Darren,

    Are you guys thinking about database setup too so we don't have to
    create databases and users for Hue, Hive, Oozie?
    We're planning to do that for Hive first, because it's more distributed
    than Hue & Oozie. But yes, it's a highly desirable feature.

    What is the timeframe for GA? Also, I don't see a way to upgrade my
    current CM installation to 4.5 beta. Is there a way?
    To upgrade, you can follow this doc: https://ccp.cloudera.com/****
    display/ENT4DOC/Upgrade+from+**C**loudera+Manager+4+to+the+**Lates**
    t+Cloudera+Manager+4.0<https://ccp.cloudera.com/display/ENT4DOC/Upgrade+from+Cloudera+Manager+4+to+the+Latest+Cloudera+Manager+4.0>
    Note that the CM45 beta bits are in a different directory. You'll need
    to pick the right distro from http://archive.cloudera.**c**om/cm45beta/<http://archive.cloudera.com/cm45beta/> and
    update your repo file accordingly.

    I can't give a concrete date for GA. It'll happen when we're satisfied
    with the quality of the product. Early next year I hope.

    Cheers,
    bc

    On Tuesday, December 18, 2012 9:41:02 AM UTC-8, Darren Lo wrote:

    Hi Ben,

    For Hive, the Beta has a Gateway role that can be assigned to any
    host. When you deploy Hive client configs, it will update the
    /etc/hive/conf on all hosts that have a Hive role. For GA (but not in
    beta), we will also support HiveServer2. Deploy client configs works on
    hosts with any kind of role, so it will work on both hosts that have a
    Gateway Role or a HiveServer2 Role.

    Hive support in Beta is very basic. Hive schema management is part of
    what we're looking to improve in the GA timeframe, and if you have
    suggestions for features in that arena, please let us know!

    Thanks,
    Darren

    On Tue, Dec 18, 2012 at 9:11 AM, Benjamin Kim wrote:

    Wong,

    I forgot to add. Is Hive Server2 included in this service? Can a
    server be a Gateway node for Hive? I want to know if Hive configurations
    can be pushed to a Gateway/Client node.

    Thanks,
    Ben
    On Monday, December 17, 2012 11:40:54 AM UTC-8, bc Wong wrote:

    On behalf of the Cloudera Manager team, I'm excited to announce the Cloudera
    Manager (Free Edition) 4.5 Beta<https://ccp.cloudera.com/display/SUPPORT/Cloudera+Manager+Downloads>.
    It is the preview of our upcoming feature-packed 4.5 release. I'll
    highlight the following:

    - *Easy CDH upgrade*. You can now manage and upgrade the CDH
    binaries on the cluster using Cloudera Manager, which supports a new
    packaging format called "parcel". Rather than manually upgrading the CDH
    packages across your cluster, you can now download, distribute and activate
    a new CDH version all from within Cloudera Manager.

    Parcels are available for CDH 4.1.2 and onwards. Cloudera
    Manager continues to work with RPM and Debian packages.

    - *Improved configuration management for role instances*. You
    can now group an arbitrary set of roles instances (of the same type, e.g.
    DataNodes) together and manage their configuration as a unit. Called "Role
    Configuration Groups", this feature simplifies configuring groups of roles
    that need to be configured differently, for example when running on nodes
    with heterogeneous hardware.

    - *Improved role to host assignment with Host Templates*. A
    host template defines a set of role configurations that can be applied to
    new hosts. This simplifies the process of adding hosts and instantiating
    the roles that should run on those hosts.

    - *Support for Hive*. Hive is now represented as its own
    service. Other services that depend on the Hive metastore (such as Impala
    and Hue) will be converted to depend on the corresponding Hive service(s),
    and will have their Hive related configuration automatically migrated over.

    - *Support for HBase REST and Thrift servers*. These can be
    added as role instances for HBase services.

    - *Automatic creation of Oozie ShareLib* when adding an Oozie
    service.

    Download: https://ccp.**cloudera******.com/display/SUPPORT/**
    Cloudera+******Manager+Downloads<https://ccp.cloudera.com/display/SUPPORT/Cloudera+Manager+Downloads>
    Installation Guide: https://ccp.cloudera.**co******
    m/display/FREE45BETADOC/**Cloude******ra+Manager+Free+Edition+**
    Instal******lation+Guide<https://ccp.cloudera.com/display/FREE45BETADOC/Cloudera+Manager+Free+Edition+Installation+Guide>
    Feedback and support: scm-...@cloudera.**org ******<
    https://groups.google.**com/a/**c****loudera.org/forum/?hl=**en&**
    fromg****roups#!forum/scm-users<https://groups.google.com/a/cloudera.org/forum/?hl=en&fromgroups#!forum/scm-users>
    **>
    Jira: https://issues.cloudera.********org/browse/cm<https://issues.cloudera.org/browse/cm>

    We encourage you to try it out, and look forward to your feedback.

    (Note that this beta release is available only for the Free Edition
    of Cloudera Manager.)

    Cheers,
    bc

    --
    Thanks,
    Darren
  • Benjamin Kim at Dec 19, 2012 at 1:48 am
    Hi Wong,

    I have decided to rollback. It's too risky to try this out now. I'll give
    it try later when it's a little more stable.

    Thanks,
    Ben
    On Tuesday, December 18, 2012 4:56:08 PM UTC-8, bc Wong wrote:

    The supervisor seems to have gone awry. That means your agents won't run,
    and hence the wizard failure. Try `/etc/init.d/cloudera-scm-agent
    hard_restart'. That resets supervisor, but note that all the running roles
    on the host will also restart.

    Cheers,
    bc

    On Tue, Dec 18, 2012 at 3:20 PM, Benjamin Kim <bbui...@gmail.com<javascript:>
    wrote:
    Herman,

    Here is the tail end of the log in question.

    18/Dec/2012 22:09:07 +0000] 5766 MainThread throttling_logger ERROR
    Failed to collect disk stats for device 'md'
    Traceback (most recent call last):
    File "/usr/lib64/cmf/agent/src/cmf/monitor/host/disk_device.py", line
    54, in collect_update
    stat_file = open(self._stat_path)
    IOError: [Errno 2] No such file or directory: '/sys/block/md/stat'
    [18/Dec/2012 22:09:07 +0000] 5766 MainThread disk_devices WARNING Failed
    to add DiskDevice for disk='md'
    [18/Dec/2012 22:09:07 +0000] 5766 MainThread disk_devices WARNING Failed
    to add DiskDevice for partition='md0'
    [18/Dec/2012 22:09:07 +0000] 5766 MainThread __init__ INFO
    Importing metric schema from file
    /usr/lib64/cmf/agent/src/cmf/monitor/schema.json
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Supervised processes will inherit the following environment:
    {'CMF_CONF_DIR': '/etc/cloudera-scm-agent', 'CMF_AGENT_HIVE_DEFAULT_XML':
    '/usr/lib/hive/conf/hive-default.xml', 'CMF_AGENT_MGMT_HOME':
    '/usr/share/cmf', 'CMF_AGENT_CDH_HTTPFS_HOME': '/usr/lib/hadoop-httpfs',
    'PATH': '/sbin:/usr/sbin:/bin:/usr/bin', 'CMF_AGENT_CDH_FLUME_HOME':
    '/usr/lib/flume-ng', 'CMF_AGENT_CDH_HBASE_HOME': '/usr/lib/hbase', 'LANG':
    'en_US.UTF-8', 'TERM': 'vt100', 'CMF_AGENT_CDH_HUE_HOME': '/usr/share/hue',
    'SHLVL': '1', 'CMF_AGENT_TOMCAT_HOME': '/usr/lib/bigtop-tomcat',
    'CMF_AGENT_CDH_PIG_HOME': '/usr/lib/pig', 'CMF_AGENT_CDH_HIVE_HOME':
    '/usr/lib/hive', 'CLOUDERA_MYSQL_CONNECTOR_JAR':
    '/usr/share/java/mysql-connector-java.jar', 'CMF_AGENT_CDH_YARN_HOME':
    '/usr/lib/hadoop-yarn', 'CMF_AGENT_CDH_HADOOP_BIN': '/usr/bin/hadoop',
    'CMF_AGENT_CDH_OOZIE_HOME': '/usr/lib/oozie',
    'CMF_AGENT_CDH_ZOOKEEPER_HOME': '/usr/lib/zookeeper',
    'CMF_AGENT_CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CMF_AGENT_JSVC_HOME':
    '/usr/libexec/bigtop-utils', 'CMF_AGENT_CDH_HDFS_HOME':
    '/usr/lib/hadoop-hdfs', 'CMF_AGENT_CDH_MR1_HOME':
    '/usr/lib/hadoop-0.20-mapreduce', 'CMF_AGENT_CDH_MR2_HOME':
    '/usr/lib/hadoop-mapreduce', 'PWD': '/', 'CMF_AGENT_CDH_HADOOP_HOME':
    '/usr/lib/hadoop', 'CMF_AGENT_CDH_IMPALA_HOME': '/usr/lib/impala'}
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Supervised processes will add the following their environment environment:
    {'CDH_MR2_HOME': '/usr/lib/hadoop-mapreduce', 'CMF_PACKAGE_DIR':
    '/usr/lib64/cmf/service', 'CDH_HADOOP_BIN': '/usr/bin/hadoop', 'MGMT_HOME':
    '/usr/share/cmf', 'CDH_IMPALA_HOME': '/usr/lib/impala', 'CDH_YARN_HOME':
    '/usr/lib/hadoop-yarn', 'CDH_HDFS_HOME': '/usr/lib/hadoop-hdfs',
    'CDH_HUE_PLUGINS_HOME': '/usr/lib/hadoop', 'CM_STATUS_CODES': u'STATUS_NONE
    HDFS_DFS_DIR_NOT_EMPTY', 'CDH_HIVE_HOME': '/usr/lib/hive', 'JSVC_HOME':
    '/usr/libexec/bigtop-utils', 'HIVE_DEFAULT_XML':
    '/usr/lib/hive/conf/hive-default.xml', 'CDH_HBASE_HOME': '/usr/lib/hbase',
    'CDH_OOZIE_HOME': '/usr/lib/oozie', 'CDH_ZOOKEEPER_HOME':
    '/usr/lib/zookeeper', 'CDH_HUE_HOME': '/usr/share/hue',
    'CLOUDERA_MYSQL_CONNECTOR_JAR': '/usr/share/java/mysql-connector-java.jar',
    'CDH_MR1_HOME': '/usr/lib/hadoop-0.20-mapreduce', 'CDH_PIG_HOME':
    '/usr/lib/pig', 'CDH_HTTPFS_HOME': '/usr/lib/hadoop-httpfs',
    'CDH_HADOOP_HOME': '/usr/lib/hadoop', 'TOMCAT_HOME':
    '/usr/lib/bigtop-tomcat', 'CDH_FLUME_HOME': '/usr/lib/flume-ng'}
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Re-using pre-existing directory: /var/run/cloudera-scm-agent/process
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Re-using pre-existing directory: /var/run/cloudera-scm-agent/supervisor
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Re-using pre-existing directory:
    /var/run/cloudera-scm-agent/supervisor/include
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO Failed
    to connect to previous supervisor.
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Deleting stale supervisor include
    /var/run/cloudera-scm-agent/supervisor/include/519-host-inspector.conf.
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Deleting stale supervisor include
    /var/run/cloudera-scm-agent/supervisor/include/738-impala-IMPALAD.conf.
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Deleting stale supervisor include
    /var/run/cloudera-scm-agent/supervisor/include/783-deploy-client-config.conf.
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Deleting stale supervisor include
    /var/run/cloudera-scm-agent/supervisor/include/351-hbase-REGIONSERVER.conf.
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Deleting stale supervisor include
    /var/run/cloudera-scm-agent/supervisor/include/806-hbase-REGIONSERVER.conf.
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Deleting stale supervisor include
    /var/run/cloudera-scm-agent/supervisor/include/726-hdfs-DATANODE.conf.
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Deleting stale supervisor include
    /var/run/cloudera-scm-agent/supervisor/include/754-mapreduce-TASKTRACKER.conf.
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Deleting stale supervisor include
    /var/run/cloudera-scm-agent/supervisor/include/374-mapreduce-TASKTRACKER.conf.
    [18/Dec/2012 22:09:08 +0000] 5766 MainThread agent INFO
    Deleting stale supervisor include
    /var/run/cloudera-scm-agent/supervisor/include/469-impala-IMPALAD.conf.
    [18/Dec/2012 22:09:09 +0000] 5766 MainThread agent INFO Trying
    to connect to newly launched supervisor (Attempt 1)
    [18/Dec/2012 22:09:09 +0000] 5766 MainThread agent ERROR
    Failed! trying again in 1 second(s)
    Traceback (most recent call last):
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1056, in
    connect_to_new_supervisor
    self.get_supervisor_process_info()
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1078, in
    get_supervisor_process_info
    self.identifier =
    self.supervisor_client.supervisor.getIdentification()
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
    File
    "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0a12-py2.6.egg/supervisor/xmlrpc.py",
    line 461, in request
    self.connection.request('POST', handler, request_body, self.headers)
    File "/usr/lib64/python2.6/httplib.py", line 914, in request
    self._send_request(method, url, body, headers)
    File "/usr/lib64/python2.6/httplib.py", line 951, in _send_request
    self.endheaders()
    File "/usr/lib64/python2.6/httplib.py", line 908, in endheaders
    self._send_output()
    File "/usr/lib64/python2.6/httplib.py", line 780, in _send_output
    self.send(msg)
    File "/usr/lib64/python2.6/httplib.py", line 739, in send
    self.connect()
    File "/usr/lib64/python2.6/httplib.py", line 720, in connect
    self.timeout)
    File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused
    [18/Dec/2012 22:09:10 +0000] 5766 MainThread agent INFO Trying
    to connect to newly launched supervisor (Attempt 2)
    [18/Dec/2012 22:09:10 +0000] 5766 MainThread agent ERROR
    Failed! trying again in 1 second(s)
    Traceback (most recent call last):
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1056, in
    connect_to_new_supervisor
    self.get_supervisor_process_info()
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1078, in
    get_supervisor_process_info
    self.identifier =
    self.supervisor_client.supervisor.getIdentification()
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
    File
    "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0a12-py2.6.egg/supervisor/xmlrpc.py",
    line 461, in request
    self.connection.request('POST', handler, request_body, self.headers)
    File "/usr/lib64/python2.6/httplib.py", line 914, in request
    self._send_request(method, url, body, headers)
    File "/usr/lib64/python2.6/httplib.py", line 951, in _send_request
    self.endheaders()
    File "/usr/lib64/python2.6/httplib.py", line 908, in endheaders
    self._send_output()
    File "/usr/lib64/python2.6/httplib.py", line 780, in _send_output
    self.send(msg)
    File "/usr/lib64/python2.6/httplib.py", line 739, in send
    self.connect()
    File "/usr/lib64/python2.6/httplib.py", line 720, in connect
    self.timeout)
    File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused
    [18/Dec/2012 22:09:11 +0000] 5766 MainThread agent INFO Trying
    to connect to newly launched supervisor (Attempt 3)
    [18/Dec/2012 22:09:11 +0000] 5766 MainThread agent ERROR
    Failed! trying again in 1 second(s)
    Traceback (most recent call last):
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1056, in
    connect_to_new_supervisor
    self.get_supervisor_process_info()
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1078, in
    get_supervisor_process_info
    self.identifier =
    self.supervisor_client.supervisor.getIdentification()
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
    File
    "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0a12-py2.6.egg/supervisor/xmlrpc.py",
    line 461, in request
    self.connection.request('POST', handler, request_body, self.headers)
    File "/usr/lib64/python2.6/httplib.py", line 914, in request
    self._send_request(method, url, body, headers)
    File "/usr/lib64/python2.6/httplib.py", line 951, in _send_request
    self.endheaders()
    File "/usr/lib64/python2.6/httplib.py", line 908, in endheaders
    self._send_output()
    File "/usr/lib64/python2.6/httplib.py", line 780, in _send_output
    self.send(msg)
    File "/usr/lib64/python2.6/httplib.py", line 739, in send
    self.connect()
    File "/usr/lib64/python2.6/httplib.py", line 720, in connect
    self.timeout)
    File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused
    [18/Dec/2012 22:09:12 +0000] 5766 MainThread agent INFO Trying
    to connect to newly launched supervisor (Attempt 4)
    [18/Dec/2012 22:09:12 +0000] 5766 MainThread agent ERROR
    Failed! trying again in 1 second(s)
    Traceback (most recent call last):
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1056, in
    connect_to_new_supervisor
    self.get_supervisor_process_info()
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1078, in
    get_supervisor_process_info
    self.identifier =
    self.supervisor_client.supervisor.getIdentification()
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
    File
    "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0a12-py2.6.egg/supervisor/xmlrpc.py",
    line 461, in request
    self.connection.request('POST', handler, request_body, self.headers)
    File "/usr/lib64/python2.6/httplib.py", line 914, in request
    self._send_request(method, url, body, headers)
    File "/usr/lib64/python2.6/httplib.py", line 951, in _send_request
    self.endheaders()
    File "/usr/lib64/python2.6/httplib.py", line 908, in endheaders
    self._send_output()
    File "/usr/lib64/python2.6/httplib.py", line 780, in _send_output
    self.send(msg)
    File "/usr/lib64/python2.6/httplib.py", line 739, in send
    self.connect()
    File "/usr/lib64/python2.6/httplib.py", line 720, in connect
    self.timeout)
    File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused
    [18/Dec/2012 22:09:13 +0000] 5766 MainThread agent INFO Trying
    to connect to newly launched supervisor (Attempt 5)
    [18/Dec/2012 22:09:13 +0000] 5766 MainThread agent ERROR
    Failed! trying again in 1 second(s)
    Traceback (most recent call last):
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1056, in
    connect_to_new_supervisor
    self.get_supervisor_process_info()
    File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1078, in
    get_supervisor_process_info
    self.identifier =
    self.supervisor_client.supervisor.getIdentification()
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
    File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
    File
    "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/supervisor-3.0a12-py2.6.egg/supervisor/xmlrpc.py",
    line 461, in request
    self.connection.request('POST', handler, request_body, self.headers)
    File "/usr/lib64/python2.6/httplib.py", line 914, in request
    self._send_request(method, url, body, headers)
    File "/usr/lib64/python2.6/httplib.py", line 951, in _send_request
    self.endheaders()
    File "/usr/lib64/python2.6/httplib.py", line 908, in endheaders
    self._send_output()
    File "/usr/lib64/python2.6/httplib.py", line 780, in _send_output
    self.send(msg)
    File "/usr/lib64/python2.6/httplib.py", line 739, in send
    self.connect()
    File "/usr/lib64/python2.6/httplib.py", line 720, in connect
    self.timeout)
    File "/usr/lib64/python2.6/socket.py", line 567, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused
    [18/Dec/2012 22:09:13 +0000] 5766 MainThread agent ERROR Failed
    to connect to newly launched supervisor. Agent will exit
    [18/Dec/2012 22:09:13 +0000] 5766 MainThread agent INFO
    Stopping agent...

    Thanks for any help.

    Cheers,
    Ben
    On Tuesday, December 18, 2012 3:07:03 PM UTC-8, Herman Chen wrote:

    Benjamin,

    You should be able to uninstall directly from the wizard -- there are
    buttons to Uninstall Failed Hosts and Abort. No the agent ports have not
    changed. To debug why your agent failed, before you uninstall them could
    you check for any errors in their log files? You can find them at
    /var/log/cloudera-scm-agent/**cloudera-scm-agent.log. Thanks.

    Herman

    On Tue, Dec 18, 2012 at 3:02 PM, Benjamin Kim wrote:

    Wong,

    I tried to upgrade, but the agent installs on the cluster had errors.
    The error message said that the heartbeat timed out so it reverted. Have
    the ports changed for the agent? If I can't resolve this, then I will have
    to uninstall. Is there documentation for that?

    Thanks,
    Ben

    On Tuesday, December 18, 2012 11:24:06 AM UTC-8, bc Wong wrote:

    [bcc: cdh-user]
    On Tue, Dec 18, 2012 at 10:19 AM, Benjamin Kim wrote:

    Hi Darren,

    Are you guys thinking about database setup too so we don't have to
    create databases and users for Hue, Hive, Oozie?
    We're planning to do that for Hive first, because it's more
    distributed than Hue & Oozie. But yes, it's a highly desirable feature.

    What is the timeframe for GA? Also, I don't see a way to upgrade my
    current CM installation to 4.5 beta. Is there a way?
    To upgrade, you can follow this doc: https://ccp.cloudera.com/****
    display/ENT4DOC/Upgrade+from+**C**loudera+Manager+4+to+the+**Lates**
    t+Cloudera+Manager+4.0<https://ccp.cloudera.com/display/ENT4DOC/Upgrade+from+Cloudera+Manager+4+to+the+Latest+Cloudera+Manager+4.0>
    Note that the CM45 beta bits are in a different directory. You'll need
    to pick the right distro from http://archive.cloudera.**c**
    om/cm45beta/ <http://archive.cloudera.com/cm45beta/> and update your
    repo file accordingly.

    I can't give a concrete date for GA. It'll happen when we're satisfied
    with the quality of the product. Early next year I hope.

    Cheers,
    bc

    On Tuesday, December 18, 2012 9:41:02 AM UTC-8, Darren Lo wrote:

    Hi Ben,

    For Hive, the Beta has a Gateway role that can be assigned to any
    host. When you deploy Hive client configs, it will update the
    /etc/hive/conf on all hosts that have a Hive role. For GA (but not in
    beta), we will also support HiveServer2. Deploy client configs works on
    hosts with any kind of role, so it will work on both hosts that have a
    Gateway Role or a HiveServer2 Role.

    Hive support in Beta is very basic. Hive schema management is part
    of what we're looking to improve in the GA timeframe, and if you have
    suggestions for features in that arena, please let us know!

    Thanks,
    Darren

    On Tue, Dec 18, 2012 at 9:11 AM, Benjamin Kim wrote:

    Wong,

    I forgot to add. Is Hive Server2 included in this service? Can a
    server be a Gateway node for Hive? I want to know if Hive configurations
    can be pushed to a Gateway/Client node.

    Thanks,
    Ben
    On Monday, December 17, 2012 11:40:54 AM UTC-8, bc Wong wrote:

    On behalf of the Cloudera Manager team, I'm excited to announce
    the Cloudera Manager (Free Edition) 4.5 Beta<https://ccp.cloudera.com/display/SUPPORT/Cloudera+Manager+Downloads>.
    It is the preview of our upcoming feature-packed 4.5 release. I'll
    highlight the following:

    - *Easy CDH upgrade*. You can now manage and upgrade the CDH
    binaries on the cluster using Cloudera Manager, which supports a new
    packaging format called "parcel". Rather than manually upgrading the CDH
    packages across your cluster, you can now download, distribute and activate
    a new CDH version all from within Cloudera Manager.

    Parcels are available for CDH 4.1.2 and onwards. Cloudera
    Manager continues to work with RPM and Debian packages.

    - *Improved configuration management for role instances*. You
    can now group an arbitrary set of roles instances (of the same type, e.g.
    DataNodes) together and manage their configuration as a unit. Called "Role
    Configuration Groups", this feature simplifies configuring groups of roles
    that need to be configured differently, for example when running on nodes
    with heterogeneous hardware.

    - *Improved role to host assignment with Host Templates*. A
    host template defines a set of role configurations that can be applied to
    new hosts. This simplifies the process of adding hosts and instantiating
    the roles that should run on those hosts.

    - *Support for Hive*. Hive is now represented as its own
    service. Other services that depend on the Hive metastore (such as Impala
    and Hue) will be converted to depend on the corresponding Hive service(s),
    and will have their Hive related configuration automatically migrated over.

    - *Support for HBase REST and Thrift servers*. These can be
    added as role instances for HBase services.

    - *Automatic creation of Oozie ShareLib* when adding an Oozie
    service.

    Download: https://ccp.**cloudera******.com/display/SUPPORT/**
    Cloudera+******Manager+Downloads<https://ccp.cloudera.com/display/SUPPORT/Cloudera+Manager+Downloads>
    Installation Guide: https://ccp.cloudera.**co******
    m/display/FREE45BETADOC/**Cloude******ra+Manager+Free+Edition+**
    Instal******lation+Guide<https://ccp.cloudera.com/display/FREE45BETADOC/Cloudera+Manager+Free+Edition+Installation+Guide>
    Feedback and support: scm-...@cloudera.**org ******<
    https://groups.google.**com/a/**c****loudera.org/forum/?hl=**en&**
    fromg****roups#!forum/scm-users<https://groups.google.com/a/cloudera.org/forum/?hl=en&fromgroups#!forum/scm-users>
    **>
    Jira: https://issues.cloudera.********org/browse/cm<https://issues.cloudera.org/browse/cm>

    We encourage you to try it out, and look forward to your feedback.

    (Note that this beta release is available only for the Free
    Edition of Cloudera Manager.)

    Cheers,
    bc

    --
    Thanks,
    Darren
  • Benjamin Kim at Dec 18, 2012 at 11:51 pm
    Herman,

    I could not Uninstall Failed Hosts because that button was not there. It
    successfully installed, but the error was caused by no heartbeat. I think I
    need to uninstall manually. Is this correct?

    Thanks,
    Ben
    On Tuesday, December 18, 2012 3:07:03 PM UTC-8, Herman Chen wrote:

    Benjamin,

    You should be able to uninstall directly from the wizard -- there are
    buttons to Uninstall Failed Hosts and Abort. No the agent ports have not
    changed. To debug why your agent failed, before you uninstall them could
    you check for any errors in their log files? You can find them at
    /var/log/cloudera-scm-agent/cloudera-scm-agent.log. Thanks.

    Herman


    On Tue, Dec 18, 2012 at 3:02 PM, Benjamin Kim <bbui...@gmail.com<javascript:>
    wrote:
    Wong,

    I tried to upgrade, but the agent installs on the cluster had errors. The
    error message said that the heartbeat timed out so it reverted. Have the
    ports changed for the agent? If I can't resolve this, then I will have to
    uninstall. Is there documentation for that?

    Thanks,
    Ben

    On Tuesday, December 18, 2012 11:24:06 AM UTC-8, bc Wong wrote:

    [bcc: cdh-user]
    On Tue, Dec 18, 2012 at 10:19 AM, Benjamin Kim wrote:

    Hi Darren,

    Are you guys thinking about database setup too so we don't have to
    create databases and users for Hue, Hive, Oozie?
    We're planning to do that for Hive first, because it's more distributed
    than Hue & Oozie. But yes, it's a highly desirable feature.

    What is the timeframe for GA? Also, I don't see a way to upgrade my
    current CM installation to 4.5 beta. Is there a way?
    To upgrade, you can follow this doc: https://ccp.cloudera.com/**
    display/ENT4DOC/Upgrade+from+**Cloudera+Manager+4+to+the+**
    Latest+Cloudera+Manager+4.0<https://ccp.cloudera.com/display/ENT4DOC/Upgrade+from+Cloudera+Manager+4+to+the+Latest+Cloudera+Manager+4.0>
    Note that the CM45 beta bits are in a different directory. You'll need
    to pick the right distro from http://archive.cloudera.**com/cm45beta/<http://archive.cloudera.com/cm45beta/> and
    update your repo file accordingly.

    I can't give a concrete date for GA. It'll happen when we're satisfied
    with the quality of the product. Early next year I hope.

    Cheers,
    bc

    On Tuesday, December 18, 2012 9:41:02 AM UTC-8, Darren Lo wrote:

    Hi Ben,

    For Hive, the Beta has a Gateway role that can be assigned to any
    host. When you deploy Hive client configs, it will update the
    /etc/hive/conf on all hosts that have a Hive role. For GA (but not in
    beta), we will also support HiveServer2. Deploy client configs works on
    hosts with any kind of role, so it will work on both hosts that have a
    Gateway Role or a HiveServer2 Role.

    Hive support in Beta is very basic. Hive schema management is part of
    what we're looking to improve in the GA timeframe, and if you have
    suggestions for features in that arena, please let us know!

    Thanks,
    Darren

    On Tue, Dec 18, 2012 at 9:11 AM, Benjamin Kim wrote:

    Wong,

    I forgot to add. Is Hive Server2 included in this service? Can a
    server be a Gateway node for Hive? I want to know if Hive configurations
    can be pushed to a Gateway/Client node.

    Thanks,
    Ben
    On Monday, December 17, 2012 11:40:54 AM UTC-8, bc Wong wrote:

    On behalf of the Cloudera Manager team, I'm excited to announce the Cloudera
    Manager (Free Edition) 4.5 Beta<https://ccp.cloudera.com/display/SUPPORT/Cloudera+Manager+Downloads>.
    It is the preview of our upcoming feature-packed 4.5 release. I'll
    highlight the following:

    - *Easy CDH upgrade*. You can now manage and upgrade the CDH
    binaries on the cluster using Cloudera Manager, which supports a new
    packaging format called "parcel". Rather than manually upgrading the CDH
    packages across your cluster, you can now download, distribute and activate
    a new CDH version all from within Cloudera Manager.

    Parcels are available for CDH 4.1.2 and onwards. Cloudera
    Manager continues to work with RPM and Debian packages.

    - *Improved configuration management for role instances*. You
    can now group an arbitrary set of roles instances (of the same type, e.g.
    DataNodes) together and manage their configuration as a unit. Called "Role
    Configuration Groups", this feature simplifies configuring groups of roles
    that need to be configured differently, for example when running on nodes
    with heterogeneous hardware.

    - *Improved role to host assignment with Host Templates*. A host
    template defines a set of role configurations that can be applied to new
    hosts. This simplifies the process of adding hosts and instantiating the
    roles that should run on those hosts.

    - *Support for Hive*. Hive is now represented as its own
    service. Other services that depend on the Hive metastore (such as Impala
    and Hue) will be converted to depend on the corresponding Hive service(s),
    and will have their Hive related configuration automatically migrated over.

    - *Support for HBase REST and Thrift servers*. These can be
    added as role instances for HBase services.

    - *Automatic creation of Oozie ShareLib* when adding an Oozie
    service.

    Download: https://ccp.**cloudera****.com/display/SUPPORT/**Cloudera+
    ****Manager+Downloads<https://ccp.cloudera.com/display/SUPPORT/Cloudera+Manager+Downloads>
    Installation Guide: https://ccp.cloudera.**co****
    m/display/FREE45BETADOC/**Cloude****ra+Manager+Free+Edition+**Instal
    ****lation+Guide<https://ccp.cloudera.com/display/FREE45BETADOC/Cloudera+Manager+Free+Edition+Installation+Guide>
    Feedback and support: scm-...@cloudera.**org ****<
    https://groups.google.**com/a/**c**loudera.org/forum/?hl=**en&**
    fromg**roups#!forum/scm-users<https://groups.google.com/a/cloudera.org/forum/?hl=en&fromgroups#!forum/scm-users>
    **>
    Jira: https://issues.cloudera.******org/browse/cm<https://issues.cloudera.org/browse/cm>

    We encourage you to try it out, and look forward to your feedback.

    (Note that this beta release is available only for the Free Edition
    of Cloudera Manager.)

    Cheers,
    bc

    --
    Thanks,
    Darren
  • Benjamin Kim at Dec 19, 2012 at 9:17 pm
    Hi Darren,

    Can you give me an idea where the Hive schema management is going so far?
    Maybe I can think of something after giving me some hints.

    So, what I gather, the hive-metastore service will be included, but the
    hive-server2 service will not in this iteration. I just want to clarify.

    Thanks,
    Ben
    On Tuesday, December 18, 2012 9:41:02 AM UTC-8, Darren Lo wrote:

    Hi Ben,

    For Hive, the Beta has a Gateway role that can be assigned to any host.
    When you deploy Hive client configs, it will update the /etc/hive/conf on
    all hosts that have a Hive role. For GA (but not in beta), we will also
    support HiveServer2. Deploy client configs works on hosts with any kind of
    role, so it will work on both hosts that have a Gateway Role or a
    HiveServer2 Role.

    Hive support in Beta is very basic. Hive schema management is part of what
    we're looking to improve in the GA timeframe, and if you have suggestions
    for features in that arena, please let us know!

    Thanks,
    Darren


    On Tue, Dec 18, 2012 at 9:11 AM, Benjamin Kim <bbui...@gmail.com<javascript:>
    wrote:
    Wong,

    I forgot to add. Is Hive Server2 included in this service? Can a server
    be a Gateway node for Hive? I want to know if Hive configurations can be
    pushed to a Gateway/Client node.

    Thanks,
    Ben
    On Monday, December 17, 2012 11:40:54 AM UTC-8, bc Wong wrote:

    On behalf of the Cloudera Manager team, I'm excited to announce the Cloudera
    Manager (Free Edition) 4.5 Beta<https://ccp.cloudera.com/display/SUPPORT/Cloudera+Manager+Downloads>.
    It is the preview of our upcoming feature-packed 4.5 release. I'll
    highlight the following:

    - *Easy CDH upgrade*. You can now manage and upgrade the CDH
    binaries on the cluster using Cloudera Manager, which supports a new
    packaging format called "parcel". Rather than manually upgrading the CDH
    packages across your cluster, you can now download, distribute and activate
    a new CDH version all from within Cloudera Manager.

    Parcels are available for CDH 4.1.2 and onwards. Cloudera Manager
    continues to work with RPM and Debian packages.

    - *Improved configuration management for role instances*. You can
    now group an arbitrary set of roles instances (of the same type, e.g.
    DataNodes) together and manage their configuration as a unit. Called "Role
    Configuration Groups", this feature simplifies configuring groups of roles
    that need to be configured differently, for example when running on nodes
    with heterogeneous hardware.

    - *Improved role to host assignment with Host Templates*. A host
    template defines a set of role configurations that can be applied to new
    hosts. This simplifies the process of adding hosts and instantiating the
    roles that should run on those hosts.

    - *Support for Hive*. Hive is now represented as its own service.
    Other services that depend on the Hive metastore (such as Impala and Hue)
    will be converted to depend on the corresponding Hive service(s), and will
    have their Hive related configuration automatically migrated over.

    - *Support for HBase REST and Thrift servers*. These can be added as
    role instances for HBase services.

    - *Automatic creation of Oozie ShareLib* when adding an Oozie
    service.

    Download: https://ccp.**cloudera.com/display/SUPPORT/**
    Cloudera+Manager+Downloads<https://ccp.cloudera.com/display/SUPPORT/Cloudera+Manager+Downloads>
    Installation Guide: https://ccp.cloudera.**com/display/FREE45BETADOC/**
    Cloudera+Manager+Free+Edition+**Installation+Guide<https://ccp.cloudera.com/display/FREE45BETADOC/Cloudera+Manager+Free+Edition+Installation+Guide>
    Feedback and support: scm-...@cloudera.**org <https://groups.google.**
    com/a/cloudera.org/forum/?hl=**en&fromgroups#!forum/scm-users<https://groups.google.com/a/cloudera.org/forum/?hl=en&fromgroups#!forum/scm-users>
    **>
    Jira: https://issues.cloudera.**org/browse/cm<https://issues.cloudera.org/browse/cm>

    We encourage you to try it out, and look forward to your feedback.

    (Note that this beta release is available only for the Free Edition of
    Cloudera Manager.)

    Cheers,
    bc

    --
    Thanks,
    Darren

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedDec 17, '12 at 7:41p
activeDec 19, '12 at 9:17p
posts13
users4
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase