FAQ
Hello,

Configuring a HDFS service with the API shows configuration validation
errors in the CM web console. The UI reports that the nodes are
configured correctly. This happens at the service level.
This is intended as a bug-report, but I'm also interested to know if
this has any impact on the functionality of the cluster.

--
Ioan Eugen Stan / CTO / http://axemblr.com

Search Discussions

  • bc Wong at Aug 27, 2012 at 2:53 pm

    On Mon, Aug 27, 2012 at 3:01 AM, Ioan Eugen Stan wrote:

    Hello,

    Configuring a HDFS service with the API shows configuration validation
    errors in the CM web console. The UI reports that the nodes are
    configured correctly. This happens at the service level.
    This is intended as a bug-report, but I'm also interested to know if
    this has any impact on the functionality of the cluster.
    Hi Eugen,

    There are some mandatory service config fields that needs to be set. When
    you go through the web ui wizard, it guides you and help you set them. When
    you use the API, you need to set those configs. You can use this call:
    http://cloudera.github.com/cm_api/apidocs/v1/path__clusters_-clusterName-_services_-serviceName-_config.html

    Cheers,
    bc
  • Ioan Eugen Stan at Aug 27, 2012 at 3:11 pm
    Hello bc,

    I already tried that without success. Please see my previous post on
    this [1]. The name-node properties can't be configured at the service
    level, although they should (my opinion and the UI's) since there is
    only one name-node per HDFS service.

    Has this changed in the meantime? The CM 4.0.4 release notes doesn't
    mention anything about API changes.

    [1] https://groups.google.com/a/cloudera.org/forum/#!topic/scm-users/w0Xq8lTbSQ8

    On Mon, Aug 27, 2012 at 5:53 PM, bc Wong wrote:
    On Mon, Aug 27, 2012 at 3:01 AM, Ioan Eugen Stan wrote:

    Hello,

    Configuring a HDFS service with the API shows configuration validation
    errors in the CM web console. The UI reports that the nodes are
    configured correctly. This happens at the service level.
    This is intended as a bug-report, but I'm also interested to know if
    this has any impact on the functionality of the cluster.

    Hi Eugen,

    There are some mandatory service config fields that needs to be set. When
    you go through the web ui wizard, it guides you and help you set them. When
    you use the API, you need to set those configs. You can use this call:
    http://cloudera.github.com/cm_api/apidocs/v1/path__clusters_-clusterName-_services_-serviceName-_config.html

    Cheers,
    bc


    --
    Ioan Eugen Stan / CTO / http://axemblr.com
  • bc Wong at Aug 27, 2012 at 3:29 pm

    On Mon, Aug 27, 2012 at 8:11 AM, Ioan Eugen Stan wrote:

    Hello bc,

    I already tried that without success. Please see my previous post on
    this [1]. The name-node properties can't be configured at the service
    level, although they should (my opinion and the UI's) since there is
    only one name-node per HDFS service.

    Has this changed in the meantime? The CM 4.0.4 release notes doesn't
    mention anything about API changes.

    [1]
    https://groups.google.com/a/cloudera.org/forum/#!topic/scm-users/w0Xq8lTbSQ8

    Hi Eugen,

    The reference in [1], as I read it, points out that the API can't install
    bits on the nodes. But it can definitely configure your name/data
    directories.

    Those missing configs are NN, SNN, and DN configs. An HDFS may have more
    than one NN & SNN. That's why they're modeled not as service config, but
    role config. You can set the roletype config template in

    http://cloudera.github.com/cm_api/apidocs/v1/path__clusters_-clusterName-_services_-serviceName-_config.html
    which will be inherited by each role of that type, or set it in every role
    instance in

    http://cloudera.github.com/cm_api/apidocs/v1/path__clusters_-clusterName-_services_-serviceName-_roles_-roleName-_config.html

    Cheers,
    bc

    On Mon, Aug 27, 2012 at 5:53 PM, bc Wong wrote:
    On Mon, Aug 27, 2012 at 3:01 AM, Ioan Eugen Stan wrote:

    Hello,

    Configuring a HDFS service with the API shows configuration validation
    errors in the CM web console. The UI reports that the nodes are
    configured correctly. This happens at the service level.
    This is intended as a bug-report, but I'm also interested to know if
    this has any impact on the functionality of the cluster.

    Hi Eugen,

    There are some mandatory service config fields that needs to be set. When
    you go through the web ui wizard, it guides you and help you set them. When
    you use the API, you need to set those configs. You can use this call:
    http://cloudera.github.com/cm_api/apidocs/v1/path__clusters_-clusterName-_services_-serviceName-_config.html
    Cheers,
    bc


    --
    Ioan Eugen Stan / CTO / http://axemblr.com
  • Ioan Eugen Stan at Aug 27, 2012 at 4:14 pm
    Hello,

    Aha, so I have to configure them at the role level. Will try that.
    Thanks for making things clear. Boy, the configuration process sure
    spans a lot of areas: host, roles, service, cluster.

    Cheers,
    On Mon, Aug 27, 2012 at 6:21 PM, bc Wong wrote:
    On Mon, Aug 27, 2012 at 8:11 AM, Ioan Eugen Stan wrote:

    Hello bc,

    I already tried that without success. Please see my previous post on
    this [1]. The name-node properties can't be configured at the service
    level, although they should (my opinion and the UI's) since there is
    only one name-node per HDFS service.

    Has this changed in the meantime? The CM 4.0.4 release notes doesn't
    mention anything about API changes.

    [1]
    https://groups.google.com/a/cloudera.org/forum/#!topic/scm-users/w0Xq8lTbSQ8

    Hi Eugen,

    The reference in [1], as I read it, points out that the API can't install
    bits on the nodes. But it can definitely configure your name/data
    directories.

    Those missing configs are NN, SNN, and DN configs. An HDFS may have more
    than one NN & SNN. That's why they're modeled not as service config, but
    role config. You can set the roletype config template in

    http://cloudera.github.com/cm_api/apidocs/v1/path__clusters_-clusterName-_services_-serviceName-_config.html
    which will be inherited by each role of that type, or set it in every role
    instance in

    http://cloudera.github.com/cm_api/apidocs/v1/path__clusters_-clusterName-_services_-serviceName-_roles_-roleName-_config.html

    Cheers,
    bc


    On Mon, Aug 27, 2012 at 5:53 PM, bc Wong wrote:
    On Mon, Aug 27, 2012 at 3:01 AM, Ioan Eugen Stan <ieugen@axemblr.com>
    wrote:
    Hello,

    Configuring a HDFS service with the API shows configuration validation
    errors in the CM web console. The UI reports that the nodes are
    configured correctly. This happens at the service level.
    This is intended as a bug-report, but I'm also interested to know if
    this has any impact on the functionality of the cluster.

    Hi Eugen,

    There are some mandatory service config fields that needs to be set.
    When
    you go through the web ui wizard, it guides you and help you set them.
    When
    you use the API, you need to set those configs. You can use this call:

    http://cloudera.github.com/cm_api/apidocs/v1/path__clusters_-clusterName-_services_-serviceName-_config.html

    Cheers,
    bc


    --
    Ioan Eugen Stan / CTO / http://axemblr.com


    --
    Ioan Eugen Stan / CTO / http://axemblr.com

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedAug 27, '12 at 10:01a
activeAug 27, '12 at 4:14p
posts5
users2
websitecloudera.com
irc#hadoop

2 users in discussion

Ioan Eugen Stan: 3 posts bc Wong: 2 posts

People

Translate

site design / logo © 2022 Grokbase