FAQ
Hello all,

I am using rackspace cloud to for CM deployment.

I have created only 4 cloud servers.

I have installed cloudera manager server on one of them successfully.

The problem appears during the hosts inspection phase:

Inspector failed on the following hosts...
<http://5.79.24.112:7180/cmf/express-wizard/inspector?submit=Continue#>

- master1-test: IOException thrown while collecting data from host:
Connection refused
- slave2-test: IOException thrown while collecting data from host:
slave2-test
- master2-test: IOException thrown while collecting data from host:
master2-test
- slave1-test: IOException thrown while collecting data from host:
slave1-test

Inspector ran on 0 hosts.

So I dunno why as for log file:
In master 1-test:
/etc/cloudera-scm-server/db.properties "btw the Cloudera Manager database
is running"
# These are database settings for CM Manager
#
com.cloudera.cmf.db.type=postgresql
com.cloudera.cmf.db.host=localhost:7432
com.cloudera.cmf.db.name=scm
com.cloudera.cmf.db.user=scm
com.cloudera.cmf.db.password=4UVl1LoidI

This the cloudera manager scm-server log file
2012-09-16 16:26:16,738 INFO [main:cmf.Main@166] Starting SCM Server.
Version: 4.0.4 (#79 built by jenkins on 20120821-1040 git:
119fe7452f0c356dcde763a1d22eda9363d24ad4)
2012-09-16 16:26:16,808 INFO [main:enterprise.CommonMain@90] Reading
database properties from /etc/cloudera-scm-server/db.properties
2012-09-16 16:26:16,927 INFO [main:enterprise.CommonMain@123] Statistics
not enabled, c3p0 JMX disabled
2012-09-16 16:26:17,511 INFO [main:common.Version@37] Hibernate Commons
Annotations 3.2.0.Final
2012-09-16 16:26:17,529 INFO [main:cfg.Environment@603] Hibernate
3.6.10.Final
2012-09-16 16:26:17,535 INFO [main:cfg.Environment@636]
hibernate.properties not found
2012-09-16 16:26:17,545 INFO [main:cfg.Environment@814] Bytecode provider
name : javassist
2012-09-16 16:26:17,554 INFO [main:cfg.Environment@695] using JDK 1.4
java.sql.Timestamp handling
2012-09-16 16:26:17,692 INFO [main:ejb.Version@42] Hibernate EntityManager
3.6.10.Final
2012-09-16 16:26:18,209 INFO [main:cfg.Configuration@789] Reading mappings
from resource : cmf.hbm.xml
2012-09-16 16:26:18,754 INFO [main:cfg.HbmBinder@353] Mapping class:
com.cloudera.cmf.model.DbHost -> HOSTS
2012-09-16 16:26:18,871 INFO [main:cfg.HbmBinder@353] Mapping class:
com.cloudera.cmf.model.DbRole -> ROLES
2012-09-16 16:26:18,894 INFO [main:cfg.HbmBinder@353] Mapping class:
com.cloudera.cmf.model.DbService -> SERVICES
2012-09-16 16:26:18,920 INFO [main:cfg.HbmBinder@353] Mapping class:
com.cloudera.cmf.model.DbConfig -> CONFIGS
2012-09-16 16:26:18,922 INFO [main:cfg.HbmBinder@353] Mapping class:
com.cloudera.cmf.model.DbCommand -> COMMANDS
2012-09-16 16:26:18,947 INFO [main:cfg.HbmBinder@353] Mapping class:
com.cloudera.cmf.model.DbProcess -> PROCESSES
2012-09-16 16:26:18,978 INFO [main:cfg.HbmBinder@353] Mapping class:
com.cloudera.cmf.model.DbMasterSettings -> SETTINGS
2012-09-16 16:26:19,021 INFO [main:cfg.HbmBinder@353] Mapping class:
com.cloudera.cmf.model.DbAudit -> AUDITS
2012-09-16 16:26:19,025 INFO [main:cfg.HbmBinder@353] Mapping class:
com.cloudera.cmf.model.DbCredential -> CREDENTIALS
2012-09-16 16:26:19,044 INFO [main:cfg.HbmBinder@353] Mapping class:
com.cloudera.cmf.model.DbUser -> USERS
2012-09-16 16:26:19,046 INFO [main:cfg.HbmBinder@353] Mapping class:
com.cloudera.cmf.model.DbUserRole -> USER_ROLES
2012-09-16 16:26:19,049 INFO [main:cfg.HbmBinder@353] Mapping class:
com.cloudera.cmf.model.DbCluster -> CLUSTERS
2012-09-16 16:26:19,050 INFO [main:cfg.HbmBinder@353] Mapping class:
com.cloudera.cmf.model.DbConfigRevision -> CONFIG_REVISIONS
2012-09-16 16:26:19,052 INFO [main:cfg.HbmBinder@353] Mapping class:
com.cloudera.cmf.model.DbClientConfig -> CLIENT_CONFIGS
2012-09-16 16:26:19,053 INFO [main:cfg.HbmBinder@353] Mapping class:
com.cloudera.cmf.model.DbConfigContainer -> CONFIG_CONTAINERS
2012-09-16 16:26:19,056 INFO [main:cfg.HbmBinder@2508] Mapping collection:
com.cloudera.cmf.model.DbHost.activeCommands -> COMMANDS
2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping collection:
com.cloudera.cmf.model.DbHost.processes -> PROCESSES
2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping collection:
com.cloudera.cmf.model.DbHost.roles -> ROLES
2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping collection:
com.cloudera.cmf.model.DbRole.processes -> PROCESSES
2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping collection:
com.cloudera.cmf.model.DbService.roles -> ROLES
2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping collection:
com.cloudera.cmf.model.DbService.activeCommands -> COMMANDS
2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping collection:
com.cloudera.cmf.model.DbService.configRevisions -> CONFIG_REVISIONS
2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping collection:
com.cloudera.cmf.model.DbService.visibleConfigRevisions -> CONFIG_REVISIONS
2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping collection:
com.cloudera.cmf.model.DbService.clientConfigs -> CLIENT_CONFIGS
2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping collection:
com.cloudera.cmf.model.DbCommand.processes -> PROCESSES
2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping collection:
com.cloudera.cmf.model.DbCommand.children -> COMMANDS
2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping collection:
com.cloudera.cmf.model.DbUser.userRoles -> USER_ROLES
2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping collection:
com.cloudera.cmf.model.DbCluster.activeCommands -> COMMANDS
2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping collection:
com.cloudera.cmf.model.DbConfigRevision.configsForDb -> CONFIGS
2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping collection:
com.cloudera.cmf.model.DbConfigContainer.configRevisions -> CONFIG_REVISIONS
2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping collection:
com.cloudera.cmf.model.DbConfigContainer.visibleConfigRevisions ->
CONFIG_REVISIONS
2012-09-16 16:26:19,071 INFO [main:cfg.Configuration@1676] Hibernate
Validator not found: ignoring
2012-09-16 16:26:19,347 INFO
[main:search.HibernateSearchEventListenerRegister@75] Unable to find
org.hibernate.search.event.FullTextIndexEventListener on the classpath.
Hibernate Search is not enabled.
2012-09-16 16:26:19,394 INFO
[main:connection.ConnectionProviderFactory@173] Initializing connection
provider: org.hibernate.connection.C3P0ConnectionProvider
2012-09-16 16:26:19,395 INFO [main:connection.C3P0ConnectionProvider@103]
C3P0 using driver: org.postgresql.Driver at URL:
jdbc:postgresql://localhost:7432/scm
2012-09-16 16:26:19,395 INFO [main:connection.C3P0ConnectionProvider@104]
Connection properties: {user=scm, password=****, autocommit=true,
release_mode=auto}
2012-09-16 16:26:19,396 INFO [main:connection.C3P0ConnectionProvider@107]
autocommit mode: true
2012-09-16 16:26:19,464 INFO [main:log.MLog@80] MLog clients using log4j
logging.
2012-09-16 16:26:19,691 INFO [main:c3p0.C3P0Registry@204] Initializing
c3p0-0.9.1 [built 16-January-2007 14:46:42; debug? true; trace: 10]
@


so any ideaaaa ????

Search Discussions

  • Philip Zeyliger at Sep 17, 2012 at 6:49 pm
    Hi Dalia,

    In some form or other, the master and agents aren't communicating
    correctly. If you go to the "Hosts" tab in CM (open up the
    http://master:7180/ in a new window) what do you see? If you click on one
    of the hosts, go to the "processes" tab, and look around at stdout/stderr,
    you might see whether or not the host inspection processes ran correctly.

    -- Philip
    On Sun, Sep 16, 2012 at 12:59 PM, Dalia Hassan wrote:

    Hello all,

    I am using rackspace cloud to for CM deployment.

    I have created only 4 cloud servers.

    I have installed cloudera manager server on one of them successfully.

    The problem appears during the hosts inspection phase:

    Inspector failed on the following hosts...
    <http://5.79.24.112:7180/cmf/express-wizard/inspector?submit=Continue#>

    - master1-test: IOException thrown while collecting data from host:
    Connection refused
    - slave2-test: IOException thrown while collecting data from host:
    slave2-test
    - master2-test: IOException thrown while collecting data from host:
    master2-test
    - slave1-test: IOException thrown while collecting data from host:
    slave1-test

    Inspector ran on 0 hosts.

    So I dunno why as for log file:
    In master 1-test:
    /etc/cloudera-scm-server/db.properties "btw the Cloudera Manager database
    is running"
    # These are database settings for CM Manager
    #
    com.cloudera.cmf.db.type=postgresql
    com.cloudera.cmf.db.host=localhost:7432
    com.cloudera.cmf.db.name=scm
    com.cloudera.cmf.db.user=scm
    com.cloudera.cmf.db.password=4UVl1LoidI

    This the cloudera manager scm-server log file
    2012-09-16 16:26:16,738 INFO [main:cmf.Main@166] Starting SCM Server.
    Version: 4.0.4 (#79 built by jenkins on 20120821-1040 git:
    119fe7452f0c356dcde763a1d22eda9363d24ad4)
    2012-09-16 16:26:16,808 INFO [main:enterprise.CommonMain@90] Reading
    database properties from /etc/cloudera-scm-server/db.properties
    2012-09-16 16:26:16,927 INFO [main:enterprise.CommonMain@123] Statistics
    not enabled, c3p0 JMX disabled
    2012-09-16 16:26:17,511 INFO [main:common.Version@37] Hibernate Commons
    Annotations 3.2.0.Final
    2012-09-16 16:26:17,529 INFO [main:cfg.Environment@603] Hibernate
    3.6.10.Final
    2012-09-16 16:26:17,535 INFO [main:cfg.Environment@636]
    hibernate.properties not found
    2012-09-16 16:26:17,545 INFO [main:cfg.Environment@814] Bytecode
    provider name : javassist
    2012-09-16 16:26:17,554 INFO [main:cfg.Environment@695] using JDK 1.4
    java.sql.Timestamp handling
    2012-09-16 16:26:17,692 INFO [main:ejb.Version@42] Hibernate
    EntityManager 3.6.10.Final
    2012-09-16 16:26:18,209 INFO [main:cfg.Configuration@789] Reading
    mappings from resource : cmf.hbm.xml
    2012-09-16 16:26:18,754 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbHost -> HOSTS
    2012-09-16 16:26:18,871 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbRole -> ROLES
    2012-09-16 16:26:18,894 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbService -> SERVICES
    2012-09-16 16:26:18,920 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbConfig -> CONFIGS
    2012-09-16 16:26:18,922 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbCommand -> COMMANDS
    2012-09-16 16:26:18,947 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbProcess -> PROCESSES
    2012-09-16 16:26:18,978 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbMasterSettings -> SETTINGS
    2012-09-16 16:26:19,021 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbAudit -> AUDITS
    2012-09-16 16:26:19,025 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbCredential -> CREDENTIALS
    2012-09-16 16:26:19,044 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbUser -> USERS
    2012-09-16 16:26:19,046 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbUserRole -> USER_ROLES
    2012-09-16 16:26:19,049 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbCluster -> CLUSTERS
    2012-09-16 16:26:19,050 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbConfigRevision -> CONFIG_REVISIONS
    2012-09-16 16:26:19,052 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbClientConfig -> CLIENT_CONFIGS
    2012-09-16 16:26:19,053 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbConfigContainer -> CONFIG_CONTAINERS
    2012-09-16 16:26:19,056 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbHost.activeCommands -> COMMANDS
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbHost.processes -> PROCESSES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbHost.roles -> ROLES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbRole.processes -> PROCESSES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.roles -> ROLES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.activeCommands -> COMMANDS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.configRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.visibleConfigRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.clientConfigs -> CLIENT_CONFIGS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbCommand.processes -> PROCESSES
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbCommand.children -> COMMANDS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbUser.userRoles -> USER_ROLES
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbCluster.activeCommands -> COMMANDS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbConfigRevision.configsForDb -> CONFIGS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbConfigContainer.configRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbConfigContainer.visibleConfigRevisions
    -> CONFIG_REVISIONS
    2012-09-16 16:26:19,071 INFO [main:cfg.Configuration@1676] Hibernate
    Validator not found: ignoring
    2012-09-16 16:26:19,347 INFO
    [main:search.HibernateSearchEventListenerRegister@75] Unable to find
    org.hibernate.search.event.FullTextIndexEventListener on the classpath.
    Hibernate Search is not enabled.
    2012-09-16 16:26:19,394 INFO
    [main:connection.ConnectionProviderFactory@173] Initializing connection
    provider: org.hibernate.connection.C3P0ConnectionProvider
    2012-09-16 16:26:19,395 INFO [main:connection.C3P0ConnectionProvider@103]
    C3P0 using driver: org.postgresql.Driver at URL:
    jdbc:postgresql://localhost:7432/scm
    2012-09-16 16:26:19,395 INFO [main:connection.C3P0ConnectionProvider@104]
    Connection properties: {user=scm, password=****, autocommit=true,
    release_mode=auto}
    2012-09-16 16:26:19,396 INFO [main:connection.C3P0ConnectionProvider@107]
    autocommit mode: true
    2012-09-16 16:26:19,464 INFO [main:log.MLog@80] MLog clients using log4j
    logging.
    2012-09-16 16:26:19,691 INFO [main:c3p0.C3P0Registry@204] Initializing
    c3p0-0.9.1 [built 16-January-2007 14:46:42; debug? true; trace: 10]
    @


    so any ideaaaa ????

  • Dalia Hassan at Sep 17, 2012 at 8:17 pm
    Hii Philip,

    the processes are not working, when I went through the cloudera manager
    server log file I found this error, any help?
    WARN [New I/O client boss #1:cmf.HeartbeatRequester@503] Error requesting
    heartbeat.
    java.net.ConnectException: Connection refused to
    http://master1-test:9000/heartbeat?token=858836ec3e3daf3704341b0c6092f9b0
    at
    com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:95)
    at
    org.jboss.netty.channel.DefaultChannelFuture.notifyListener(DefaultChannelFuture.java:381)
    at
    org.jboss.netty.channel.DefaultChannelFuture.notifyListeners(DefaultChannelFuture.java:372)
    at
    org.jboss.netty.channel.DefaultChannelFuture.setFailure(DefaultChannelFuture.java:334)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:389)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
    at
    org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    at
    org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
    Caused by: java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at
    sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
    .
    .
    .
    INFO [CommandPusher:command.CommandHelpers@198] Command (Inspector, 5) has
    finished on service null.
    2012-09-16 16:46:52,391 INFO [StaleEntityEviction:model.HeartbeatStore@107]
    Reaped 0 process heartbeats and 0 host heartbeats.
    2012-09-16 16:46:52,399 INFO
    [StaleEntityEviction:cmf.StaleEntityEvictionThread@135] Wizard is active,
    not reaping scanners or configurators
    2012-09-16 16:56:52,405 INFO [StaleEntityEviction:model.HeartbeatStore@107]
    Reaped 0 process heartbeats and 0 host heartbeats.

    On Mon, Sep 17, 2012 at 8:49 PM, Philip Zeyliger wrote:

    Hi Dalia,

    In some form or other, the master and agents aren't communicating
    correctly. If you go to the "Hosts" tab in CM (open up the
    http://master:7180/ in a new window) what do you see? If you click on
    one of the hosts, go to the "processes" tab, and look around at
    stdout/stderr, you might see whether or not the host inspection processes
    ran correctly.

    -- Philip

    On Sun, Sep 16, 2012 at 12:59 PM, Dalia Hassan wrote:

    Hello all,

    I am using rackspace cloud to for CM deployment.

    I have created only 4 cloud servers.

    I have installed cloudera manager server on one of them successfully.

    The problem appears during the hosts inspection phase:

    Inspector failed on the following hosts...
    <http://5.79.24.112:7180/cmf/express-wizard/inspector?submit=Continue#>

    - master1-test: IOException thrown while collecting data from host:
    Connection refused
    - slave2-test: IOException thrown while collecting data from host:
    slave2-test
    - master2-test: IOException thrown while collecting data from host:
    master2-test
    - slave1-test: IOException thrown while collecting data from host:
    slave1-test

    Inspector ran on 0 hosts.

    So I dunno why as for log file:
    In master 1-test:
    /etc/cloudera-scm-server/db.properties "btw the Cloudera Manager database
    is running"
    # These are database settings for CM Manager
    #
    com.cloudera.cmf.db.type=postgresql
    com.cloudera.cmf.db.host=localhost:7432
    com.cloudera.cmf.db.name=scm
    com.cloudera.cmf.db.user=scm
    com.cloudera.cmf.db.password=4UVl1LoidI

    This the cloudera manager scm-server log file
    2012-09-16 16:26:16,738 INFO [main:cmf.Main@166] Starting SCM Server.
    Version: 4.0.4 (#79 built by jenkins on 20120821-1040 git:
    119fe7452f0c356dcde763a1d22eda9363d24ad4)
    2012-09-16 16:26:16,808 INFO [main:enterprise.CommonMain@90] Reading
    database properties from /etc/cloudera-scm-server/db.properties
    2012-09-16 16:26:16,927 INFO [main:enterprise.CommonMain@123]
    Statistics not enabled, c3p0 JMX disabled
    2012-09-16 16:26:17,511 INFO [main:common.Version@37] Hibernate Commons
    Annotations 3.2.0.Final
    2012-09-16 16:26:17,529 INFO [main:cfg.Environment@603] Hibernate
    3.6.10.Final
    2012-09-16 16:26:17,535 INFO [main:cfg.Environment@636]
    hibernate.properties not found
    2012-09-16 16:26:17,545 INFO [main:cfg.Environment@814] Bytecode
    provider name : javassist
    2012-09-16 16:26:17,554 INFO [main:cfg.Environment@695] using JDK 1.4
    java.sql.Timestamp handling
    2012-09-16 16:26:17,692 INFO [main:ejb.Version@42] Hibernate
    EntityManager 3.6.10.Final
    2012-09-16 16:26:18,209 INFO [main:cfg.Configuration@789] Reading
    mappings from resource : cmf.hbm.xml
    2012-09-16 16:26:18,754 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbHost -> HOSTS
    2012-09-16 16:26:18,871 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbRole -> ROLES
    2012-09-16 16:26:18,894 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbService -> SERVICES
    2012-09-16 16:26:18,920 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbConfig -> CONFIGS
    2012-09-16 16:26:18,922 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbCommand -> COMMANDS
    2012-09-16 16:26:18,947 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbProcess -> PROCESSES
    2012-09-16 16:26:18,978 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbMasterSettings -> SETTINGS
    2012-09-16 16:26:19,021 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbAudit -> AUDITS
    2012-09-16 16:26:19,025 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbCredential -> CREDENTIALS
    2012-09-16 16:26:19,044 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbUser -> USERS
    2012-09-16 16:26:19,046 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbUserRole -> USER_ROLES
    2012-09-16 16:26:19,049 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbCluster -> CLUSTERS
    2012-09-16 16:26:19,050 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbConfigRevision -> CONFIG_REVISIONS
    2012-09-16 16:26:19,052 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbClientConfig -> CLIENT_CONFIGS
    2012-09-16 16:26:19,053 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbConfigContainer -> CONFIG_CONTAINERS
    2012-09-16 16:26:19,056 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbHost.activeCommands -> COMMANDS
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbHost.processes -> PROCESSES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbHost.roles -> ROLES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbRole.processes -> PROCESSES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.roles -> ROLES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.activeCommands -> COMMANDS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.configRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.visibleConfigRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.clientConfigs -> CLIENT_CONFIGS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbCommand.processes -> PROCESSES
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbCommand.children -> COMMANDS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbUser.userRoles -> USER_ROLES
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbCluster.activeCommands -> COMMANDS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbConfigRevision.configsForDb -> CONFIGS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbConfigContainer.configRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbConfigContainer.visibleConfigRevisions
    -> CONFIG_REVISIONS
    2012-09-16 16:26:19,071 INFO [main:cfg.Configuration@1676] Hibernate
    Validator not found: ignoring
    2012-09-16 16:26:19,347 INFO
    [main:search.HibernateSearchEventListenerRegister@75] Unable to find
    org.hibernate.search.event.FullTextIndexEventListener on the classpath.
    Hibernate Search is not enabled.
    2012-09-16 16:26:19,394 INFO
    [main:connection.ConnectionProviderFactory@173] Initializing connection
    provider: org.hibernate.connection.C3P0ConnectionProvider
    2012-09-16 16:26:19,395 INFO [main:connection.C3P0ConnectionProvider@103]
    C3P0 using driver: org.postgresql.Driver at URL:
    jdbc:postgresql://localhost:7432/scm
    2012-09-16 16:26:19,395 INFO [main:connection.C3P0ConnectionProvider@104]
    Connection properties: {user=scm, password=****, autocommit=true,
    release_mode=auto}
    2012-09-16 16:26:19,396 INFO [main:connection.C3P0ConnectionProvider@107]
    autocommit mode: true
    2012-09-16 16:26:19,464 INFO [main:log.MLog@80] MLog clients using
    log4j logging.
    2012-09-16 16:26:19,691 INFO [main:c3p0.C3P0Registry@204] Initializing
    c3p0-0.9.1 [built 16-January-2007 14:46:42; debug? true; trace: 10]
    @


    so any ideaaaa ????

  • Philip Zeyliger at Sep 17, 2012 at 7:47 pm
    CM is trying to talk to the agent running on that host over port 9000, and
    is failing to do so. I suspect that there are firewalls involved.

    -- Philip
    On Mon, Sep 17, 2012 at 12:45 PM, Dalia Hassan wrote:

    Hii Philip,

    the processes are not working, when I went through the cloudera manager
    server log file I found this error, any help?
    WARN [New I/O client boss #1:cmf.HeartbeatRequester@503] Error requesting
    heartbeat.
    java.net.ConnectException: Connection refused to
    http://master1-test:9000/heartbeat?token=858836ec3e3daf3704341b0c6092f9b0
    at
    com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:95)
    at
    org.jboss.netty.channel.DefaultChannelFuture.notifyListener(DefaultChannelFuture.java:381)
    at
    org.jboss.netty.channel.DefaultChannelFuture.notifyListeners(DefaultChannelFuture.java:372)
    at
    org.jboss.netty.channel.DefaultChannelFuture.setFailure(DefaultChannelFuture.java:334)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:389)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
    at
    org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    at
    org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
    Caused by: java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at
    sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
    .
    .
    .
    INFO [CommandPusher:command.CommandHelpers@198] Command (Inspector, 5)
    has finished on service null.
    2012-09-16 16:46:52,391 INFO [StaleEntityEviction:model.HeartbeatStore@107]
    Reaped 0 process heartbeats and 0 host heartbeats.
    2012-09-16 16:46:52,399 INFO
    [StaleEntityEviction:cmf.StaleEntityEvictionThread@135] Wizard is active,
    not reaping scanners or configurators
    2012-09-16 16:56:52,405 INFO [StaleEntityEviction:model.HeartbeatStore@107]
    Reaped 0 process heartbeats and 0 host heartbeats.


    On Mon, Sep 17, 2012 at 8:49 PM, Philip Zeyliger wrote:

    Hi Dalia,

    In some form or other, the master and agents aren't communicating
    correctly. If you go to the "Hosts" tab in CM (open up the
    http://master:7180/ in a new window) what do you see? If you click on
    one of the hosts, go to the "processes" tab, and look around at
    stdout/stderr, you might see whether or not the host inspection processes
    ran correctly.

    -- Philip

    On Sun, Sep 16, 2012 at 12:59 PM, Dalia Hassan wrote:

    Hello all,

    I am using rackspace cloud to for CM deployment.

    I have created only 4 cloud servers.

    I have installed cloudera manager server on one of them successfully.

    The problem appears during the hosts inspection phase:

    Inspector failed on the following hosts...
    <http://5.79.24.112:7180/cmf/express-wizard/inspector?submit=Continue#>

    - master1-test: IOException thrown while collecting data from host:
    Connection refused
    - slave2-test: IOException thrown while collecting data from host:
    slave2-test
    - master2-test: IOException thrown while collecting data from host:
    master2-test
    - slave1-test: IOException thrown while collecting data from host:
    slave1-test

    Inspector ran on 0 hosts.

    So I dunno why as for log file:
    In master 1-test:
    /etc/cloudera-scm-server/db.properties "btw the Cloudera Manager
    database is running"
    # These are database settings for CM Manager
    #
    com.cloudera.cmf.db.type=postgresql
    com.cloudera.cmf.db.host=localhost:7432
    com.cloudera.cmf.db.name=scm
    com.cloudera.cmf.db.user=scm
    com.cloudera.cmf.db.password=4UVl1LoidI

    This the cloudera manager scm-server log file
    2012-09-16 16:26:16,738 INFO [main:cmf.Main@166] Starting SCM Server.
    Version: 4.0.4 (#79 built by jenkins on 20120821-1040 git:
    119fe7452f0c356dcde763a1d22eda9363d24ad4)
    2012-09-16 16:26:16,808 INFO [main:enterprise.CommonMain@90] Reading
    database properties from /etc/cloudera-scm-server/db.properties
    2012-09-16 16:26:16,927 INFO [main:enterprise.CommonMain@123]
    Statistics not enabled, c3p0 JMX disabled
    2012-09-16 16:26:17,511 INFO [main:common.Version@37] Hibernate
    Commons Annotations 3.2.0.Final
    2012-09-16 16:26:17,529 INFO [main:cfg.Environment@603] Hibernate
    3.6.10.Final
    2012-09-16 16:26:17,535 INFO [main:cfg.Environment@636]
    hibernate.properties not found
    2012-09-16 16:26:17,545 INFO [main:cfg.Environment@814] Bytecode
    provider name : javassist
    2012-09-16 16:26:17,554 INFO [main:cfg.Environment@695] using JDK 1.4
    java.sql.Timestamp handling
    2012-09-16 16:26:17,692 INFO [main:ejb.Version@42] Hibernate
    EntityManager 3.6.10.Final
    2012-09-16 16:26:18,209 INFO [main:cfg.Configuration@789] Reading
    mappings from resource : cmf.hbm.xml
    2012-09-16 16:26:18,754 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbHost -> HOSTS
    2012-09-16 16:26:18,871 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbRole -> ROLES
    2012-09-16 16:26:18,894 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbService -> SERVICES
    2012-09-16 16:26:18,920 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbConfig -> CONFIGS
    2012-09-16 16:26:18,922 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbCommand -> COMMANDS
    2012-09-16 16:26:18,947 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbProcess -> PROCESSES
    2012-09-16 16:26:18,978 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbMasterSettings -> SETTINGS
    2012-09-16 16:26:19,021 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbAudit -> AUDITS
    2012-09-16 16:26:19,025 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbCredential -> CREDENTIALS
    2012-09-16 16:26:19,044 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbUser -> USERS
    2012-09-16 16:26:19,046 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbUserRole -> USER_ROLES
    2012-09-16 16:26:19,049 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbCluster -> CLUSTERS
    2012-09-16 16:26:19,050 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbConfigRevision -> CONFIG_REVISIONS
    2012-09-16 16:26:19,052 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbClientConfig -> CLIENT_CONFIGS
    2012-09-16 16:26:19,053 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbConfigContainer -> CONFIG_CONTAINERS
    2012-09-16 16:26:19,056 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbHost.activeCommands -> COMMANDS
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbHost.processes -> PROCESSES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbHost.roles -> ROLES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbRole.processes -> PROCESSES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.roles -> ROLES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.activeCommands -> COMMANDS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.configRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.visibleConfigRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.clientConfigs -> CLIENT_CONFIGS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbCommand.processes -> PROCESSES
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbCommand.children -> COMMANDS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbUser.userRoles -> USER_ROLES
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbCluster.activeCommands -> COMMANDS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbConfigRevision.configsForDb -> CONFIGS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbConfigContainer.configRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbConfigContainer.visibleConfigRevisions
    -> CONFIG_REVISIONS
    2012-09-16 16:26:19,071 INFO [main:cfg.Configuration@1676] Hibernate
    Validator not found: ignoring
    2012-09-16 16:26:19,347 INFO
    [main:search.HibernateSearchEventListenerRegister@75] Unable to find
    org.hibernate.search.event.FullTextIndexEventListener on the classpath.
    Hibernate Search is not enabled.
    2012-09-16 16:26:19,394 INFO
    [main:connection.ConnectionProviderFactory@173] Initializing connection
    provider: org.hibernate.connection.C3P0ConnectionProvider
    2012-09-16 16:26:19,395 INFO [main:connection.C3P0ConnectionProvider@103]
    C3P0 using driver: org.postgresql.Driver at URL:
    jdbc:postgresql://localhost:7432/scm
    2012-09-16 16:26:19,395 INFO [main:connection.C3P0ConnectionProvider@104]
    Connection properties: {user=scm, password=****, autocommit=true,
    release_mode=auto}
    2012-09-16 16:26:19,396 INFO [main:connection.C3P0ConnectionProvider@107]
    autocommit mode: true
    2012-09-16 16:26:19,464 INFO [main:log.MLog@80] MLog clients using
    log4j logging.
    2012-09-16 16:26:19,691 INFO [main:c3p0.C3P0Registry@204] Initializing
    c3p0-0.9.1 [built 16-January-2007 14:46:42; debug? true; trace: 10]
    @


    so any ideaaaa ????

  • Philip Zeyliger at Sep 17, 2012 at 8:53 pm
    The relevant logs are /var/log/cloudera-scm-agent/*.log; anything
    interesting in there? When you go to the hosts tab, what do you see?

    -- Philip
    On Mon, Sep 17, 2012 at 12:58 PM, Dalia Hassan wrote:

    I already disabled the firewall by typing ufw disable

    $ufw status
    Status: inactive

    So what do u think?

    On Mon, Sep 17, 2012 at 9:46 PM, Philip Zeyliger wrote:

    CM is trying to talk to the agent running on that host over port 9000,
    and is failing to do so. I suspect that there are firewalls involved.

    -- Philip

    On Mon, Sep 17, 2012 at 12:45 PM, Dalia Hassan wrote:

    Hii Philip,

    the processes are not working, when I went through the cloudera manager
    server log file I found this error, any help?
    WARN [New I/O client boss #1:cmf.HeartbeatRequester@503] Error
    requesting heartbeat.
    java.net.ConnectException: Connection refused to
    http://master1-test:9000/heartbeat?token=858836ec3e3daf3704341b0c6092f9b0
    at
    com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:95)
    at
    org.jboss.netty.channel.DefaultChannelFuture.notifyListener(DefaultChannelFuture.java:381)
    at
    org.jboss.netty.channel.DefaultChannelFuture.notifyListeners(DefaultChannelFuture.java:372)
    at
    org.jboss.netty.channel.DefaultChannelFuture.setFailure(DefaultChannelFuture.java:334)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:389)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
    at
    org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    at
    org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
    Caused by: java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at
    sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
    .
    .
    .
    INFO [CommandPusher:command.CommandHelpers@198] Command (Inspector, 5)
    has finished on service null.
    2012-09-16 16:46:52,391 INFO
    [StaleEntityEviction:model.HeartbeatStore@107] Reaped 0 process
    heartbeats and 0 host heartbeats.
    2012-09-16 16:46:52,399 INFO
    [StaleEntityEviction:cmf.StaleEntityEvictionThread@135] Wizard is
    active, not reaping scanners or configurators
    2012-09-16 16:56:52,405 INFO
    [StaleEntityEviction:model.HeartbeatStore@107] Reaped 0 process
    heartbeats and 0 host heartbeats.


    On Mon, Sep 17, 2012 at 8:49 PM, Philip Zeyliger wrote:

    Hi Dalia,

    In some form or other, the master and agents aren't communicating
    correctly. If you go to the "Hosts" tab in CM (open up the
    http://master:7180/ in a new window) what do you see? If you click on
    one of the hosts, go to the "processes" tab, and look around at
    stdout/stderr, you might see whether or not the host inspection processes
    ran correctly.

    -- Philip

    On Sun, Sep 16, 2012 at 12:59 PM, Dalia Hassan wrote:

    Hello all,

    I am using rackspace cloud to for CM deployment.

    I have created only 4 cloud servers.

    I have installed cloudera manager server on one of them successfully.

    The problem appears during the hosts inspection phase:

    Inspector failed on the following hosts...
    <http://5.79.24.112:7180/cmf/express-wizard/inspector?submit=Continue#>

    - master1-test: IOException thrown while collecting data from
    host: Connection refused
    - slave2-test: IOException thrown while collecting data from host:
    slave2-test
    - master2-test: IOException thrown while collecting data from
    host: master2-test
    - slave1-test: IOException thrown while collecting data from host:
    slave1-test

    Inspector ran on 0 hosts.

    So I dunno why as for log file:
    In master 1-test:
    /etc/cloudera-scm-server/db.properties "btw the Cloudera Manager
    database is running"
    # These are database settings for CM Manager
    #
    com.cloudera.cmf.db.type=postgresql
    com.cloudera.cmf.db.host=localhost:7432
    com.cloudera.cmf.db.name=scm
    com.cloudera.cmf.db.user=scm
    com.cloudera.cmf.db.password=4UVl1LoidI

    This the cloudera manager scm-server log file
    2012-09-16 16:26:16,738 INFO [main:cmf.Main@166] Starting SCM
    Server. Version: 4.0.4 (#79 built by jenkins on 20120821-1040 git:
    119fe7452f0c356dcde763a1d22eda9363d24ad4)
    2012-09-16 16:26:16,808 INFO [main:enterprise.CommonMain@90] Reading
    database properties from /etc/cloudera-scm-server/db.properties
    2012-09-16 16:26:16,927 INFO [main:enterprise.CommonMain@123]
    Statistics not enabled, c3p0 JMX disabled
    2012-09-16 16:26:17,511 INFO [main:common.Version@37] Hibernate
    Commons Annotations 3.2.0.Final
    2012-09-16 16:26:17,529 INFO [main:cfg.Environment@603] Hibernate
    3.6.10.Final
    2012-09-16 16:26:17,535 INFO [main:cfg.Environment@636]
    hibernate.properties not found
    2012-09-16 16:26:17,545 INFO [main:cfg.Environment@814] Bytecode
    provider name : javassist
    2012-09-16 16:26:17,554 INFO [main:cfg.Environment@695] using JDK
    1.4 java.sql.Timestamp handling
    2012-09-16 16:26:17,692 INFO [main:ejb.Version@42] Hibernate
    EntityManager 3.6.10.Final
    2012-09-16 16:26:18,209 INFO [main:cfg.Configuration@789] Reading
    mappings from resource : cmf.hbm.xml
    2012-09-16 16:26:18,754 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbHost -> HOSTS
    2012-09-16 16:26:18,871 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbRole -> ROLES
    2012-09-16 16:26:18,894 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbService -> SERVICES
    2012-09-16 16:26:18,920 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbConfig -> CONFIGS
    2012-09-16 16:26:18,922 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbCommand -> COMMANDS
    2012-09-16 16:26:18,947 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbProcess -> PROCESSES
    2012-09-16 16:26:18,978 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbMasterSettings -> SETTINGS
    2012-09-16 16:26:19,021 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbAudit -> AUDITS
    2012-09-16 16:26:19,025 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbCredential -> CREDENTIALS
    2012-09-16 16:26:19,044 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbUser -> USERS
    2012-09-16 16:26:19,046 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbUserRole -> USER_ROLES
    2012-09-16 16:26:19,049 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbCluster -> CLUSTERS
    2012-09-16 16:26:19,050 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbConfigRevision -> CONFIG_REVISIONS
    2012-09-16 16:26:19,052 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbClientConfig -> CLIENT_CONFIGS
    2012-09-16 16:26:19,053 INFO [main:cfg.HbmBinder@353] Mapping class:
    com.cloudera.cmf.model.DbConfigContainer -> CONFIG_CONTAINERS
    2012-09-16 16:26:19,056 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbHost.activeCommands -> COMMANDS
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbHost.processes -> PROCESSES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbHost.roles -> ROLES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbRole.processes -> PROCESSES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.roles -> ROLES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.activeCommands -> COMMANDS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.configRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.visibleConfigRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.clientConfigs -> CLIENT_CONFIGS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbCommand.processes -> PROCESSES
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbCommand.children -> COMMANDS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbUser.userRoles -> USER_ROLES
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbCluster.activeCommands -> COMMANDS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbConfigRevision.configsForDb -> CONFIGS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbConfigContainer.configRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbConfigContainer.visibleConfigRevisions
    -> CONFIG_REVISIONS
    2012-09-16 16:26:19,071 INFO [main:cfg.Configuration@1676] Hibernate
    Validator not found: ignoring
    2012-09-16 16:26:19,347 INFO
    [main:search.HibernateSearchEventListenerRegister@75] Unable to find
    org.hibernate.search.event.FullTextIndexEventListener on the classpath.
    Hibernate Search is not enabled.
    2012-09-16 16:26:19,394 INFO
    [main:connection.ConnectionProviderFactory@173] Initializing
    connection provider: org.hibernate.connection.C3P0ConnectionProvider
    2012-09-16 16:26:19,395 INFO
    [main:connection.C3P0ConnectionProvider@103] C3P0 using driver:
    org.postgresql.Driver at URL: jdbc:postgresql://localhost:7432/scm
    2012-09-16 16:26:19,395 INFO
    [main:connection.C3P0ConnectionProvider@104] Connection properties:
    {user=scm, password=****, autocommit=true, release_mode=auto}
    2012-09-16 16:26:19,396 INFO
    [main:connection.C3P0ConnectionProvider@107] autocommit mode: true
    2012-09-16 16:26:19,464 INFO [main:log.MLog@80] MLog clients using
    log4j logging.
    2012-09-16 16:26:19,691 INFO [main:c3p0.C3P0Registry@204]
    Initializing c3p0-0.9.1 [built 16-January-2007 14:46:42; debug? true;
    trace: 10]
    @


    so any ideaaaa ????

  • Philip Zeyliger at Sep 17, 2012 at 9:08 pm
    Please continue to reply-all so that other list members can help.

    Your agent and your server aren't heartbeating. You should manually see if
    you can connect from one to another, on the given ports.

    -- Philip
    On Mon, Sep 17, 2012 at 2:00 PM, Dalia Hassan wrote:

    in agent log:
    [17/Sep/2012 00:03:38 +0000] 642 MainThread agent ERROR
    Heartbeating to 10.177.196.147:7182 failed.
    Traceback (most recent call last):
    File "/usr/lib/cmf/agent/src/cmf/agent.py", line 558, in send_heartbeat
    self.master_port)
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py",
    line 471, in __init__
    self.conn.connect()
    File "/usr/lib/python2.6/httplib.py", line 716, in connect
    self.timeout)
    File "/usr/lib/python2.6/socket.py", line 514, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused

    /etc/hosts

    127.0.0.1 localhost
    2a00:1a48:7803:0107:2d9a:7ff0:ff08:0ef3 slave2-test

    10.177.196.5 slave2-test
    5.79.25.31 slave2-test







    On Mon, Sep 17, 2012 at 10:53 PM, Philip Zeyliger wrote:

    The relevant logs are /var/log/cloudera-scm-agent/*.log; anything
    interesting in there? When you go to the hosts tab, what do you see?

    -- Philip

    On Mon, Sep 17, 2012 at 12:58 PM, Dalia Hassan wrote:

    I already disabled the firewall by typing ufw disable

    $ufw status
    Status: inactive

    So what do u think?

    On Mon, Sep 17, 2012 at 9:46 PM, Philip Zeyliger wrote:

    CM is trying to talk to the agent running on that host over port 9000,
    and is failing to do so. I suspect that there are firewalls involved.

    -- Philip

    On Mon, Sep 17, 2012 at 12:45 PM, Dalia Hassan wrote:

    Hii Philip,

    the processes are not working, when I went through the cloudera
    manager server log file I found this error, any help?
    WARN [New I/O client boss #1:cmf.HeartbeatRequester@503] Error
    requesting heartbeat.
    java.net.ConnectException: Connection refused to
    http://master1-test:9000/heartbeat?token=858836ec3e3daf3704341b0c6092f9b0
    at
    com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:95)
    at
    org.jboss.netty.channel.DefaultChannelFuture.notifyListener(DefaultChannelFuture.java:381)
    at
    org.jboss.netty.channel.DefaultChannelFuture.notifyListeners(DefaultChannelFuture.java:372)
    at
    org.jboss.netty.channel.DefaultChannelFuture.setFailure(DefaultChannelFuture.java:334)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:389)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
    at
    org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    at
    org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
    Caused by: java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at
    sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
    .
    .
    .
    INFO [CommandPusher:command.CommandHelpers@198] Command (Inspector,
    5) has finished on service null.
    2012-09-16 16:46:52,391 INFO
    [StaleEntityEviction:model.HeartbeatStore@107] Reaped 0 process
    heartbeats and 0 host heartbeats.
    2012-09-16 16:46:52,399 INFO
    [StaleEntityEviction:cmf.StaleEntityEvictionThread@135] Wizard is
    active, not reaping scanners or configurators
    2012-09-16 16:56:52,405 INFO
    [StaleEntityEviction:model.HeartbeatStore@107] Reaped 0 process
    heartbeats and 0 host heartbeats.


    On Mon, Sep 17, 2012 at 8:49 PM, Philip Zeyliger wrote:

    Hi Dalia,

    In some form or other, the master and agents aren't communicating
    correctly. If you go to the "Hosts" tab in CM (open up the
    http://master:7180/ in a new window) what do you see? If you click
    on one of the hosts, go to the "processes" tab, and look around at
    stdout/stderr, you might see whether or not the host inspection processes
    ran correctly.

    -- Philip


    On Sun, Sep 16, 2012 at 12:59 PM, Dalia Hassan <
    daliahassan0@gmail.com> wrote:
    Hello all,

    I am using rackspace cloud to for CM deployment.

    I have created only 4 cloud servers.

    I have installed cloudera manager server on one of them successfully.

    The problem appears during the hosts inspection phase:

    Inspector failed on the following hosts...
    <http://5.79.24.112:7180/cmf/express-wizard/inspector?submit=Continue#>

    - master1-test: IOException thrown while collecting data from
    host: Connection refused
    - slave2-test: IOException thrown while collecting data from
    host: slave2-test
    - master2-test: IOException thrown while collecting data from
    host: master2-test
    - slave1-test: IOException thrown while collecting data from
    host: slave1-test

    Inspector ran on 0 hosts.

    So I dunno why as for log file:
    In master 1-test:
    /etc/cloudera-scm-server/db.properties "btw the Cloudera Manager
    database is running"
    # These are database settings for CM Manager
    #
    com.cloudera.cmf.db.type=postgresql
    com.cloudera.cmf.db.host=localhost:7432
    com.cloudera.cmf.db.name=scm
    com.cloudera.cmf.db.user=scm
    com.cloudera.cmf.db.password=4UVl1LoidI

    This the cloudera manager scm-server log file
    2012-09-16 16:26:16,738 INFO [main:cmf.Main@166] Starting SCM
    Server. Version: 4.0.4 (#79 built by jenkins on 20120821-1040 git:
    119fe7452f0c356dcde763a1d22eda9363d24ad4)
    2012-09-16 16:26:16,808 INFO [main:enterprise.CommonMain@90]
    Reading database properties from /etc/cloudera-scm-server/db.properties
    2012-09-16 16:26:16,927 INFO [main:enterprise.CommonMain@123]
    Statistics not enabled, c3p0 JMX disabled
    2012-09-16 16:26:17,511 INFO [main:common.Version@37] Hibernate
    Commons Annotations 3.2.0.Final
    2012-09-16 16:26:17,529 INFO [main:cfg.Environment@603] Hibernate
    3.6.10.Final
    2012-09-16 16:26:17,535 INFO [main:cfg.Environment@636]
    hibernate.properties not found
    2012-09-16 16:26:17,545 INFO [main:cfg.Environment@814] Bytecode
    provider name : javassist
    2012-09-16 16:26:17,554 INFO [main:cfg.Environment@695] using JDK
    1.4 java.sql.Timestamp handling
    2012-09-16 16:26:17,692 INFO [main:ejb.Version@42] Hibernate
    EntityManager 3.6.10.Final
    2012-09-16 16:26:18,209 INFO [main:cfg.Configuration@789] Reading
    mappings from resource : cmf.hbm.xml
    2012-09-16 16:26:18,754 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbHost -> HOSTS
    2012-09-16 16:26:18,871 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbRole -> ROLES
    2012-09-16 16:26:18,894 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbService -> SERVICES
    2012-09-16 16:26:18,920 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbConfig -> CONFIGS
    2012-09-16 16:26:18,922 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbCommand -> COMMANDS
    2012-09-16 16:26:18,947 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbProcess -> PROCESSES
    2012-09-16 16:26:18,978 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbMasterSettings -> SETTINGS
    2012-09-16 16:26:19,021 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbAudit -> AUDITS
    2012-09-16 16:26:19,025 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbCredential -> CREDENTIALS
    2012-09-16 16:26:19,044 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbUser -> USERS
    2012-09-16 16:26:19,046 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbUserRole -> USER_ROLES
    2012-09-16 16:26:19,049 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbCluster -> CLUSTERS
    2012-09-16 16:26:19,050 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbConfigRevision -> CONFIG_REVISIONS
    2012-09-16 16:26:19,052 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbClientConfig -> CLIENT_CONFIGS
    2012-09-16 16:26:19,053 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbConfigContainer -> CONFIG_CONTAINERS
    2012-09-16 16:26:19,056 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbHost.activeCommands -> COMMANDS
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbHost.processes -> PROCESSES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbHost.roles -> ROLES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbRole.processes -> PROCESSES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.roles -> ROLES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.activeCommands -> COMMANDS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.configRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.visibleConfigRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.clientConfigs -> CLIENT_CONFIGS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbCommand.processes -> PROCESSES
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbCommand.children -> COMMANDS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbUser.userRoles -> USER_ROLES
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbCluster.activeCommands -> COMMANDS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbConfigRevision.configsForDb -> CONFIGS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbConfigContainer.configRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbConfigContainer.visibleConfigRevisions
    -> CONFIG_REVISIONS
    2012-09-16 16:26:19,071 INFO [main:cfg.Configuration@1676]
    Hibernate Validator not found: ignoring
    2012-09-16 16:26:19,347 INFO
    [main:search.HibernateSearchEventListenerRegister@75] Unable to
    find org.hibernate.search.event.FullTextIndexEventListener on the
    classpath. Hibernate Search is not enabled.
    2012-09-16 16:26:19,394 INFO
    [main:connection.ConnectionProviderFactory@173] Initializing
    connection provider: org.hibernate.connection.C3P0ConnectionProvider
    2012-09-16 16:26:19,395 INFO
    [main:connection.C3P0ConnectionProvider@103] C3P0 using driver:
    org.postgresql.Driver at URL: jdbc:postgresql://localhost:7432/scm
    2012-09-16 16:26:19,395 INFO
    [main:connection.C3P0ConnectionProvider@104] Connection properties:
    {user=scm, password=****, autocommit=true, release_mode=auto}
    2012-09-16 16:26:19,396 INFO
    [main:connection.C3P0ConnectionProvider@107] autocommit mode: true
    2012-09-16 16:26:19,464 INFO [main:log.MLog@80] MLog clients using
    log4j logging.
    2012-09-16 16:26:19,691 INFO [main:c3p0.C3P0Registry@204]
    Initializing c3p0-0.9.1 [built 16-January-2007 14:46:42; debug? true;
    trace: 10]
    @


    so any ideaaaa ????

  • Dalia Hassan at Sep 17, 2012 at 9:34 pm
    how??
    On Mon, Sep 17, 2012 at 11:08 PM, Philip Zeyliger wrote:

    Please continue to reply-all so that other list members can help.

    Your agent and your server aren't heartbeating. You should manually see
    if you can connect from one to another, on the given ports.

    -- Philip

    On Mon, Sep 17, 2012 at 2:00 PM, Dalia Hassan wrote:

    in agent log:
    [17/Sep/2012 00:03:38 +0000] 642 MainThread agent ERROR
    Heartbeating to 10.177.196.147:7182 failed.
    Traceback (most recent call last):
    File "/usr/lib/cmf/agent/src/cmf/agent.py", line 558, in send_heartbeat
    self.master_port)
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py",
    line 471, in __init__
    self.conn.connect()
    File "/usr/lib/python2.6/httplib.py", line 716, in connect
    self.timeout)
    File "/usr/lib/python2.6/socket.py", line 514, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused

    /etc/hosts

    127.0.0.1 localhost
    2a00:1a48:7803:0107:2d9a:7ff0:ff08:0ef3 slave2-test

    10.177.196.5 slave2-test
    5.79.25.31 slave2-test







    On Mon, Sep 17, 2012 at 10:53 PM, Philip Zeyliger wrote:

    The relevant logs are /var/log/cloudera-scm-agent/*.log; anything
    interesting in there? When you go to the hosts tab, what do you see?

    -- Philip

    On Mon, Sep 17, 2012 at 12:58 PM, Dalia Hassan wrote:

    I already disabled the firewall by typing ufw disable

    $ufw status
    Status: inactive

    So what do u think?

    On Mon, Sep 17, 2012 at 9:46 PM, Philip Zeyliger wrote:

    CM is trying to talk to the agent running on that host over port 9000,
    and is failing to do so. I suspect that there are firewalls involved.

    -- Philip


    On Mon, Sep 17, 2012 at 12:45 PM, Dalia Hassan <daliahassan0@gmail.com
    wrote:
    Hii Philip,

    the processes are not working, when I went through the cloudera
    manager server log file I found this error, any help?
    WARN [New I/O client boss #1:cmf.HeartbeatRequester@503] Error
    requesting heartbeat.
    java.net.ConnectException: Connection refused to
    http://master1-test:9000/heartbeat?token=858836ec3e3daf3704341b0c6092f9b0
    at
    com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:95)
    at
    org.jboss.netty.channel.DefaultChannelFuture.notifyListener(DefaultChannelFuture.java:381)
    at
    org.jboss.netty.channel.DefaultChannelFuture.notifyListeners(DefaultChannelFuture.java:372)
    at
    org.jboss.netty.channel.DefaultChannelFuture.setFailure(DefaultChannelFuture.java:334)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:389)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
    at
    org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    at
    org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
    Caused by: java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at
    sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
    .
    .
    .
    INFO [CommandPusher:command.CommandHelpers@198] Command (Inspector,
    5) has finished on service null.
    2012-09-16 16:46:52,391 INFO
    [StaleEntityEviction:model.HeartbeatStore@107] Reaped 0 process
    heartbeats and 0 host heartbeats.
    2012-09-16 16:46:52,399 INFO
    [StaleEntityEviction:cmf.StaleEntityEvictionThread@135] Wizard is
    active, not reaping scanners or configurators
    2012-09-16 16:56:52,405 INFO
    [StaleEntityEviction:model.HeartbeatStore@107] Reaped 0 process
    heartbeats and 0 host heartbeats.



    On Mon, Sep 17, 2012 at 8:49 PM, Philip Zeyliger <philip@cloudera.com
    wrote:
    Hi Dalia,

    In some form or other, the master and agents aren't communicating
    correctly. If you go to the "Hosts" tab in CM (open up the
    http://master:7180/ in a new window) what do you see? If you click
    on one of the hosts, go to the "processes" tab, and look around at
    stdout/stderr, you might see whether or not the host inspection processes
    ran correctly.

    -- Philip


    On Sun, Sep 16, 2012 at 12:59 PM, Dalia Hassan <
    daliahassan0@gmail.com> wrote:
    Hello all,

    I am using rackspace cloud to for CM deployment.

    I have created only 4 cloud servers.

    I have installed cloudera manager server on one of them
    successfully.

    The problem appears during the hosts inspection phase:

    Inspector failed on the following hosts...
    <http://5.79.24.112:7180/cmf/express-wizard/inspector?submit=Continue#>

    - master1-test: IOException thrown while collecting data from
    host: Connection refused
    - slave2-test: IOException thrown while collecting data from
    host: slave2-test
    - master2-test: IOException thrown while collecting data from
    host: master2-test
    - slave1-test: IOException thrown while collecting data from
    host: slave1-test

    Inspector ran on 0 hosts.

    So I dunno why as for log file:
    In master 1-test:
    /etc/cloudera-scm-server/db.properties "btw the Cloudera Manager
    database is running"
    # These are database settings for CM Manager
    #
    com.cloudera.cmf.db.type=postgresql
    com.cloudera.cmf.db.host=localhost:7432
    com.cloudera.cmf.db.name=scm
    com.cloudera.cmf.db.user=scm
    com.cloudera.cmf.db.password=4UVl1LoidI

    This the cloudera manager scm-server log file
    2012-09-16 16:26:16,738 INFO [main:cmf.Main@166] Starting SCM
    Server. Version: 4.0.4 (#79 built by jenkins on 20120821-1040 git:
    119fe7452f0c356dcde763a1d22eda9363d24ad4)
    2012-09-16 16:26:16,808 INFO [main:enterprise.CommonMain@90]
    Reading database properties from /etc/cloudera-scm-server/db.properties
    2012-09-16 16:26:16,927 INFO [main:enterprise.CommonMain@123]
    Statistics not enabled, c3p0 JMX disabled
    2012-09-16 16:26:17,511 INFO [main:common.Version@37] Hibernate
    Commons Annotations 3.2.0.Final
    2012-09-16 16:26:17,529 INFO [main:cfg.Environment@603] Hibernate
    3.6.10.Final
    2012-09-16 16:26:17,535 INFO [main:cfg.Environment@636]
    hibernate.properties not found
    2012-09-16 16:26:17,545 INFO [main:cfg.Environment@814] Bytecode
    provider name : javassist
    2012-09-16 16:26:17,554 INFO [main:cfg.Environment@695] using JDK
    1.4 java.sql.Timestamp handling
    2012-09-16 16:26:17,692 INFO [main:ejb.Version@42] Hibernate
    EntityManager 3.6.10.Final
    2012-09-16 16:26:18,209 INFO [main:cfg.Configuration@789] Reading
    mappings from resource : cmf.hbm.xml
    2012-09-16 16:26:18,754 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbHost -> HOSTS
    2012-09-16 16:26:18,871 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbRole -> ROLES
    2012-09-16 16:26:18,894 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbService -> SERVICES
    2012-09-16 16:26:18,920 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbConfig -> CONFIGS
    2012-09-16 16:26:18,922 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbCommand -> COMMANDS
    2012-09-16 16:26:18,947 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbProcess -> PROCESSES
    2012-09-16 16:26:18,978 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbMasterSettings -> SETTINGS
    2012-09-16 16:26:19,021 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbAudit -> AUDITS
    2012-09-16 16:26:19,025 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbCredential -> CREDENTIALS
    2012-09-16 16:26:19,044 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbUser -> USERS
    2012-09-16 16:26:19,046 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbUserRole -> USER_ROLES
    2012-09-16 16:26:19,049 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbCluster -> CLUSTERS
    2012-09-16 16:26:19,050 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbConfigRevision -> CONFIG_REVISIONS
    2012-09-16 16:26:19,052 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbClientConfig -> CLIENT_CONFIGS
    2012-09-16 16:26:19,053 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbConfigContainer -> CONFIG_CONTAINERS
    2012-09-16 16:26:19,056 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbHost.activeCommands -> COMMANDS
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbHost.processes -> PROCESSES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbHost.roles -> ROLES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbRole.processes -> PROCESSES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.roles -> ROLES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.activeCommands -> COMMANDS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.configRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.visibleConfigRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.clientConfigs -> CLIENT_CONFIGS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbCommand.processes -> PROCESSES
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbCommand.children -> COMMANDS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbUser.userRoles -> USER_ROLES
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbCluster.activeCommands -> COMMANDS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbConfigRevision.configsForDb -> CONFIGS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbConfigContainer.configRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbConfigContainer.visibleConfigRevisions
    -> CONFIG_REVISIONS
    2012-09-16 16:26:19,071 INFO [main:cfg.Configuration@1676]
    Hibernate Validator not found: ignoring
    2012-09-16 16:26:19,347 INFO
    [main:search.HibernateSearchEventListenerRegister@75] Unable to
    find org.hibernate.search.event.FullTextIndexEventListener on the
    classpath. Hibernate Search is not enabled.
    2012-09-16 16:26:19,394 INFO
    [main:connection.ConnectionProviderFactory@173] Initializing
    connection provider: org.hibernate.connection.C3P0ConnectionProvider
    2012-09-16 16:26:19,395 INFO
    [main:connection.C3P0ConnectionProvider@103] C3P0 using driver:
    org.postgresql.Driver at URL: jdbc:postgresql://localhost:7432/scm
    2012-09-16 16:26:19,395 INFO
    [main:connection.C3P0ConnectionProvider@104] Connection
    properties: {user=scm, password=****, autocommit=true, release_mode=auto}
    2012-09-16 16:26:19,396 INFO
    [main:connection.C3P0ConnectionProvider@107] autocommit mode: true
    2012-09-16 16:26:19,464 INFO [main:log.MLog@80] MLog clients
    using log4j logging.
    2012-09-16 16:26:19,691 INFO [main:c3p0.C3P0Registry@204]
    Initializing c3p0-0.9.1 [built 16-January-2007 14:46:42; debug? true;
    trace: 10]
    @


    so any ideaaaa ????

  • Dalia Hassan at Sep 17, 2012 at 9:11 pm
    in agent log:
    [17/Sep/2012 00:03:38 +0000] 642 MainThread agent ERROR
    Heartbeating to 10.177.196.147:7182 failed.
    Traceback (most recent call last):
    File "/usr/lib/cmf/agent/src/cmf/
    agent.py", line 558, in send_heartbeat
    self.master_port)
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py",
    line 471, in __init__
    self.conn.connect()
    File "/usr/lib/python2.6/httplib.py", line 716, in connect
    self.timeout)
    File "/usr/lib/python2.6/socket.py", line 514, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused

    /etc/hosts

    127.0.0.1 localhost
    2a00:1a48:7803:0107:2d9a:7ff0:ff08:0ef3 slave2-test

    10.177.196.5 slave2-test
    5.79.25.31 slave2-test

    On Mon, Sep 17, 2012 at 11:10 PM, Dalia Hassan wrote:

    how??

    On Mon, Sep 17, 2012 at 11:08 PM, Philip Zeyliger wrote:

    Please continue to reply-all so that other list members can help.

    Your agent and your server aren't heartbeating. You should manually see
    if you can connect from one to another, on the given ports.

    -- Philip

    On Mon, Sep 17, 2012 at 2:00 PM, Dalia Hassan wrote:

    in agent log:
    [17/Sep/2012 00:03:38 +0000] 642 MainThread agent ERROR
    Heartbeating to 10.177.196.147:7182 failed.
    Traceback (most recent call last):
    File "/usr/lib/cmf/agent/src/cmf/agent.py", line 558, in send_heartbeat
    self.master_port)
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py",
    line 471, in __init__
    self.conn.connect()
    File "/usr/lib/python2.6/httplib.py", line 716, in connect
    self.timeout)
    File "/usr/lib/python2.6/socket.py", line 514, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused

    /etc/hosts

    127.0.0.1 localhost
    2a00:1a48:7803:0107:2d9a:7ff0:ff08:0ef3 slave2-test

    10.177.196.5 slave2-test
    5.79.25.31 slave2-test







    On Mon, Sep 17, 2012 at 10:53 PM, Philip Zeyliger wrote:

    The relevant logs are /var/log/cloudera-scm-agent/*.log; anything
    interesting in there? When you go to the hosts tab, what do you see?

    -- Philip

    On Mon, Sep 17, 2012 at 12:58 PM, Dalia Hassan wrote:

    I already disabled the firewall by typing ufw disable

    $ufw status
    Status: inactive

    So what do u think?

    On Mon, Sep 17, 2012 at 9:46 PM, Philip Zeyliger wrote:

    CM is trying to talk to the agent running on that host over port
    9000, and is failing to do so. I suspect that there are firewalls involved.

    -- Philip


    On Mon, Sep 17, 2012 at 12:45 PM, Dalia Hassan <
    daliahassan0@gmail.com> wrote:
    Hii Philip,

    the processes are not working, when I went through the cloudera
    manager server log file I found this error, any help?
    WARN [New I/O client boss #1:cmf.HeartbeatRequester@503] Error
    requesting heartbeat.
    java.net.ConnectException: Connection refused to
    http://master1-test:9000/heartbeat?token=858836ec3e3daf3704341b0c6092f9b0
    at
    com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:95)
    at
    org.jboss.netty.channel.DefaultChannelFuture.notifyListener(DefaultChannelFuture.java:381)
    at
    org.jboss.netty.channel.DefaultChannelFuture.notifyListeners(DefaultChannelFuture.java:372)
    at
    org.jboss.netty.channel.DefaultChannelFuture.setFailure(DefaultChannelFuture.java:334)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:389)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
    at
    org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    at
    org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
    Caused by: java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at
    sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
    .
    .
    .
    INFO [CommandPusher:command.CommandHelpers@198] Command
    (Inspector, 5) has finished on service null.
    2012-09-16 16:46:52,391 INFO
    [StaleEntityEviction:model.HeartbeatStore@107] Reaped 0 process
    heartbeats and 0 host heartbeats.
    2012-09-16 16:46:52,399 INFO
    [StaleEntityEviction:cmf.StaleEntityEvictionThread@135] Wizard is
    active, not reaping scanners or configurators
    2012-09-16 16:56:52,405 INFO
    [StaleEntityEviction:model.HeartbeatStore@107] Reaped 0 process
    heartbeats and 0 host heartbeats.



    On Mon, Sep 17, 2012 at 8:49 PM, Philip Zeyliger <
    philip@cloudera.com> wrote:
    Hi Dalia,

    In some form or other, the master and agents aren't communicating
    correctly. If you go to the "Hosts" tab in CM (open up the
    http://master:7180/ in a new window) what do you see? If you
    click on one of the hosts, go to the "processes" tab, and look around at
    stdout/stderr, you might see whether or not the host inspection processes
    ran correctly.

    -- Philip


    On Sun, Sep 16, 2012 at 12:59 PM, Dalia Hassan <
    daliahassan0@gmail.com> wrote:
    Hello all,

    I am using rackspace cloud to for CM deployment.

    I have created only 4 cloud servers.

    I have installed cloudera manager server on one of them
    successfully.

    The problem appears during the hosts inspection phase:

    Inspector failed on the following hosts...
    <http://5.79.24.112:7180/cmf/express-wizard/inspector?submit=Continue#>

    - master1-test: IOException thrown while collecting data from
    host: Connection refused
    - slave2-test: IOException thrown while collecting data from
    host: slave2-test
    - master2-test: IOException thrown while collecting data from
    host: master2-test
    - slave1-test: IOException thrown while collecting data from
    host: slave1-test

    Inspector ran on 0 hosts.

    So I dunno why as for log file:
    In master 1-test:
    /etc/cloudera-scm-server/db.properties "btw the Cloudera Manager
    database is running"
    # These are database settings for CM Manager
    #
    com.cloudera.cmf.db.type=postgresql
    com.cloudera.cmf.db.host=localhost:7432
    com.cloudera.cmf.db.name=scm
    com.cloudera.cmf.db.user=scm
    com.cloudera.cmf.db.password=4UVl1LoidI

    This the cloudera manager scm-server log file
    2012-09-16 16:26:16,738 INFO [main:cmf.Main@166] Starting SCM
    Server. Version: 4.0.4 (#79 built by jenkins on 20120821-1040 git:
    119fe7452f0c356dcde763a1d22eda9363d24ad4)
    2012-09-16 16:26:16,808 INFO [main:enterprise.CommonMain@90]
    Reading database properties from /etc/cloudera-scm-server/db.properties
    2012-09-16 16:26:16,927 INFO [main:enterprise.CommonMain@123]
    Statistics not enabled, c3p0 JMX disabled
    2012-09-16 16:26:17,511 INFO [main:common.Version@37] Hibernate
    Commons Annotations 3.2.0.Final
    2012-09-16 16:26:17,529 INFO [main:cfg.Environment@603]
    Hibernate 3.6.10.Final
    2012-09-16 16:26:17,535 INFO [main:cfg.Environment@636]
    hibernate.properties not found
    2012-09-16 16:26:17,545 INFO [main:cfg.Environment@814] Bytecode
    provider name : javassist
    2012-09-16 16:26:17,554 INFO [main:cfg.Environment@695] using
    JDK 1.4 java.sql.Timestamp handling
    2012-09-16 16:26:17,692 INFO [main:ejb.Version@42] Hibernate
    EntityManager 3.6.10.Final
    2012-09-16 16:26:18,209 INFO [main:cfg.Configuration@789]
    Reading mappings from resource : cmf.hbm.xml
    2012-09-16 16:26:18,754 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbHost -> HOSTS
    2012-09-16 16:26:18,871 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbRole -> ROLES
    2012-09-16 16:26:18,894 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbService -> SERVICES
    2012-09-16 16:26:18,920 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbConfig -> CONFIGS
    2012-09-16 16:26:18,922 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbCommand -> COMMANDS
    2012-09-16 16:26:18,947 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbProcess -> PROCESSES
    2012-09-16 16:26:18,978 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbMasterSettings -> SETTINGS
    2012-09-16 16:26:19,021 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbAudit -> AUDITS
    2012-09-16 16:26:19,025 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbCredential -> CREDENTIALS
    2012-09-16 16:26:19,044 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbUser -> USERS
    2012-09-16 16:26:19,046 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbUserRole -> USER_ROLES
    2012-09-16 16:26:19,049 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbCluster -> CLUSTERS
    2012-09-16 16:26:19,050 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbConfigRevision -> CONFIG_REVISIONS
    2012-09-16 16:26:19,052 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbClientConfig -> CLIENT_CONFIGS
    2012-09-16 16:26:19,053 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbConfigContainer -> CONFIG_CONTAINERS
    2012-09-16 16:26:19,056 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbHost.activeCommands -> COMMANDS
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbHost.processes -> PROCESSES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbHost.roles -> ROLES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbRole.processes -> PROCESSES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.roles -> ROLES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.activeCommands -> COMMANDS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.configRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.visibleConfigRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.clientConfigs -> CLIENT_CONFIGS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbCommand.processes -> PROCESSES
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbCommand.children -> COMMANDS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbUser.userRoles -> USER_ROLES
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbCluster.activeCommands -> COMMANDS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbConfigRevision.configsForDb -> CONFIGS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbConfigContainer.configRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbConfigContainer.visibleConfigRevisions
    -> CONFIG_REVISIONS
    2012-09-16 16:26:19,071 INFO [main:cfg.Configuration@1676]
    Hibernate Validator not found: ignoring
    2012-09-16 16:26:19,347 INFO
    [main:search.HibernateSearchEventListenerRegister@75] Unable to
    find org.hibernate.search.event.FullTextIndexEventListener on the
    classpath. Hibernate Search is not enabled.
    2012-09-16 16:26:19,394 INFO
    [main:connection.ConnectionProviderFactory@173] Initializing
    connection provider: org.hibernate.connection.C3P0ConnectionProvider
    2012-09-16 16:26:19,395 INFO
    [main:connection.C3P0ConnectionProvider@103] C3P0 using driver:
    org.postgresql.Driver at URL: jdbc:postgresql://localhost:7432/scm
    2012-09-16 16:26:19,395 INFO
    [main:connection.C3P0ConnectionProvider@104] Connection
    properties: {user=scm, password=****, autocommit=true, release_mode=auto}
    2012-09-16 16:26:19,396 INFO
    [main:connection.C3P0ConnectionProvider@107] autocommit mode: true
    2012-09-16 16:26:19,464 INFO [main:log.MLog@80] MLog clients
    using log4j logging.
    2012-09-16 16:26:19,691 INFO [main:c3p0.C3P0Registry@204]
    Initializing c3p0-0.9.1 [built 16-January-2007 14:46:42; debug? true;
    trace: 10]
    @


    so any ideaaaa ????

  • Dalia Hassan at Sep 19, 2012 at 12:22 am
    Hi Philip and all,

    any new additions am still facing the same problem..

    Do I have to use a Fully qualified domain name for the hosts ???


    On Mon, Sep 17, 2012 at 11:11 PM, Dalia Hassan wrote:

    in agent log:
    [17/Sep/2012 00:03:38 +0000] 642 MainThread agent ERROR
    Heartbeating to 10.177.196.147:7182 failed.
    Traceback (most recent call last):
    File "/usr/lib/cmf/agent/src/cmf/
    agent.py", line 558, in send_heartbeat
    self.master_port)
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py",
    line 471, in __init__
    self.conn.connect()
    File "/usr/lib/python2.6/httplib.py", line 716, in connect
    self.timeout)
    File "/usr/lib/python2.6/socket.py", line 514, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused

    /etc/hosts

    127.0.0.1 localhost
    2a00:1a48:7803:0107:2d9a:7ff0:ff08:0ef3 slave2-test

    10.177.196.5 slave2-test
    5.79.25.31 slave2-test

    On Mon, Sep 17, 2012 at 11:10 PM, Dalia Hassan wrote:

    how??

    On Mon, Sep 17, 2012 at 11:08 PM, Philip Zeyliger wrote:

    Please continue to reply-all so that other list members can help.

    Your agent and your server aren't heartbeating. You should manually see
    if you can connect from one to another, on the given ports.

    -- Philip

    On Mon, Sep 17, 2012 at 2:00 PM, Dalia Hassan wrote:

    in agent log:
    [17/Sep/2012 00:03:38 +0000] 642 MainThread agent ERROR
    Heartbeating to 10.177.196.147:7182 failed.
    Traceback (most recent call last):
    File "/usr/lib/cmf/agent/src/cmf/agent.py", line 558, in
    send_heartbeat
    self.master_port)
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py",
    line 471, in __init__
    self.conn.connect()
    File "/usr/lib/python2.6/httplib.py", line 716, in connect
    self.timeout)
    File "/usr/lib/python2.6/socket.py", line 514, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused

    /etc/hosts

    127.0.0.1 localhost
    2a00:1a48:7803:0107:2d9a:7ff0:ff08:0ef3 slave2-test

    10.177.196.5 slave2-test
    5.79.25.31 slave2-test







    On Mon, Sep 17, 2012 at 10:53 PM, Philip Zeyliger wrote:

    The relevant logs are /var/log/cloudera-scm-agent/*.log; anything
    interesting in there? When you go to the hosts tab, what do you see?

    -- Philip


    On Mon, Sep 17, 2012 at 12:58 PM, Dalia Hassan <daliahassan0@gmail.com
    wrote:
    I already disabled the firewall by typing ufw disable

    $ufw status
    Status: inactive

    So what do u think?


    On Mon, Sep 17, 2012 at 9:46 PM, Philip Zeyliger <philip@cloudera.com
    wrote:
    CM is trying to talk to the agent running on that host over port
    9000, and is failing to do so. I suspect that there are firewalls involved.

    -- Philip


    On Mon, Sep 17, 2012 at 12:45 PM, Dalia Hassan <
    daliahassan0@gmail.com> wrote:
    Hii Philip,

    the processes are not working, when I went through the cloudera
    manager server log file I found this error, any help?
    WARN [New I/O client boss #1:cmf.HeartbeatRequester@503] Error
    requesting heartbeat.
    java.net.ConnectException: Connection refused to
    http://master1-test:9000/heartbeat?token=858836ec3e3daf3704341b0c6092f9b0
    at
    com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:95)
    at
    org.jboss.netty.channel.DefaultChannelFuture.notifyListener(DefaultChannelFuture.java:381)
    at
    org.jboss.netty.channel.DefaultChannelFuture.notifyListeners(DefaultChannelFuture.java:372)
    at
    org.jboss.netty.channel.DefaultChannelFuture.setFailure(DefaultChannelFuture.java:334)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:389)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
    at
    org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    at
    org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
    Caused by: java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at
    sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
    .
    .
    .
    INFO [CommandPusher:command.CommandHelpers@198] Command
    (Inspector, 5) has finished on service null.
    2012-09-16 16:46:52,391 INFO
    [StaleEntityEviction:model.HeartbeatStore@107] Reaped 0 process
    heartbeats and 0 host heartbeats.
    2012-09-16 16:46:52,399 INFO
    [StaleEntityEviction:cmf.StaleEntityEvictionThread@135] Wizard is
    active, not reaping scanners or configurators
    2012-09-16 16:56:52,405 INFO
    [StaleEntityEviction:model.HeartbeatStore@107] Reaped 0 process
    heartbeats and 0 host heartbeats.



    On Mon, Sep 17, 2012 at 8:49 PM, Philip Zeyliger <
    philip@cloudera.com> wrote:
    Hi Dalia,

    In some form or other, the master and agents aren't communicating
    correctly. If you go to the "Hosts" tab in CM (open up the
    http://master:7180/ in a new window) what do you see? If you
    click on one of the hosts, go to the "processes" tab, and look around at
    stdout/stderr, you might see whether or not the host inspection processes
    ran correctly.

    -- Philip


    On Sun, Sep 16, 2012 at 12:59 PM, Dalia Hassan <
    daliahassan0@gmail.com> wrote:
    Hello all,

    I am using rackspace cloud to for CM deployment.

    I have created only 4 cloud servers.

    I have installed cloudera manager server on one of them
    successfully.

    The problem appears during the hosts inspection phase:

    Inspector failed on the following hosts...
    <http://5.79.24.112:7180/cmf/express-wizard/inspector?submit=Continue#>

    - master1-test: IOException thrown while collecting data from
    host: Connection refused
    - slave2-test: IOException thrown while collecting data from
    host: slave2-test
    - master2-test: IOException thrown while collecting data from
    host: master2-test
    - slave1-test: IOException thrown while collecting data from
    host: slave1-test

    Inspector ran on 0 hosts.

    So I dunno why as for log file:
    In master 1-test:
    /etc/cloudera-scm-server/db.properties "btw the Cloudera Manager
    database is running"
    # These are database settings for CM Manager
    #
    com.cloudera.cmf.db.type=postgresql
    com.cloudera.cmf.db.host=localhost:7432
    com.cloudera.cmf.db.name=scm
    com.cloudera.cmf.db.user=scm
    com.cloudera.cmf.db.password=4UVl1LoidI

    This the cloudera manager scm-server log file
    2012-09-16 16:26:16,738 INFO [main:cmf.Main@166] Starting SCM
    Server. Version: 4.0.4 (#79 built by jenkins on 20120821-1040 git:
    119fe7452f0c356dcde763a1d22eda9363d24ad4)
    2012-09-16 16:26:16,808 INFO [main:enterprise.CommonMain@90]
    Reading database properties from /etc/cloudera-scm-server/db.properties
    2012-09-16 16:26:16,927 INFO [main:enterprise.CommonMain@123]
    Statistics not enabled, c3p0 JMX disabled
    2012-09-16 16:26:17,511 INFO [main:common.Version@37] Hibernate
    Commons Annotations 3.2.0.Final
    2012-09-16 16:26:17,529 INFO [main:cfg.Environment@603]
    Hibernate 3.6.10.Final
    2012-09-16 16:26:17,535 INFO [main:cfg.Environment@636]
    hibernate.properties not found
    2012-09-16 16:26:17,545 INFO [main:cfg.Environment@814]
    Bytecode provider name : javassist
    2012-09-16 16:26:17,554 INFO [main:cfg.Environment@695] using
    JDK 1.4 java.sql.Timestamp handling
    2012-09-16 16:26:17,692 INFO [main:ejb.Version@42] Hibernate
    EntityManager 3.6.10.Final
    2012-09-16 16:26:18,209 INFO [main:cfg.Configuration@789]
    Reading mappings from resource : cmf.hbm.xml
    2012-09-16 16:26:18,754 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbHost -> HOSTS
    2012-09-16 16:26:18,871 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbRole -> ROLES
    2012-09-16 16:26:18,894 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbService -> SERVICES
    2012-09-16 16:26:18,920 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbConfig -> CONFIGS
    2012-09-16 16:26:18,922 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbCommand -> COMMANDS
    2012-09-16 16:26:18,947 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbProcess -> PROCESSES
    2012-09-16 16:26:18,978 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbMasterSettings -> SETTINGS
    2012-09-16 16:26:19,021 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbAudit -> AUDITS
    2012-09-16 16:26:19,025 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbCredential -> CREDENTIALS
    2012-09-16 16:26:19,044 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbUser -> USERS
    2012-09-16 16:26:19,046 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbUserRole -> USER_ROLES
    2012-09-16 16:26:19,049 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbCluster -> CLUSTERS
    2012-09-16 16:26:19,050 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbConfigRevision -> CONFIG_REVISIONS
    2012-09-16 16:26:19,052 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbClientConfig -> CLIENT_CONFIGS
    2012-09-16 16:26:19,053 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbConfigContainer -> CONFIG_CONTAINERS
    2012-09-16 16:26:19,056 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbHost.activeCommands -> COMMANDS
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbHost.processes -> PROCESSES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbHost.roles -> ROLES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbRole.processes -> PROCESSES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.roles -> ROLES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.activeCommands -> COMMANDS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.configRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.visibleConfigRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.clientConfigs -> CLIENT_CONFIGS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbCommand.processes -> PROCESSES
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbCommand.children -> COMMANDS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbUser.userRoles -> USER_ROLES
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbCluster.activeCommands -> COMMANDS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbConfigRevision.configsForDb -> CONFIGS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbConfigContainer.configRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbConfigContainer.visibleConfigRevisions
    -> CONFIG_REVISIONS
    2012-09-16 16:26:19,071 INFO [main:cfg.Configuration@1676]
    Hibernate Validator not found: ignoring
    2012-09-16 16:26:19,347 INFO
    [main:search.HibernateSearchEventListenerRegister@75] Unable to
    find org.hibernate.search.event.FullTextIndexEventListener on the
    classpath. Hibernate Search is not enabled.
    2012-09-16 16:26:19,394 INFO
    [main:connection.ConnectionProviderFactory@173] Initializing
    connection provider: org.hibernate.connection.C3P0ConnectionProvider
    2012-09-16 16:26:19,395 INFO
    [main:connection.C3P0ConnectionProvider@103] C3P0 using driver:
    org.postgresql.Driver at URL: jdbc:postgresql://localhost:7432/scm
    2012-09-16 16:26:19,395 INFO
    [main:connection.C3P0ConnectionProvider@104] Connection
    properties: {user=scm, password=****, autocommit=true, release_mode=auto}
    2012-09-16 16:26:19,396 INFO
    [main:connection.C3P0ConnectionProvider@107] autocommit mode:
    true
    2012-09-16 16:26:19,464 INFO [main:log.MLog@80] MLog clients
    using log4j logging.
    2012-09-16 16:26:19,691 INFO [main:c3p0.C3P0Registry@204]
    Initializing c3p0-0.9.1 [built 16-January-2007 14:46:42; debug? true;
    trace: 10]
    @


    so any ideaaaa ????

  • Philip Zeyliger at Sep 19, 2012 at 1:32 am
    Hi Dalia,

    I know very very little of your setup, so it's proving tricky to help you.
    You can check whether a host has network connectivity to another by
    running "nc -l <port>" on one host and "nc <host> <port>" on the other.
    You'll see things you type on the latter appear on the former. The "nc"
    is "netcat."

    Yes, typically you do need working DNS for things to work.

    -- Philip
    On Tue, Sep 18, 2012 at 5:22 PM, Dalia Hassan wrote:

    Hi Philip and all,

    any new additions am still facing the same problem..

    Do I have to use a Fully qualified domain name for the hosts ???


    On Mon, Sep 17, 2012 at 11:11 PM, Dalia Hassan wrote:

    in agent log:
    [17/Sep/2012 00:03:38 +0000] 642 MainThread agent ERROR
    Heartbeating to 10.177.196.147:7182 failed.
    Traceback (most recent call last):
    File "/usr/lib/cmf/agent/src/cmf/
    agent.py", line 558, in send_heartbeat
    self.master_port)
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py",
    line 471, in __init__
    self.conn.connect()
    File "/usr/lib/python2.6/httplib.py", line 716, in connect
    self.timeout)
    File "/usr/lib/python2.6/socket.py", line 514, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused

    /etc/hosts

    127.0.0.1 localhost
    2a00:1a48:7803:0107:2d9a:7ff0:ff08:0ef3 slave2-test

    10.177.196.5 slave2-test
    5.79.25.31 slave2-test

    On Mon, Sep 17, 2012 at 11:10 PM, Dalia Hassan wrote:

    how??

    On Mon, Sep 17, 2012 at 11:08 PM, Philip Zeyliger wrote:

    Please continue to reply-all so that other list members can help.

    Your agent and your server aren't heartbeating. You should manually
    see if you can connect from one to another, on the given ports.

    -- Philip

    On Mon, Sep 17, 2012 at 2:00 PM, Dalia Hassan wrote:

    in agent log:
    [17/Sep/2012 00:03:38 +0000] 642 MainThread agent ERROR
    Heartbeating to 10.177.196.147:7182 failed.
    Traceback (most recent call last):
    File "/usr/lib/cmf/agent/src/cmf/agent.py", line 558, in
    send_heartbeat
    self.master_port)
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py",
    line 471, in __init__
    self.conn.connect()
    File "/usr/lib/python2.6/httplib.py", line 716, in connect
    self.timeout)
    File "/usr/lib/python2.6/socket.py", line 514, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused

    /etc/hosts

    127.0.0.1 localhost
    2a00:1a48:7803:0107:2d9a:7ff0:ff08:0ef3 slave2-test

    10.177.196.5 slave2-test
    5.79.25.31 slave2-test








    On Mon, Sep 17, 2012 at 10:53 PM, Philip Zeyliger <philip@cloudera.com
    wrote:
    The relevant logs are /var/log/cloudera-scm-agent/*.log; anything
    interesting in there? When you go to the hosts tab, what do you see?

    -- Philip


    On Mon, Sep 17, 2012 at 12:58 PM, Dalia Hassan <
    daliahassan0@gmail.com> wrote:
    I already disabled the firewall by typing ufw disable

    $ufw status
    Status: inactive

    So what do u think?


    On Mon, Sep 17, 2012 at 9:46 PM, Philip Zeyliger <
    philip@cloudera.com> wrote:
    CM is trying to talk to the agent running on that host over port
    9000, and is failing to do so. I suspect that there are firewalls involved.

    -- Philip


    On Mon, Sep 17, 2012 at 12:45 PM, Dalia Hassan <
    daliahassan0@gmail.com> wrote:
    Hii Philip,

    the processes are not working, when I went through the cloudera
    manager server log file I found this error, any help?
    WARN [New I/O client boss #1:cmf.HeartbeatRequester@503] Error
    requesting heartbeat.
    java.net.ConnectException: Connection refused to
    http://master1-test:9000/heartbeat?token=858836ec3e3daf3704341b0c6092f9b0
    at
    com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:95)
    at
    org.jboss.netty.channel.DefaultChannelFuture.notifyListener(DefaultChannelFuture.java:381)
    at
    org.jboss.netty.channel.DefaultChannelFuture.notifyListeners(DefaultChannelFuture.java:372)
    at
    org.jboss.netty.channel.DefaultChannelFuture.setFailure(DefaultChannelFuture.java:334)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:389)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
    at
    org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    at
    org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
    Caused by: java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at
    sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
    .
    .
    .
    INFO [CommandPusher:command.CommandHelpers@198] Command
    (Inspector, 5) has finished on service null.
    2012-09-16 16:46:52,391 INFO
    [StaleEntityEviction:model.HeartbeatStore@107] Reaped 0 process
    heartbeats and 0 host heartbeats.
    2012-09-16 16:46:52,399 INFO
    [StaleEntityEviction:cmf.StaleEntityEvictionThread@135] Wizard is
    active, not reaping scanners or configurators
    2012-09-16 16:56:52,405 INFO
    [StaleEntityEviction:model.HeartbeatStore@107] Reaped 0 process
    heartbeats and 0 host heartbeats.



    On Mon, Sep 17, 2012 at 8:49 PM, Philip Zeyliger <
    philip@cloudera.com> wrote:
    Hi Dalia,

    In some form or other, the master and agents aren't communicating
    correctly. If you go to the "Hosts" tab in CM (open up the
    http://master:7180/ in a new window) what do you see? If you
    click on one of the hosts, go to the "processes" tab, and look around at
    stdout/stderr, you might see whether or not the host inspection processes
    ran correctly.

    -- Philip


    On Sun, Sep 16, 2012 at 12:59 PM, Dalia Hassan <
    daliahassan0@gmail.com> wrote:
    Hello all,

    I am using rackspace cloud to for CM deployment.

    I have created only 4 cloud servers.

    I have installed cloudera manager server on one of them
    successfully.

    The problem appears during the hosts inspection phase:

    Inspector failed on the following hosts...
    <http://5.79.24.112:7180/cmf/express-wizard/inspector?submit=Continue#>

    - master1-test: IOException thrown while collecting data
    from host: Connection refused
    - slave2-test: IOException thrown while collecting data from
    host: slave2-test
    - master2-test: IOException thrown while collecting data
    from host: master2-test
    - slave1-test: IOException thrown while collecting data from
    host: slave1-test

    Inspector ran on 0 hosts.

    So I dunno why as for log file:
    In master 1-test:
    /etc/cloudera-scm-server/db.properties "btw the Cloudera Manager
    database is running"
    # These are database settings for CM Manager
    #
    com.cloudera.cmf.db.type=postgresql
    com.cloudera.cmf.db.host=localhost:7432
    com.cloudera.cmf.db.name=scm
    com.cloudera.cmf.db.user=scm
    com.cloudera.cmf.db.password=4UVl1LoidI

    This the cloudera manager scm-server log file
    2012-09-16 16:26:16,738 INFO [main:cmf.Main@166] Starting SCM
    Server. Version: 4.0.4 (#79 built by jenkins on 20120821-1040 git:
    119fe7452f0c356dcde763a1d22eda9363d24ad4)
    2012-09-16 16:26:16,808 INFO [main:enterprise.CommonMain@90]
    Reading database properties from /etc/cloudera-scm-server/db.properties
    2012-09-16 16:26:16,927 INFO [main:enterprise.CommonMain@123]
    Statistics not enabled, c3p0 JMX disabled
    2012-09-16 16:26:17,511 INFO [main:common.Version@37]
    Hibernate Commons Annotations 3.2.0.Final
    2012-09-16 16:26:17,529 INFO [main:cfg.Environment@603]
    Hibernate 3.6.10.Final
    2012-09-16 16:26:17,535 INFO [main:cfg.Environment@636]
    hibernate.properties not found
    2012-09-16 16:26:17,545 INFO [main:cfg.Environment@814]
    Bytecode provider name : javassist
    2012-09-16 16:26:17,554 INFO [main:cfg.Environment@695] using
    JDK 1.4 java.sql.Timestamp handling
    2012-09-16 16:26:17,692 INFO [main:ejb.Version@42] Hibernate
    EntityManager 3.6.10.Final
    2012-09-16 16:26:18,209 INFO [main:cfg.Configuration@789]
    Reading mappings from resource : cmf.hbm.xml
    2012-09-16 16:26:18,754 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbHost -> HOSTS
    2012-09-16 16:26:18,871 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbRole -> ROLES
    2012-09-16 16:26:18,894 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbService -> SERVICES
    2012-09-16 16:26:18,920 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbConfig -> CONFIGS
    2012-09-16 16:26:18,922 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbCommand -> COMMANDS
    2012-09-16 16:26:18,947 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbProcess -> PROCESSES
    2012-09-16 16:26:18,978 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbMasterSettings -> SETTINGS
    2012-09-16 16:26:19,021 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbAudit -> AUDITS
    2012-09-16 16:26:19,025 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbCredential -> CREDENTIALS
    2012-09-16 16:26:19,044 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbUser -> USERS
    2012-09-16 16:26:19,046 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbUserRole -> USER_ROLES
    2012-09-16 16:26:19,049 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbCluster -> CLUSTERS
    2012-09-16 16:26:19,050 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbConfigRevision -> CONFIG_REVISIONS
    2012-09-16 16:26:19,052 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbClientConfig -> CLIENT_CONFIGS
    2012-09-16 16:26:19,053 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbConfigContainer -> CONFIG_CONTAINERS
    2012-09-16 16:26:19,056 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbHost.activeCommands -> COMMANDS
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbHost.processes -> PROCESSES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbHost.roles -> ROLES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbRole.processes -> PROCESSES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.roles -> ROLES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.activeCommands -> COMMANDS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.configRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.visibleConfigRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbService.clientConfigs -> CLIENT_CONFIGS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbCommand.processes -> PROCESSES
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbCommand.children -> COMMANDS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbUser.userRoles -> USER_ROLES
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbCluster.activeCommands -> COMMANDS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbConfigRevision.configsForDb -> CONFIGS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbConfigContainer.configRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508] Mapping
    collection: com.cloudera.cmf.model.DbConfigContainer.visibleConfigRevisions
    -> CONFIG_REVISIONS
    2012-09-16 16:26:19,071 INFO [main:cfg.Configuration@1676]
    Hibernate Validator not found: ignoring
    2012-09-16 16:26:19,347 INFO
    [main:search.HibernateSearchEventListenerRegister@75] Unable to
    find org.hibernate.search.event.FullTextIndexEventListener on the
    classpath. Hibernate Search is not enabled.
    2012-09-16 16:26:19,394 INFO
    [main:connection.ConnectionProviderFactory@173] Initializing
    connection provider: org.hibernate.connection.C3P0ConnectionProvider
    2012-09-16 16:26:19,395 INFO
    [main:connection.C3P0ConnectionProvider@103] C3P0 using driver:
    org.postgresql.Driver at URL: jdbc:postgresql://localhost:7432/scm
    2012-09-16 16:26:19,395 INFO
    [main:connection.C3P0ConnectionProvider@104] Connection
    properties: {user=scm, password=****, autocommit=true, release_mode=auto}
    2012-09-16 16:26:19,396 INFO
    [main:connection.C3P0ConnectionProvider@107] autocommit mode:
    true
    2012-09-16 16:26:19,464 INFO [main:log.MLog@80] MLog clients
    using log4j logging.
    2012-09-16 16:26:19,691 INFO [main:c3p0.C3P0Registry@204]
    Initializing c3p0-0.9.1 [built 16-January-2007 14:46:42; debug? true;
    trace: 10]
    @


    so any ideaaaa ????

  • Dalia Hassan at Sep 19, 2012 at 7:34 pm
    Hi philip and all,

    the inspector is running normally and port 9000 is running but there are
    some failures:

    Inspector ran on all 4 hosts. Individual hosts resolved their own
    hostnames correctly. No errors where found while checking
    /etc/hosts. All hosts resolved localhost to 127.0.0.1. The
    following failures were observed in checking hostnames. Showing first 1000
    failures only... ** <http://5.79.24.218:7180/cmf/inspector?commandId=110#>

    - DNS reverse lookup of IP 5.79.6.217 on host master1 failed. Expected
    master2 but got 5.79.6.217.
    - DNS reverse lookup of IP 5.79.7.103 on host master1 failed. Expected
    slave1 but got 5.79.7.103.
    - DNS reverse lookup of IP 5.79.26.192 on host master1 failed. Expected
    slave2 but got 5.79.26.192.
    - DNS reverse lookup of IP 5.79.24.218 on host slave2 failed. Expected
    master1 but got 5.79.24.218.
    - DNS reverse lookup of IP 5.79.6.217 on host slave2 failed. Expected
    master2 but got 5.79.6.217.
    - DNS reverse lookup of IP 5.79.7.103 on host slave2 failed. Expected
    slave1 but got 5.79.7.103.
    - DNS reverse lookup of IP 5.79.24.218 on host master2 failed. Expected
    master1 but got 5.79.24.218.
    - DNS reverse lookup of IP 5.79.7.103 on host master2 failed. Expected
    slave1 but got 5.79.7.103.
    - DNS reverse lookup of IP 5.79.26.192 on host master2 failed. Expected
    slave2 but got 5.79.26.192.
    - DNS reverse lookup of IP 5.79.24.218 on host slave1 failed. Expected
    master1 but got 5.79.24.218.
    - DNS reverse lookup of IP 5.79.6.217 on host slave1 failed. Expected
    master2 but got 5.79.6.217.
    - DNS reverse lookup of IP 5.79.26.192 on host slave1 failed. Expected
    slave2 but got 5.79.26.192.

    could you provide me with any clues?


    Thanks for your support and help..
    On Wed, Sep 19, 2012 at 3:31 AM, Philip Zeyliger wrote:

    Hi Dalia,

    I know very very little of your setup, so it's proving tricky to help you.
    You can check whether a host has network connectivity to another by
    running "nc -l <port>" on one host and "nc <host> <port>" on the other.
    You'll see things you type on the latter appear on the former. The "nc"
    is "netcat."

    Yes, typically you do need working DNS for things to work.

    -- Philip

    On Tue, Sep 18, 2012 at 5:22 PM, Dalia Hassan wrote:

    Hi Philip and all,

    any new additions am still facing the same problem..

    Do I have to use a Fully qualified domain name for the hosts ???


    On Mon, Sep 17, 2012 at 11:11 PM, Dalia Hassan wrote:

    in agent log:
    [17/Sep/2012 00:03:38 +0000] 642 MainThread agent ERROR
    Heartbeating to 10.177.196.147:7182 failed.
    Traceback (most recent call last):
    File "/usr/lib/cmf/agent/src/cmf/
    agent.py", line 558, in send_heartbeat
    self.master_port)
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py",
    line 471, in __init__
    self.conn.connect()
    File "/usr/lib/python2.6/httplib.py", line 716, in connect
    self.timeout)
    File "/usr/lib/python2.6/socket.py", line 514, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused

    /etc/hosts

    127.0.0.1 localhost
    2a00:1a48:7803:0107:2d9a:7ff0:ff08:0ef3 slave2-test

    10.177.196.5 slave2-test
    5.79.25.31 slave2-test

    On Mon, Sep 17, 2012 at 11:10 PM, Dalia Hassan wrote:

    how??

    On Mon, Sep 17, 2012 at 11:08 PM, Philip Zeyliger wrote:

    Please continue to reply-all so that other list members can help.

    Your agent and your server aren't heartbeating. You should manually
    see if you can connect from one to another, on the given ports.

    -- Philip

    On Mon, Sep 17, 2012 at 2:00 PM, Dalia Hassan wrote:

    in agent log:
    [17/Sep/2012 00:03:38 +0000] 642 MainThread agent ERROR
    Heartbeating to 10.177.196.147:7182 failed.
    Traceback (most recent call last):
    File "/usr/lib/cmf/agent/src/cmf/agent.py", line 558, in
    send_heartbeat
    self.master_port)
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py",
    line 471, in __init__
    self.conn.connect()
    File "/usr/lib/python2.6/httplib.py", line 716, in connect
    self.timeout)
    File "/usr/lib/python2.6/socket.py", line 514, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused

    /etc/hosts

    127.0.0.1 localhost
    2a00:1a48:7803:0107:2d9a:7ff0:ff08:0ef3 slave2-test

    10.177.196.5 slave2-test
    5.79.25.31 slave2-test








    On Mon, Sep 17, 2012 at 10:53 PM, Philip Zeyliger <
    philip@cloudera.com> wrote:
    The relevant logs are /var/log/cloudera-scm-agent/*.log; anything
    interesting in there? When you go to the hosts tab, what do you see?

    -- Philip


    On Mon, Sep 17, 2012 at 12:58 PM, Dalia Hassan <
    daliahassan0@gmail.com> wrote:
    I already disabled the firewall by typing ufw disable

    $ufw status
    Status: inactive

    So what do u think?


    On Mon, Sep 17, 2012 at 9:46 PM, Philip Zeyliger <
    philip@cloudera.com> wrote:
    CM is trying to talk to the agent running on that host over port
    9000, and is failing to do so. I suspect that there are firewalls involved.

    -- Philip


    On Mon, Sep 17, 2012 at 12:45 PM, Dalia Hassan <
    daliahassan0@gmail.com> wrote:
    Hii Philip,

    the processes are not working, when I went through the cloudera
    manager server log file I found this error, any help?
    WARN [New I/O client boss #1:cmf.HeartbeatRequester@503] Error
    requesting heartbeat.
    java.net.ConnectException: Connection refused to
    http://master1-test:9000/heartbeat?token=858836ec3e3daf3704341b0c6092f9b0
    at
    com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:95)
    at
    org.jboss.netty.channel.DefaultChannelFuture.notifyListener(DefaultChannelFuture.java:381)
    at
    org.jboss.netty.channel.DefaultChannelFuture.notifyListeners(DefaultChannelFuture.java:372)
    at
    org.jboss.netty.channel.DefaultChannelFuture.setFailure(DefaultChannelFuture.java:334)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:389)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
    at
    org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    at
    org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
    Caused by: java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native
    Method)
    at
    sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
    .
    .
    .
    INFO [CommandPusher:command.CommandHelpers@198] Command
    (Inspector, 5) has finished on service null.
    2012-09-16 16:46:52,391 INFO
    [StaleEntityEviction:model.HeartbeatStore@107] Reaped 0 process
    heartbeats and 0 host heartbeats.
    2012-09-16 16:46:52,399 INFO
    [StaleEntityEviction:cmf.StaleEntityEvictionThread@135] Wizard
    is active, not reaping scanners or configurators
    2012-09-16 16:56:52,405 INFO
    [StaleEntityEviction:model.HeartbeatStore@107] Reaped 0 process
    heartbeats and 0 host heartbeats.



    On Mon, Sep 17, 2012 at 8:49 PM, Philip Zeyliger <
    philip@cloudera.com> wrote:
    Hi Dalia,

    In some form or other, the master and agents aren't
    communicating correctly. If you go to the "Hosts" tab in CM (open up the
    http://master:7180/ in a new window) what do you see? If you
    click on one of the hosts, go to the "processes" tab, and look around at
    stdout/stderr, you might see whether or not the host inspection processes
    ran correctly.

    -- Philip


    On Sun, Sep 16, 2012 at 12:59 PM, Dalia Hassan <
    daliahassan0@gmail.com> wrote:
    Hello all,

    I am using rackspace cloud to for CM deployment.

    I have created only 4 cloud servers.

    I have installed cloudera manager server on one of them
    successfully.

    The problem appears during the hosts inspection phase:

    Inspector failed on the following hosts...
    <http://5.79.24.112:7180/cmf/express-wizard/inspector?submit=Continue#>

    - master1-test: IOException thrown while collecting data
    from host: Connection refused
    - slave2-test: IOException thrown while collecting data
    from host: slave2-test
    - master2-test: IOException thrown while collecting data
    from host: master2-test
    - slave1-test: IOException thrown while collecting data
    from host: slave1-test

    Inspector ran on 0 hosts.

    So I dunno why as for log file:
    In master 1-test:
    /etc/cloudera-scm-server/db.properties "btw the Cloudera
    Manager database is running"
    # These are database settings for CM Manager
    #
    com.cloudera.cmf.db.type=postgresql
    com.cloudera.cmf.db.host=localhost:7432
    com.cloudera.cmf.db.name=scm
    com.cloudera.cmf.db.user=scm
    com.cloudera.cmf.db.password=4UVl1LoidI

    This the cloudera manager scm-server log file
    2012-09-16 16:26:16,738 INFO [main:cmf.Main@166] Starting SCM
    Server. Version: 4.0.4 (#79 built by jenkins on 20120821-1040 git:
    119fe7452f0c356dcde763a1d22eda9363d24ad4)
    2012-09-16 16:26:16,808 INFO [main:enterprise.CommonMain@90]
    Reading database properties from /etc/cloudera-scm-server/db.properties
    2012-09-16 16:26:16,927 INFO [main:enterprise.CommonMain@123]
    Statistics not enabled, c3p0 JMX disabled
    2012-09-16 16:26:17,511 INFO [main:common.Version@37]
    Hibernate Commons Annotations 3.2.0.Final
    2012-09-16 16:26:17,529 INFO [main:cfg.Environment@603]
    Hibernate 3.6.10.Final
    2012-09-16 16:26:17,535 INFO [main:cfg.Environment@636]
    hibernate.properties not found
    2012-09-16 16:26:17,545 INFO [main:cfg.Environment@814]
    Bytecode provider name : javassist
    2012-09-16 16:26:17,554 INFO [main:cfg.Environment@695] using
    JDK 1.4 java.sql.Timestamp handling
    2012-09-16 16:26:17,692 INFO [main:ejb.Version@42] Hibernate
    EntityManager 3.6.10.Final
    2012-09-16 16:26:18,209 INFO [main:cfg.Configuration@789]
    Reading mappings from resource : cmf.hbm.xml
    2012-09-16 16:26:18,754 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbHost -> HOSTS
    2012-09-16 16:26:18,871 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbRole -> ROLES
    2012-09-16 16:26:18,894 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbService -> SERVICES
    2012-09-16 16:26:18,920 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbConfig -> CONFIGS
    2012-09-16 16:26:18,922 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbCommand -> COMMANDS
    2012-09-16 16:26:18,947 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbProcess -> PROCESSES
    2012-09-16 16:26:18,978 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbMasterSettings -> SETTINGS
    2012-09-16 16:26:19,021 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbAudit -> AUDITS
    2012-09-16 16:26:19,025 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbCredential -> CREDENTIALS
    2012-09-16 16:26:19,044 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbUser -> USERS
    2012-09-16 16:26:19,046 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbUserRole -> USER_ROLES
    2012-09-16 16:26:19,049 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbCluster -> CLUSTERS
    2012-09-16 16:26:19,050 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbConfigRevision -> CONFIG_REVISIONS
    2012-09-16 16:26:19,052 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbClientConfig -> CLIENT_CONFIGS
    2012-09-16 16:26:19,053 INFO [main:cfg.HbmBinder@353] Mapping
    class: com.cloudera.cmf.model.DbConfigContainer -> CONFIG_CONTAINERS
    2012-09-16 16:26:19,056 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbHost.activeCommands -> COMMANDS
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbHost.processes -> PROCESSES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbHost.roles -> ROLES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbRole.processes -> PROCESSES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbService.roles -> ROLES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbService.activeCommands ->
    COMMANDS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbService.configRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbService.visibleConfigRevisions
    -> CONFIG_REVISIONS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbService.clientConfigs ->
    CLIENT_CONFIGS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbCommand.processes -> PROCESSES
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbCommand.children -> COMMANDS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbUser.userRoles -> USER_ROLES
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbCluster.activeCommands ->
    COMMANDS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbConfigRevision.configsForDb ->
    CONFIGS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508]
    Mapping collection:
    com.cloudera.cmf.model.DbConfigContainer.configRevisions -> CONFIG_REVISIONS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508]
    Mapping collection:
    com.cloudera.cmf.model.DbConfigContainer.visibleConfigRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,071 INFO [main:cfg.Configuration@1676]
    Hibernate Validator not found: ignoring
    2012-09-16 16:26:19,347 INFO
    [main:search.HibernateSearchEventListenerRegister@75] Unable
    to find org.hibernate.search.event.FullTextIndexEventListener on the
    classpath. Hibernate Search is not enabled.
    2012-09-16 16:26:19,394 INFO
    [main:connection.ConnectionProviderFactory@173] Initializing
    connection provider: org.hibernate.connection.C3P0ConnectionProvider
    2012-09-16 16:26:19,395 INFO
    [main:connection.C3P0ConnectionProvider@103] C3P0 using
    driver: org.postgresql.Driver at URL: jdbc:postgresql://localhost:7432/scm
    2012-09-16 16:26:19,395 INFO
    [main:connection.C3P0ConnectionProvider@104] Connection
    properties: {user=scm, password=****, autocommit=true, release_mode=auto}
    2012-09-16 16:26:19,396 INFO
    [main:connection.C3P0ConnectionProvider@107] autocommit mode:
    true
    2012-09-16 16:26:19,464 INFO [main:log.MLog@80] MLog clients
    using log4j logging.
    2012-09-16 16:26:19,691 INFO [main:c3p0.C3P0Registry@204]
    Initializing c3p0-0.9.1 [built 16-January-2007 14:46:42; debug? true;
    trace: 10]
    @


    so any ideaaaa ????

  • Philip Zeyliger at Sep 19, 2012 at 6:08 pm
    Hi Dalia,

    Earlier the inspector wasn't working at all; what's changed?

    Your reverse DNS isn't working. When we tried to look up the reverse of
    "5.79.6.217" we expected to get a name, but got an IP address. You'll run
    into trouble later; Hadoop expects reverse DNS to work.

    -- Philip
    On Wed, Sep 19, 2012 at 10:55 AM, Dalia Hassan wrote:

    Hi philip and all,

    the inspector is running normally and port 9000 is running but there are
    some failures:

    Inspector ran on all 4 hosts. Individual hosts resolved their own
    hostnames correctly. No errors where found while checking /etc/hosts. All hosts resolved localhost to 127.0.0.1. The
    following failures were observed in checking hostnames. Showing first 1000
    failures only... ** <http://5.79.24.218:7180/cmf/inspector?commandId=110#>

    - DNS reverse lookup of IP 5.79.6.217 on host master1 failed. Expected
    master2 but got 5.79.6.217.
    - DNS reverse lookup of IP 5.79.7.103 on host master1 failed. Expected
    slave1 but got 5.79.7.103.
    - DNS reverse lookup of IP 5.79.26.192 on host master1 failed.
    Expected slave2 but got 5.79.26.192.
    - DNS reverse lookup of IP 5.79.24.218 on host slave2 failed. Expected
    master1 but got 5.79.24.218.
    - DNS reverse lookup of IP 5.79.6.217 on host slave2 failed. Expected
    master2 but got 5.79.6.217.
    - DNS reverse lookup of IP 5.79.7.103 on host slave2 failed. Expected
    slave1 but got 5.79.7.103.
    - DNS reverse lookup of IP 5.79.24.218 on host master2 failed.
    Expected master1 but got 5.79.24.218.
    - DNS reverse lookup of IP 5.79.7.103 on host master2 failed. Expected
    slave1 but got 5.79.7.103.
    - DNS reverse lookup of IP 5.79.26.192 on host master2 failed.
    Expected slave2 but got 5.79.26.192.
    - DNS reverse lookup of IP 5.79.24.218 on host slave1 failed. Expected
    master1 but got 5.79.24.218.
    - DNS reverse lookup of IP 5.79.6.217 on host slave1 failed. Expected
    master2 but got 5.79.6.217.
    - DNS reverse lookup of IP 5.79.26.192 on host slave1 failed. Expected
    slave2 but got 5.79.26.192.

    could you provide me with any clues?


    Thanks for your support and help..
    On Wed, Sep 19, 2012 at 3:31 AM, Philip Zeyliger wrote:

    Hi Dalia,

    I know very very little of your setup, so it's proving tricky to help
    you. You can check whether a host has network connectivity to another by
    running "nc -l <port>" on one host and "nc <host> <port>" on the other.
    You'll see things you type on the latter appear on the former. The "nc"
    is "netcat."

    Yes, typically you do need working DNS for things to work.

    -- Philip

    On Tue, Sep 18, 2012 at 5:22 PM, Dalia Hassan wrote:

    Hi Philip and all,

    any new additions am still facing the same problem..

    Do I have to use a Fully qualified domain name for the hosts ???


    On Mon, Sep 17, 2012 at 11:11 PM, Dalia Hassan wrote:

    in agent log:
    [17/Sep/2012 00:03:38 +0000] 642 MainThread agent ERROR
    Heartbeating to 10.177.196.147:7182 failed.
    Traceback (most recent call last):
    File "/usr/lib/cmf/agent/src/cmf/
    agent.py", line 558, in send_heartbeat
    self.master_port)
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py",
    line 471, in __init__
    self.conn.connect()
    File "/usr/lib/python2.6/httplib.py", line 716, in connect
    self.timeout)
    File "/usr/lib/python2.6/socket.py", line 514, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused

    /etc/hosts

    127.0.0.1 localhost
    2a00:1a48:7803:0107:2d9a:7ff0:ff08:0ef3 slave2-test

    10.177.196.5 slave2-test
    5.79.25.31 slave2-test

    On Mon, Sep 17, 2012 at 11:10 PM, Dalia Hassan wrote:

    how??


    On Mon, Sep 17, 2012 at 11:08 PM, Philip Zeyliger <philip@cloudera.com
    wrote:
    Please continue to reply-all so that other list members can help.

    Your agent and your server aren't heartbeating. You should manually
    see if you can connect from one to another, on the given ports.

    -- Philip


    On Mon, Sep 17, 2012 at 2:00 PM, Dalia Hassan <daliahassan0@gmail.com
    wrote:
    in agent log:
    [17/Sep/2012 00:03:38 +0000] 642 MainThread agent ERROR
    Heartbeating to 10.177.196.147:7182 failed.
    Traceback (most recent call last):
    File "/usr/lib/cmf/agent/src/cmf/agent.py", line 558, in
    send_heartbeat
    self.master_port)
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py",
    line 471, in __init__
    self.conn.connect()
    File "/usr/lib/python2.6/httplib.py", line 716, in connect
    self.timeout)
    File "/usr/lib/python2.6/socket.py", line 514, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused

    /etc/hosts

    127.0.0.1 localhost
    2a00:1a48:7803:0107:2d9a:7ff0:ff08:0ef3 slave2-test

    10.177.196.5 slave2-test
    5.79.25.31 slave2-test








    On Mon, Sep 17, 2012 at 10:53 PM, Philip Zeyliger <
    philip@cloudera.com> wrote:
    The relevant logs are /var/log/cloudera-scm-agent/*.log; anything
    interesting in there? When you go to the hosts tab, what do you see?

    -- Philip


    On Mon, Sep 17, 2012 at 12:58 PM, Dalia Hassan <
    daliahassan0@gmail.com> wrote:
    I already disabled the firewall by typing ufw disable

    $ufw status
    Status: inactive

    So what do u think?


    On Mon, Sep 17, 2012 at 9:46 PM, Philip Zeyliger <
    philip@cloudera.com> wrote:
    CM is trying to talk to the agent running on that host over port
    9000, and is failing to do so. I suspect that there are firewalls involved.

    -- Philip


    On Mon, Sep 17, 2012 at 12:45 PM, Dalia Hassan <
    daliahassan0@gmail.com> wrote:
    Hii Philip,

    the processes are not working, when I went through the cloudera
    manager server log file I found this error, any help?
    WARN [New I/O client boss #1:cmf.HeartbeatRequester@503] Error
    requesting heartbeat.
    java.net.ConnectException: Connection refused to
    http://master1-test:9000/heartbeat?token=858836ec3e3daf3704341b0c6092f9b0
    at
    com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:95)
    at
    org.jboss.netty.channel.DefaultChannelFuture.notifyListener(DefaultChannelFuture.java:381)
    at
    org.jboss.netty.channel.DefaultChannelFuture.notifyListeners(DefaultChannelFuture.java:372)
    at
    org.jboss.netty.channel.DefaultChannelFuture.setFailure(DefaultChannelFuture.java:334)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:389)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
    at
    org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    at
    org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
    Caused by: java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native
    Method)
    at
    sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
    .
    .
    .
    INFO [CommandPusher:command.CommandHelpers@198] Command
    (Inspector, 5) has finished on service null.
    2012-09-16 16:46:52,391 INFO
    [StaleEntityEviction:model.HeartbeatStore@107] Reaped 0 process
    heartbeats and 0 host heartbeats.
    2012-09-16 16:46:52,399 INFO
    [StaleEntityEviction:cmf.StaleEntityEvictionThread@135] Wizard
    is active, not reaping scanners or configurators
    2012-09-16 16:56:52,405 INFO
    [StaleEntityEviction:model.HeartbeatStore@107] Reaped 0 process
    heartbeats and 0 host heartbeats.



    On Mon, Sep 17, 2012 at 8:49 PM, Philip Zeyliger <
    philip@cloudera.com> wrote:
    Hi Dalia,

    In some form or other, the master and agents aren't
    communicating correctly. If you go to the "Hosts" tab in CM (open up the
    http://master:7180/ in a new window) what do you see? If you
    click on one of the hosts, go to the "processes" tab, and look around at
    stdout/stderr, you might see whether or not the host inspection processes
    ran correctly.

    -- Philip


    On Sun, Sep 16, 2012 at 12:59 PM, Dalia Hassan <
    daliahassan0@gmail.com> wrote:
    Hello all,

    I am using rackspace cloud to for CM deployment.

    I have created only 4 cloud servers.

    I have installed cloudera manager server on one of them
    successfully.

    The problem appears during the hosts inspection phase:

    Inspector failed on the following hosts...
    <http://5.79.24.112:7180/cmf/express-wizard/inspector?submit=Continue#>

    - master1-test: IOException thrown while collecting data
    from host: Connection refused
    - slave2-test: IOException thrown while collecting data
    from host: slave2-test
    - master2-test: IOException thrown while collecting data
    from host: master2-test
    - slave1-test: IOException thrown while collecting data
    from host: slave1-test

    Inspector ran on 0 hosts.

    So I dunno why as for log file:
    In master 1-test:
    /etc/cloudera-scm-server/db.properties "btw the Cloudera
    Manager database is running"
    # These are database settings for CM Manager
    #
    com.cloudera.cmf.db.type=postgresql
    com.cloudera.cmf.db.host=localhost:7432
    com.cloudera.cmf.db.name=scm
    com.cloudera.cmf.db.user=scm
    com.cloudera.cmf.db.password=4UVl1LoidI

    This the cloudera manager scm-server log file
    2012-09-16 16:26:16,738 INFO [main:cmf.Main@166] Starting
    SCM Server. Version: 4.0.4 (#79 built by jenkins on 20120821-1040 git:
    119fe7452f0c356dcde763a1d22eda9363d24ad4)
    2012-09-16 16:26:16,808 INFO [main:enterprise.CommonMain@90]
    Reading database properties from /etc/cloudera-scm-server/db.properties
    2012-09-16 16:26:16,927 INFO [main:enterprise.CommonMain@123]
    Statistics not enabled, c3p0 JMX disabled
    2012-09-16 16:26:17,511 INFO [main:common.Version@37]
    Hibernate Commons Annotations 3.2.0.Final
    2012-09-16 16:26:17,529 INFO [main:cfg.Environment@603]
    Hibernate 3.6.10.Final
    2012-09-16 16:26:17,535 INFO [main:cfg.Environment@636]
    hibernate.properties not found
    2012-09-16 16:26:17,545 INFO [main:cfg.Environment@814]
    Bytecode provider name : javassist
    2012-09-16 16:26:17,554 INFO [main:cfg.Environment@695]
    using JDK 1.4 java.sql.Timestamp handling
    2012-09-16 16:26:17,692 INFO [main:ejb.Version@42] Hibernate
    EntityManager 3.6.10.Final
    2012-09-16 16:26:18,209 INFO [main:cfg.Configuration@789]
    Reading mappings from resource : cmf.hbm.xml
    2012-09-16 16:26:18,754 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbHost -> HOSTS
    2012-09-16 16:26:18,871 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbRole -> ROLES
    2012-09-16 16:26:18,894 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbService -> SERVICES
    2012-09-16 16:26:18,920 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbConfig -> CONFIGS
    2012-09-16 16:26:18,922 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbCommand -> COMMANDS
    2012-09-16 16:26:18,947 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbProcess -> PROCESSES
    2012-09-16 16:26:18,978 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbMasterSettings -> SETTINGS
    2012-09-16 16:26:19,021 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbAudit -> AUDITS
    2012-09-16 16:26:19,025 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbCredential -> CREDENTIALS
    2012-09-16 16:26:19,044 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbUser -> USERS
    2012-09-16 16:26:19,046 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbUserRole -> USER_ROLES
    2012-09-16 16:26:19,049 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbCluster -> CLUSTERS
    2012-09-16 16:26:19,050 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbConfigRevision -> CONFIG_REVISIONS
    2012-09-16 16:26:19,052 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbClientConfig -> CLIENT_CONFIGS
    2012-09-16 16:26:19,053 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbConfigContainer -> CONFIG_CONTAINERS
    2012-09-16 16:26:19,056 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbHost.activeCommands -> COMMANDS
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbHost.processes -> PROCESSES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbHost.roles -> ROLES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbRole.processes -> PROCESSES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbService.roles -> ROLES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbService.activeCommands ->
    COMMANDS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbService.configRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbService.visibleConfigRevisions
    -> CONFIG_REVISIONS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbService.clientConfigs ->
    CLIENT_CONFIGS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbCommand.processes -> PROCESSES
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbCommand.children -> COMMANDS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbUser.userRoles -> USER_ROLES
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbCluster.activeCommands ->
    COMMANDS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbConfigRevision.configsForDb ->
    CONFIGS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508]
    Mapping collection:
    com.cloudera.cmf.model.DbConfigContainer.configRevisions -> CONFIG_REVISIONS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508]
    Mapping collection:
    com.cloudera.cmf.model.DbConfigContainer.visibleConfigRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,071 INFO [main:cfg.Configuration@1676]
    Hibernate Validator not found: ignoring
    2012-09-16 16:26:19,347 INFO
    [main:search.HibernateSearchEventListenerRegister@75] Unable
    to find org.hibernate.search.event.FullTextIndexEventListener on the
    classpath. Hibernate Search is not enabled.
    2012-09-16 16:26:19,394 INFO
    [main:connection.ConnectionProviderFactory@173] Initializing
    connection provider: org.hibernate.connection.C3P0ConnectionProvider
    2012-09-16 16:26:19,395 INFO
    [main:connection.C3P0ConnectionProvider@103] C3P0 using
    driver: org.postgresql.Driver at URL: jdbc:postgresql://localhost:7432/scm
    2012-09-16 16:26:19,395 INFO
    [main:connection.C3P0ConnectionProvider@104] Connection
    properties: {user=scm, password=****, autocommit=true, release_mode=auto}
    2012-09-16 16:26:19,396 INFO
    [main:connection.C3P0ConnectionProvider@107] autocommit mode:
    true
    2012-09-16 16:26:19,464 INFO [main:log.MLog@80] MLog clients
    using log4j logging.
    2012-09-16 16:26:19,691 INFO [main:c3p0.C3P0Registry@204]
    Initializing c3p0-0.9.1 [built 16-January-2007 14:46:42; debug? true;
    trace: 10]
    @


    so any ideaaaa ????

  • Dalia Hassan at Sep 19, 2012 at 6:31 pm
    In /etc/hosts

    There were 3 IPs for the host as for master1 --> public IP, private IP, IPv6

    so I removed all and kept the public one, thats why it worked out

    ex: telnet master1 9000
    Trying 5.79.24.218...
    Connected to master1.
    Escape character is '^]'.

    telnet master1.medcloud.com 9000
    Trying 5.79.24.218...
    Connected to master1.medcloud.com.
    Escape character is '^]'.

    On Wed, Sep 19, 2012 at 8:07 PM, Philip Zeyliger wrote:

    Hi Dalia,

    Earlier the inspector wasn't working at all; what's changed?

    Your reverse DNS isn't working. When we tried to look up the reverse of
    "5.79.6.217" we expected to get a name, but got an IP address. You'll run
    into trouble later; Hadoop expects reverse DNS to work.

    -- Philip

    On Wed, Sep 19, 2012 at 10:55 AM, Dalia Hassan wrote:

    Hi philip and all,

    the inspector is running normally and port 9000 is running but there are
    some failures:

    Inspector ran on all 4 hosts. Individual hosts resolved their own
    hostnames correctly. No errors where found while checking /etc/hosts. All hosts resolved localhost to 127.0.0.1. The
    following failures were observed in checking hostnames. Showing first 1000
    failures only... **
    <http://5.79.24.218:7180/cmf/inspector?commandId=110#>

    - DNS reverse lookup of IP 5.79.6.217 on host master1 failed.
    Expected master2 but got 5.79.6.217.
    - DNS reverse lookup of IP 5.79.7.103 on host master1 failed.
    Expected slave1 but got 5.79.7.103.
    - DNS reverse lookup of IP 5.79.26.192 on host master1 failed.
    Expected slave2 but got 5.79.26.192.
    - DNS reverse lookup of IP 5.79.24.218 on host slave2 failed.
    Expected master1 but got 5.79.24.218.
    - DNS reverse lookup of IP 5.79.6.217 on host slave2 failed. Expected
    master2 but got 5.79.6.217.
    - DNS reverse lookup of IP 5.79.7.103 on host slave2 failed. Expected
    slave1 but got 5.79.7.103.
    - DNS reverse lookup of IP 5.79.24.218 on host master2 failed.
    Expected master1 but got 5.79.24.218.
    - DNS reverse lookup of IP 5.79.7.103 on host master2 failed.
    Expected slave1 but got 5.79.7.103.
    - DNS reverse lookup of IP 5.79.26.192 on host master2 failed.
    Expected slave2 but got 5.79.26.192.
    - DNS reverse lookup of IP 5.79.24.218 on host slave1 failed.
    Expected master1 but got 5.79.24.218.
    - DNS reverse lookup of IP 5.79.6.217 on host slave1 failed. Expected
    master2 but got 5.79.6.217.
    - DNS reverse lookup of IP 5.79.26.192 on host slave1 failed.
    Expected slave2 but got 5.79.26.192.

    could you provide me with any clues?


    Thanks for your support and help..
    On Wed, Sep 19, 2012 at 3:31 AM, Philip Zeyliger wrote:

    Hi Dalia,

    I know very very little of your setup, so it's proving tricky to help
    you. You can check whether a host has network connectivity to another by
    running "nc -l <port>" on one host and "nc <host> <port>" on the other.
    You'll see things you type on the latter appear on the former. The "nc"
    is "netcat."

    Yes, typically you do need working DNS for things to work.

    -- Philip

    On Tue, Sep 18, 2012 at 5:22 PM, Dalia Hassan wrote:

    Hi Philip and all,

    any new additions am still facing the same problem..

    Do I have to use a Fully qualified domain name for the hosts ???


    On Mon, Sep 17, 2012 at 11:11 PM, Dalia Hassan wrote:

    in agent log:
    [17/Sep/2012 00:03:38 +0000] 642 MainThread agent ERROR
    Heartbeating to 10.177.196.147:7182 failed.
    Traceback (most recent call last):
    File "/usr/lib/cmf/agent/src/cmf/
    agent.py", line 558, in send_heartbeat
    self.master_port)
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py",
    line 471, in __init__
    self.conn.connect()
    File "/usr/lib/python2.6/httplib.py", line 716, in connect
    self.timeout)
    File "/usr/lib/python2.6/socket.py", line 514, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused

    /etc/hosts

    127.0.0.1 localhost
    2a00:1a48:7803:0107:2d9a:7ff0:ff08:0ef3 slave2-test

    10.177.196.5 slave2-test
    5.79.25.31 slave2-test


    On Mon, Sep 17, 2012 at 11:10 PM, Dalia Hassan <daliahassan0@gmail.com
    wrote:
    how??


    On Mon, Sep 17, 2012 at 11:08 PM, Philip Zeyliger <
    philip@cloudera.com> wrote:
    Please continue to reply-all so that other list members can help.

    Your agent and your server aren't heartbeating. You should manually
    see if you can connect from one to another, on the given ports.

    -- Philip


    On Mon, Sep 17, 2012 at 2:00 PM, Dalia Hassan <
    daliahassan0@gmail.com> wrote:
    in agent log:
    [17/Sep/2012 00:03:38 +0000] 642 MainThread agent ERROR
    Heartbeating to 10.177.196.147:7182 failed.
    Traceback (most recent call last):
    File "/usr/lib/cmf/agent/src/cmf/agent.py", line 558, in
    send_heartbeat
    self.master_port)
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py",
    line 471, in __init__
    self.conn.connect()
    File "/usr/lib/python2.6/httplib.py", line 716, in connect
    self.timeout)
    File "/usr/lib/python2.6/socket.py", line 514, in
    create_connection
    raise error, msg
    error: [Errno 111] Connection refused

    /etc/hosts

    127.0.0.1 localhost
    2a00:1a48:7803:0107:2d9a:7ff0:ff08:0ef3 slave2-test

    10.177.196.5 slave2-test
    5.79.25.31 slave2-test








    On Mon, Sep 17, 2012 at 10:53 PM, Philip Zeyliger <
    philip@cloudera.com> wrote:
    The relevant logs are /var/log/cloudera-scm-agent/*.log; anything
    interesting in there? When you go to the hosts tab, what do you see?

    -- Philip


    On Mon, Sep 17, 2012 at 12:58 PM, Dalia Hassan <
    daliahassan0@gmail.com> wrote:
    I already disabled the firewall by typing ufw disable

    $ufw status
    Status: inactive

    So what do u think?


    On Mon, Sep 17, 2012 at 9:46 PM, Philip Zeyliger <
    philip@cloudera.com> wrote:
    CM is trying to talk to the agent running on that host over port
    9000, and is failing to do so. I suspect that there are firewalls involved.

    -- Philip


    On Mon, Sep 17, 2012 at 12:45 PM, Dalia Hassan <
    daliahassan0@gmail.com> wrote:
    Hii Philip,

    the processes are not working, when I went through the cloudera
    manager server log file I found this error, any help?
    WARN [New I/O client boss #1:cmf.HeartbeatRequester@503] Error
    requesting heartbeat.
    java.net.ConnectException: Connection refused to
    http://master1-test:9000/heartbeat?token=858836ec3e3daf3704341b0c6092f9b0
    at
    com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:95)
    at
    org.jboss.netty.channel.DefaultChannelFuture.notifyListener(DefaultChannelFuture.java:381)
    at
    org.jboss.netty.channel.DefaultChannelFuture.notifyListeners(DefaultChannelFuture.java:372)
    at
    org.jboss.netty.channel.DefaultChannelFuture.setFailure(DefaultChannelFuture.java:334)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:389)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
    at
    org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    at
    org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
    Caused by: java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native
    Method)
    at
    sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
    .
    .
    .
    INFO [CommandPusher:command.CommandHelpers@198] Command
    (Inspector, 5) has finished on service null.
    2012-09-16 16:46:52,391 INFO
    [StaleEntityEviction:model.HeartbeatStore@107] Reaped 0
    process heartbeats and 0 host heartbeats.
    2012-09-16 16:46:52,399 INFO
    [StaleEntityEviction:cmf.StaleEntityEvictionThread@135] Wizard
    is active, not reaping scanners or configurators
    2012-09-16 16:56:52,405 INFO
    [StaleEntityEviction:model.HeartbeatStore@107] Reaped 0
    process heartbeats and 0 host heartbeats.



    On Mon, Sep 17, 2012 at 8:49 PM, Philip Zeyliger <
    philip@cloudera.com> wrote:
    Hi Dalia,

    In some form or other, the master and agents aren't
    communicating correctly. If you go to the "Hosts" tab in CM (open up the
    http://master:7180/ in a new window) what do you see? If you
    click on one of the hosts, go to the "processes" tab, and look around at
    stdout/stderr, you might see whether or not the host inspection processes
    ran correctly.

    -- Philip


    On Sun, Sep 16, 2012 at 12:59 PM, Dalia Hassan <
    daliahassan0@gmail.com> wrote:
    Hello all,

    I am using rackspace cloud to for CM deployment.

    I have created only 4 cloud servers.

    I have installed cloudera manager server on one of them
    successfully.

    The problem appears during the hosts inspection phase:

    Inspector failed on the following hosts...
    <http://5.79.24.112:7180/cmf/express-wizard/inspector?submit=Continue#>

    - master1-test: IOException thrown while collecting data
    from host: Connection refused
    - slave2-test: IOException thrown while collecting data
    from host: slave2-test
    - master2-test: IOException thrown while collecting data
    from host: master2-test
    - slave1-test: IOException thrown while collecting data
    from host: slave1-test

    Inspector ran on 0 hosts.

    So I dunno why as for log file:
    In master 1-test:
    /etc/cloudera-scm-server/db.properties "btw the Cloudera
    Manager database is running"
    # These are database settings for CM Manager
    #
    com.cloudera.cmf.db.type=postgresql
    com.cloudera.cmf.db.host=localhost:7432
    com.cloudera.cmf.db.name=scm
    com.cloudera.cmf.db.user=scm
    com.cloudera.cmf.db.password=4UVl1LoidI

    This the cloudera manager scm-server log file
    2012-09-16 16:26:16,738 INFO [main:cmf.Main@166] Starting
    SCM Server. Version: 4.0.4 (#79 built by jenkins on 20120821-1040 git:
    119fe7452f0c356dcde763a1d22eda9363d24ad4)
    2012-09-16 16:26:16,808 INFO [main:enterprise.CommonMain@90]
    Reading database properties from /etc/cloudera-scm-server/db.properties
    2012-09-16 16:26:16,927 INFO [main:enterprise.CommonMain@123]
    Statistics not enabled, c3p0 JMX disabled
    2012-09-16 16:26:17,511 INFO [main:common.Version@37]
    Hibernate Commons Annotations 3.2.0.Final
    2012-09-16 16:26:17,529 INFO [main:cfg.Environment@603]
    Hibernate 3.6.10.Final
    2012-09-16 16:26:17,535 INFO [main:cfg.Environment@636]
    hibernate.properties not found
    2012-09-16 16:26:17,545 INFO [main:cfg.Environment@814]
    Bytecode provider name : javassist
    2012-09-16 16:26:17,554 INFO [main:cfg.Environment@695]
    using JDK 1.4 java.sql.Timestamp handling
    2012-09-16 16:26:17,692 INFO [main:ejb.Version@42]
    Hibernate EntityManager 3.6.10.Final
    2012-09-16 16:26:18,209 INFO [main:cfg.Configuration@789]
    Reading mappings from resource : cmf.hbm.xml
    2012-09-16 16:26:18,754 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbHost -> HOSTS
    2012-09-16 16:26:18,871 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbRole -> ROLES
    2012-09-16 16:26:18,894 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbService -> SERVICES
    2012-09-16 16:26:18,920 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbConfig -> CONFIGS
    2012-09-16 16:26:18,922 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbCommand -> COMMANDS
    2012-09-16 16:26:18,947 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbProcess -> PROCESSES
    2012-09-16 16:26:18,978 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbMasterSettings -> SETTINGS
    2012-09-16 16:26:19,021 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbAudit -> AUDITS
    2012-09-16 16:26:19,025 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbCredential -> CREDENTIALS
    2012-09-16 16:26:19,044 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbUser -> USERS
    2012-09-16 16:26:19,046 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbUserRole -> USER_ROLES
    2012-09-16 16:26:19,049 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbCluster -> CLUSTERS
    2012-09-16 16:26:19,050 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbConfigRevision -> CONFIG_REVISIONS
    2012-09-16 16:26:19,052 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbClientConfig -> CLIENT_CONFIGS
    2012-09-16 16:26:19,053 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbConfigContainer -> CONFIG_CONTAINERS
    2012-09-16 16:26:19,056 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbHost.activeCommands -> COMMANDS
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbHost.processes -> PROCESSES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbHost.roles -> ROLES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbRole.processes -> PROCESSES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbService.roles -> ROLES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbService.activeCommands ->
    COMMANDS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbService.configRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbService.visibleConfigRevisions
    -> CONFIG_REVISIONS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbService.clientConfigs ->
    CLIENT_CONFIGS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbCommand.processes -> PROCESSES
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbCommand.children -> COMMANDS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbUser.userRoles -> USER_ROLES
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbCluster.activeCommands ->
    COMMANDS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbConfigRevision.configsForDb ->
    CONFIGS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508]
    Mapping collection:
    com.cloudera.cmf.model.DbConfigContainer.configRevisions -> CONFIG_REVISIONS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508]
    Mapping collection:
    com.cloudera.cmf.model.DbConfigContainer.visibleConfigRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,071 INFO [main:cfg.Configuration@1676]
    Hibernate Validator not found: ignoring
    2012-09-16 16:26:19,347 INFO
    [main:search.HibernateSearchEventListenerRegister@75] Unable
    to find org.hibernate.search.event.FullTextIndexEventListener on the
    classpath. Hibernate Search is not enabled.
    2012-09-16 16:26:19,394 INFO
    [main:connection.ConnectionProviderFactory@173] Initializing
    connection provider: org.hibernate.connection.C3P0ConnectionProvider
    2012-09-16 16:26:19,395 INFO
    [main:connection.C3P0ConnectionProvider@103] C3P0 using
    driver: org.postgresql.Driver at URL: jdbc:postgresql://localhost:7432/scm
    2012-09-16 16:26:19,395 INFO
    [main:connection.C3P0ConnectionProvider@104] Connection
    properties: {user=scm, password=****, autocommit=true, release_mode=auto}
    2012-09-16 16:26:19,396 INFO
    [main:connection.C3P0ConnectionProvider@107] autocommit
    mode: true
    2012-09-16 16:26:19,464 INFO [main:log.MLog@80] MLog
    clients using log4j logging.
    2012-09-16 16:26:19,691 INFO [main:c3p0.C3P0Registry@204]
    Initializing c3p0-0.9.1 [built 16-January-2007 14:46:42; debug? true;
    trace: 10]
    @


    so any ideaaaa ????

  • Mehal Patel at Nov 20, 2012 at 3:43 pm
    Hi Dalia,

    were you able to get resolution of this DNS reverse lookup failed error
    that you encountered ?

    Please do let me know.

    Mehal
    On Wednesday, 19 September 2012 11:23:52 UTC-7, Dalia Hassan wrote:

    In /etc/hosts

    There were 3 IPs for the host as for master1 --> public IP, private IP,
    IPv6

    so I removed all and kept the public one, thats why it worked out

    ex: telnet master1 9000
    Trying 5.79.24.218...
    Connected to master1.
    Escape character is '^]'.

    telnet master1.medcloud.com 9000
    Trying 5.79.24.218...
    Connected to master1.medcloud.com.
    Escape character is '^]'.


    On Wed, Sep 19, 2012 at 8:07 PM, Philip Zeyliger <phi...@cloudera.com<javascript:>
    wrote:
    Hi Dalia,

    Earlier the inspector wasn't working at all; what's changed?

    Your reverse DNS isn't working. When we tried to look up the reverse of
    "5.79.6.217" we expected to get a name, but got an IP address. You'll run
    into trouble later; Hadoop expects reverse DNS to work.

    -- Philip


    On Wed, Sep 19, 2012 at 10:55 AM, Dalia Hassan <daliah...@gmail.com<javascript:>
    wrote:
    Hi philip and all,

    the inspector is running normally and port 9000 is running but there are
    some failures:

    Inspector ran on all 4 hosts. Individual hosts resolved their own
    hostnames correctly. No errors where found while checking
    /etc/hosts. All hosts resolved localhost to 127.0.0.1. The
    following failures were observed in checking hostnames. Showing first 1000
    failures only... **
    <http://5.79.24.218:7180/cmf/inspector?commandId=110#>

    - DNS reverse lookup of IP 5.79.6.217 on host master1 failed.
    Expected master2 but got 5.79.6.217.
    - DNS reverse lookup of IP 5.79.7.103 on host master1 failed.
    Expected slave1 but got 5.79.7.103.
    - DNS reverse lookup of IP 5.79.26.192 on host master1 failed.
    Expected slave2 but got 5.79.26.192.
    - DNS reverse lookup of IP 5.79.24.218 on host slave2 failed.
    Expected master1 but got 5.79.24.218.
    - DNS reverse lookup of IP 5.79.6.217 on host slave2 failed.
    Expected master2 but got 5.79.6.217.
    - DNS reverse lookup of IP 5.79.7.103 on host slave2 failed.
    Expected slave1 but got 5.79.7.103.
    - DNS reverse lookup of IP 5.79.24.218 on host master2 failed.
    Expected master1 but got 5.79.24.218.
    - DNS reverse lookup of IP 5.79.7.103 on host master2 failed.
    Expected slave1 but got 5.79.7.103.
    - DNS reverse lookup of IP 5.79.26.192 on host master2 failed.
    Expected slave2 but got 5.79.26.192.
    - DNS reverse lookup of IP 5.79.24.218 on host slave1 failed.
    Expected master1 but got 5.79.24.218.
    - DNS reverse lookup of IP 5.79.6.217 on host slave1 failed.
    Expected master2 but got 5.79.6.217.
    - DNS reverse lookup of IP 5.79.26.192 on host slave1 failed.
    Expected slave2 but got 5.79.26.192.

    could you provide me with any clues?


    Thanks for your support and help..

    On Wed, Sep 19, 2012 at 3:31 AM, Philip Zeyliger <phi...@cloudera.com<javascript:>
    wrote:
    Hi Dalia,

    I know very very little of your setup, so it's proving tricky to help
    you. You can check whether a host has network connectivity to another by
    running "nc -l <port>" on one host and "nc <host> <port>" on the other.
    You'll see things you type on the latter appear on the former. The "nc"
    is "netcat."

    Yes, typically you do need working DNS for things to work.

    -- Philip


    On Tue, Sep 18, 2012 at 5:22 PM, Dalia Hassan <daliah...@gmail.com<javascript:>
    wrote:
    Hi Philip and all,

    any new additions am still facing the same problem..

    Do I have to use a Fully qualified domain name for the hosts ???



    On Mon, Sep 17, 2012 at 11:11 PM, Dalia Hassan <daliah...@gmail.com<javascript:>
    wrote:
    in agent log:
    [17/Sep/2012 00:03:38 +0000] 642 MainThread agent ERROR
    Heartbeating to 10.177.196.147:7182 failed.
    Traceback (most recent call last):
    File "/usr/lib/cmf/agent/src/cmf/
    agent.py", line 558, in send_heartbeat
    self.master_port)
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py",
    line 471, in __init__
    self.conn.connect()
    File "/usr/lib/python2.6/httplib.py", line 716, in connect
    self.timeout)
    File "/usr/lib/python2.6/socket.py", line 514, in create_connection
    raise error, msg
    error: [Errno 111] Connection refused

    /etc/hosts

    127.0.0.1 localhost
    2a00:1a48:7803:0107:2d9a:7ff0:ff08:0ef3 slave2-test

    10.177.196.5 slave2-test
    5.79.25.31 slave2-test


    On Mon, Sep 17, 2012 at 11:10 PM, Dalia Hassan <daliah...@gmail.com<javascript:>
    wrote:
    how??


    On Mon, Sep 17, 2012 at 11:08 PM, Philip Zeyliger <
    phi...@cloudera.com <javascript:>> wrote:
    Please continue to reply-all so that other list members can help.

    Your agent and your server aren't heartbeating. You should
    manually see if you can connect from one to another, on the given ports.

    -- Philip


    On Mon, Sep 17, 2012 at 2:00 PM, Dalia Hassan <daliah...@gmail.com<javascript:>
    wrote:
    in agent log:
    [17/Sep/2012 00:03:38 +0000] 642 MainThread agent ERROR
    Heartbeating to 10.177.196.147:7182 failed.
    Traceback (most recent call last):
    File "/usr/lib/cmf/agent/src/cmf/agent.py", line 558, in
    send_heartbeat
    self.master_port)
    File
    "/usr/lib/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py",
    line 471, in __init__
    self.conn.connect()
    File "/usr/lib/python2.6/httplib.py", line 716, in connect
    self.timeout)
    File "/usr/lib/python2.6/socket.py", line 514, in
    create_connection
    raise error, msg
    error: [Errno 111] Connection refused

    /etc/hosts

    127.0.0.1 localhost
    2a00:1a48:7803:0107:2d9a:7ff0:ff08:0ef3 slave2-test

    10.177.196.5 slave2-test
    5.79.25.31 slave2-test








    On Mon, Sep 17, 2012 at 10:53 PM, Philip Zeyliger <
    phi...@cloudera.com <javascript:>> wrote:
    The relevant logs are /var/log/cloudera-scm-agent/*.log; anything
    interesting in there? When you go to the hosts tab, what do you see?

    -- Philip


    On Mon, Sep 17, 2012 at 12:58 PM, Dalia Hassan <
    daliah...@gmail.com <javascript:>> wrote:
    I already disabled the firewall by typing ufw disable

    $ufw status
    Status: inactive

    So what do u think?


    On Mon, Sep 17, 2012 at 9:46 PM, Philip Zeyliger <
    phi...@cloudera.com <javascript:>> wrote:
    CM is trying to talk to the agent running on that host over
    port 9000, and is failing to do so. I suspect that there are firewalls
    involved.

    -- Philip


    On Mon, Sep 17, 2012 at 12:45 PM, Dalia Hassan <
    daliah...@gmail.com <javascript:>> wrote:
    Hii Philip,

    the processes are not working, when I went through the
    cloudera manager server log file I found this error, any help?
    WARN [New I/O client boss #1:cmf.HeartbeatRequester@503] Error
    requesting heartbeat.
    java.net.ConnectException: Connection refused to
    http://master1-test:9000/heartbeat?token=858836ec3e3daf3704341b0c6092f9b0
    at
    com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:95)
    at
    org.jboss.netty.channel.DefaultChannelFuture.notifyListener(DefaultChannelFuture.java:381)
    at
    org.jboss.netty.channel.DefaultChannelFuture.notifyListeners(DefaultChannelFuture.java:372)
    at
    org.jboss.netty.channel.DefaultChannelFuture.setFailure(DefaultChannelFuture.java:334)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:389)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276)
    at
    org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    at
    org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
    Caused by: java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native
    Method)
    at
    sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
    at
    org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384)
    .
    .
    .
    INFO [CommandPusher:command.CommandHelpers@198] Command
    (Inspector, 5) has finished on service null.
    2012-09-16 16:46:52,391 INFO
    [StaleEntityEviction:model.HeartbeatStore@107] Reaped 0 process heartbeats
    and 0 host heartbeats.
    2012-09-16 16:46:52,399 INFO
    [StaleEntityEviction:cmf.StaleEntityEvictionThread@135] Wizard is active,
    not reaping scanners or configurators
    2012-09-16 16:56:52,405 INFO
    [StaleEntityEviction:model.HeartbeatStore@107] Reaped 0 process heartbeats
    and 0 host heartbeats.



    On Mon, Sep 17, 2012 at 8:49 PM, Philip Zeyliger <
    phi...@cloudera.com <javascript:>> wrote:
    Hi Dalia,

    In some form or other, the master and agents aren't
    communicating correctly. If you go to the "Hosts" tab in CM (open up the
    http://master:7180/ in a new window) what do you see? If
    you click on one of the hosts, go to the "processes" tab, and look around
    at stdout/stderr, you might see whether or not the host inspection
    processes ran correctly.

    -- Philip


    On Sun, Sep 16, 2012 at 12:59 PM, Dalia Hassan <
    daliah...@gmail.com <javascript:>> wrote:
    Hello all,

    I am using rackspace cloud to for CM deployment.

    I have created only 4 cloud servers.

    I have installed cloudera manager server on one of them
    successfully.

    The problem appears during the hosts inspection phase:

    Inspector failed on the following hosts...
    <http://5.79.24.112:7180/cmf/express-wizard/inspector?submit=Continue#>

    - master1-test: IOException thrown while collecting data
    from host: Connection refused
    - slave2-test: IOException thrown while collecting data
    from host: slave2-test
    - master2-test: IOException thrown while collecting data
    from host: master2-test
    - slave1-test: IOException thrown while collecting data
    from host: slave1-test

    Inspector ran on 0 hosts.

    So I dunno why as for log file:
    In master 1-test:
    /etc/cloudera-scm-server/db.properties "btw the Cloudera
    Manager database is running"
    # These are database settings for CM Manager
    #
    com.cloudera.cmf.db.type=postgresql
    com.cloudera.cmf.db.host=localhost:7432
    com.cloudera.cmf.db.name=scm
    com.cloudera.cmf.db.user=scm
    com.cloudera.cmf.db.password=4UVl1LoidI

    This the cloudera manager scm-server log file
    2012-09-16 16:26:16,738 INFO [main:cmf.Main@166] Starting
    SCM Server. Version: 4.0.4 (#79 built by jenkins on 20120821-1040 git:
    119fe7452f0c356dcde763a1d22eda9363d24ad4)
    2012-09-16 16:26:16,808 INFO
    [main:enterprise.CommonMain@90] Reading database properties from
    /etc/cloudera-scm-server/db.properties
    2012-09-16 16:26:16,927 INFO
    [main:enterprise.CommonMain@123] Statistics not enabled, c3p0 JMX disabled
    2012-09-16 16:26:17,511 INFO [main:common.Version@37]
    Hibernate Commons Annotations 3.2.0.Final
    2012-09-16 16:26:17,529 INFO [main:cfg.Environment@603]
    Hibernate 3.6.10.Final
    2012-09-16 16:26:17,535 INFO [main:cfg.Environment@636]
    hibernate.properties not found
    2012-09-16 16:26:17,545 INFO [main:cfg.Environment@814]
    Bytecode provider name : javassist
    2012-09-16 16:26:17,554 INFO [main:cfg.Environment@695]
    using JDK 1.4 java.sql.Timestamp handling
    2012-09-16 16:26:17,692 INFO [main:ejb.Version@42]
    Hibernate EntityManager 3.6.10.Final
    2012-09-16 16:26:18,209 INFO [main:cfg.Configuration@789]
    Reading mappings from resource : cmf.hbm.xml
    2012-09-16 16:26:18,754 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbHost -> HOSTS
    2012-09-16 16:26:18,871 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbRole -> ROLES
    2012-09-16 16:26:18,894 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbService -> SERVICES
    2012-09-16 16:26:18,920 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbConfig -> CONFIGS
    2012-09-16 16:26:18,922 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbCommand -> COMMANDS
    2012-09-16 16:26:18,947 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbProcess -> PROCESSES
    2012-09-16 16:26:18,978 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbMasterSettings -> SETTINGS
    2012-09-16 16:26:19,021 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbAudit -> AUDITS
    2012-09-16 16:26:19,025 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbCredential -> CREDENTIALS
    2012-09-16 16:26:19,044 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbUser -> USERS
    2012-09-16 16:26:19,046 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbUserRole -> USER_ROLES
    2012-09-16 16:26:19,049 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbCluster -> CLUSTERS
    2012-09-16 16:26:19,050 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbConfigRevision -> CONFIG_REVISIONS
    2012-09-16 16:26:19,052 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbClientConfig -> CLIENT_CONFIGS
    2012-09-16 16:26:19,053 INFO [main:cfg.HbmBinder@353]
    Mapping class: com.cloudera.cmf.model.DbConfigContainer -> CONFIG_CONTAINERS
    2012-09-16 16:26:19,056 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbHost.activeCommands -> COMMANDS
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbHost.processes -> PROCESSES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbHost.roles -> ROLES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbRole.processes -> PROCESSES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbService.roles -> ROLES
    2012-09-16 16:26:19,067 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbService.activeCommands ->
    COMMANDS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbService.configRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbService.visibleConfigRevisions
    -> CONFIG_REVISIONS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbService.clientConfigs ->
    CLIENT_CONFIGS
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbCommand.processes -> PROCESSES
    2012-09-16 16:26:19,068 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbCommand.children -> COMMANDS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbUser.userRoles -> USER_ROLES
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbCluster.activeCommands ->
    COMMANDS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508]
    Mapping collection: com.cloudera.cmf.model.DbConfigRevision.configsForDb ->
    CONFIGS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508]
    Mapping collection:
    com.cloudera.cmf.model.DbConfigContainer.configRevisions -> CONFIG_REVISIONS
    2012-09-16 16:26:19,069 INFO [main:cfg.HbmBinder@2508]
    Mapping collection:
    com.cloudera.cmf.model.DbConfigContainer.visibleConfigRevisions ->
    CONFIG_REVISIONS
    2012-09-16 16:26:19,071 INFO [main:cfg.Configuration@1676]
    Hibernate Validator not found: ignoring
    2012-09-16 16:26:19,347 INFO
    [main:search.HibernateSearchEventListenerRegister@75] Unable to find
    org.hibernate.search.event.FullTextIndexEventListener on the classpath.
    Hibernate Search is not enabled.
    2012-09-16 16:26:19,394 INFO
    [main:connection.ConnectionProviderFactory@173] Initializing connection
    provider: org.hibernate.connection.C3P0ConnectionProvider
    2012-09-16 16:26:19,395 INFO
    [main:connection.C3P0ConnectionProvider@103] C3P0 using driver:
    org.postgresql.Driver at URL: jdbc:postgresql://localhost:7432/scm
    2012-09-16 16:26:19,395 INFO
    [main:connection.C3P0ConnectionProvider@104] Connection properties:
    {user=scm, password=****, autocommit=true, release_mode=auto}
    2012-09-16 16:26:19,396 INFO
    [main:connection.C3P0ConnectionProvider@107] autocommit mode: true
    2012-09-16 16:26:19,464 INFO [main:log.MLog@80] MLog
    clients using log4j logging.
    2012-09-16 16:26:19,691 INFO [main:c3p0.C3P0Registry@204]
    Initializing c3p0-0.9.1 [built 16-January-2007 14:46:42; debug? true;
    trace: 10]
    @


    so any ideaaaa ????

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedSep 16, '12 at 7:59p
activeNov 20, '12 at 3:43p
posts14
users3
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase