FAQ
I have a Flume service installed on 3 different hosts (meaning it uses the
same configuration settings). So, service flume1 has 3 instances.

The first instance, which is installed on the same system as the zookeeper
service, connects to my Hbase instance as expected.

The other two instances on the other hosts keeps on trying to connect to
0.0.0.0:2181 instead of connecting to the zookeeper instance.

The reason I suspect zookeeper is the following from the log output of the
other two agents:
12:54:17.293 AMINFOorg.apache.zookeeper.ClientCnxn

Opening socket connection to server localhost/127.0.0.1:2181. Will not
attempt to authenticate using SASL (Unable to locate a login
configuration)

12:54:17.294 AMWARNorg.apache.zookeeper.ClientCnxn

Session 0x0 for server null, unexpected error, closing socket
connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1047)

Search Discussions

  • Patrick Ethier at Nov 20, 2012 at 3:43 pm
    It seems that uninstalling zookeeper and re-installing it then gave me the
    option to "Update Client Configurations" that seems to push out the zoo.cfg
    to all the Flume hosts properly.

    Once I did this, Flume finds my Hbase.

    On Sat, Nov 17, 2012 at 12:57 AM, Patrick Ethier wrote:

    I have a Flume service installed on 3 different hosts (meaning it uses the
    same configuration settings). So, service flume1 has 3 instances.

    The first instance, which is installed on the same system as the zookeeper
    service, connects to my Hbase instance as expected.

    The other two instances on the other hosts keeps on trying to connect to
    0.0.0.0:2181 instead of connecting to the zookeeper instance.

    The reason I suspect zookeeper is the following from the log output of the
    other two agents:
    12:54:17.293 AM INFO org.apache.zookeeper.ClientCnxn

    Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (Unable to locate a login configuration)

    12:54:17.294 AM WARN org.apache.zookeeper.ClientCnxn

    Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
    java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)

    at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1047)
  • Patrick Ethier at Nov 20, 2012 at 3:43 pm
    not to sound annoying but I got the same issue on a new node in my cluster.

    It seems that using Flume with an HBase sink won't work unless Flume is
    installed on an HBase region server. It seems that it's dependent on the
    region server's zoo.cfg.

    I don't know what to do in order to fix this. Perhaps giving Flume it's own
    zoo.cfg file independent of HBase's classpath?

    Anyhow, temporary fix is to install a Region Server role along with Flume
    and it makes it work.

    Pat


    On Sat, Nov 17, 2012 at 12:38 PM, Patrick Ethier wrote:

    It seems that uninstalling zookeeper and re-installing it then gave me the
    option to "Update Client Configurations" that seems to push out the zoo.cfg
    to all the Flume hosts properly.

    Once I did this, Flume finds my Hbase.

    On Sat, Nov 17, 2012 at 12:57 AM, Patrick Ethier wrote:

    I have a Flume service installed on 3 different hosts (meaning it uses
    the same configuration settings). So, service flume1 has 3 instances.

    The first instance, which is installed on the same system as the
    zookeeper service, connects to my Hbase instance as expected.

    The other two instances on the other hosts keeps on trying to connect to
    0.0.0.0:2181 instead of connecting to the zookeeper instance.

    The reason I suspect zookeeper is the following from the log output of
    the other two agents:
    12:54:17.293 AM INFO org.apache.zookeeper.ClientCnxn

    Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (Unable to locate a login configuration)

    12:54:17.294 AM WARN org.apache.zookeeper.ClientCnxn

    Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
    java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)


    at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1047)
  • Vikram Srivastava at Nov 20, 2012 at 4:47 pm
    Hey Patrick,

    You don't need to add a region server role to Flume Agent's host. You can
    add an Hbase "Gateway" role to the host and then "Deploy Client
    Configuration" for HBase service (from the Actions menu on HBase service
    page).

    Regards,

    Vikram
    On Sat, Nov 17, 2012 at 10:54 AM, Patrick Ethier wrote:

    not to sound annoying but I got the same issue on a new node in my cluster.

    It seems that using Flume with an HBase sink won't work unless Flume is
    installed on an HBase region server. It seems that it's dependent on the
    region server's zoo.cfg.

    I don't know what to do in order to fix this. Perhaps giving Flume it's
    own zoo.cfg file independent of HBase's classpath?

    Anyhow, temporary fix is to install a Region Server role along with Flume
    and it makes it work.

    Pat


    On Sat, Nov 17, 2012 at 12:38 PM, Patrick Ethier wrote:

    It seems that uninstalling zookeeper and re-installing it then gave me
    the option to "Update Client Configurations" that seems to push out the
    zoo.cfg to all the Flume hosts properly.

    Once I did this, Flume finds my Hbase.

    On Sat, Nov 17, 2012 at 12:57 AM, Patrick Ethier wrote:

    I have a Flume service installed on 3 different hosts (meaning it uses
    the same configuration settings). So, service flume1 has 3 instances.

    The first instance, which is installed on the same system as the
    zookeeper service, connects to my Hbase instance as expected.

    The other two instances on the other hosts keeps on trying to connect to
    0.0.0.0:2181 instead of connecting to the zookeeper instance.

    The reason I suspect zookeeper is the following from the log output of
    the other two agents:
    12:54:17.293 AM INFO org.apache.zookeeper.ClientCnxn

    Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (Unable to locate a login configuration)

    12:54:17.294 AM WARN org.apache.zookeeper.ClientCnxn

    Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
    java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)



    at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:286)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1047)

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedNov 20, '12 at 3:43p
activeNov 20, '12 at 4:47p
posts4
users2
websitecloudera.com
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase