FAQ
Hi,

I just set up a storm cluster with storm-deploy by following
wiki https://github.com/nathanmarz/storm-deploy/wiki. However, when I
tried to run WordCountTopology on the cluster, I got "Connection refused"
error.

$ storm jar target/storm-starter-0.0.1-SNAPSHOT-jar-with-dependencies.jar
storm.starter.WordCountTopology wordcount
Exception in thread "main" java.lang.RuntimeException:
org.apache.thrift7.transport.TTransportException:
java.net.ConnectException: Connection refused
at backtype.storm.utils.NimbusClient.(NimbusClient.java:17)
at
backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:53)
at storm.starter.WordCountTopology.main(WordCountTopology.java:78)
Caused by: org.apache.thrift7.transport.TTransportException:
java.net.ConnectException: Connection refused
at org.apache.thrift7.transport.TSocket.open(TSocket.java:183)
at
org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:34)
... 3 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(Unknown Source)
at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at org.apache.thrift7.transport.TSocket.open(TSocket.java:178)
... 5 more

I verified that port 22 on nimbus host is reachable. So I guess this may
be a key related issue. AWS console shows that my nimbus host uses key pair
"jclouds#nimbus-mycluster#us-east-1#87". I believe this key pair was
created by storm-deploy. My questions are:
1. Where do I find the private key of this key pair. How do I ssh to my
nimbus host?
2. What can I do to trouble shoot the connection error?

Really appreciate if you can give me some help. Thanks!

Terry

Search Discussions

  • Michael Rose at Jan 13, 2013 at 6:53 am
    You'll need to verify nimbus is running and further that port 6627 (storm thrift port) is open.

    The fact that it's connection refused leads me to believe that perhaps your Nimbus server didn't actually start.

    You can ssh into your nimbus host with the storm user. The keypair its self should have been specified during setup (it would have been copied as "jclouds#nimbus-mycluster etc".

    ssh -i /path/to/my/keypair.pem storm@mynimbushostname

    --
    Michael Rose (@Xorlev (https://twitter.com/xorlev))
    Senior Platform Engineer, FullContact (http://fullcontact.com/)
    michael@fullcontact.com (mailto:michael@fullcontact.com)

    On Friday, January 11, 2013 at 5:07 PM, Terry Zhuo wrote:

    Hi,

    I just set up a storm cluster with storm-deploy by following wiki https://github.com/nathanmarz/storm-deploy/wiki. However, when I tried to run WordCountTopology on the cluster, I got "Connection refused" error.

    $ storm jar target/storm-starter-0.0.1-SNAPSHOT-jar-with-dependencies.jar storm.starter.WordCountTopology wordcount
    Exception in thread "main" java.lang.RuntimeException: org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused
    at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:36)
    at backtype.storm.utils.NimbusClient.getConfiguredClient(NimbusClient.java:17)
    at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:53)
    at storm.starter.WordCountTopology.main(WordCountTopology.java:78)
    Caused by: org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused
    at org.apache.thrift7.transport.TSocket.open(TSocket.java:183)
    at org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
    at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:34)
    ... 3 more
    Caused by: java.net.ConnectException: Connection refused
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.PlainSocketImpl.doConnect(Unknown Source)
    at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
    at java.net.PlainSocketImpl.connect(Unknown Source)
    at java.net.SocksSocketImpl.connect(Unknown Source)
    at java.net.Socket.connect(Unknown Source)
    at org.apache.thrift7.transport.TSocket.open(TSocket.java:178)
    ... 5 more


    I verified that port 22 on nimbus host is reachable. So I guess this may be a key related issue. AWS console shows that my nimbus host uses key pair "jclouds#nimbus-mycluster#us-east-1#87". I believe this key pair was created by storm-deploy. My questions are:
    1. Where do I find the private key of this key pair. How do I ssh to my nimbus host?
    2. What can I do to trouble shoot the connection error?

    Really appreciate if you can give me some help. Thanks!

    Terry
  • Terry Zhuo at Jan 16, 2013 at 11:49 pm
    Yes. I can see jclouds created key pairs like
    "jclouds#zookeeper-gat#us-east-1#16". But I don't know where does it keep
    the private key.

    Where does jclouds store the private keys? Without the private keys, I
    cannot access my nimbus hosts.

    Terry
    On Saturday, January 12, 2013 10:53:06 PM UTC-8, Michael Rose wrote:

    You'll need to verify nimbus is running and further that port 6627 (storm
    thrift port) is open.

    The fact that it's connection refused leads me to believe that perhaps
    your Nimbus server didn't actually start.

    You can ssh into your nimbus host with the storm user. The keypair its
    self should have been specified during setup (it would have been copied as
    "jclouds#nimbus-mycluster etc".

    ssh -i /path/to/my/keypair.pem storm@mynimbushostname

    --
    Michael Rose (@Xorlev <https://twitter.com/xorlev>)
    Senior Platform Engineer, FullContact <http://fullcontact.com/>
    mic...@fullcontact.com <javascript:>

    On Friday, January 11, 2013 at 5:07 PM, Terry Zhuo wrote:

    Hi,

    I just set up a storm cluster with storm-deploy by following wiki
    https://github.com/nathanmarz/storm-deploy/wiki. However, when I tried
    to run WordCountTopology on the cluster, I got "Connection refused" error.

    $ storm jar target/storm-starter-0.0.1-SNAPSHOT-jar-with-dependencies.jar
    storm.starter.WordCountTopology wordcount
    Exception in thread "main" java.lang.RuntimeException:
    org.apache.thrift7.transport.TTransportException:
    java.net.ConnectException: Connection refused
    at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:36)
    at
    backtype.storm.utils.NimbusClient.getConfiguredClient(NimbusClient.java:17)
    at
    backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:53)
    at storm.starter.WordCountTopology.main(WordCountTopology.java:78)
    Caused by: org.apache.thrift7.transport.TTransportException:
    java.net.ConnectException: Connection refused
    at org.apache.thrift7.transport.TSocket.open(TSocket.java:183)
    at
    org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
    at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:34)
    ... 3 more
    Caused by: java.net.ConnectException: Connection refused
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.PlainSocketImpl.doConnect(Unknown Source)
    at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
    at java.net.PlainSocketImpl.connect(Unknown Source)
    at java.net.SocksSocketImpl.connect(Unknown Source)
    at java.net.Socket.connect(Unknown Source)
    at org.apache.thrift7.transport.TSocket.open(TSocket.java:178)
    ... 5 more

    I verified that port 22 on nimbus host is reachable. So I guess this may
    be a key related issue. AWS console shows that my nimbus host uses key pair
    "jclouds#nimbus-mycluster#us-east-1#87". I believe this key pair was
    created by storm-deploy. My questions are:
    1. Where do I find the private key of this key pair. How do I ssh to my
    nimbus host?
    2. What can I do to trouble shoot the connection error?

    Really appreciate if you can give me some help. Thanks!

    Terry

  • Justin Searles at Jan 17, 2013 at 3:29 am
    If you used storm-deploy, you're private key should be the same that was
    used in the .pallet/config.clj file.
    On Wednesday, January 16, 2013 6:49:09 PM UTC-5, Terry Zhuo wrote:

    Yes. I can see jclouds created key pairs like
    "jclouds#zookeeper-gat#us-east-1#16". But I don't know where does it keep
    the private key.

    Where does jclouds store the private keys? Without the private keys, I
    cannot access my nimbus hosts.

    Terry
    On Saturday, January 12, 2013 10:53:06 PM UTC-8, Michael Rose wrote:

    You'll need to verify nimbus is running and further that port 6627
    (storm thrift port) is open.

    The fact that it's connection refused leads me to believe that perhaps
    your Nimbus server didn't actually start.

    You can ssh into your nimbus host with the storm user. The keypair its
    self should have been specified during setup (it would have been copied as
    "jclouds#nimbus-mycluster etc".

    ssh -i /path/to/my/keypair.pem storm@mynimbushostname

    --
    Michael Rose (@Xorlev <https://twitter.com/xorlev>)
    Senior Platform Engineer, FullContact <http://fullcontact.com/>
    mic...@fullcontact.com

    On Friday, January 11, 2013 at 5:07 PM, Terry Zhuo wrote:

    Hi,

    I just set up a storm cluster with storm-deploy by following wiki
    https://github.com/nathanmarz/storm-deploy/wiki. However, when I tried
    to run WordCountTopology on the cluster, I got "Connection refused" error.

    $ storm jar target/storm-starter-0.0.1-SNAPSHOT-jar-with-dependencies.jar
    storm.starter.WordCountTopology wordcount
    Exception in thread "main" java.lang.RuntimeException:
    org.apache.thrift7.transport.TTransportException:
    java.net.ConnectException: Connection refused
    at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:36)
    at
    backtype.storm.utils.NimbusClient.getConfiguredClient(NimbusClient.java:17)
    at
    backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:53)
    at storm.starter.WordCountTopology.main(WordCountTopology.java:78)
    Caused by: org.apache.thrift7.transport.TTransportException:
    java.net.ConnectException: Connection refused
    at org.apache.thrift7.transport.TSocket.open(TSocket.java:183)
    at
    org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
    at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:34)
    ... 3 more
    Caused by: java.net.ConnectException: Connection refused
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.PlainSocketImpl.doConnect(Unknown Source)
    at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
    at java.net.PlainSocketImpl.connect(Unknown Source)
    at java.net.SocksSocketImpl.connect(Unknown Source)
    at java.net.Socket.connect(Unknown Source)
    at org.apache.thrift7.transport.TSocket.open(TSocket.java:178)
    ... 5 more

    I verified that port 22 on nimbus host is reachable. So I guess this may
    be a key related issue. AWS console shows that my nimbus host uses key pair
    "jclouds#nimbus-mycluster#us-east-1#87". I believe this key pair was
    created by storm-deploy. My questions are:
    1. Where do I find the private key of this key pair. How do I ssh to my
    nimbus host?
    2. What can I do to trouble shoot the connection error?

    Really appreciate if you can give me some help. Thanks!

    Terry

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupstorm-user @
postedJan 13, '13 at 5:32a
activeJan 17, '13 at 3:29a
posts4
users3
websitestorm-project.net
irc#storm-user

People

Translate

site design / logo © 2022 Grokbase