FAQ
Update Aug 25, 2014:

We changed
akka.remote.transport-failure-detector.acceptable-heartbeat-pause = 10 s
instead of 5 s, the WARN message gone. I guess the [Disassociated] WARN
might be caused by network delay or GC pause (Full GC lasts 3+ secs now on
our system) etc. The setting is

akka.remote {

  transport-failure-detector {

    heartbeat-interval = 30 s # default 4s

    acceptable-heartbeat-pause = 10 s # default 10s

  }
}

But, that could not explain the periodic "Disassociated" WARN occurred
before, which, seems could not be recovered from Disassociated state.
On Monday, August 11, 2014 12:08:00 AM UTC+8, Caoyuan wrote:

We have an akka cluster with 10 nodes. it works almost smoothly except
periodic firing "Disassociated" WARN log, which seems cannot be recovered:

The following is the log records.

......
2014-08-10 00:00:09,253 WARN a.remote.ReliableDeliverySupervisor
akka.tcp://
ClusterSystem@10.0.69.169:2551/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%4010.0.65.3%3A2552-5
- Association with remote system [akka.tcp://ClusterSystem@10.0.65.3:2552]
has failed, address is now gated for [5000] ms. Reason is: [Disassociated].

2014-08-10 00:00:44,292 WARN a.remote.ReliableDeliverySupervisor
akka.tcp://
ClusterSystem@10.0.69.169:2551/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%4010.0.65.3%3A2552-5
- Association with remote system [akka.tcp://ClusterSystem@10.0.65.3:2552]
has failed, address is now gated for [5000] ms. Reason is: [Disassociated].

2014-08-10 00:01:49,332 WARN a.remote.ReliableDeliverySupervisor
akka.tcp://
ClusterSystem@10.0.69.169:2551/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%4010.0.65.3%3A2552-5
- Association with remote system [akka.tcp://ClusterSystem@10.0.65.3:2552]
has failed, address is now gated for [5000] ms. Reason is: [Disassociated].

2014-08-10 00:02:24,373 WARN a.remote.ReliableDeliverySupervisor
akka.tcp://
ClusterSystem@10.0.69.169:2551/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%4010.0.65.3%3A2552-5
- Association with remote system [akka.tcp://ClusterSystem@10.0.65.3:2552]
has failed, address is now gated for [5000] ms. Reason is: [Disassociated].

2014-08-10 00:02:59,412 WARN a.remote.ReliableDeliverySupervisor
akka.tcp://
ClusterSystem@10.0.69.169:2551/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%4010.0.65.3%3A2552-5
- Association with remote system [akka.tcp://ClusterSystem@10.0.65.3:2552]
has failed, address is now gated for [5000] ms. Reason is: [Disassociated].

2014-08-10 00:03:34,452 WARN a.remote.ReliableDeliverySupervisor
akka.tcp://
ClusterSystem@10.0.69.169:2551/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%4010.0.65.3%3A2552-5
- Association with remote system [akka.tcp://ClusterSystem@10.0.65.3:2552]
has failed, address is now gated for [5000] ms. Reason is: [Disassociated].
......


The warning continually occurred almost all day, with the period 35
seconds (30 + 5 s) or 65 seconds (30 + 30 + 5 s), which is exactly the
setting of akka.remote's transport failure detector:

akka.remote {

transport-failure-detector {

heartbeat-interval = 30 s # default 4s

acceptable-heartbeat-pause = 5 s # default 10s

}

Where, the failure-detector mark it unavailable after heartbeat-interval
+ acceptable-heartbeat-pause period (35 s).

We're using akka-2.3.3. the node which logged is at 10.0.69.169:2551, and
the remote node is at 10.0.65.3:2552

I tried to dig via the akka.remoting source code, but with no progressing.

Thoughts ?

-Caoyuan Deng
--
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscribe@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 4 of 10 | next ›
Discussion Overview
groupakka-user @
categoriesscala
postedAug 10, '14 at 9:22p
activeAug 27, '14 at 3:20p
posts10
users5
websiteakka.io
irc#akka

People

Translate

site design / logo © 2019 Grokbase