FAQ
Hi,

I've been looking at the SimpleConsumer example, and that I've noticed that it always reads from the leader, and reacts to leader changes by reconnecting to the new leader. Is it possible to read from a replica in ISR that's not the leader? If so, how does the consumer get notified the replica it's reading from is no longer in ISR?

Thanks,
SimonC

Search Discussions

  • Jun Rao at Nov 27, 2013 at 4:01 pm
    Currently, regular consumers can only fetch from the leader replica.
    Otherwise, they will get an error in response. We allow some special
    consumers to read from follower replicas, but this is really for testing.

    Are you thinking of load balancing? Currently, we do load balancing across
    partitions. We assume that there are many partitions in a cluster and try
    to spread the leaders evenly among brokers. That way, even for a particular
    partition, all clients have to talk to a single broker, the overall
    workload is still balanced.

    Thanks,

    Jun

    On Wed, Nov 27, 2013 at 2:13 AM, Simon Cooper wrote:

    Hi,

    I've been looking at the SimpleConsumer example, and that I've noticed
    that it always reads from the leader, and reacts to leader changes by
    reconnecting to the new leader. Is it possible to read from a replica in
    ISR that's not the leader? If so, how does the consumer get notified the
    replica it's reading from is no longer in ISR?

    Thanks,
    SimonC
  • Hsy541 at Nov 27, 2013 at 6:12 pm
    What I did for my project is I have a thread send metadata request to a
    random broker and monitor the metadata change periodically. The good thing
    is, to my knowledge, any broker in the cluster know the metadata for all
    the topics served in this cluster. Another options is you can always query
    zookeeper to pull the latest information about the cluster

    On Wed, Nov 27, 2013 at 7:57 AM, Jun Rao wrote:

    Currently, regular consumers can only fetch from the leader replica.
    Otherwise, they will get an error in response. We allow some special
    consumers to read from follower replicas, but this is really for testing.

    Are you thinking of load balancing? Currently, we do load balancing across
    partitions. We assume that there are many partitions in a cluster and try
    to spread the leaders evenly among brokers. That way, even for a particular
    partition, all clients have to talk to a single broker, the overall
    workload is still balanced.

    Thanks,

    Jun


    On Wed, Nov 27, 2013 at 2:13 AM, Simon Cooper <
    simon.cooper@featurespace.co.uk> wrote:
    Hi,

    I've been looking at the SimpleConsumer example, and that I've noticed
    that it always reads from the leader, and reacts to leader changes by
    reconnecting to the new leader. Is it possible to read from a replica in
    ISR that's not the leader? If so, how does the consumer get notified the
    replica it's reading from is no longer in ISR?

    Thanks,
    SimonC

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupusers @
categorieskafka
postedNov 27, '13 at 10:14a
activeNov 27, '13 at 6:12p
posts3
users3
websitekafka.apache.org
irc#kafka

3 users in discussion

Hsy541: 1 post Simon Cooper: 1 post Jun Rao: 1 post

People

Translate

site design / logo © 2019 Grokbase