I'm currently looking into implementing a load shedding strategy into
Flink's Kafka consumer.
Therefore, I would like to allow users to request the latest offset of the
subscribed TopicPartitions, so that they can
a) determine the lag
b) maybe set the next fetch offset to the latest offset (or anything in
With the SimpleConsumer, there is the getOffsetsBefore() method, which
allows to request the largest offset.
For the new KafkaConsumer, I only found the seekToEnd() and
seekToBeginning() methods, but they don't allow to get the actual end or
Are there plans to add such an API to the KafkaConsumer again?
Can you suggest any workarounds?