Grokbase Groups Kafka dev June 2016
[ ]

Ismael Juma commented on KAFKA-1789:

To add to Gwen's comment, the Scala producers have been deprecated and the new Java producer is recommended instead.
Issue with Async producer

Key: KAFKA-1789
Project: Kafka
Issue Type: Bug
Affects Versions: 0.8.0, 0.8.1
Reporter: devendra tagare
Priority: Critical

We are using an async producer to send data to a kafka cluster.The event rate at peak is around 250 events/second of size 25KB each.
In the producer code base we have added specific debug statements to capture the time taken to create a producer,create a keyed message with a byte payload & send the message.
We have added the below properties to the producerConfig
Based on the documentation, producer.send() queues the message on the async producer's queue.
So, ideally if the queue is full then the enqueue operation should result in an kafka.common.QueueFullException in 20 ms.
The logs indicate that the enqueue operation is taking more than 20ms (takes around 250ms) without throwing any exceptions.
Is there any other property that could conflict with which is causing this behavior ?
Or is it possible that the queue is not full & yet the producer.send() call is still taking around 200ms under peak load ?
Also, could you suggest any other alternatives so that we can either enforce a timeout or throw an exception in-case the async producer is taking more than a specified amount of time.

This message was sent by Atlassian JIRA

Search Discussions

Discussion Posts


Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 2 of 2 | next ›
Discussion Overview
groupdev @
postedJun 18, '16 at 11:32a
activeJun 18, '16 at 1:32p

1 user in discussion

Ismael Juma (JIRA): 2 posts



site design / logo © 2022 Grokbase