Grokbase Groups Kafka users
FAQ

Search Discussions

4,701 discussions - 19,880 posts

  • I send every message with timestamp, and when I receive a message,I do a subtraction between current timestamp and message’s timestamp. then I get the consumer’s delay.
    KafkaKafka
    Jun 18, 2016 at 2:26 pm
    Jun 18, 2016 at 2:26 pm
  • hello,I have done a series of tests on kafka 0.9.0,and one of the results confused me. test enviroment: kafka cluster: 3 brokers,8core cpu / 8g mem /1g netcard client:4core cpu/4g mem topic:6 ...
    KafkaKafka
    Jun 18, 2016 at 3:27 am
    Jun 18, 2016 at 3:27 am
  • Does anybody have good RHEL 7 systemd scripts for the broker, zookeeper, schema registry, etc ... ? This e-mail and any files transmitted with it are confidential, may contain sensitive information, ...
    Tauzell, DaveTauzell, Dave
    Jun 17, 2016 at 6:51 pm
    Jun 17, 2016 at 6:51 pm
  • If you have a consumer listening on a topic and that topic is deleted is the consumer made aware -- perhaps by some exception -- or does it continue listening, blissfully unaware that it will never ...
    Chris BarlockChris Barlock
    Jun 17, 2016 at 5:56 pm
    Jun 17, 2016 at 5:56 pm
  • Hi. I am trying to use an embedded Kafka server to allow me to create tests in jUnit using a real Kafka implementation, instead of a stub or a mock. I am using Kafka version 0.9.0.0. The test works, ...
    Rodrigo OtteroRodrigo Ottero
    Jun 17, 2016 at 4:34 pm
    Jun 17, 2016 at 4:34 pm
  • Hello, I'm currently working with kafka 0.10.0.0 in Java 8 and facing the following issue: When I create multiple topics with the AdminUtil one by one and immediately start an KafkaConsumer ...
    Lukas HauserLukas Hauser
    Jun 17, 2016 at 3:16 pm
    Jun 17, 2016 at 3:16 pm
  • Hello people, I'm running one single kafka broker from within a docker container. The folder where kafka writes the logs is mounted as *docker volume* on my system. As soon as I try to create a topic ...
    OGrandeDiEnneOGrandeDiEnne
    Jun 17, 2016 at 11:25 am
    Jun 17, 2016 at 1:19 pm
  • I am using Kafka 0.9.0.1 with ZK 3.5.0-alpha I am seeing this error intermittently which goes away after several reboots. Any ideas? Sent from my iPad
    Nomar MoradoNomar Morado
    Jun 17, 2016 at 11:14 am
    Jun 17, 2016 at 1:57 pm
  • Hello! I would like to know what are the configurations/properties for the producer/consumer in order fail fast when the connection to the entire broker cluster is lost. For example if we can set up ...
    Spico FlorinSpico Florin
    Jun 17, 2016 at 10:28 am
    Jun 17, 2016 at 10:28 am
  • Hello, As part of the log cleaning which compacts the topic we saw that it encountered the following exception kafka.message.InvalidMessageException: Message found with corrupt size (0) How do we ...
    Anishek AgarwalAnishek Agarwal
    Jun 17, 2016 at 4:53 am
    Jun 17, 2016 at 4:53 am
  • Hi Users, I am facing message loss while using kafka v 0.8.2.2. Please see details below and help me if you can. Issue: 2 messages produced to same partition one by one – Kafka producer returns same ...
    Gulia, VikramGulia, Vikram
    Jun 17, 2016 at 3:19 am
    Jun 17, 2016 at 4:47 pm
  • Hi all, I'm trying to do a KTable-KTable join to compute an average within a tumbling window. Here's the KStreams code (I've put a fully working example in a gist ...
    Tim RennerTim Renner
    Jun 16, 2016 at 11:12 pm
    Jun 18, 2016 at 3:20 am
  • Hi all, I would like to start a discussion on making Java 8 a minimum requirement for Kafka's next feature release (let's say Kafka 0.10.1.0 for now). This is the first discussion on the topic so the ...
    Ismael JumaIsmael Juma
    Jun 16, 2016 at 8:46 pm
    Jun 17, 2016 at 6:14 pm
  • We encountered a corrupted topic and when we attempt to delete it, it comes back with some unusable defaults. It's really, really annoying. We are shutting down all the kafka brokers, removing the ...
    Jason KaniaJason Kania
    Jun 16, 2016 at 7:12 pm
    Jun 16, 2016 at 7:39 pm
  • The issue had apparently existed and is apparently resolved, but so far it does not work for me: https://issues.apache.org/jira/browse/KAFKA-2203. I issue same command as Stevo Slavic with JDK ...
    Philippe DeromePhilippe Derome
    Jun 16, 2016 at 4:03 pm
    Jun 17, 2016 at 7:08 am
  • Hi, I am trying to use groupby operator in simple example. However, I get strange results. I have inputs on "test" topic like: (Long, String) 1 Message_1 1 Message_1 2 Message_2 3 Message_3 4 ...
    Davood RafieiDavood Rafiei
    Jun 16, 2016 at 2:21 pm
    Jun 17, 2016 at 3:33 pm
  • Hi, I am curious about the relationship of KTables and state store in Kafka Streams. When I create a table with the `table` method, it uses the original topic as a changelog, thus persisting a table ...
    Jan EhrhardtJan Ehrhardt
    Jun 16, 2016 at 11:45 am
    Jun 16, 2016 at 6:49 pm
  • Hi, We've noticed that we have some partitions receiving more messages than others. What I've done to learn that is: * In Kafka Manager, per a given topic, the list of Partition Information is ...
    Asaf MesikaAsaf Mesika
    Jun 16, 2016 at 9:39 am
    Jun 16, 2016 at 9:39 am
  • My team has published a post comparing python kafka clients. Might be of interest to python users. http://activisiongamescience.github.io/2016/06/15/Kafka-Client-Benchmarking/
    John DennisonJohn Dennison
    Jun 16, 2016 at 1:40 am
    Jun 16, 2016 at 2:33 pm
  • Hi All, I am embedding Kafka 0.10.0 and corresponding zookeeper in java process. In this process, I start zookeeper first and then wait for 10 seconds and then start kafka. These are all running in ...
    Subhash AgrawalSubhash Agrawal
    Jun 15, 2016 at 8:56 pm
    Jun 15, 2016 at 10:39 pm
  • Hi, I was following the Quickstart guide and I have noticed that ConsoleProducer does not publish all messages (the number of messages published differs from one run to another) and happens mostly on ...
    Radu RadutiuRadu Radutiu
    Jun 15, 2016 at 4:21 pm
    Jun 17, 2016 at 8:06 am
  • Hi community, Probably it is very basic question, as I am new to Kafka Streams. I am trying to initialize KTable or KStream from kafka topic. However, I don't know how to avoid getting null keys. So, ...
    Adrienne KoleAdrienne Kole
    Jun 15, 2016 at 4:21 pm
    Jun 16, 2016 at 3:07 pm
  • I have a very simple driver which loads a textFile and filters a sub-string from each line in the textfile. When the collect action is executed , I am getting an exception. (The file is only 90 MB - ...
    VGVG
    Jun 15, 2016 at 2:19 pm
    Jun 15, 2016 at 2:19 pm
  • Hello, I am running stream processing job with Kafka and Flink. Flink reads records from Kafka. My software versions are: - Kafka broker: 0.9.0.2.4 (HDP 2.4.0.0 version) - Kafka client library ...
    Hironori OgibayashiHironori Ogibayashi
    Jun 15, 2016 at 12:41 pm
    Jun 15, 2016 at 1:51 pm
  • Hello Guys. We are going to install Apache Kafka in our local data center and different producers which are distributed across different locations will be connected to this server. Our Producers will ...
    AliAli
    Jun 15, 2016 at 7:03 am
    Jun 15, 2016 at 7:03 am
  • I have the HdfsSink reading from a topic. If it finds a message that it cannot deal with then it stops processing. Is there a way to tell the HdfsSink to put that message somewhere else and continue ...
    Tauzell, DaveTauzell, Dave
    Jun 14, 2016 at 6:58 pm
    Jun 15, 2016 at 12:09 am
  • Hello, I'm processing events from several users but one user should not affect the other user processing throughput. My initial idea was: one single topic partitioned by the userId. So if I have 5000 ...
    Francisco LopesFrancisco Lopes
    Jun 14, 2016 at 6:51 pm
    Jun 14, 2016 at 7:59 pm
  • I have been able to get my C# client to put avro records to a Kafka topic and have the HdfsSink read and save them in files. I am confused about interaction with the registry. The kafka message ...
    Tauzell, DaveTauzell, Dave
    Jun 14, 2016 at 3:10 pm
    Jun 15, 2016 at 2:10 pm
  • Hi everyone, I would like to know if there is a way to shutdown a connector programmatically ? On my project we have developped a sink-connector to write messages into GZIP files for testing ...
    Florian HussonnoisFlorian Hussonnois
    Jun 14, 2016 at 2:04 pm
    Jun 14, 2016 at 11:52 pm
  • Hi, I started working on Apache Kafka and want to be included in users group. Please include me. Thank you. Best Regards, Srikanth.
    Srikanth HugarSrikanth Hugar
    Jun 14, 2016 at 1:35 pm
    Jun 14, 2016 at 6:09 pm
  • Hello everyone, I have use kafka_2.11-0.10.0.0, I write a simple consumer code. But I can't find consumer offset in zookeeper(I have check ...
    ThinkingThinking
    Jun 14, 2016 at 1:34 pm
    Jun 14, 2016 at 1:36 pm
  • Hey, How can I delete particular messages from particular topic?Is that possible? Thanks, Mudit
    Mudit KumarMudit Kumar
    Jun 14, 2016 at 12:29 pm
    Jun 14, 2016 at 5:33 pm
  • Hi, I have a cluster of 3 brokers and 1 topic which has 1 partition and replication factor of 3. There are also 3 consumers consuming from that topic. Now all the docs I have seen say that if number ...
    Al-Isawi RamiAl-Isawi Rami
    Jun 14, 2016 at 10:59 am
    Jun 14, 2016 at 12:41 pm
  • Hi All, I am using kafka version kafka_2.10-0.9.0.1/ At some random time we are not able to send message to kafka It gives error as "Failed to send message after three times." is it network error? ...
    Snehalata NagajeSnehalata Nagaje
    Jun 14, 2016 at 10:41 am
    Jun 14, 2016 at 11:17 am
  • a very good book on big data interview preparation https://notionpress.com/read/big-data-interview-faqs chaturvedi
    Chaturvedi CholaChaturvedi Chola
    Jun 14, 2016 at 5:56 am
    Jun 14, 2016 at 5:56 am
  • As part of testing v0.9 Kafka at least once guarantees, we tried disconnecting Producer network and found that retries=10000000 are not happening. We get a WARN kafka-producer-network-thread | ...
    R KrishnaR Krishna
    Jun 14, 2016 at 3:32 am
    Jun 15, 2016 at 9:04 am
  • Hi Everyone, Is there a way to start Kafka Connector via JMX? Thanks, Abhinav
    Abhinav SolanAbhinav Solan
    Jun 14, 2016 at 12:42 am
    Jun 14, 2016 at 6:04 am
  • In JMX for Kafka producer there are metrics for both request, record, and batch size Max + Avg. What is the difference between these concepts? In the logging use case: I assume record is the single ...
    Allen chanAllen chan
    Jun 13, 2016 at 8:43 pm
    Jun 16, 2016 at 8:02 pm
  • Hi Community, I am new to kafka-streams and trying to run WordCount example. I get this error whatever I do: Exception in thread "StreamThread-1" ...
    Jeyhun KarimovJeyhun Karimov
    Jun 13, 2016 at 7:27 pm
    Jun 17, 2016 at 6:12 am
  • Hello, I have a .Net client (Microsoft KafkaNet) putting serialized Avro messages onto Kafka. I started up the Kafka Connect HDFS connector but it fails with a "No magic byte!" error. After some ...
    Tauzell, DaveTauzell, Dave
    Jun 13, 2016 at 3:45 pm
    Jun 14, 2016 at 5:16 am
  • I'm getting the following exception when trying to register a schema: [2016-06-13 11:37:14,531] INFO 172.23.147.101 - - [13/Jun/2016:11:37:14 -0400] "POST /subjects/GenericAuditRecord/versions ...
    Tauzell, DaveTauzell, Dave
    Jun 13, 2016 at 3:40 pm
    Jun 13, 2016 at 5:38 pm
  • Hi - does anyone know if this is compatible? we like to use ZK's dynamic config but i think it is not available until 3.5.x Thanks.
    Nomar MoradoNomar Morado
    Jun 13, 2016 at 12:03 pm
    Jun 13, 2016 at 12:03 pm
  • Hi, I would like to process a stream with a tumbling window of 5secs, create aggregated stats for keys and push the final aggregates at the end of each window period to a analytics backend. I have ...
    Clive CoxClive Cox
    Jun 13, 2016 at 10:13 am
    Jun 13, 2016 at 3:03 pm
  • https://softwaremill.com/kafka-streams-how-does-it-fit-stream-landscape/ It discusses its key features, abstractions(KStream and KTable); when to, and when not to use kafka streams. Its written by ...
    Prajod VettiyattilPrajod Vettiyattil
    Jun 13, 2016 at 8:12 am
    Jun 13, 2016 at 8:12 am
  • Hello Experts, As per documentation in kafka docs - *Windowing* is a common prerequisite for stateful transformations which group records in a stream, for example, by their timestamps. A local state ...
    Pariksheet BarapatrePariksheet Barapatre
    Jun 13, 2016 at 7:31 am
    Jun 13, 2016 at 11:03 am
  • In connect-distributed.properties I have values like: session.timeout.ms=120000 But at startup I get warnings like: The configuration session.timeout.ms = 120000 was supplied but isn't a I'm getting ...
    Barry KaplanBarry Kaplan
    Jun 13, 2016 at 4:45 am
    Jun 14, 2016 at 5:42 am
  • Hi, I'm building a topology where I am connecting twitter with another application (ex: appA) appA also consists of the following graph model (similar to facebook/twitter) where a user can have ...
    KanaghaKanagha
    Jun 12, 2016 at 11:26 pm
    Jun 12, 2016 at 11:26 pm
  • Hi, I'm running the TwitterProducer task as per https://github.com/Eneco/kafka-connect-twitter connect-standalone <path /connect-source-standalone.properties <path /twitter-source.properties I see ...
    KanaghaKanagha
    Jun 12, 2016 at 9:36 pm
    Jun 14, 2016 at 5:35 am
  • Hello Apache Kafka community, Is it intentional that not all states (like ReplicaDeletionIneligible) are documented on https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Controller+Internals or ...
    Stevo SlavićStevo Slavić
    Jun 12, 2016 at 7:58 am
    Jun 12, 2016 at 7:58 am
  • I'd like to add a notification API to Kafka so that I can receive the following notifications for each broker in the cluster: 1. An event notification when a broker is promoted from follower to ...
    Dominic ChambersDominic Chambers
    Jun 11, 2016 at 11:59 pm
    Jun 11, 2016 at 11:59 pm
Group Navigation
period‹ prev | Latest | first ›
Group Overview
groupusers @
categorieskafka
discussions4,701
posts19,880
users2,183
websitekafka.apache.org
irc#kafka

Top users

Jun Rao: 1479 posts Guozhang Wang: 899 posts Neha Narkhede: 859 posts Jay Kreps: 423 posts Jason Rosenberg: 363 posts Joel Koshy: 307 posts Joe Stein: 298 posts Gshapira: 247 posts Jun Rao: 209 posts Tao xiao: 204 posts Ewen Cheslack-Postava: 193 posts Yu, Libo: 184 posts Jiangjie Qin: 172 posts Gwen Shapira: 158 posts Jason Gustafson: 148 posts Ismael Juma: 131 posts Stevo Slavić: 130 posts Otis Gospodnetic: 130 posts Gharatmayuresh15: 119 posts Bhavesh Mistry: 108 posts
show more