FAQ
I'm looking at the kafka server.properties that is in /config folder and
didn't find any reference to topics or partitions.

Are topics and optionally partitions dynamic in nature or do they have to
be defined before starting the broker? (and consequently producers can't
send to a topic and partition that isn't predefined).

Search Discussions

  • Neha Narkhede at Jan 27, 2013 at 8:37 pm
    In Kafka 0.7, topics and partitions are dynamic in the sense that a
    partition is created when the broker receives the first message for that
    partition.
    In Kafka 0.8, there are 2 ways of creating a new topic -
    1. Turn on auto.create.topics.enable option on the broker. When the broker
    receives the first message for a new topic, it creates that topic with
    num.partitions and default.replication.factor.
    2. Use the admin command bin/kafka-create-topic.sh

    Thanks,
    Neha

    On Sun, Jan 27, 2013 at 12:23 PM, S Ahmed wrote:

    I'm looking at the kafka server.properties that is in /config folder and
    didn't find any reference to topics or partitions.

    Are topics and optionally partitions dynamic in nature or do they have to
    be defined before starting the broker? (and consequently producers can't
    send to a topic and partition that isn't predefined).
  • S Ahmed at Jan 27, 2013 at 8:40 pm
    So from what I understand, a single broker will contain all the messages
    for a given topic (and its partitions). Who decides which broker will own
    a particular topic?

    On Sun, Jan 27, 2013 at 3:36 PM, Neha Narkhede wrote:

    In Kafka 0.7, topics and partitions are dynamic in the sense that a
    partition is created when the broker receives the first message for that
    partition.
    In Kafka 0.8, there are 2 ways of creating a new topic -
    1. Turn on auto.create.topics.enable option on the broker. When the broker
    receives the first message for a new topic, it creates that topic with
    num.partitions and default.replication.factor.
    2. Use the admin command bin/kafka-create-topic.sh

    Thanks,
    Neha

    On Sun, Jan 27, 2013 at 12:23 PM, S Ahmed wrote:

    I'm looking at the kafka server.properties that is in /config folder and
    didn't find any reference to topics or partitions.

    Are topics and optionally partitions dynamic in nature or do they have to
    be defined before starting the broker? (and consequently producers can't
    send to a topic and partition that isn't predefined).
  • Jun Rao at Jan 27, 2013 at 11:50 pm
    In 0.8, when a topic is created, we decide how each replica of a partition
    is mapped to a broker. In general, we try to spread the replicas across all
    brokers.

    Thanks,

    Jun
    On Sun, Jan 27, 2013 at 12:40 PM, S Ahmed wrote:

    So from what I understand, a single broker will contain all the messages
    for a given topic (and its partitions). Who decides which broker will own
    a particular topic?


    On Sun, Jan 27, 2013 at 3:36 PM, Neha Narkhede <neha.narkhede@gmail.com
    wrote:
    In Kafka 0.7, topics and partitions are dynamic in the sense that a
    partition is created when the broker receives the first message for that
    partition.
    In Kafka 0.8, there are 2 ways of creating a new topic -
    1. Turn on auto.create.topics.enable option on the broker. When the broker
    receives the first message for a new topic, it creates that topic with
    num.partitions and default.replication.factor.
    2. Use the admin command bin/kafka-create-topic.sh

    Thanks,
    Neha

    On Sun, Jan 27, 2013 at 12:23 PM, S Ahmed wrote:

    I'm looking at the kafka server.properties that is in /config folder
    and
    didn't find any reference to topics or partitions.

    Are topics and optionally partitions dynamic in nature or do they have
    to
    be defined before starting the broker? (and consequently producers
    can't
    send to a topic and partition that isn't predefined).
  • Arjun at Dec 4, 2013 at 3:13 pm
    hi,

    I was just looking at kafka 0.8, I could not find any option in
    config/server.properties with key "auto.create.topics.enable" or
    "default.replication.factor".

    Can some one help me out,where i can find these. I want my topics to be
    created dynamically.


    thanks
    Arjun Narasimha Kota
  • Magnus Edenhill at Dec 4, 2013 at 3:17 pm
    See the configuration reference here:

    https://kafka.apache.org/documentation.html#configuration


    2013/12/4 Arjun <arjun@socialtwist.com>
    hi,

    I was just looking at kafka 0.8, I could not find any option in
    config/server.properties with key "auto.create.topics.enable" or
    "default.replication.factor".

    Can some one help me out,where i can find these. I want my topics to be
    created dynamically.


    thanks
    Arjun Narasimha Kota
  • 小宇 at Dec 4, 2013 at 3:26 pm
    "auto.create.topics.enable" are default true, so you don't need care about
    it. And you can set it false in the server.properties


    2013/12/4 Magnus Edenhill <magnus@edenhill.se>
    See the configuration reference here:

    https://kafka.apache.org/documentation.html#configuration


    2013/12/4 Arjun <arjun@socialtwist.com>
    hi,

    I was just looking at kafka 0.8, I could not find any option in
    config/server.properties with key "auto.create.topics.enable" or
    "default.replication.factor".

    Can some one help me out,where i can find these. I want my topics to be
    created dynamically.


    thanks
    Arjun Narasimha Kota

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupusers @
categorieskafka
postedJan 27, '13 at 8:24p
activeDec 4, '13 at 3:26p
posts7
users6
websitekafka.apache.org
irc#kafka

People

Translate

site design / logo © 2017 Grokbase