Key Concepts in Kafka

Apache Kafka is a publish-subscribe messaging system which lets you send messages between processes, applications, and servers. Applications which need to read data from Kafka use a KafkaConsumer to subscribe to Kafka topics and receive messages from these topics.

Reading data from Kafka is a bit different than reading data from other messaging systems, and there are few unique concepts involved. Let’s first review some of the key concepts in Kafka.

1.Topics

Similar to a table in database, topics is a particular stream of data. Topics consist of one or more partitions, ordered, immutable sequences of messages to which Kafka appends new messages.

2. Partitions and Offset

Topics are split in partitions. Each message within a partition gets an incremental id, called offset.