What happens if Kafka topic is full?

Deepika Sahu answered on December 6, 2022 Popularity 7/10 Helpfulness 2/10

answer What happens if Kafka topic is full?

What happens if Kafka topic is full?

Comment

cleanup.policy property from topic config which by default is delete, says that "The delete policy will discard old segments when their retention time or size limit has been reached."

So, if you send record with producer api and topic got full, it will discard old segments.

How can I know if a topic is getting full and logs are being deleted before being consumed? Is there a way to monitor or expose a metric when a topic is getting full?

You can get Partition size using below script:

/bin/kafka-log-dirs.sh --describe --bootstrap-server : --topic-list
You will need to develop a script that will run above script for fetching current size of topic and send it periodically to Datadog. In Datadog, you can create widget that will trigger appropriate action(e.g. sending email alerts) once size reaches a particular threshold.

Popularity 7/10 Helpfulness 2/10 Language whatever

Source: Grepper

Tags: kafka-topic whatever

Link to this answer
Share Copy Link

Contributed on Dec 06 2022

Deepika Sahu

0 Answers Avg Quality 2/10

What happens if Kafka topic is full?

Contents

More Related Answers

What happens if Kafka topic is full?

Grepper

Documentation

Social

Legal

Contact

Oops, You will need to install Grepper and log-in to perform this action.