In Kafka, there are three (and just three) ways to deliver messages

Pragya Keshap answered on February 11, 2023 Popularity 1/10 Helpfulness 1/10

answer In Kafka, there are three (and just three) ways to deliver messages

In Kafka, there are three (and just three) ways to deliver messages

Comment

Never redelivered: The messages may be lost because, once delivered, they are not sent again.

May be redelivered: The messages are never lost because, if it is not received, the message can be sent again.

Delivered once: The message is delivered exactly once. This is the most difficult form of delivery; since the message is only sent once and never redelivered, it implies that there is zero loss of any message.

private Map<TopicPartition, OffsetAndMetadata> currentOffsets =
    new HashMap<>();

private class HandleRebalance implements ConsumerRebalanceListener { 1
    public void onPartitionsAssigned(Collection<TopicPartition>
        partitions) { 2
    }

    public void onPartitionsRevoked(Collection<TopicPartition> partitions) {
        System.out.println("Lost partitions in rebalance. " +
            "Committing current offsets:" + currentOffsets);
        consumer.commitSync(currentOffsets); 3
    }
}

try {
    consumer.subscribe(topics, new HandleRebalance()); 4

    while (true) {
        ConsumerRecords<String, String> records = consumer.poll(100);
        for (ConsumerRecord<String, String> record : records) {
            System.out.printf("topic = %s, partition = %s, offset = %d,
                 customer = %s, country = %s\n",
                 record.topic(), record.partition(), record.offset(),
                 record.key(), record.value());
             currentOffsets.put(
                 new TopicPartition(record.topic(), record.partition()),
                 new OffsetAndMetadata(record.offset()+1, null));
        }
        consumer.commitAsync(currentOffsets, null);
    }
} catch (WakeupException e) {
    // ignore, we're closing
} catch (Exception e) {
    log.error("Unexpected error", e);
} finally {
    try {
        consumer.commitSync(currentOffsets);
    } finally {
        consumer.close();
        System.out.println("Closed consumer and we are done");
    }
}

https://learning.oreilly.com/library/view/kafka-the-definitive/9781491936153/ch04.html

Popularity 1/10 Helpfulness 1/10 Language java

Source: Grepper

Tags: java messages

Link to this answer
Share Copy Link

Contributed on Feb 11 2023

Pragya Keshap

0 Answers Avg Quality 2/10

In Kafka, there are three (and just three) ways to deliver messages

Contents

More Related Answers

In Kafka, there are three (and just three) ways to deliver messages

Grepper

Documentation

Social

Legal

Contact

Oops, You will need to install Grepper and log-in to perform this action.