Offset Management: Auto-Commit vs Manual Acknowledgment

Part 15 of 37

May 04, 2026 Abhay 5 min read

Offset Management: Auto-Commit vs Manual Acknowledgment

Why Offset Management Matters

The committed offset determines what happens when a consumer restarts. If the offset is committed too early, a crash before processing completes means events are lost. If it is committed too late, a crash after processing but before committing means events are re-processed.

flowchart TD
    subgraph TooEarly["Commit too early → Data Loss"]
        E1["Commit offset 43"] --> E2["Process record 42"] --> E3["Crash!"]
        E4["Restart: resume from 43"] --> E5["Record 42 NEVER processed ❌"]
    end

    subgraph TooLate["Commit after processing → At-Least-Once"]
        L1["Process record 42 ✓"] --> L2["Commit offset 43"] --> L3["Crash before commit!"]
        L4["Restart: resume from 42"] --> L5["Record 42 processed AGAIN ✓\n(acceptable with idempotent logic)"]
    end

Most production systems accept at-least-once delivery: process first, commit after. Make your processing logic idempotent to handle re-delivery safely.

Auto-Commit (Default)

With enable.auto.commit=true (Spring Kafka default when no AckMode is set), the consumer commits the highest fetched offset on a timer.

sequenceDiagram
    participant Consumer
    participant Broker

    Consumer->>Broker: poll() → records [40, 41, 42, 43, 44]
    Consumer->>Consumer: Process record 40 ✓
    Consumer->>Consumer: Process record 41 ✓
    Note over Consumer: auto.commit.interval.ms = 5000ms elapses
    Consumer->>Broker: commitOffset(44+1=45) ← auto
    Consumer->>Consumer: Process record 42 ✓
    Consumer->>Consumer: Process record 43 — CRASH 💥

    Consumer->>Broker: Restart → fetch committed offset
    Broker-->>Consumer: offset = 45
    Note over Consumer: Records 43 and 44 are LOST\n(committed before processing)

Auto-commit is dangerous for any data that matters. It provides at-most-once semantics: the offset is committed regardless of processing success.

Disable auto-commit:

spring.kafka.consumer.enable-auto-commit=false

When you set a Spring Kafka AckMode, auto-commit is automatically disabled — Spring Kafka takes over offset management.

Spring Kafka AckMode

Spring Kafka’s AckMode replaces auto-commit with explicit, controlled offset commits:

flowchart TD
    AckMode["spring.kafka.listener.ack-mode"]
    RECORD["RECORD\nCommit after each record"]
    BATCH["BATCH\nCommit after each poll batch"]
    TIME["TIME\nCommit every N milliseconds"]
    COUNT["COUNT\nCommit every N records"]
    COUNT_TIME["COUNT_TIME\nCommit on whichever\ncomes first: count or time"]
    MANUAL["MANUAL\nCommit when you call\nAcknowledgment.acknowledge()"]
    MANUAL_IMMEDIATE["MANUAL_IMMEDIATE\nCommit immediately\n(synchronously) on acknowledge()"]

    AckMode --> RECORD
    AckMode --> BATCH
    AckMode --> TIME
    AckMode --> COUNT
    AckMode --> COUNT_TIME
    AckMode --> MANUAL
    AckMode --> MANUAL_IMMEDIATE

AckMode.RECORD — After Each Record

spring.kafka.listener.ack-mode=record

sequenceDiagram
    participant Consumer
    participant Broker

    Consumer->>Broker: poll() → [40, 41, 42]
    Consumer->>Consumer: Process 40 ✓
    Consumer->>Broker: commitOffset(41)
    Consumer->>Consumer: Process 41 ✓
    Consumer->>Broker: commitOffset(42)
    Consumer->>Consumer: Process 42 ✓
    Consumer->>Broker: commitOffset(43)
    Note over Consumer: Safe but slow — 3 commits per poll

Safest option (one record at a time) but generates the most commit traffic. Good for low-volume, high-value events.

AckMode.BATCH — After Each Poll Batch

spring.kafka.listener.ack-mode=batch

sequenceDiagram
    participant Consumer
    participant Broker

    Consumer->>Broker: poll() → [40, 41, 42, 43, 44]
    Consumer->>Consumer: Process 40 ✓
    Consumer->>Consumer: Process 41 ✓
    Consumer->>Consumer: Process 42 ✓
    Consumer->>Consumer: Process 43 ✓
    Consumer->>Consumer: Process 44 ✓
    Consumer->>Broker: commitOffset(45) ← one commit for entire batch
    Note over Consumer: Efficient — but if crash after processing 42\nrecords 40-42 are reprocessed on restart

Best balance of safety and throughput for most applications. Spring Kafka’s default AckMode when enable.auto.commit=false.

AckMode.MANUAL — You Control Every Commit

spring.kafka.listener.ack-mode=manual

Inject Acknowledgment and call .acknowledge() when ready:

@KafkaListener(topics = "orders", groupId = "inventory-service")
public void onOrderPlaced(
        OrderPlacedEvent event,
        Acknowledgment ack) {

    try {
        inventoryService.reserveStock(event);
        ack.acknowledge();  // commit THIS record's offset
    } catch (TransientException e) {
        // Do NOT acknowledge — will be redelivered after restart
        log.warn("Transient error, not committing offset: {}", e.getMessage());
    }
}

Important: MANUAL mode batches acknowledgments — it calls commitAsync() on the next poll, not immediately. Use MANUAL_IMMEDIATE if you need synchronous commits.

Manual Commit with MANUAL_IMMEDIATE

@KafkaListener(topics = "orders", groupId = "inventory-service")
public void onOrderPlaced(OrderPlacedEvent event, Acknowledgment ack) {
    inventoryService.reserveStock(event);
    ack.acknowledge();  // blocks until broker confirms commit (synchronous)
}

spring.kafka.listener.ack-mode=manual_immediate

Synchronous commits add latency but guarantee the offset is committed before your method returns.

Manual Batch Acknowledgment

For batch listeners, acknowledge the entire batch or individual records:

@KafkaListener(topics = "orders", groupId = "inventory-service-batch")
public void onOrderBatch(
        List<ConsumerRecord<String, OrderPlacedEvent>> records,
        Acknowledgment ack) {

    List<OrderPlacedEvent> successful = new ArrayList<>();

    for (ConsumerRecord<String, OrderPlacedEvent> record : records) {
        try {
            inventoryService.reserveStock(record.value());
            successful.add(record.value());
        } catch (Exception e) {
            log.error("Failed at offset={}", record.offset(), e);
            // nack from this record forward — will retry from this position
            ack.nack(records.indexOf(record), Duration.ofSeconds(5));
            return;
        }
    }

    ack.acknowledge();  // commit entire batch
}

ack.nack(index, sleepDuration) seeks the partition back to the failed record’s offset — it will be redelivered after sleepDuration.

Choosing the Right AckMode

flowchart TD
    Q1{Processing\ncritical data?}
    Q2{Need bulk\ndatabase ops?}
    Q3{Complex retry\nlogic needed?}

    Q1 -->|No — analytics, logs| BatchAuto["BATCH\n(default, good enough)"]
    Q1 -->|Yes — orders, payments| Q2

    Q2 -->|Yes — save 100 records at once| Batch["BATCH\nCommit after all succeed"]
    Q2 -->|No — process one at a time| Q3

    Q3 -->|Yes — selective retry/DLQ| Manual["MANUAL\nor MANUAL_IMMEDIATE"]
    Q3 -->|No — simple processing| Record["RECORD\n(simple, safe)"]

AckMode Configuration in @Bean

@Bean
public ConcurrentKafkaListenerContainerFactory<String, Object> kafkaListenerContainerFactory(
        ConsumerFactory<String, Object> consumerFactory) {

    ConcurrentKafkaListenerContainerFactory<String, Object> factory =
        new ConcurrentKafkaListenerContainerFactory<>();

    factory.setConsumerFactory(consumerFactory);
    factory.setConcurrency(3);
    factory.getContainerProperties().setAckMode(ContainerProperties.AckMode.MANUAL);

    return factory;
}

Commit Semantics Summary

quadrantChart
    title Delivery Semantics by AckMode
    x-axis Low Safety --> High Safety
    y-axis Low Throughput --> High Throughput
    quadrant-1 High safety, lower throughput
    quadrant-2 High safety, high throughput
    quadrant-3 Low safety, low throughput
    quadrant-4 Low safety, high throughput
    AUTO_COMMIT: [0.15, 0.85]
    BATCH: [0.55, 0.75]
    RECORD: [0.7, 0.35]
    MANUAL_IMMEDIATE: [0.85, 0.2]
    MANUAL: [0.8, 0.5]

AckMode	Delivery	Throughput	Use case
AUTO_COMMIT	At-most-once	Highest	Non-critical metrics
BATCH	At-least-once	High	Default for most apps
RECORD	At-least-once	Medium	Low-volume critical events
MANUAL	At-least-once	Medium-High	Custom retry logic
MANUAL_IMMEDIATE	At-least-once	Lower	Strict ordering + commit

Key Takeaways

Auto-commit commits offsets on a timer regardless of processing result — it provides at-most-once semantics and risks data loss
Always set enable.auto.commit=false for production and use Spring Kafka’s AckMode instead
BATCH is the right default: commit after all records in a poll succeed, high throughput, at-least-once
MANUAL / MANUAL_IMMEDIATE give full control: commit only after confirmed processing, selective nack for failed records
ack.nack(index, duration) in batch mode seeks back to the failed offset for targeted redelivery
Make your consumers idempotent — with at-least-once semantics, any record can be delivered more than once

Next: Seeking to Specific Offsets: Replay, Recovery, and Time-Based Seeking — programmatically seek consumer positions to replay events, skip bad records, or resume from a specific timestamp.