Spring Kafka Tutorial — Complete Guide

A complete, hands-on Apache Kafka and Spring Kafka tutorial series. Master event streaming from first principles — brokers, topics, producers, consumers, error handling, transactions, Avro serialization, Kafka Streams, and production patterns.

Spring Kafka Tutorial — Complete Guide

Apache Kafka is the backbone of modern event-driven architectures. This series dismantles it from the ground up — starting from what a broker actually is, through every producer and consumer knob, to exactly-once transactions and Kafka Streams. Every article uses a consistent e-commerce order-processing system as the running example.

37 articles. Real architecture. Production-grade patterns throughout.

Part 1: Apache Kafka Fundamentals

  1. What Is Apache Kafka: Event Streaming From First Principles
  2. Kafka Architecture: Brokers, Topics, Partitions, and Replicas
  3. Consumer Groups, Offsets, and the __consumer_offsets Topic
  4. KRaft Mode: Running Kafka Without ZooKeeper
  5. Starting a Kafka Cluster: Single-Broker and 3-Broker with KRaft
  6. Kafka CLI: Creating Topics, Producing, and Consuming Messages

Part 2: Spring Kafka Producers

  1. Kafka Producer in Spring Boot: KafkaTemplate Basics
  2. Sending Messages with Keys, Headers, and Custom Partitioning
  3. Producer @Bean Configuration: Beyond application.properties
  4. Producer Acknowledgments: acks, min.insync.replicas, and Data Durability
  5. Producer Retries: Backoff, Timeouts, and Retry Strategies
  6. Idempotent Producers: Eliminating Duplicate Messages

Part 3: Spring Kafka Consumers

  1. Kafka Consumer in Spring Boot: @KafkaListener Basics
  2. Consumer Groups: Parallel Processing and Partition Assignment Strategies
  3. Offset Management: Auto-Commit vs Manual Acknowledgment
  4. Seeking to Specific Offsets: Replay, Recovery, and Time-Based Seeking
  5. Consumer @Bean Configuration: ConcurrentKafkaListenerContainerFactory
  6. Filtering Messages with RecordFilterStrategy
  7. Pausing, Resuming, and Stopping Listener Containers

Part 4: Serialization and Message Handling

  1. JSON Serialization: JsonSerializer, JsonDeserializer, and Type Mapping
  2. Avro Serialization with Confluent Schema Registry
  3. Custom Serializers and Deserializers
  4. Message Headers: Metadata, Routing, and Custom Header Propagation

Part 5: Error Handling and Resilience

  1. Error Handling Basics: DefaultErrorHandler and CommonErrorHandler
  2. Retryable vs Non-Retryable Exceptions: Custom Exception Classification
  3. Dead Letter Topics: Routing Failed Messages with DeadLetterPublishingRecoverer
  4. Handling Deserialization Errors Gracefully
  5. Non-Blocking Retries: @RetryableTopic, BackOff, and the Retry Topic Chain
  6. Kafka Transactions and Exactly-Once Semantics

Part 6: Advanced Patterns

  1. @SendTo and @KafkaHandler: Chaining Consumers and Multi-Type Dispatch
  2. Request-Reply Pattern with ReplyingKafkaTemplate
  3. Dynamic Listener Containers and Programmatic Topic Registration
  4. Kafka Streams with Spring Boot: Stateless and Stateful Processing

Part 7: Administration, Testing, and Monitoring

  1. KafkaAdmin and AdminClient: Managing Topics Programmatically
  2. Testing Kafka Applications: EmbeddedKafka and Testcontainers
  3. Monitoring: Consumer Lag, Micrometer Metrics, and Actuator Integration

Part 8: Production Best Practices

  1. Spring Kafka Production Checklist and Best Practices

What You Will Learn

  • How Apache Kafka stores and replays events — brokers, topics, partitions, offsets
  • How to run Kafka locally with KRaft mode (no ZooKeeper) using Docker Compose
  • Building producers: KafkaTemplate, acknowledgments, retries, idempotence
  • Building consumers: @KafkaListener, consumer groups, offset management, manual commits
  • Serialization: JSON, Avro with Schema Registry, custom serializers
  • Error handling: DefaultErrorHandler, dead letter topics, non-blocking retries
  • Exactly-once semantics with Kafka transactions
  • Advanced patterns: request-reply, dynamic listeners, Kafka Streams
  • Testing: @EmbeddedKafka, Testcontainers, integration test patterns
  • Production: monitoring consumer lag, metrics, and the production launch checklist

Prerequisites

  • Java 11+ (examples use Java 21)
  • Spring Boot basics (dependency injection, REST controllers)
  • Basic understanding of messaging concepts (helpful but not required)
  • Docker Desktop installed (for running Kafka locally)

No prior Kafka experience needed.