How to Install Apache Kafka on Docker (Step-by-Step Guide)

Introduction

Apache Kafka is one of the most powerful distributed streaming platforms for real-time data processing. However, setting up Kafka manually on a local or cloud server can be tedious.
Docker simplifies this process by allowing you to spin up Kafka containers instantly with minimal configuration.

In this guide, you’ll learn how to install Apache Kafka on Docker, verify it’s running correctly, and follow best practices for stability and performance — all within a containerized environment.

Why Use Docker for Kafka?

Docker enables isolated, reproducible environments for your software components — including Kafka.
Here are key advantages:

✅ Easy setup: No need to manually install Java, Zookeeper, or Kafka binaries.
✅ Consistency: Run the same Kafka setup across different machines.
✅ Quick teardown: Stop and remove everything with a single command.
✅ Ideal for testing: Perfect for development or staging environments.

However, note that containerized Kafka is not recommended for large-scale production unless you configure volumes, network rules, and replication properly.

Prerequisites

Before getting started, make sure you have:

1. Docker Engine installed on your system
   → Get Docker
2. Docker Compose (comes pre-installed with Docker Desktop)
3. Internet access to pull Kafka images from Docker Hub
4. At least 2 GB of RAM and 2 CPU cores for smooth operation

Check your setup with:

docker --version
docker compose version

Step 1 – Pull the Kafka Docker Image

Let’s start by pulling the official Apache Kafka image from Docker Hub:

docker pull apache/kafka:latest

This image supports the KRaft mode (Kafka without ZooKeeper), making it simpler and faster to deploy.

Once downloaded, verify it’s available:

docker images

You should see an entry similar to:

REPOSITORY     TAG       IMAGE ID       CREATED        SIZE
apache/kafka   latest    1a2b3c4d5e6f   2 days ago     512MB

Step 2 – Run a Single Kafka Container

Start a Kafka broker in detached mode:

docker run -d \
  --name kafka \
  -p 9092:9092 \
  apache/kafka:latest

Check logs to confirm it’s running:

docker logs kafka

You’ll see something like:

INFO [KafkaServer id=1] started (kafka.server.KafkaServer)

Test It

You can run producer and consumer commands inside the container:

docker exec -it kafka /opt/kafka/bin/kafka-topics.sh \
  --create --topic test-topic --bootstrap-server localhost:9092

Then test:

docker exec -it kafka /opt/kafka/bin/kafka-console-producer.sh \
  --topic test-topic --bootstrap-server localhost:9092

Open another terminal:

docker exec -it kafka /opt/kafka/bin/kafka-console-consumer.sh \
  --topic test-topic --from-beginning --bootstrap-server localhost:9092

Step 3 – Use Docker Compose for Multi-Service Setup

Running Kafka with Docker Compose is cleaner and scalable.

Create a file named docker-compose.yml:

version: '3'
services:
  kafka:
    image: apache/kafka:latest
    container_name: kafka
    ports:
      - "9092:9092"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_PROCESS_ROLES: broker,controller
      KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka:9093
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092,CONTROLLER://0.0.0.0:9093
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_LOG_DIRS: /tmp/kraft-logs

Then start it:

docker compose up -d

Check running containers:

docker ps

Step 4 – Verify the Kafka Setup

Let’s test our running broker.

Create a topic:

docker exec -it kafka /opt/kafka/bin/kafka-topics.sh \
  --create --topic demo --bootstrap-server localhost:9092

List topics:

docker exec -it kafka /opt/kafka/bin/kafka-topics.sh \
  --list --bootstrap-server localhost:9092

You should see:

demo

Produce and consume messages as shown in Step 2.
If messages appear correctly, your Kafka container setup is working fine.

Step 5 – Add Persistence (Optional)

By default, all Kafka data (topics, offsets, logs) are stored in the container’s temporary filesystem.
Once the container is removed, all data is lost.

To persist data, update the Compose file:

volumes:
  kafka-data:
services:
  kafka:
    ...
    volumes:
      - kafka-data:/var/lib/kafka/data

Now, Kafka logs and topic data will remain intact even after container restarts.

Step 6 – Troubleshooting Common Issues

Issue	Cause	Solution
Clients can’t connect	Wrong KAFKA_ADVERTISED_LISTENERS	Ensure it’s set to PLAINTEXT://localhost:9092
Messages lost after restart	No persistent volume	Add Docker volumes for data
Container keeps restarting	Low memory	Increase Docker resources
Producer stuck	Wrong network config	Check listener ports and firewall

Pro Tip 💡: When connecting another containerized app (e.g., a Spring Boot microservice) to Kafka, use the container name (kafka:9092) instead of localhost.

Best Practices

• Always use official or trusted images (like apache/kafka or bitnami/kafka).
• Configure listeners properly for both local and networked access.
• Use Docker volumes for persistence.
• Use Docker Compose for reproducibility.
• Monitor your broker with tools like Prometheus or Kafka-UI.

Step 7 – Stop and Clean Up

To stop all containers:

docker compose down

To remove everything including volumes:

docker compose down -v

This ensures a clean workspace.

Conclusion

Running Apache Kafka on Docker is one of the fastest ways to start learning or prototyping Kafka-based systems.
It removes manual installation headaches, ensures consistency across environments, and is perfect for development and testing.

By following the steps in this guide — from pulling the image to using Docker Compose — you now have a fully functional Kafka instance ready for streaming experiments.

Next steps:

• Connect your applications to Kafka using Java or Python clients.
• Try adding Kafka UI for visual topic management.
• Explore multi-broker clusters using Docker Compose.