Kafka - Install Kafka-4.1.1

 

frank@ZZHPC:~/download$ sudo adduser --system --no-create-home --group kafka

frank@ZZHPC:~/download$ wget https://dlcdn.apache.org/kafka/4.1.1/kafka_2.13-4.1.1.tgz

frank@ZZHPC:~/download$ tar -xzf kafka_2.13-4.1.1.tgz

frank@ZZHPC:~/download$ sudo mv kafka_2.13-4.1.1 /opt

 

export KAFKA_HOME=/opt/kafka_2.13-4.1.1
export PATH=$PATH:$SPARK_HOME/sbin:$SPARK_HOME/bin:$FLINK_HOME/bin:$KAFKA_HOME/bin

 

Here is a complete, copy-paste-ready setup for a single-node Kafka 4.1.1 KRaft cluster on Ubuntu.
This is the simplest reliable configuration for development or local testing.

---

✅ 1. Create a minimal KRaft config file

Create:

$KAFKA_HOME/config/kraft/server.properties

Paste this exact file:

process.roles=broker,controller
node.id=1
controller.quorum.voters=1@localhost:9093

# REQUIRED: controller listener must be specified
controller.listener.names=CONTROLLER

listeners=PLAINTEXT://localhost:9092,CONTROLLER://localhost:9093
listener.security.protocol.map=CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT

log.dirs=/tmp/kraft-combined-logs

broker.id=1

num.partitions=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1

 

---

✅ 2. Generate a cluster ID

$KAFKA_HOME/bin/kafka-storage.sh random-uuid

You will get output like:

frank@ZZHPC:~$ kafka-storage.sh random-uuid
AUnCC4-AT4K8oqYowqp4Fg

 

---

✅ 3. Format the KRaft metadata directory

Use the generated cluster ID here:

$KAFKA_HOME/bin/kafka-storage.sh format \
  -t Gv4j8tB8QFu8m6BQ8fQQAA \
  -c $KAFKA_HOME/config/kraft/server.properties

You should see:

frank@ZZHPC:~$ kafka-storage.sh format -t AUnCC4-AT4K8oqYowqp4Fg -c $KAFKA_HOME/config/kraft/server.properties
Formatting metadata directory /tmp/kraft-combined-logs with metadata.version 4.1-IV1.

 

---

✅ 4. Start Kafka (no ZooKeeper required)

frank@ZZHPC:~$ kafka-server-start.sh $KAFKA_HOME/config/kraft/server.properties

Kafka 4.1.1 is now running in combined controller + broker mode.

---

🎉 5. Test your Kafka instance

Create a topic:

frank@ZZHPC:~$ kafka-topics.sh --create --topic test-topic --bootstrap-server localhost:9092
Created topic test-topic.

Produce messages:

frank@ZZHPC:~$ kafka-console-producer.sh --topic test-topic --bootstrap-server localhost:9092
>msg1
>msg2
>

(type messages, press Enter)

Consume messages:

frank@ZZHPC:~$ kafka-console-consumer.sh --topic test-topic --from-beginning --bootstrap-server localhost:9092
msg1
msg2

 

---

🚀 You now have a working single-node Kafka 4.1.1 setup!

If you'd like, I can also provide:

• Systemd service files to run Kafka automatically on boot

• A docker-compose setup for Kafka 4.x

• A multi-node KRaft cluster config

 

frank@ZZHPC:~$ kafka-topics.sh --delete --topic test-topic --bootstrap-server localhost:9092