-->

Linux系统安装Kafka单机教程（带有kerberos和scram认证）

1.搭建Kafka环境

1.1 下载Kafka压缩包

  1. 去Kafka官网下载自己需要的版本安装包：https://kafka.apache.org/community/downloads/
    image
    我这里直接下的中间的Scala 2.12 - kafka_2.12-3.5.1.tgz (asc, sha512)
    当然，也可以通过wget命令直接在linux上下载，我是window上下载完传输到Linux上

  2. 在Linux上解压缩

我这里是打算将所有Kafka相关安装到/home/cloud_nms/kafka/文件夹下，所以将压缩包放在/home/cloud_nms/kafka/下，执行命令如下

tar -zxvf kafka_2.12-3.5.0.tgz

image

1.2 修改配置文件

Kafka需要安装zookeeper使用，但kafka集成zookeeper,在单机搭建时可直接使用自带的Kafka。使用需配置kafka_2.12-2.5.0/config 下的“zookeeper.properties”

cd kafka_2.12-3.5.0/config/

注:经过部署发现 kafka必须配置zookeeper 否则无法启动 无论是单机还是集群,必须,必须,必须开启

配置"zookeeper.properties"。修改dataDir和clientPort。前者是快照存放地址(自己随意配置)，后者是客户端连接zookeeper服务的端口。默认端口2181 最好默认不修改

vim zookeeper.properties

我的修改如下：

image

配置kafka_2.12-2.5.0/config下的“server.properties”，修改log.dirs和zookeeper.connect。前者是日志存放文件夹，后者是zookeeper连接地址（端口和clientPort保持一致）。

vim server.properties

image
image
地址配置的是centos（centos是我在/etc/hosts文件里面配置的127.0.0.1），这里localhost只能本机连接使用，若想其他连接建议更换成自定义的，我后续换为了centos，下面截图不在更改，还是localhost，大家记得别用就好，用自定义的
image

注意！！！！！！！！！！！！：listeners=SASL_PLAINTEXT://centos:9092，这里我配置的是centos，需要在客户端（要连kafka的机器）的/etc/hosts文件中加入如下：

139.19.71.86   centos

image
前面为ip，后面为自定义名称

若想不在客户端加配置，这里需要更改 server.properties配置为如下（）：
SASL认证：

listeners=SASL_PLAINTEXT://0.0.0.0:9092
advertised.listeners=SASL_PLAINTEXT://139.19.71.86:9092

账号密码或无认证：

listeners=PLAINTEXT://0.0.0.0:9092
advertised.listeners=PLAINTEXT://139.19.71.86:9092

1.3 启动Kafka和Zookeeper

(1) 开启kafka自带zookeeper

前台运行：

./bin/zookeeper-server-start.sh ./config/zookeeper.properties

后台运行：

./bin/zookeeper-server-start.sh -daemon ./config/zookeeper.properties

关闭zookeeper（仅用于出错关闭重启，实际上启动成功就行了）：

 bin/zookeeper-server-stop.sh

(2) 开启kafka

以下命令是在安装路径下执行的，我这里是/home/cloud_nms/kafka/

前台运行：

bin/kafka-server-start.sh config/server.properties

后台运行：

./bin/kafka-server-start.sh -daemon ./config/server.properties

启动后server.log报错如下：

点击查看报错信息 ``` [2026-09-20 09:22:22,543] INFO Creating /brokers/ids/0 (is it secure? false) (kafka.zk.KafkaZkClient) [2026-09-20 09:22:22,555] ERROR Error while creating ephemeral at /brokers/ids/0, node already exists and owner '0x1000f234e620001' does not match current session '0x10000045c430000' (kafka.zk.KafkaZkClient$CheckedEphemeral) [2026-09-20 09:22:22,567] ERROR [KafkaServer id=0] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists at org.apache.zookeeper.KeeperException.create(KeeperException.java:126) at kafka.zk.KafkaZkClient$CheckedEphemeral.getAfterNodeExists(KafkaZkClient.scala:2189) at kafka.zk.KafkaZkClient$CheckedEphemeral.create(KafkaZkClient.scala:2127) at kafka.zk.KafkaZkClient.checkedEphemeralCreate(KafkaZkClient.scala:2094) at kafka.zk.KafkaZkClient.registerBroker(KafkaZkClient.scala:106) at kafka.server.KafkaServer.startup(KafkaServer.scala:365) at kafka.Kafka$.main(Kafka.scala:113) at kafka.Kafka.main(Kafka.scala) [2026-09-20 09:22:22,568] INFO [KafkaServer id=0] shutting down (kafka.server.KafkaServer) [2026-09-20 09:22:22,569] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Stopping socket server request processors (kafka.network.SocketServer) [2026-09-20 09:22:22,570] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Stopped socket server request processors (kafka.network.SocketServer) [2026-09-20 09:22:22,572] INFO [ReplicaManager broker=0] Shutting down (kafka.server.ReplicaManager) [2026-09-20 09:22:22,572] INFO [LogDirFailureHandler]: Shutting down (kafka.server.ReplicaManager$LogDirFailureHandler) [2026-09-20 09:22:22,572] INFO [LogDirFailureHandler]: Stopped (kafka.server.ReplicaManager$LogDirFailureHandler) [2026-09-20 09:22:22,572] INFO [LogDirFailureHandler]: Shutdown completed (kafka.server.ReplicaManager$LogDirFailureHandler) [2026-09-20 09:22:22,572] INFO [ReplicaFetcherManager on broker 0] shutting down (kafka.server.ReplicaFetcherManager) [2026-09-20 09:22:22,573] INFO [ReplicaFetcherManager on broker 0] shutdown completed (kafka.server.ReplicaFetcherManager) [2026-09-20 09:22:22,573] INFO [ReplicaAlterLogDirsManager on broker 0] shutting down (kafka.server.ReplicaAlterLogDirsManager) [2026-09-20 09:22:22,573] INFO [ReplicaAlterLogDirsManager on broker 0] shutdown completed (kafka.server.ReplicaAlterLogDirsManager) [2026-09-20 09:22:22,573] INFO [ExpirationReaper-0-Fetch]: Shutting down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [2026-09-20 09:22:22,574] INFO [ExpirationReaper-0-Fetch]: Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [2026-09-20 09:22:22,574] INFO [ExpirationReaper-0-Fetch]: Shutdown completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [2026-09-20 09:22:22,574] INFO [ExpirationReaper-0-Produce]: Shutting down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [2026-09-20 09:22:22,574] INFO [ExpirationReaper-0-Produce]: Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [2026-09-20 09:22:22,574] INFO [ExpirationReaper-0-Produce]: Shutdown completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [2026-09-20 09:22:22,574] INFO [ExpirationReaper-0-DeleteRecords]: Shutting down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [2026-09-20 09:22:22,575] INFO [ExpirationReaper-0-DeleteRecords]: Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [2026-09-20 09:22:22,575] INFO [ExpirationReaper-0-DeleteRecords]: Shutdown completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [2026-09-20 09:22:22,575] INFO [ExpirationReaper-0-ElectLeader]: Shutting down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [2026-09-20 09:22:22,575] INFO [ExpirationReaper-0-ElectLeader]: Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [2026-09-20 09:22:22,575] INFO [ExpirationReaper-0-ElectLeader]: Shutdown completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper) [2026-09-20 09:22:22,578] INFO [ReplicaManager broker=0] Shut down completely (kafka.server.ReplicaManager) [2026-09-20 09:22:22,578] INFO [zk-broker-0-to-controller-alter-partition-channel-manager]: Shutting down (kafka.server.BrokerToControllerRequestThread) [2026-09-20 09:22:22,579] INFO [zk-broker-0-to-controller-alter-partition-channel-manager]: Stopped (kafka.server.BrokerToControllerRequestThread) [2026-09-20 09:22:22,579] INFO [zk-broker-0-to-controller-alter-partition-channel-manager]: Shutdown completed (kafka.server.BrokerToControllerRequestThread) [2026-09-20 09:22:22,579] INFO Broker to controller channel manager for alter-partition shutdown (kafka.server.BrokerToControllerChannelManagerImpl) [2026-09-20 09:22:22,579] INFO [zk-broker-0-to-controller-forwarding-channel-manager]: Shutting down (kafka.server.BrokerToControllerRequestThread) [2026-09-20 09:22:22,579] INFO [zk-broker-0-to-controller-forwarding-channel-manager]: Stopped (kafka.server.BrokerToControllerRequestThread) [2026-09-20 09:22:22,579] INFO [zk-broker-0-to-controller-forwarding-channel-manager]: Shutdown completed (kafka.server.BrokerToControllerRequestThread) [2026-09-20 09:22:22,580] INFO Broker to controller channel manager for forwarding shutdown (kafka.server.BrokerToControllerChannelManagerImpl) [2026-09-20 09:22:22,580] INFO Shutting down. (kafka.log.LogManager) [2026-09-20 09:22:22,581] INFO [kafka-log-cleaner-thread-0]: Shutting down (kafka.log.LogCleaner$CleanerThread) [2026-09-20 09:22:22,582] INFO [kafka-log-cleaner-thread-0]: Stopped (kafka.log.LogCleaner$CleanerThread) [2026-09-20 09:22:22,582] INFO [kafka-log-cleaner-thread-0]: Shutdown completed (kafka.log.LogCleaner$CleanerThread) [2026-09-20 09:22:22,603] INFO Shutdown complete. (kafka.log.LogManager) [2026-09-20 09:22:22,604] INFO [feature-zk-node-event-process-thread]: Shutting down (kafka.server.FinalizedFeatureChangeListener$ChangeNotificationProcessorThread) [2026-09-20 09:22:22,605] INFO [feature-zk-node-event-process-thread]: Stopped (kafka.server.FinalizedFeatureChangeListener$ChangeNotificationProcessorThread) [2026-09-20 09:22:22,605] INFO [feature-zk-node-event-process-thread]: Shutdown completed (kafka.server.FinalizedFeatureChangeListener$ChangeNotificationProcessorThread) [2026-09-20 09:22:22,605] INFO [ZooKeeperClient Kafka server] Closing. (kafka.zookeeper.ZooKeeperClient) [2026-09-20 09:22:22,708] INFO Session: 0x10000045c430000 closed (org.apache.zookeeper.ZooKeeper) [2026-09-20 09:22:22,708] INFO EventThread shut down for session: 0x10000045c430000 (org.apache.zookeeper.ClientCnxn) [2026-09-20 09:22:22,708] INFO [ZooKeeperClient Kafka server] Closed. (kafka.zookeeper.ZooKeeperClient) [2026-09-20 09:22:22,708] INFO [ThrottledChannelReaper-Fetch]: Shutting down (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2026-09-20 09:22:22,709] INFO [ThrottledChannelReaper-Fetch]: Stopped (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2026-09-20 09:22:22,709] INFO [ThrottledChannelReaper-Fetch]: Shutdown completed (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2026-09-20 09:22:22,709] INFO [ThrottledChannelReaper-Produce]: Shutting down (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2026-09-20 09:22:22,709] INFO [ThrottledChannelReaper-Produce]: Stopped (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2026-09-20 09:22:22,709] INFO [ThrottledChannelReaper-Produce]: Shutdown completed (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2026-09-20 09:22:22,709] INFO [ThrottledChannelReaper-Request]: Shutting down (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2026-09-20 09:22:22,710] INFO [ThrottledChannelReaper-Request]: Stopped (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2026-09-20 09:22:22,710] INFO [ThrottledChannelReaper-Request]: Shutdown completed (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2026-09-20 09:22:22,710] INFO [ThrottledChannelReaper-ControllerMutation]: Shutting down (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2026-09-20 09:22:22,710] INFO [ThrottledChannelReaper-ControllerMutation]: Stopped (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2026-09-20 09:22:22,710] INFO [ThrottledChannelReaper-ControllerMutation]: Shutdown completed (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2026-09-20 09:22:22,711] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Shutting down socket server (kafka.network.SocketServer) [2026-09-20 09:22:22,722] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Shutdown completed (kafka.network.SocketServer) [2026-09-20 09:22:22,723] INFO Metrics scheduler closed (org.apache.kafka.common.metrics.Metrics) [2026-09-20 09:22:22,723] INFO Metrics reporters closed (org.apache.kafka.common.metrics.Metrics) [2026-09-20 09:22:22,725] INFO Broker and topic stats closed (kafka.server.BrokerTopicStats) [2026-09-20 09:22:22,727] INFO App info kafka.server for 0 unregistered (org.apache.kafka.common.utils.AppInfoParser) [2026-09-20 09:22:22,727] INFO [KafkaServer id=0] shut down completed (kafka.server.KafkaServer) [2026-09-20 09:22:22,727] ERROR Exiting Kafka due to fatal exception during startup. (kafka.Kafka$) org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists at org.apache.zookeeper.KeeperException.create(KeeperException.java:126) at kafka.zk.KafkaZkClient$CheckedEphemeral.getAfterNodeExists(KafkaZkClient.scala:2189) at kafka.zk.KafkaZkClient$CheckedEphemeral.create(KafkaZkClient.scala:2127) at kafka.zk.KafkaZkClient.checkedEphemeralCreate(KafkaZkClient.scala:2094) at kafka.zk.KafkaZkClient.registerBroker(KafkaZkClient.scala:106) at kafka.server.KafkaServer.startup(KafkaServer.scala:365) at kafka.Kafka$.main(Kafka.scala:113) at kafka.Kafka.main(Kafka.scala) [2026-09-20 09:22:22,728] INFO [KafkaServer id=0] shutting down (kafka.server.KafkaServer) [2026-09-20 09:22:28,011] INFO Expiring session 0x1000f234e620001, timeout of 18000ms exceeded (org.apache.zookeeper.server.ZooKeeperServer) ```
这是一个 Kafka 启动时 ZooKeeper 中残留了旧的 broker 临时节点（ephemeral node） 导致的启动失败，与你项目代码无关，属于 Kafka/ZooKeeper 运维问题。

解决：在过期时间INFO Expiring session 0x1000f234e620001, timeout of 18000ms exceeded后，重启kafka

关闭Kafka（仅用于出错关闭重启，实际上启动成功就行了）：

bin/kafka-server-stop.sh

使用jps命令查看是否正常了

jps

注意：jps 只显示"当前用户"的 JVM，所以你用root启动，也就是sudo，在普通用户里jps看不到，需要su root

image

1.4 Kafka测试

(1)创建kafka主题

貌似不同版本的命令有所不同，我这里用的3.5，就碰到了不支持zookeeper
旧版本命令：

./bin/kafka-topics.sh --create --zookeeper 192.168.118.128:2181 --replication-factor 1 --partitions 1 --topic test
 
#也可以使用下面命令 与上面命令等效
./bin/kafka-topics.sh --create --zookeeper web:2181 --replication-factor 1 --partitions 1 --topic test

报错如下：
image

原因：新版本的kafka，已经不需要依赖zookeeper来创建，可以使用如下

./bin/kafka-topics.sh --create --bootstrap-server centos:9092 --replication-factor 1 --partitions 1 --topic test

image

旧板与新版的主要差别在于--zookeeper变为了--bootstrap-server，后面命令只介绍我是用的3.5版本，旧命令不再赘述，想用自己替换一下就好了
(2)显示kafka所有主题

./bin/kafka-topics.sh --list --bootstrap-server centos:9092

image

(3)查看某个主题的详细信息：

 ./bin/kafka-topics.sh --bootstrap-server centos:9092 --describe --topic test

image

(4)创建kafka生产者：

 ./bin/kafka-console-producer.sh --broker-list centos:9092 --topic test

image

Ctrl + D 发送文件结束符 (EOF)：这个组合键告诉生产者进程“输入流已经结束”。生产者接收到这个信号后，会触发一个优雅的关闭过程。它会等待内部缓冲区（RecordAccumulator）中所有尚未发送的消息被后台的 Sender 线程发送出去，并收到 Kafka 集群的确认（根据你的 acks 设置），然后才正常退出。这确保了数据的完整发送

(5)创建kafka消费者

./bin/kafka-console-consumer.sh --bootstrap-server centos:9092 --topic test --from-beginning

--from-beginning 是从头开始消费
image

Ctrl + C停止退出

(6)删除topic

./bin/kafka-topics.sh  --bootstrap-server centos:9092 --delete --topic test

image

到此，Kafka单机安装完毕

2.安装kerberos

2.1 安装kerberos服务端

前提：必须能连接外网，因为网上教程全是直接用yum安装的，内网的话，直接传安装包，解压出来和教程对不上

yum install krb5-server

image

发现是镜像仓库不对，按照网上教程禁用fastestmirror和重新下载阿里云、清华镜像都不能用，直到询问同事，发现是镜像文件和服务器不匹配。。。。。

解决方法如下：

uname -r

image
我这是ARM服务器，使用的一直是x86镜像，所以出问题，如果查询出来有x86就可以直接用这个镜像了

2.2 重新配置yum镜像源：

如果上述yum install krb5-server执行成功可以跳过这一步

  1. 备份原yum镜像：
cd /etc/yum.repos.d
mkdir backup
mv *.repo backup/
  1. 下载最新yum源：

对应服务器下载镜像源，自己的是哪个，可以通过uname -r查看

x86：

wget -O /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo

arm：

wget -O /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-altarch-7.repo
  1. 清除缓存和重新生成缓存
yum clean all
yum makecache

这里，如果yum源文件是错的，在执行yum makecache，会报错404，按照上述改一下就行了，报错如下
image

源文件是对的，执行如下：
image

问题解决，继续执行安装kerberos指令

sudo yum install krb5-server

image
中途Is this ok [y/d/N]: 直接输入y

点击查看报错信息 
Running transaction check
Running transaction test


Transaction check error:
  file /usr/bin/sclient from install of krb5-server-1.15.1-55.el7_9.x86_64 conflicts with file from package krb5-devel-1.11.3-49.el7.x86_64
  file /usr/sbin/sserver from install of krb5-server-1.15.1-55.el7_9.x86_64 conflicts with file from package krb5-devel-1.11.3-49.el7.x86_64
  file /usr/share/man/man1/sclient.1.gz from install of krb5-server-1.15.1-55.el7_9.x86_64 conflicts with file from package krb5-devel-1.11.3-49.el7.x86_64
  file /usr/share/man/man8/sserver.8.gz from install of krb5-server-1.15.1-55.el7_9.x86_64 conflicts with file from package krb5-devel-1.11.3-49.el7.x86_64

错误概要
-------------
你要装的是 krb5-server 1.15.1，但系统里已经装了 旧版 krb5-devel 1.11.3。在 krb5 1.15+ 版本中，下面这几个文件的归属从 krb5-devel 转移到了 krb5-server

解决方案：让所有 krb5-* 组件升到同一版本，冲突自然消失

sudo yum update krb5-libs krb5-workstation krb5-server krb5-devel krb5-pkinit

或者直接：

sudo yum update krb5-*

更新完后重新执行上面安装kerberos指令，没有报错跳过即可

  1. 修改配置文件
sudo vim /etc/krb5.conf

这里我只改了如下配置：
image

相关参数说明：

[logging]：日志的位置
[libdefaults]：每种连接的默认配置
　　dns_lookup_realm：是否通过 dns 查找需使用的 releam
　　ticket_lifetime：凭证的有效时限，一般为 24 小时
　　renew_lifetime：凭证最长可以被延期的时限，一般为一周。当凭证过期之后，对安全认证的服务后续访问就会失败
　　forwardable：ticket 是否可以被转发（如果用户已经有了一个TGT，当他登入到另一个远程系统，KDC会为他重新创建一个TGT，而不需要让用户重新进行身份认证）
　　rdns：如果为 true，则除根据 hostname 正向查找外，同时反向查找对应的 principal。如果 dns_canonicalize_hostname 设置为 false，则此标志不起作用。默认值为 true。
　　pkinit_anchors：受信任锚（根）证书的位置；如果用户在命令行上指定X509_anchors，则不使用该配置。
　　default_realm：默认的 realm，必须跟要配置的 realm 名称一致
　　default_ccache_name：指定默认凭据缓存的名称。默认值为 DEFCCNAME
[realms]：列举使用的 realm
　　kdc：kdc 运行的机器
　　admin_server：kdc 数据库管理服务运行的机器
[domain_realm]：配置 domain name 或 hostname 对应的 releam

详细解释如下：krb5.conf官方文档

sudo vim /var/kerberos/krb5kdc/kdc.conf

这个文件我没有更改
image

相关参数说明：

[kdcdefaults]：KDC 默认配置
kdc_ports：UDP 端口号
kdc_tcp_ports：TCP 端口号
[realms]：realm 数据库配置
master_key_type：主密钥的密钥类型；默认值为 aes256-cts-hmac-sha1-96。
acl_file：用于指定哪些用户可以访问 kdc 数据库的控制文件；如果不需要现在用户访问，该值可以设为空
dict_file：字典文件位置，该文件中的单词不能被用于密码；如果文件为空，或者没有为用户分配策略，则不会执行密码字典检查。
admin_keytab：KDC 进行校验的 keytab。
supported_enctypes：支持的加密方式，默认为 aes256-cts-hmac-sha1-96:normal aes128-cts-hmac-sha1-96:normal

详细解释如下：kdc.conf官方文档

  1. 创建数据库
kdb5_util create -s -r KAFKA.COM

这里创建的数据库名字要和krb5.conf中配置的一样

若初始错误数据库：删库命令如下：

sudo kdb5_util destroy -f -r KAFKA.COM

image
这里创建报错配置有问题，无效的配置，原因：krb5.conf少加了结尾的}，正常不会报错

  1. 启动服务
#开启自启动
[root@kafka01 ~]# systemctl enable krb5kdc.service 
Created symlink /etc/systemd/system/multi-user.target.wants/krb5kdc.service → /usr/lib/systemd/system/krb5kdc.service.
#开启Kerberos服务
[root@kafka01 ~]# systemctl start krb5kdc.service 

#开启kadmin服务
[root@kafka01 ~]# systemctl enable kadmin.service 
Created symlink /etc/systemd/system/multi-user.target.wants/kadmin.service → /usr/lib/systemd/system/kadmin.service.
[root@kafka01 ~]# systemctl start kadmin.service

启动报错如下：
image

查看报错详细信息命令如下：

sudo journalctl -xe -u krb5kdc.service

image
可以看到是因为没有初始化KAFKA.COM数据库，如果初始化和上面配置文件中一样，就不会有这个问题

  1. 创建账号

创建存储位置和赋权，后面导出账号秘钥要用：

# 以 root 身份创建目录
sudo mkdir -p /root/data
# 确保目录权限允许写入（root 默认就有）
sudo chmod 700 /root/data

进入kadmin.local界面：

sudo kadmin.local

image

创建账号（kadmin.local界面）：

add_principal root/admin@KAFKA.COM
add_principal kafka-client@KAFKA.COM

image
image

输入完，自己输入两次相同的密码即可，我这里就只先添加了这一个账号

导出账号秘钥（kadmin.local界面）：

xst -norandkey -k /root/data/root.keytab root/admin@KAFKA.COM
xst -norandkey -k  /root/data/kafka-client.keytab kafka-client@KAFKA.COM

exit是退出kadmin.local命令

2.3 安装kerberos客户端

yum install krb5-workstation

image

#客户端可以是用kadmin命令
[root@kafka01 ~]# kadmin 

kinit(在客户端认证用户)
[root@kafka02 ~]# kinit root/admin@KAFKA.COM   #输入密码认证完成

#查看当前的认证用户
[root@kafka01 ~]# klist

#kdestroy(删除当前的认证缓存)
[root@kafka01 ~]# kdestroy

image

3.kafka开启kerberos认证

3.1 创建 keytab

  1. 创建keytab存放路径,并给路径写入权限
 sudo mkdir -p /root/data
 sudo chmod 700 /root/data
  1. 进入kerberos服务器端（Kerberos 服务端上使用 kadmin.local，安装了 Kerberos Client 的机器上可以使用 kadmin）这里建议使用 kadmin.local便于账号管理，理论上kadmin也可以进行后续，就是有权限限制，比较麻烦，所以还是用kadmin.local吧
sudo kadmin.local
  1. 创建客户端和服务器端账号和导出keytab文件
kadmin.local:  add_principal -randkey kafka-server/centos@KAFKA.COM
kadmin.local:  add_principal -randkey kafka-client/centos@KAFKA.COM
kadmin.local:  xst -k /root/data/kafka-server.keytab kafka-server/centos@KAFKA.COM
kadmin.local:  xst -k /root/data/kafka-client.keytab kafka-client/centos@KAFKA.COM

执行addprinc命令，用于创建账号，如果没指定-randkey或-nokey参数，需要设置密码
执行xst导出命令，用于导出指定账号对应的keytab文件，如果没有使用-norandkey，会导致密码被随机重置

这里的创建不建议使用上述使用的root/adminlocalhost@KAFKA，容易乱，还是老实建两个吧，还有这里的localhost是因为我的/etc/hosts文件里配的localhost，如需要更改名称，可以自行配置，也可以直接用ip

3.2 配置

  1. 拷贝krb5.conf及keytab文件到所有安装Kafka的机器，把文件都放到Kafka的 config/kerberos 目录下(kerberos 目录需新建)

将keytab文件复制到config/kerberos下

sudo cp /root/data/kafka-server.keytab /home/cloud_nms/kafka/kafka_2.12-3.5.0/config/kerberos/
sudo cp /root/data/kafka-client.keytab /home/cloud_nms/kafka/kafka_2.12-3.5.0/config/kerberos/

将krb5.conf文件复制到config/kerberos下

sudo cp /etc/krb5.conf /home/cloud_nms/kafka/kafka_2.12-3.5.0/config/kerberos/
  1. 配置权限：

给复制过来的加读权限，一定要加，因为我调了一天，一直报没有kafkaclient

sudo chmod 644 /home/cloud_nms/kafka/kafka_2.12-3.5.0/config/kerberos/kafka-server.keytab
sudo chmod 644 /home/cloud_nms/kafka/kafka_2.12-3.5.0/config/kerberos/kafka-client.keytab
sudo chmod 644 /home/cloud_nms/kafka/kafka_2.12-3.5.0/config/kerberos/krb5.conf

3.2.1 Kafka服务端配置

  1. 复制 config/server.properties（复制动作能帮助我们在 认证/非认证 模式自由切换）重命名为 config/server-sasl.properties：
sudo cp /home/cloud_nms/kafka/kafka_2.12-3.5.0/config/server.properties /home/cloud_nms/kafka/kafka_2.12-3.5.0/config/kerberos/server-sasl.properties

这里也可以选择不复制，直接在原文件更改，但是建议复制，因为确实能 认证/非认证 模式自由切换，认证启动使用sasl文件，否则使用原文件

  1. 在复制的server-sasl.properties下加如下配置：
listeners=SASL_PLAINTEXT://centos:9092
security.inter.broker.protocol=SASL_PLAINTEXT
sasl.mechanism.inter.broker.protocol=GSSAPI
sasl.enabled.mechanisms=GSSAPI
sasl.kerberos.service.name=kafka-server

这里第一行在之前安装kerberos时已更改过，这次更改为SASL_PLAINTEXT://localhost:9092就行，因为之前配的是PLAINTEXT://localhost:9092，还有这里的sasl.kerberos.service.name=kafka-server要记住，而且这里的service.name要和kafka-server-jaas.conf中principal的/前名称一致，后面要用

因为这里创建的是GSSAPI认证方式，在server-sasl.properties加入上述配置后，记得上面的listeners和advertised.listeners也要更改为对应的SASL_PLAINTEXT，如下图：
image

  1. 新建 kafka-server-jaas.conf 文件，该文件也放到 Kafka 的 config/kerberos 目录下
    在 config/kerberos 目录下输入创建文件命令
sudo touch kafka-server-jaas.conf
  1. 加入如下内容：
KafkaServer {
    com.sun.security.auth.module.Krb5LoginModule required
    useKeyTab=true
    keyTab="/home/cloud_nms/kafka/kafka_2.12-3.5.0/config/kerberos/kafka-server.keytab"
    storeKey=true
    useTicketCache=false
    principal="kafka-server/centos@KAFKA.COM";
};

这里不建议加#注释，感觉有时候识别不出来
我的如下：
image

  1. 复制 bin/kafka-server-start.sh 脚本重命名为 bin/kafka-server-start-sasl.sh
sudo cp /home/cloud_nms/kafka/kafka_2.12-3.5.0/bin/kafka-server-start.sh /home/cloud_nms/kafka/kafka_2.12-3.5.0/bin/kafka-server-start-sasl.sh
  1. kafka-server-start-sasl.sh倒数第二行增加如下配置
    修改文件命令：
sudo vim kafka-server-start-sasl.sh

添加如下内容：

#-Dzookeeper.sasl.client=false zk没有开启认证就设置false
export KAFKA_OPTS="-Dzookeeper.sasl.client=false -Dzookeeper.sasl.client.username=zk-server -Djava.security.krb5.conf=/home/cloud_nms/kafka/kafka_2.12-3.5.0/config/kerberos/krb5.conf -Djava.security.auth.login.config=/home/cloud_nms/kafka/kafka_2.12-3.5.0/config/kerberos/kafka-server-jaas.conf -Dsun.security.krb5.debug=true"

-Dsun.security.krb5.debug=true是用来调试看报错的，可以不加
结果如下：
image

3.2.2 Kafka客户端配置

该配置主要为了使用 bin/kafka-topics.sh、bin/kafka-console-consumer.sh、kafka-console-producer.sh 等命令。

  1. 新建 client.properties 文件，该文件也放到 Kafka 的 config/kerberos 目录下：
    在config/kerberos 目录下，执行创建文件：
sudo touch client.properties

加入如下内容：

security.protocol=SASL_PLAINTEXT
sasl.mechanism=GSSAPI
sasl.kerberos.service.name=kafka-server
inter.broker.listener.name=SASL_PLAINTEXT

这里的sasl.kerberos.service.name一定要和server-sasl.properties中一致

  1. 新建 kafka-client-jaas.conf 文件，该文件也放到 Kafka 的 config/kerberos 目录下
    在config/kerberos 目录下，执行创建文件：
sudo touch kafka-client-jaas.conf

加入如下内容：

KafkaClient {
   com.sun.security.auth.module.Krb5LoginModule required
   useKeyTab=true
   keyTab="/home/cloud_nms/kafka/kafka_2.12-3.5.0/config/kerberos/kafka-client.keytab"
   storeKey=true
   useTicketCache=false
   principal="kafka-client/centos@KAFKA.COM";
};
  1. 复制 bin/kafka-topics.sh、kafka-console-producer.sh、bin/kafka-console-consumer.sh 脚本，分别重命名为 bin/kafka-topics-sasl.sh、kafka-console-producer-sasl.sh、bin/kafka-console-consumer-sasl.sh
sudo cp /home/cloud_nms/kafka/kafka_2.12-3.5.0/bin/kafka-topics.sh /home/cloud_nms/kafka/kafka_2.12-3.5.0/bin/kafka-topics-sasl.sh
sudo cp /home/cloud_nms/kafka/kafka_2.12-3.5.0/bin/kafka-console-producer.sh /home/cloud_nms/kafka/kafka_2.12-3.5.0/bin/kafka-console-producer-sasl.sh
sudo cp /home/cloud_nms/kafka/kafka_2.12-3.5.0/bin/kafka-console-consumer.sh /home/cloud_nms/kafka/kafka_2.12-3.5.0/bin/kafka-console-consumer-sasl.sh
  1. kafka-topics-sasl.sh、kafka-console-producer-sasl.sh和kafka-console-consumer-sasl.sh倒数第二行增加如下配置：
#三个文件倒数第二行 新增以下内容
export KAFKA_OPTS="-Djava.security.krb5.conf=/home/cloud_nms/kafka/kafka_2.12-3.5.0/config/kerberos/krb5.conf -Djava.security.auth.login.config=/home/cloud_nms/kafka/kafka_2.12-3.5.0/config/kerberos/kafka-client-jaas.conf"

效果如下：
image

3.3 启动测试

  1. 运行 bin/kafka-server-start-sasl.sh 启动 Kafka 集群：
cd /home/cloud_nms/kafka/kafka_2.12-3.5.0
./bin/kafka-server-start-sasl.sh -daemon config/kerberos/server-sasl.properties

image

  1. 运行 bin/kafka-topics-sasl.sh 查看 topic 列表信息：
./bin/kafka-topics-sasl.sh --list --bootstrap-server centos:9092 --command-config config/kerberos/client.properties

image

点击查看可能报错 ``` [root@siteA_node1 kafka_2.12-3.5.0]# ./bin/kafka-topics-sasl.sh --list --bootstrap-server 139.19.8.80:9092 --command-config config/kerberos/client.properties [2026-09-20 14:52:04,076] ERROR [AdminClient clientId=adminclient-1] Connection to node -1 (siteA_node1/139.19.8.80:9092) failed authentication due to: An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - LOOKING_UP_SERVER)]) occurred when evaluating SASL token received from the Kafka Broker. This may be caused by Java's being unable to resolve the Kafka Broker's hostname correctly. You may want to try to adding '-Dsun.net.spi.nameservice.provider.1=dns,sun' to your client's JVMFLAGS environment. Users must configure FQDN of kafka brokers when authenticating using SASL and `socketChannel.socket().getInetAddress().getHostName()` must match the hostname in `principal/hostname@realm` Kafka Client will go to AUTHENTICATION_FAILED state. (org.apache.kafka.clients.NetworkClient) [2026-09-20 14:52:04,076] WARN [AdminClient clientId=adminclient-1] Metadata update failed due to authentication error (org.apache.kafka.clients.admin.internals.AdminMetadataManager) org.apache.kafka.common.errors.SaslAuthenticationException: An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - LOOKING_UP_SERVER)]) occurred when evaluating SASL token received from the Kafka Broker. This may be caused by Java's being unable to resolve the Kafka Broker's hostname correctly. You may want to try to adding '-Dsun.net.spi.nameservice.provider.1=dns,sun' to your client's JVMFLAGS environment. Users must configure FQDN of kafka brokers when authenticating using SASL and `socketChannel.socket().getInetAddress().getHostName()` must match the hostname in `principal/hostname@realm` Kafka Client will go to AUTHENTICATION_FAILED state. Caused by: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - LOOKING_UP_SERVER)] at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.lambda$createSaslToken$1(SaslClientAuthenticator.java:534) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.createSaslToken(SaslClientAuthenticator.java:534) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.sendSaslClientToken(SaslClientAuthenticator.java:433) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.sendInitialToken(SaslClientAuthenticator.java:332) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.authenticate(SaslClientAuthenticator.java:273) at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:181) at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:543) at org.apache.kafka.common.network.Selector.poll(Selector.java:481) at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:571) at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.processRequests(KafkaAdminClient.java:1413) at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1344) at java.lang.Thread.run(Thread.java:750) Caused by: GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - LOOKING_UP_SERVER) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:772) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192) ... 14 more Caused by: KrbException: Server not found in Kerberos database (7) - LOOKING_UP_SERVER at sun.security.krb5.KrbTgsRep.(KrbTgsRep.java:73) at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:225) at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:236) at sun.security.krb5.internal.CredentialsUtil.serviceCredsSingle(CredentialsUtil.java:477) at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:340) at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:314) at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:169) at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:490) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:695) ... 17 more Caused by: KrbException: Identifier doesn't match expected value (906) at sun.security.krb5.internal.KDCRep.init(KDCRep.java:140) at sun.security.krb5.internal.TGSRep.init(TGSRep.java:65) at sun.security.krb5.internal.TGSRep.(TGSRep.java:60) at sun.security.krb5.KrbTgsRep.(KrbTgsRep.java:55) ... 25 more ```

上述报错原因：Kerberos 认证失败：KDC 里找不到 Kafka broker 的服务 principal，我这里是因为之前配过集群，主机名不一致，139.19.8.80映射为其他名称，导致KDC找不到，解决方案是从 nsswitch.conf 移除 myhostname，让 /etc/hosts 完全接管，命令如下：

  • sudo vi /etc/nsswitch.conf
  • 把hosts: files dns myhostname改为hosts: files dns
  • 同时确保/etc/hosts没有其他映射
  1. 运行 kafka-console-producer-sasl.sh 生产消息
./bin/kafka-console-producer-sasl.sh --broker-list centos:9092 --topic test --producer.config config/kerberos/client.properties

image

  1. 运行 bin/kafka-console-consumer-sasl.sh 消费消息：
 ./bin/kafka-console-consumer-sasl.sh --bootstrap-server centos:9092 --topic test --from-beginning --consumer.config config/kerberos/client.properties

image

至此，kafka开启kerberos认证完结，抱歉，后面开启认证不够详细，因为被权限的问题困扰一天，整到晚上8点才发现权限问题，中途遇到的报错我尽可能的写在下面，希望对你们有帮助

4. kafka由kerberos认证改为SCRAM认证

4.1 准备和创建用户凭证 (SCRAM机制的核心优势)

SASL/SCRAM 的一个主要优势是可以动态管理用户。使用 Kafka 自带的 kafka-configs.sh 脚本进行创建。

  • 如果kafka开启kerberos认证开启着，关闭启动原始kafka

停止命令：

./bin/kafka-server-stop.sh

启动原始kafka（因为上述kerberos单独建立了kafka-server-start-sasl.sh ，所以启动kafka-server-start.sh就是原始 ）：

 ./bin/kafka-server-start.sh -daemon ./config/server.properties
  • 创建管理员用户 (admin)：用于后续的 Broker 间通信。

执行如下命令的前提是server.properties中配置了advertised.listeners，否则会报连不上node，我的配置如下
image

 ./bin/kafka-configs.sh --bootstrap-server 139.19.71.51:9092 --alter --add-config 'SCRAM-SHA-256=[password=Marine.123],SCRAM-SHA-512=[password=Marine.123]' --entity-type users --entity-name admin

注意：对于 KRaft 模式下的 Kafka 集群，需要使用 kafka-storage 命令的 --add-scram 选项

  • 创建普通用户 (alice)：用于客户端（生产者/消费者）的连接。
 ./bin/kafka-configs.sh --bootstrap-server 139.19.71.51:9092 --alter --add-config 'SCRAM-SHA-256=[password=Marine.123],SCRAM-SHA-512=[password=Marine.123]' --entity-type users --entity-name alice

4.2 修改 Kafka 服务端 (server.properties)

对配置文件进行以下增、删、改操作：

  • 修改与启用 (listeners 保持 SASL_PLAINTEXT 或 SASL_SSL 不变)：
    我这里同样复制一份server.properties，仅用于SCRAM认证
sudo cp config/kerberos/server-sasl.properties config/scram/server-scram.properties

修改server-scram.properties为如下内容：

sasl.enabled.mechanisms=SCRAM-SHA-256,SCRAM-SHA-512
sasl.mechanism.inter.broker.protocol=SCRAM-SHA-256
  • 删除：移除任何与 Kerberos 相关的配置行，例如 sasl.kerberos.service.name。

4.3 配置 Broker 的 JAAS (kafka-server-jaas.conf和kafka-client-jaas.conf)

创建或编辑 JAAS 配置文件（通常在 config/ 目录下），用于 Broker 自身的登录认证
同样的，复制一份kafka-server-jaas.conf，仅用于SCRAM认证：

我这里提前创建了个和kerberos同级的scram文件夹，用于scram认证

sudo cp config/kerberos/kafka-server-jaas.conf config/scram/kafka-server-jaas.conf
sudo cp config/kerberos/kafka-client-jaas.conf config/scram/kafka-client-jaas.conf

编辑scram/kafka-server-jaas.conf，为如下内容：

sudo vim config/scram/kafka-server-jaas.conf

KafkaServer {
   org.apache.kafka.common.security.scram.ScramLoginModule required
   username="admin"
   password="Marine.123";
};

编辑scram/kafka-client-jaas.conf，为如下内容：

sudo vim config/scram/kafka-client-jaas.conf

KafkaClient {
    org.apache.kafka.common.security.scram.ScramLoginModule required
    username="alice"
    password="Marine.123";
};

小提示：KafkaServer 是固定的 section 名称，Kafka 进程会去寻找它。JAAS 配置也可以通过 listener.name.{listenerName}.scram-sha-256.sasl.jaas.config 这样的格式直接在 server.properties 文件中定义，但使用独立的 JAAS 文件管理起来更清晰。

4.4 配置客户端 (client.properties)

sudo cp config/kerberos/client.properties config/scram/client.properties
sudo vim config/scram/client.properties

修改scram/client.properties为如下内容：

security.protocol=SASL_PLAINTEXT
sasl.mechanism=SCRAM-SHA-256
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
    username="alice" \
    password="Marine.123";
bootstrap.servers=139.19.71.51:9092

4.5 修改kafka脚本（kafka-server-start-sasl.sh、kafka-topics.sh、kafka-console-producer.sh和kafka-console-consumer.sh）

sudo cp bin/kafka-server-start-sasl.sh bin/kafka-server-start-scram.sh
sudo cp bin/kafka-topics-sasl.sh bin/kafka-topics-scram.sh
sudo cp bin/kafka-console-producer-sasl.sh bin/kafka-console-producer-scram.sh
sudo cp bin/kafka-console-consumer-sasl.sh bin/kafka-console-consumer-scram.sh

kafka-server-start-scram.sh：注掉kerberos配置，更改JAAS路径，如下：
image

kafka-topics-scram.sh：更改JAAS路径，如下：
image

kafka-console-producer-scram.sh：更改JAAS路径，如下：
image

kafka-console-consumer-scram.sh：更改JAAS路径，如下：
image

至此，所有配置更改完成

4.6 启动验证

启动：

./bin/kafka-server-start-scram.sh -daemon config/scram/server-scram.properties

查询topic：

./bin/kafka-topics-scram.sh --list --bootstrap-server 139.19.71.51:9092 --command-config config/scram/client.properties

生产者：

bin/kafka-console-producer-scram.sh --bootstrap-server 139.19.71.51:9092 --topic test --producer.config config/scram/client.properties

消费者：

bin/kafka-console-consumer-scram.sh --bootstrap-server 139.19.71.51:9092 --topic test --from-beginning --consumer.config config/scram/client.properties

--from-beginning为查看所有，不需要可以不添加

5. 遇到的报错问题

  1. keytab文件有误，或者无法获取
Exception in thread "main" org.apache.kafka.common.KafkaException: Failed to create new KafkaAdminClient
        at org.apache.kafka.clients.admin.KafkaAdminClient.createInternal(KafkaAdminClient.java:551)
        at org.apache.kafka.clients.admin.KafkaAdminClient.createInternal(KafkaAdminClient.java:488)
        at org.apache.kafka.clients.admin.Admin.create(Admin.java:134)
        at kafka.admin.TopicCommand$TopicService$.createAdminClient(TopicCommand.scala:203)
        at kafka.admin.TopicCommand$TopicService$.apply(TopicCommand.scala:207)
        at kafka.admin.TopicCommand$.main(TopicCommand.scala:51)
        at kafka.admin.TopicCommand.main(TopicCommand.scala)
Caused by: java.lang.SecurityException: java.io.IOException: 配置错误:

原因：keytab文件有误，或者无法获取

kafka的server.log报错：
client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner authentication information from the us

解决办法：

  1. 验证keytab文件：
    检查 broker 的 keytab 文件：
# 查看 keytab 内容
klist -kt /home/cloud_nms/kafka/kafka_2.12-3.5.0/config/kerberos/kafka-server.keytab
  • 确认输出的 principal 为 kafka-server/localhost@KAFKA.COM（与你 JAAS 中配置的一致）
  • 如果 principal 不对，需要重新生成 keytab。
    测试 keytab 有效性：
kinit -kt /home/cloud_nms/kafka/kafka_2.12-3.5.0/config/kerberos/kafka-server.keytab kafka-server/centos@KAFKA.COM
  • 如果成功，klist 会显示票据；如果失败，会给出具体错误（如 kvno 不匹配），需要重新生成 keytab。
  1. 检查kafka-server-jaas.conf文件内容
  2. 检查防火墙状态和端口状态
  3. 检查keytab文件权限（直接改权限644就行了）
    最有效的办法，查看kafka启动报错信息，我的位置在/home/cloud_nms/kafka/kafka_2.12-3.5.0/logs/server.log中，查看kerberos启动日志，在/var/log/krb5kdc.log和/var/log/kadmind.log，一般后者不报错，主要看server.log，然后解决办法就是网页查询加问AI

server.log报错：
2. Message stream modified
解决办法：删除客户端使用的配置文件中的krb5.conf中的renew_lifetime配置项，也可以注释

  1. [Kerberos]Message stream modified (41)错误
[2026-03-27 13:59:52,805] ERROR Exiting Kafka due to fatal exception during startup. (kafka.Kafka$)
org.apache.kafka.common.KafkaException: javax.security.auth.login.LoginException: Message stream modified (41)
	at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:184)
	at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:192)
	at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:81)
	at kafka.server.BrokerToControllerChannelManagerImpl.buildNetworkClient$1(BrokerToControllerChannelManager.scala:204)
	at kafka.server.BrokerToControllerChannelManagerImpl.newRequestThread(BrokerToControllerChannelManager.scala:243)
	at kafka.server.BrokerToControllerChannelManagerImpl.<init>(BrokerToControllerChannelManager.scala:183)
	at kafka.server.BrokerToControllerChannelManager$.apply(BrokerToControllerChannelManager.scala:149)
	at kafka.server.KafkaServer.startup(KafkaServer.scala:316)
	at kafka.Kafka$.main(Kafka.scala:113)
	at kafka.Kafka.main(Kafka.scala)
Caused by: javax.security.auth.login.LoginException: Message stream modified (41)
	at jdk.security.auth/com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:785)
	at jdk.security.auth/com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:597)
	at java.base/javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
	at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:679)
	at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:677)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
	at java.base/javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:677)
	at java.base/javax.security.auth.login.LoginContext.login(LoginContext.java:587)
	at org.apache.kafka.common.security.authenticator.AbstractLogin.login(AbstractLogin.java:60)
	at org.apache.kafka.common.security.kerberos.KerberosLogin.login(KerberosLogin.java:103)
	at org.apache.kafka.common.security.authenticator.LoginManager.<init>(LoginManager.java:62)
	at org.apache.kafka.common.security.authenticator.LoginManager.acquireLoginManager(LoginManager.java:112)
	at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:170)
	... 9 more
Caused by: KrbException: Message stream modified (41)
	at java.security.jgss/sun.security.krb5.KrbKdcRep.check(KrbKdcRep.java:103)
	at java.security.jgss/sun.security.krb5.KrbAsRep.decrypt(KrbAsRep.java:159)
	at java.security.jgss/sun.security.krb5.KrbAsRep.decryptUsingKeyTab(KrbAsRep.java:121)
	at java.security.jgss/sun.security.krb5.KrbAsReqBuilder.resolve(KrbAsReqBuilder.java:310)
	at java.security.jgss/sun.security.krb5.KrbAsReqBuilder.action(KrbAsReqBuilder.java:498)
	at jdk.security.auth/com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:761)
	... 21 more
[2026-03-27 13:59:52,806] INFO [KafkaServer id=0] shutting down (kafka.server.KafkaServer)

解决：
删除 krb5.conf 配置文件里的 renew_lifetime = xxx 这行配置即可

  1. 服务名不匹配
[2026-03-27 15:43:17,490] WARN [Controller id=0, targetBrokerId=0] Connection to node 0 (139.19.71.51/139.19.71.51:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2026-03-27 15:43:17,495] INFO [Controller id=0, targetBrokerId=0] Client requested connection close from node 0 (org.apache.kafka.clients.NetworkClient)
[2026-03-27 15:43:17,718] INFO [Controller id=0, targetBrokerId=0] Node 0 disconnected. (org.apache.kafka.clients.NetworkClient)
[2026-03-27 15:43:17,718] WARN [Controller id=0, targetBrokerId=0] Connection to node 0 (139.19.71.51/139.19.71.51:9092) terminated during authentication. This may happen due to any of the following reasons: (1) Authentication failed due to invalid credentials with brokers older than 1.0.0, (2) Firewall blocking Kafka TLS traffic (eg it may only allow HTTPS traffic), (3) Transient network issue. (org.apache.kafka.clients.NetworkClient)
[2026-03-27 15:43:17,718] INFO [Controller id=0, targetBrokerId=0] Client requested connection close from node 0 (org.apache.kafka.clients.NetworkClient)
[2026-03-27 15:43:17,887] INFO [Controller id=0, targetBrokerId=0] Node 0 disconnected. (org.apache.kafka.clients.NetworkClient)
[2026-03-27 15:43:17,887] WARN [Controller id=0, targetBrokerId=0] Connection to node 0 (139.19.71.51/139.19.71.51:9092) terminated during authentication. This may happen due to any of the following reasons: (1) Authentication failed due to invalid credentials with brokers older than 1.0.0, (2) Firewall blocking Kafka TLS traffic (eg it may only allow HTTPS traffic), (3) Transient network issue. (org.apache.kafka.clients.NetworkClient)
[2026-03-27 15:43:17,887] INFO [Controlle

解决：

  • 如果 Broker principal 是 kafka/host@REALM，则 sasl.kerberos.service.name 应为 kafka（这是默认值，若未配置则使用此默认）。

  • 所有 Broker 的 security.inter.broker.protocol 必须相同。

  1. kerberos的缓存机制

  2. 操作系统级别：Kerberos 客户端库（如 MIT Kerberos、Heimdal）在首次调用时会读取 krb5.conf，并将其解析结果缓存在进程内存中。后续的认证请求直接使用内存中的配置，不会重新读取文件。

  3. Java 环境：JRE 中的 sun.security.krb5.Config 类采用单例模式，第一次调用 Config.getInstance() 时加载并解析 krb5.conf，之后一直复用该实例，直至 JVM 退出。

这意味着，如果你直接修改了 krb5.conf 文件，正在运行的 Java 应用不会自动感知变化。

在设置 krb5 配置文件中添加清除缓存的代码：
jdk8：

        try {
            LOG.info("KAFKA -> setKrb5Config -> refresh");
            Field configField = Config.class.getDeclaredField("singleton");
            configField.setAccessible(true);
            configField.set(null, null); // 强制置空
            // 重新获取实例以触发重新加载
            Config.refresh();
            return Config.getInstance().getDefaultRealm();
        } catch (Exception e) {
            LOG.error("KAFKA -> setKrb5Config -> ERROR", e);
        }

jdk17：因为17的封装不能使用jdk8的方式

    private static void refreshJaasConfiguration() {
        try {
            System.clearProperty("java.security.krb5.realm");
            System.clearProperty("java.security.krb5.kdc");
            // 使用标准的 Configuration 类刷新
//            Configuration.setConfiguration(null);
            Class<?> configClass = Class.forName("sun.security.krb5.Config");
            Method getInstance = configClass.getMethod("getInstance");
            Object krb5Config = getInstance.invoke(null);
            Method refresh = configClass.getMethod("refresh");
            refresh.invoke(krb5Config);
            LOG.info("成功刷新 JAAS 配置");
        } catch (Exception e) {
            LOG.warn("刷新 JAAS 配置失败：{}", e.getMessage());
        }
    }
  1. 防火墙未关闭或未开放9092端口导致Controller启动失败
[2026-09-18 16:16:08,930] WARN [RequestSendThread controllerId=0] Controller 0's connection to broker 139.19.8.80:9092 (id: 0 rack: null) was unsuccessful (kafka.controller.RequestSendThread)
java.io.IOException: Connection to 139.19.8.80:9092 (id: 0 rack: null) failed.
	at org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
	at kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
	at kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
	at org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)

解决：关闭防火墙，或者防火墙上开通9092端口
查看防火墙开放端口：

sudo firewall-cmd --list-ports

开通端口：

sudo firewall-cmd --zone=public --add-port=9092/tcp --permanent

参考文献：kafka安装参考：https://blog.csdn.net/u013416034/article/details/123875299
kerberos安装参考：https://www.cnblogs.com/wuyongyin/p/15634397.html
kafka开启kerberos参考：https://www.cnblogs.com/myownswordsman/p/kafka-security-kerberos.html
https://blog.csdn.net/mandiandengwo/article/details/144831039
https://www.cnblogs.com/lizhaoxian/p/16454980.html
错误报错参考：https://developer.aliyun.com/article/1604481

posted @ 2026-03-06 11:25  ꧁ʚ星月天空ɞ꧂  阅读(129)  评论(0)    收藏  举报