kafka关于修改副本数和分区的数的案例实战（也可用作leader节点均衡案例）

一、关于topic分区数的修改

1、创建1个分区1个topic,名称为rabin-channel

[root@Hexindai-C11-73 ~]# kafka-topics.sh --create --zookeeper Hexindai-C11-73:2181/kafka --replication-factor 1 --partitions 1 --topic rabin-channel                 
Created topic "rabin-channel".
[root@Hexindai-C11-73 ~]# 

2、查看topic的信息

[root@Hexindai-C11-73 ~]# kafka-topics.sh --describe --zookeeper Hexindai-C11-73:2181/kafka --topic rabin-channel
Topic:rabin-channel     PartitionCount:1        ReplicationFactor:1     Configs:
        Topic: rabin-channel    Partition: 0    Leader: 74      Replicas: 74    Isr: 74
[root@Hexindai-C11-73 ~]# 

3、将之前创建的topic修改为3个分区

[root@Hexindai-C11-73 ~]# kafka-topics.sh --alter --zookeeper Hexindai-C11-73:2181/kafka --topic rabin-channel --partitions 3         
WARNING: If partitions are increased for a topic that has a key, the partition logic or ordering of the messages will be affected
Adding partitions succeeded!
[root@Hexindai-C11-73 ~]# 

4、再次查看topic的分区数

[root@Hexindai-C11-73 ~]# kafka-topics.sh --describe --zookeeper Hexindai-C11-73:2181/kafka --topic rabin-channel            
Topic:rabin-channel     PartitionCount:3        ReplicationFactor:1     Configs:
        Topic: rabin-channel    Partition: 0    Leader: 74      Replicas: 74    Isr: 74
        Topic: rabin-channel    Partition: 1    Leader: 71      Replicas: 71    Isr: 71
        Topic: rabin-channel    Partition: 2    Leader: 72      Replicas: 72    Isr: 72
[root@Hexindai-C11-73 ~]# 

二、关于topic副本数的修改

1、编写分配脚本

[root@Hexindai-C11-73 ~]# cat addReplicas.json
{"topics":
        [{"topic":"rabin-channel"}],
        "version":1
}
[root@Hexindai-C11-73 ~]# 

2、执行分配计划，用于生成json格式的文件

[root@Hexindai-C11-73 ~]# kafka-reassign-partitions.sh --zookeeper Hexindai-C11-71:2181/kafka --topics-to-move-json-file addReplicas.json --broker-list "71,72,74" 
--generate
Current partition replica assignment       #这是当前的分区情况，可以结合--describe参数查看当前的分区情况
{"version":1,"partitions":[{"topic":"rabin-channel","partition":2,"replicas":[72]},{"topic":"rabin-channel","partition":1,"replicas":[71]},{"topic":"rabin-channel",
"partition":0,"replicas":[74]}]}

Proposed partition reassignment configuration  #这是推荐分区计划
{"version":1,"partitions":[{"topic":"rabin-channel","partition":2,"replicas":[74]},{"topic":"rabin-channel","partition":1,"replicas":[72]},{"topic":"rabin-channel",
"partition":0,"replicas":[71]}]}
[root@Hexindai-C11-73 ~]# 

[root@Hexindai-C11-73 ~]# kafka-topics.sh --describe --zookeeper Hexindai-C11-73:2181/kafka --topic rabin-channel         #查看当前分区的情况                                              
Topic:rabin-channel     PartitionCount:3        ReplicationFactor:1     Configs:
        Topic: rabin-channel    Partition: 0    Leader: 74      Replicas: 74    Isr: 74
        Topic: rabin-channel    Partition: 1    Leader: 71      Replicas: 71    Isr: 71
        Topic: rabin-channel    Partition: 2    Leader: 72      Replicas: 72    Isr: 72
[root@Hexindai-C11-73 ~]# 

3、Proposed partition reassignment configuration 后是根据命令行的指定的brokerlist生成的分区分配计划json格式。将 Proposed partition reassignment configuration的配置copy保存到一个文件中topic-reassignment.json并对它进行相应的修改

[root@Hexindai-C11-73 ~]# cat topic-reasssignment.json
{"version":1,"partitions":[{"topic":"rabin-channel","partition":0,"replicas":[73,71]},{"topic":"rabin-channel","partition":1,"replicas":[73,74]},
{"topic":"rabin-channel","partition":2,"replicas":[71,74]}]}
[root@Hexindai-C11-73 ~]# 

4、根据上一步生成的分配计划配置json文件topic-reassignment.json，进行topic的重新分配

[root@Hexindai-C11-73 ~]# kafka-reassign-partitions.sh --zookeeper Hexindai-C11-73:2181/kafka --reassignment-json-file topic-reasssignment.json --execute
Current partition replica assignment

{"version":1,"partitions":[{"topic":"rabin-channel","partition":2,"replicas":[72]},{"topic":"rabin-channel","partition":1,"replicas":[71]},{"topic":"rabin-channel","partition":0,"replicas":[74]}]}

Save this to use as the --reassignment-json-file option during rollback
Successfully started reassignment of partitions.
[root@Hexindai-C11-73 ~]# 

　　提示：上述方法不仅仅可以用来修改副本数，还可以用来修改leader节点，下图是用来均衡leader节点的截图：

6、若分配完成，我们再次查看

[root@Hexindai-C11-73 ~]# kafka-topics.sh --describe --zookeeper Hexindai-C11-73:2181/kafka --topic rabin-channel     #还没有分配之前的状态                                                       
Topic:rabin-channel     PartitionCount:3        ReplicationFactor:1     Configs:
        Topic: rabin-channel    Partition: 0    Leader: 74      Replicas: 74    Isr: 74
        Topic: rabin-channel    Partition: 1    Leader: 71      Replicas: 71    Isr: 71
        Topic: rabin-channel    Partition: 2    Leader: 72      Replicas: 72    Isr: 72
[root@Hexindai-C11-73 ~]#
[root@Hexindai-C11-73 ~]# kafka-topics.sh --describe --zookeeper Hexindai-C11-73:2181/kafka --topic rabin-channel  #分配之后的状态
Topic:rabin-channel     PartitionCount:3        ReplicationFactor:2     Configs:
        Topic: rabin-channel    Partition: 0    Leader: 73      Replicas: 73,71 Isr: 73,71
        Topic: rabin-channel    Partition: 1    Leader: 73      Replicas: 73,74 Isr: 74,73
        Topic: rabin-channel    Partition: 2    Leader: 71      Replicas: 71,74 Isr: 74,71
[root@Hexindai-C11-73 ~]#
#分配后的状态是不是变成了两个副本呢？