Broker_服役新节点

1. 将node1拍摄快照new_broker_pre(注意再关机状态下)

2. 启动node1节点

3. 从node2上将/opt/kafka目录拷贝到node1上一份

   xxxxxxxxxx
   [root@node2 ~]# scp -r /opt/kafka/ node1:/opt
   

4. 修改node1上server.properties，将broker.id的值改为3

   xxxxxxxxxx
   [root@node1 ~]# cd /opt/kafka/config/
   [root@node1 config]# vim server.properties
   broker.id=3
   

5. 删除 node1中 kafka 下logs

   xxxxxxxxxx
   [root@node1 config]# cd ../
   [root@node1 kafka]# ls
   bin  config  libs  LICENSE  licenses  logs  NOTICE  site-docs
   [root@node1 kafka]# rm -rf logs/
   

6. 单独启动 node1中的 kafka,如果zk集群和node2,3,4的kafka没有启动先启动它们。

   xxxxxxxxxx
   [root@node1 kafka]# bin/kafka-server-start.sh -daemon config/server.properties
   [root@node1 kafka]# jps
   2180 Kafka
   2253 Jps
   

7. 在node2的/opt/kafka目录下创建jsons目录

   xxxxxxxxxx
   [root@node2 ~]# cd /opt/kafka/
   [root@node2 kafka]# mkdir jsons
   

8. 编写一个json文件，配置想要被负载均衡的主题（最好是有数据的主题）

   xxxxxxxxxx
   [root@node2 kafka]# vim jsons/topics_to_move.json
   {
    "topics": [
    {"topic": "topicA"}
    ],
    "version": 1
   }
   

9. 根据8编写的json文件生成一个负载均衡的计划

   xxxxxxxxxx
   [root@node2 kafka]# kafka-reassign-partitions.sh --bootstrap-server node2:9092 --topics-to-move-json-file jsons/topics_to_move.json --broker-list "0,1,2,3" --generate
   Current partition replica assignment
   {"version":1,"partitions":[{"topic":"topicA","partition":0,"replicas":[2,0],"log_dirs":["any","any"]},{"topic":"topicA","partition":1,"replicas":[1,2],"log_dirs":["any","any"]},{"topic":"topicA","partition":2,"replicas":[1,2],"log_dirs":["any","any"]}]}
   
   Proposed partition reassignment configuration
   {"version":1,"partitions":[{"topic":"topicA","partition":0,"replicas":[3,0],"log_dirs":["any","any"]},{"topic":"topicA","partition":1,"replicas":[0,1],"log_dirs":["any","any"]},{"topic":"topicA","partition":2,"replicas":[1,2],"log_dirs":["any","any"]}]}
   

10. 创建副本存储计划（所有副本存储在 broker0、broker1、broker2、broker3 中）

    xxxxxxxxxx
    [root@node2 kafka]# vim jsons/increase-replication-factor.json
    {"version":1,"partitions":[{"topic":"topicA","partition":0,"replicas":[3,0],"log_dirs":["any","any"]},{"topic":"topicA","partition":1,"replicas":[0,1],"log_dirs":["any","any"]},{"topic":"topicA","partition":2,"replicas":[1,2],"log_dirs":["any","any"]}]}
    

11. 执行副本存储计划

    xxxxxxxxxx
    [root@node2 kafka]# kafka-reassign-partitions.sh --bootstrap-server node2:9092 --reassignment-json-file jsons/increase-replication-factor.json --execute
    

12. 验证副本存储计划

    xxxxxxxxxx
    [root@node2 kafka]# kafka-reassign-partitions.sh --bootstrap-server node2:9092 --reassignment-json-file jsons/increase-replication-factor.json --verify
    

    结果信息：

    xxxxxxxxxx
    Status of partition reassignment:
    Reassignment of partition topicA-0 is complete.
    Reassignment of partition topicA-1 is complete.
    Reassignment of partition topicA-2 is complete.
    Clearing broker-level throttles on brokers 0,1,2,3
    Clearing topic-level throttles on topic topicA