运行模式_StandaloneHA测试

1. node1上启动spark集群

xxxxxxxxxx
[root@node1 sbin]# pwd
/opt/spark-3.2.1/sbin
[root@node1 sbin]# ./start-all.sh 

2. node1上启动历史日志进程(如已启动忽略此步)

xxxxxxxxxx
[root@node1 sbin]# start-history-server.sh

3. node2上启动备用的master

xxxxxxxxxx
[root@node2 conf]# cd ../sbin/
[root@node2 sbin]# start-master.sh

4. WebUI检查启动情况

http://node1:8080/ 如果8080被占用则测试8081、8082...

http://node2:8081/

5. node2上提交一个spark任务到当前alivemaster上

xxxxxxxxxx
spark-submit --master spark://node1:7077 \
--class org.apache.spark.examples.SparkPi \
/opt/spark-3.2.1/examples/jars/spark-examples_2.12-3.2.1.jar 800

6. 在提交成功后, 将alivemaster直接kill掉，不会影响程序运行

xxxxxxxxxx
[root@node1 ~]# stop-master.sh 
stopping org.apache.spark.deploy.master.Master

7. 返回node2界面上查看日志：

xxxxxxxxxx
06/21 20:44:41 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
06/21 20:45:02 WARN StandaloneAppClient$ClientEndpoint: Connection to node1:7077 failed; waiting for master to reconnect...#发现node1上的master down了，寻找新的主master
06/21 20:45:02 WARN StandaloneSchedulerBackend: Disconnected from Spark cluster! Waiting for reconnection...
06/21 20:45:02 WARN StandaloneAppClient$ClientEndpoint: Connection to node1:7077 failed; waiting for master to reconnect...
Pi is roughly 3.137800

当新的master接收集群后, 程序继续运行, 正常得到结果。结论 HA模式下, 主备切换 不会影响到正在运行的程序，最大的影响是会让它中断大约30秒左右。

8. http://node2:8082/