spark standalone ha spark submit

when you build a spark standalone ha cluster, when you submit your app,  you should send it to the leader master,

not the standby master, how to decided the status of the two masters?

a simple way, not judge the status which master is the active, you can just add

spark.master spark://master1:port,master2:port

in the $SPARK_HOME/conf/spark-defaults.conf

or else,

you can curl the web-ui for your spark master.

like curl sparkmaster:web-ui-port |grep STANDBY?

 

posted on 2016-09-19 15:45  tneduts  阅读(249)  评论(0编辑  收藏  举报

导航