大叔经验分享（4）Yarn ResourceManager页面如何实现主被自动切换

hdfs、yarn、hbase这些组件的master支持多个，实现自动主备切换，其中hdfs、hbase无论访问主master或者备master都可以正常访问页面，但是yarn比较特别，只有主master的页面可以访问，备master会返回Refresh，3s后重定向；

一种方式是提供两个域名，分别对应两个yarn的master，一旦有master切换，需要手工切换到另外一个，有没有更好的方式？

 

访问备master过程如下：

curl http://standby_ip:8088/cluster -v

* About to connect() to standby_ip port 8088 (#0)

*   Trying standby_ip...

* Connected to standby_ip (standby_ip) port 8088 (#0)

> GET /cluster HTTP/1.1

> User-Agent: curl/7.29.0

> Host: standby_ip:8088

> Accept: */*

>

< HTTP/1.1 200 OK

< Cache-Control: no-cache

< Expires: Tue, 25 Sep 2018 03:59:22 GMT

< Date: Tue, 25 Sep 2018 03:59:22 GMT

< Pragma: no-cache

< Expires: Tue, 25 Sep 2018 03:59:22 GMT

< Date: Tue, 25 Sep 2018 03:59:22 GMT

< Pragma: no-cache

< Content-Type: text/plain; charset=UTF-8

< Refresh: 3; url=http://active_ip:8088/cluster

< Content-Length: 103

< Server: Jetty(6.1.26)

<

This is standby RM. Redirecting to the current active RM: http://active_ip:8088/cluster

* Connection #0 to host standby_ip left intact

可见备master响应http status为200，包含Refresh头，同时body为This is standby RM. Redirecting to the current active RM:***

有没有可能在load balancer（比如nginx）上配置实现自动切换？这里非法响应需要判断header包含‘Refresh’或者判断body包含‘This is standby RM’

 

1）被动切换，支持http status以及timeout等判断，不满足

http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream

proxy_next_upstream error | timeout | invalid_header | http_500 | http_502 | http_503 | http_504 | http_403 | http_404 | http_429 | non_idempotent | off ...;

 

2）主动健康检查，最常用的开源module，只支持http status级别的健康检查，不满足

https://github.com/yaoweibin/nginx_upstream_check_module

  check_http_expect_alive

    syntax: *check_http_expect_alive [ http_2xx | http_3xx | http_4xx |

    http_5xx ]*

    default: *http_2xx | http_3xx*

    context: *upstream*

    description: These status codes indicate the upstream server's http

    response is ok, the backend is alive.

 

3）nginx收费版本的ngx_http_upstream_hc_module，支持header和body的判断，可以满足

Dynamically configurable group with periodic health checks is available as part of our commercial subscription:

http://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html

    match active {

        body !~ "This is standby RM";

    }