Envoy-负载均衡之全局负载均衡策略之优先级、位置权重【八】
节点优先级及优先级调度
优先级调度
Panic阈值
优先级调度示例
优先级实验部署
1.Readme
五个Service:
envoy：Front Proxy,地址为172.31.29.2
webserver01：第一个后端服务
webserver01-sidecar：第一个后端服务的Sidecar Proxy,地址为172.31.29.11, 别名为red和webservice1
webserver02：第二个后端服务
webserver02-sidecar：第一个后端服务的Sidecar Proxy,地址为172.31.29.12, 别名为blue和webservice1
webserver03：第三个后端服务
webserver03-sidecar：第一个后端服务的Sidecar Proxy,地址为172.31.29.13, 别名为green和webservice1
webserver04：第四个后端服务
webserver04-sidecar：第四个后端服务的Sidecar Proxy,地址为172.31.29.14, 别名为gray和webservice2
webserver05：第五个后端服务
webserver05-sidecar：第五个后端服务的Sidecar Proxy,地址为172.31.29.15, 别名为black和webservice2
运行和测试
创建
docker-compose up
测试
持续请求服务，可发现，请求均被调度至优先级为0的webservice1相关的后端端点之上；
while true; do curl 172.31.29.2; sleep .5; done
# 等确定服务的调度结果后，另启一个终端，修改webservice1中任何一个后端端点的/livez响应为非"OK"值，例如，修改第一个后端端点;
curl -X POST -d 'livez=FAIL' http://172.31.29.11/livez
# 而后通过请求的响应结果可发现，因过载因子为1.4，客户端的请求仍然始终只发往webservice1的后端端点blue和green之上；
# 等确定服务的调度结果后，再修改其中任何一个服务的/livez响应为非"OK"值，例如，修改第一个后端端点;
curl -X POST -d 'livez=FAIL' http://172.31.29.12/livez
# 请求中，可以看出第一个端点因响应5xx的响应码，每次被加回之后，会再次弹出，除非使用类似如下命令修改为正常响应结果；
curl -X POST -d 'livez=OK' http://172.31.29.11/livez
# 而后通过请求的响应结果可发现，因过载因子为1.4，优先级为0的webserver1已然无法锁住所有的客户端请求，于是，客户端的请求的部分流量将被转发至webservice2的端点之上；
停止后清理
docker-compose down2.docker-compose.yaml
此实验 front-envoy基于：envoy-alpine:v1.13-latest 版本
[root@xksmaster1 locality-weighted]# cat docker-compose.yaml
version: '3'
services:
front-envoy:
image: envoyproxy/envoy-alpine:v1.21-latest
environment:
- ENVOY_UID=0
- ENVOY_GID=0
volumes:
- ./front-envoy.yaml:/etc/envoy/envoy.yaml
networks:
- envoymesh
expose:
# Expose ports 80 (for general traffic) and 9901 (for the admin server)
- "80"
- "9901"
webserver01-sidecar:
image: envoyproxy/envoy-alpine:v1.21-latest
environment:
- ENVOY_UID=0
- ENVOY_GID=0
volumes:
- ./envoy-sidecar-proxy.yaml:/etc/envoy/envoy.yaml
hostname: red
networks:
envoymesh:
ipv4_address: 172.31.31.11
aliases:
- webservice1
- red
webserver01:
image: ikubernetes/demoapp:v1.0
environment:
- PORT=8080
- HOST=127.0.0.1
network_mode: "service:webserver01-sidecar"
depends_on:
- webserver01-sidecar
webserver02-sidecar:
image: envoyproxy/envoy-alpine:v1.21-latest
environment:
- ENVOY_UID=0
- ENVOY_GID=0
volumes:
- ./envoy-sidecar-proxy.yaml:/etc/envoy/envoy.yaml
hostname: blue
networks:
envoymesh:
ipv4_address: 172.31.31.12
aliases:
- webservice1
- blue
webserver02:
image: ikubernetes/demoapp:v1.0
environment:
- PORT=8080
- HOST=127.0.0.1
network_mode: "service:webserver02-sidecar"
depends_on:
- webserver02-sidecar
webserver03-sidecar:
image: envoyproxy/envoy-alpine:v1.21-latest
environment:
- ENVOY_UID=0
- ENVOY_GID=0
volumes:
- ./envoy-sidecar-proxy.yaml:/etc/envoy/envoy.yaml
hostname: green
networks:
envoymesh:
ipv4_address: 172.31.31.13
aliases:
- webservice1
- green
webserver03:
image: ikubernetes/demoapp:v1.0
environment:
- PORT=8080
- HOST=127.0.0.1
network_mode: "service:webserver03-sidecar"
depends_on:
- webserver03-sidecar
webserver04-sidecar:
image: envoyproxy/envoy-alpine:v1.21-latest
environment:
- ENVOY_UID=0
- ENVOY_GID=0
volumes:
- ./envoy-sidecar-proxy.yaml:/etc/envoy/envoy.yaml
hostname: gray
networks:
envoymesh:
ipv4_address: 172.31.31.14
aliases:
- webservice2
- gray
webserver04:
image: ikubernetes/demoapp:v1.0
environment:
- PORT=8080
- HOST=127.0.0.1
network_mode: "service:webserver04-sidecar"
depends_on:
- webserver04-sidecar
webserver05-sidecar:
image: envoyproxy/envoy-alpine:v1.21-latest
environment:
- ENVOY_UID=0
- ENVOY_GID=0
volumes:
- ./envoy-sidecar-proxy.yaml:/etc/envoy/envoy.yaml
hostname: black
networks:
envoymesh:
ipv4_address: 172.31.31.15
aliases:
- webservice2
- black
webserver05:
image: ikubernetes/demoapp:v1.0
environment:
- PORT=8080
- HOST=127.0.0.1
network_mode: "service:webserver05-sidecar"
depends_on:
- webserver05-sidecar
networks:
envoymesh:
driver: bridge
ipam:
config:
- subnet: 172.31.31.0/243.front-envoy.yaml
因子为1.4：overprovisioning_factor: 140
对每个service的/livez进行健康检查,返回如果不是OK就认为失败.
两个集群cn-north-1和cn-north-2,cn-north-1优先级是0,cn-north-2优先级是1.所以当cn-north-1全部正常的情况下cn-north-2不分担流量admin:
access_log_path: "/dev/null"
address:
socket_address:
address: 0.0.0.0
port_value: 9901
static_resources:
listeners:
- address:
socket_address:
address: 0.0.0.0
port_value: 80
name: listener_http
filter_chains:
- filters:
- name: envoy.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager
codec_type: auto
stat_prefix: ingress_http
route_config:
name: local_route
virtual_hosts:
- name: backend
domains:
- "*"
routes:
- match:
prefix: "/"
route:
cluster: webcluster1
http_filters:
- name: envoy.router
clusters:
- name: webcluster1
connect_timeout: 0.5s
type: STRICT_DNS
lb_policy: ROUND_ROBIN
http2_protocol_options: {}
load_assignment:
cluster_name: webcluster1
policy:
overprovisioning_factor: 140
endpoints:
- locality:
region: cn-north-1
priority: 0
lb_endpoints:
- endpoint:
address:
socket_address:
address: webservice1
port_value: 80
- locality:
region: cn-north-2
priority: 1
lb_endpoints:
- endpoint:
address:
socket_address:
address: webservice2
port_value: 80
health_checks:
- timeout: 5s
interval: 10s
unhealthy_threshold: 2
healthy_threshold: 1
http_health_check:
path: /livez4.envoy-sidercar-proxy.yaml
admin:
profile_path: /tmp/envoy.prof
access_log_path: /tmp/admin_access.log
address:
socket_address:
address: 0.0.0.0
port_value: 9901
static_resources:
listeners:
- name: listener_0
address:
socket_address: { address: 0.0.0.0, port_value: 80 }
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
codec_type: AUTO
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains: ["*"]
routes:
- match: { prefix: "/" }
route: { cluster: local_cluster }
http_filters:
- name: envoy.filters.http.router
clusters:
- name: local_cluster
connect_timeout: 0.25s
type: STATIC
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: local_cluster
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address: { address: 127.0.0.1, port_value: 8080 }5.部署测试
5.1 此时由于cn-north-1的3台服务器都正常,所以由cn-north-1的服务器响应所有请求
#docker-compose up
# 持续请求服务，可发现，请求均被调度至优先级为0的webservice1相关的后端端点之上；
[root@xksmaster1 priority-levels]# while true; do curl 172.31.29.2; sleep .5; done
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: blue, ServerIP: 172.31.29.12!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: red, ServerIP: 172.31.29.11!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: red, ServerIP: 172.31.29.11!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!5.2 让blue服务 FAIL
当将cn-north-1中的服务器172.31.29.12/livez设置为非OK状态,此时触发健康检查失败,返回值506、
当连续3次506后,故障节点172.31.29.12被标记为不可用,不再将流量转发至172.31.29.12
此时cn-north-1可用率是66.66%,乘以1.4的超配因子(overprovisioning_factor),此时93.324%的流量由cn-north-1负载,其余6.676%的流量由cn-north-2负担.所以此时会看到172.31.29.14,172.31.29.15进行响应# 等确定服务的调度结果后，另启一个终端，修改webservice1中任何一个后端端点的/livez响应为非"OK"值，例如，修改第一个后端端点;
#curl -X POST -d 'livez=FAIL' http://172.31.29.11/livez
# 而后通过请求的响应结果可发现，因过载因子为1.4，客户端的请求仍然始终只发往webservice1的后端端点blue和green之上；
#让blue服务172.31.29.12 FAIL 请求往其他red、green、gray、black流量上转
#经过3次506之后就会有一定几率往172.31.29.14和172.31.29.15进行流量的调度
#算法：100/3=33.3% 、100%-33.3%*1.4（因子）=0.938 、0.938-1=-0.062=6% 流量会转移到gray和black上
root@k8s-node-1:~# curl -XPOST -d "livez=FAIL" 172.31.29.12/livez
webserver02_1 | 127.0.0.1 - - [12/May/2023 08:01:43] "GET /livez HTTP/1.1" 506 -
webserver01_1 | 127.0.0.1 - - [12/May/2023 08:01:43] "GET /livez HTTP/1.1" 200 -
webserver03_1 | 127.0.0.1 - - [12/May/2023 08:01:43] "GET /livez HTTP/1.1" 200 -
webserver04_1 | 127.0.0.1 - - [12/May/2023 08:01:43] "GET /livez HTTP/1.1" 200 -
webserver05_1 | 127.0.0.1 - - [12/May/2023 08:01:43] "GET /livez HTTP/1.1" 200 -
#此时有6%的请求到区域0的 11、13 区域1 14、15
[root@xksmaster1 priority-levels]# while true; do curl 172.31.29.2; sleep .5; done
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: red, ServerIP: 172.31.29.11!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: red, ServerIP: 172.31.29.11!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: gray, ServerIP: 172.31.29.14!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
...
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: red, ServerIP: 172.31.29.11!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: red, ServerIP: 172.31.29.11!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!5.3 blue服务FAIL同时、再让red服务FAIL
[root@xksmaster1 priority-levels]# curl -XPOST -d "livez=FAIL" 172.31.29.11/livez
此时优先级为0 整个区域健康值小于50%到了Panic值
流量转到
green(此时流量少)
black(此时流量多)
gray(此时流量多)
[root@xksmaster1 priority-levels]# while true; do curl 172.31.29.2; sleep .5; done
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: gray, ServerIP: 172.31.29.14!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: gray, ServerIP: 172.31.29.14!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: gray, ServerIP: 172.31.29.14!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: gray, ServerIP: 172.31.29.14!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: gray, ServerIP: 172.31.29.14!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: gray, ServerIP: 172.31.29.14!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: gray, ServerIP: 172.31.29.14!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: gray, ServerIP: 172.31.29.14!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: gray, ServerIP: 172.31.29.14!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: gray, ServerIP: 172.31.29.14!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: gray, ServerIP: 172.31.29.14!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: gray, ServerIP: 172.31.29.14!5.4 blue服务FAIL、red服务FAIL同时、再让gray服务FAIL
root@k8s-node-1:~# curl -XPOST -d "livez=FAIL" 172.31.29.14/livez
webserver02_1 | 127.0.0.1 - - [12/May/2023 08:15:13] "GET /livez HTTP/1.1" 506 -
webserver01_1 | 127.0.0.1 - - [12/May/2023 08:15:13] "GET /livez HTTP/1.1" 506 -
webserver03_1 | 127.0.0.1 - - [12/May/2023 08:15:13] "GET /livez HTTP/1.1" 200 -
webserver05_1 | 127.0.0.1 - - [12/May/2023 08:15:13] "GET /livez HTTP/1.1" 200 -
webserver04_1 | 127.0.0.1 - - [12/May/2023 08:15:13] "GET /livez HTTP/1.1" 506 -
此时优先级为0 整个区域健康值小于50%到了Panic值、优先级1 整个区域=50%
流量重新分配转到
green、black
[root@xksmaster1 priority-levels]# curl -X POST -d 'livez=FAIL' http://172.31.29.14/livez [root@xksmaster1 priority-levels]# while true; do curl 172.31.29.2; sleep .5; done iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: black, ServerIP: 172.31.29.15!5.5 恢复
此时cn-north-1恢复到100%状态,由cn-north-1的服务器响应所有请求,cn-north-2不再参与响应
root@k8s-node-1:~# curl -XPOST -d "livez=OK" 172.31.29.11/livez
root@k8s-node-1:~# curl -XPOST -d "livez=OK" 172.31.29.12/livez
root@k8s-node-1:~# curl -XPOST -d "livez=OK" 172.31.29.14/livez
webserver02_1 | 127.0.0.1 - - [12/May/2023 08:16:13] "GET /livez HTTP/1.1" 200 -
webserver01_1 | 127.0.0.1 - - [12/May/2023 08:16:13] "GET /livez HTTP/1.1" 200 -
webserver03_1 | 127.0.0.1 - - [12/May/2023 08:16:13] "GET /livez HTTP/1.1" 200 -
webserver04_1 | 127.0.0.1 - - [12/May/2023 08:16:13] "GET /livez HTTP/1.1" 200 -
webserver05_1 | 127.0.0.1 - - [12/May/2023 08:16:13] "GET /livez HTTP/1.1" 200 -
## 此时流量又恢复到原先11.12.13上
[root@xksmaster1 priority-levels]# while true; do curl 172.31.29.2; sleep .5; done
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: blue, ServerIP: 172.31.29.12!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: red, ServerIP: 172.31.29.11!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: blue, ServerIP: 172.31.29.12!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: red, ServerIP: 172.31.29.11!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: blue, ServerIP: 172.31.29.12!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: red, ServerIP: 172.31.29.11!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: blue, ServerIP: 172.31.29.12!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: blue, ServerIP: 172.31.29.12!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: red, ServerIP: 172.31.29.11!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: red, ServerIP: 172.31.29.11!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: green, ServerIP: 172.31.29.13!
iKubernetes demoapp v1.0 !! ClientIP: 127.0.0.1, ServerName: blue, ServerIP: 172.31.29.12!位置加权负载均衡1
位置加权负载均衡2
位置加权负载均衡配置示例
部署实验
1.Readme
Locality Weighted Cluster Demo
环境说明
五个Service:
envoy：Front Proxy,地址为172.31.31.2
webserver01：第一个后端服务
webserver01-sidecar：第一个后端服务的Sidecar Proxy,地址为172.31.31.11, 别名为red和webservice1
webserver02：第二个后端服务
webserver02-sidecar：第一个后端服务的Sidecar Proxy,地址为172.31.31.12, 别名为blue和webservice1
webserver03：第三个后端服务
webserver03-sidecar：第一个后端服务的Sidecar Proxy,地址为172.31.31.13, 别名为green和webservice1
webserver04：第四个后端服务
webserver04-sidecar：第四个后端服务的Sidecar Proxy,地址为172.31.31.14, 别名为gray和webservice2
webserver05：第五个后端服务
webserver05-sidecar：第五个后端服务的Sidecar Proxy,地址为172.31.31.15, 别名为black和webservice2
运行和测试
创建
docker-compose up
测试
# 通过send-requests.sh脚本进行测试，可发现，用户请求被按权重分配至不同的locality之上，每个locality内部再按负载均衡算法进行调度；
./send-requests.sh 172.31.31.2
# 可以试着将权重较高的一组中的某一主机的健康状态团置为不可用；
停止后清理
docker-compose down2.docker-compose.yaml
[root@k8s-master01 locality-weighted]# cat docker-compose.yaml
version: '3'
services:
front-envoy:
image: envoyproxy/envoy-alpine:v1.21-latest
environment:
- ENVOY_UID=0
- ENVOY_GID=0
volumes:
- ./front-envoy.yaml:/etc/envoy/envoy.yaml
networks:
- envoymesh
expose:
# Expose ports 80 (for general traffic) and 9901 (for the admin server)
- "80"
- "9901"
webserver01-sidecar:
image: envoyproxy/envoy-alpine:v1.21-latest
environment:
- ENVOY_UID=0
- ENVOY_GID=0
volumes:
- ./envoy-sidecar-proxy.yaml:/etc/envoy/envoy.yaml
hostname: red
networks:
envoymesh:
ipv4_address: 172.31.31.11
aliases:
- webservice1
- red
webserver01:
image: ikubernetes/demoapp:v1.0
environment:
- PORT=8080
- HOST=127.0.0.1
network_mode: "service:webserver01-sidecar"
depends_on:
- webserver01-sidecar
webserver02-sidecar:
image: envoyproxy/envoy-alpine:v1.21-latest
environment:
- ENVOY_UID=0
- ENVOY_GID=0
volumes:
- ./envoy-sidecar-proxy.yaml:/etc/envoy/envoy.yaml
hostname: blue
networks:
envoymesh:
ipv4_address: 172.31.31.12
aliases:
- webservice1
- blue
webserver02:
image: ikubernetes/demoapp:v1.0
environment:
- PORT=8080
- HOST=127.0.0.1
network_mode: "service:webserver02-sidecar"
depends_on:
- webserver02-sidecar
webserver03-sidecar:
image: envoyproxy/envoy-alpine:v1.21-latest
environment:
- ENVOY_UID=0
- ENVOY_GID=0
volumes:
- ./envoy-sidecar-proxy.yaml:/etc/envoy/envoy.yaml
hostname: green
networks:
envoymesh:
ipv4_address: 172.31.31.13
aliases:
- webservice1
- green
webserver03:
image: ikubernetes/demoapp:v1.0
environment:
- PORT=8080
- HOST=127.0.0.1
network_mode: "service:webserver03-sidecar"
depends_on:
- webserver03-sidecar
webserver04-sidecar:
image: envoyproxy/envoy-alpine:v1.21-latest
environment:
- ENVOY_UID=0
- ENVOY_GID=0
volumes:
- ./envoy-sidecar-proxy.yaml:/etc/envoy/envoy.yaml
hostname: gray
networks:
envoymesh:
ipv4_address: 172.31.31.14
aliases:
- webservice2
- gray
webserver04:
image: ikubernetes/demoapp:v1.0
environment:
- PORT=8080
- HOST=127.0.0.1
network_mode: "service:webserver04-sidecar"
depends_on:
- webserver04-sidecar
webserver05-sidecar:
image: envoyproxy/envoy-alpine:v1.21-latest
environment:
- ENVOY_UID=0
- ENVOY_GID=0
volumes:
- ./envoy-sidecar-proxy.yaml:/etc/envoy/envoy.yaml
hostname: black
networks:
envoymesh:
ipv4_address: 172.31.31.15
aliases:
- webservice2
- black
webserver05:
image: ikubernetes/demoapp:v1.0
environment:
- PORT=8080
- HOST=127.0.0.1
network_mode: "service:webserver05-sidecar"
depends_on:
- webserver05-sidecar
networks:
envoymesh:
driver: bridge
ipam:
config:
- subnet: 172.31.31.0/243.envoy-sidercar-proxy.yaml
admin:
profile_path: /tmp/envoy.prof
access_log_path: /tmp/admin_access.log
address:
socket_address:
address: 0.0.0.0
port_value: 9901
static_resources:
listeners:
- name: listener_0
address:
socket_address: { address: 0.0.0.0, port_value: 80 }
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
codec_type: AUTO
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains: ["*"]
routes:
- match: { prefix: "/" }
route: { cluster: local_cluster }
http_filters:
- name: envoy.filters.http.router
clusters:
- name: local_cluster
connect_timeout: 0.25s
type: STATIC
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: local_cluster
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address: { address: 127.0.0.1, port_value: 8080 }4.front-proxy.yaml
#此时连个区域的优先级都为0 进行位置权重
priority: 0
load_balancing_weight: 10
load_balancing_weight: 20admin:
access_log_path: "/dev/null"
address:
socket_address: { address: 0.0.0.0, port_value: 9901 }
static_resources:
listeners:
- address:
socket_address: { address: 0.0.0.0, port_value: 80 }
name: listener_http
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
codec_type: auto
stat_prefix: ingress_http
route_config:
name: local_route
virtual_hosts:
- name: backend
domains:
- "*"
routes:
- match:
prefix: "/"
route:
cluster: webcluster1
http_filters:
- name: envoy.filters.http.router
clusters:
- name: webcluster1
connect_timeout: 0.25s
type: STRICT_DNS
lb_policy: ROUND_ROBIN
http2_protocol_options: {}
load_assignment:
cluster_name: webcluster1
policy:
overprovisioning_factor: 140
endpoints:
- locality:
region: cn-north-1
priority: 0
load_balancing_weight: 10
lb_endpoints:
- endpoint:
address:
socket_address: { address: webservice1, port_value: 80 }
- locality:
region: cn-north-2
priority: 0
load_balancing_weight: 20
lb_endpoints:
- endpoint:
address:
socket_address: { address: webservice2, port_value: 80 }
health_checks:
- timeout: 5s
interval: 10s
unhealthy_threshold: 2
healthy_threshold: 1
http_health_check:
path: /livez
expected_statuses:
start: 200
end: 399测试脚本
[root@xksmaster1 locality-weighted]# cat send-request.sh
#!/bin/bash
declare -i colored=0
declare -i colorless=0
interval="0.1"
while true; do
if curl -s http://$1/hostname | grep -E "red|blue|green" &> /dev/null; then
# $1 is the host address of the front-envoy.
colored=$[$colored+1]
else
colorless=$[$colorless+1]
fi
echo $colored:$colorless
sleep $interval
done
[root@xksmaster1 locality-weighted]# ./send-request.sh 172.31.31.2
1:0
2:0
3:0
4:0
5:0
5:1
6:1
6:2
6:3
6:4
7:4
8:4
9:4
10:4
10:5
11:5
11:6
12:6
13:6
14:6
15:6
15:7
15:8
15:9
15:10
16:10
17:10
18:10
19:10
20:10
20:11
21:11
21:12
22:12
23:12
23:13
24:13
24:14
浙公网安备 33010602011771号