lvs+keepalived部署
lvs介绍
lvs做负载均衡是工作的4层，而nginx做反向代理是工作再7层。由于lvs工作在4层，其处理的报文要简单很多，处理的速度很快，所以用lvs要比nginx的效率要高很多。
如果是独立域名的话直接用lvs4层负载均衡就可，如果非独立域名的话，lvs就实现不了了，就只能用nginx做反向代理了。这时候可以在lvs后再挂nginx反向代理来实现。
lvs做负载均衡可以用ipvsadm工具来实现，也可以keepalived来实现（推荐用keepalived部署lvs）
lvs工作模式有NAT模式、TUNNLE模式、FULLNAT模式、DR模式等，这几个模式相比较DR模式效率最高，同时也是工作中常用的LVS模式
lvs体系结构：
lvs的DR模式为：
用ipvsadm部署lvs
1、安装lvs: yum install ipvsadm -y
2、添加vip：ip addr add 192.168.182.200/24 dev eth0 label eth0:1
3、通过lsmod |grep ip_vs来查看lvs是否生效
4、查看ipvsadm支持的参数：
[root@mysql-server ~]# ipvsadm --help ipvsadm v1.26 2008/5/15 (compiled with popt and IPVS v1.2.1) Usage: ipvsadm -A|E -t|u|f service-address [-s scheduler] [-p [timeout]] [-M netmask] [--pe persistence_engine] ipvsadm -D -t|u|f service-address ipvsadm -C ipvsadm -R ipvsadm -S [-n] ipvsadm -a|e -t|u|f service-address -r server-address [options] ipvsadm -d -t|u|f service-address -r server-address ipvsadm -L|l [options] ipvsadm -Z [-t|u|f service-address] ipvsadm --set tcp tcpfin udp ipvsadm --start-daemon state [--mcast-interface interface] [--syncid sid] ipvsadm --stop-daemon state ipvsadm -h Commands: Either long or short options are allowed. --add-service -A add virtual service with options --edit-service -E edit virtual service with options --delete-service -D delete virtual service --clear -C clear the whole table --restore -R restore rules from stdin --save -S save rules to stdout --add-server -a add real server with options --edit-server -e edit real server with options --delete-server -d delete real server --list -L|-l list the table --zero -Z zero counters in a service or all services --set tcp tcpfin udp set connection timeout values --start-daemon start connection sync daemon --stop-daemon stop connection sync daemon --help -h display this help message Options: --tcp-service -t service-address service-address is host[:port] --udp-service -u service-address service-address is host[:port] --fwmark-service -f fwmark fwmark is an integer greater than zero --ipv6 -6 fwmark entry uses IPv6 --scheduler -s scheduler one of rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq, the default scheduler is wlc. --pe engine alternate persistence engine may be sip, not set by default. --persistent -p [timeout] persistent service --netmask -M netmask persistent granularity mask --real-server -r server-address server-address is host (and port) --gatewaying -g gatewaying (direct routing) (default) --ipip -i ipip encapsulation (tunneling) --masquerading -m masquerading (NAT) --weight -w weight capacity of real server --u-threshold -x uthreshold upper threshold of connections --l-threshold -y lthreshold lower threshold of connections --mcast-interface interface multicast interface for connection sync --syncid sid syncid for connection sync (default=255) --connection -c output of current IPVS connections --timeout output of timeout (tcp tcpfin udp) --daemon output of daemon information --stats output of statistics information --rate output of rate information --exact expand numbers (display exact values) --thresholds output of thresholds information --persistent-conn output of persistent connection info --nosort disable sorting output of service/server entries --sort does nothing, for backwards compatibility --ops -o one-packet scheduling --numeric -n numeric output of addresses and ports
5、创建vip服务：ipvsadm -A -t 192.168.182.200:80 -s rr
6、添加vip服务下的节点服务 ：ipvsadm -a -t 192.168.182.200:80 -r 192.168.182.142:80 -g
7、查看lvs状态：
[root@mysql-server ~]# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.182.200:80 rr -> 192.168.182.142:80 Route 1 0 0 -> 192.168.182.143:80 Route 1 0 0
当然这里也可使用ipvsadm -Ln --stats来查看lvs的详细信息
[root@mysql-server ~]# ipvsadm -Ln --stats IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Conns InPkts OutPkts InBytes OutBytes -> RemoteAddress:Port TCP 192.168.182.200:80 0 0 0 0 0 -> 192.168.182.142:80 0 0 0 0 0 -> 192.168.182.143:80 0 0 0 0 0
8、修改hosts文件绑定vip和域名的关系 192.168.182.200 www.goser.com
9、这时候客户端访问DR服务器的vip地址的时候是无法访问节点的web服务的，因为这时候DR将客户端放到web节点的时候，web节点查看vip地址不是自己的，就会让此报文抛掉不管。
所以这里应该在web节点服务的lo回环网卡上添加一个vip地址，并且要保证arp抑制(如果没有做arp抑制的话，lvs就起不到作用了)
添加lo网卡的vip地址：ip addr add 192.168.182.200/32 dev lo label lo:200
抑制arp
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
这样ipvsadm部署lvs就完成了,客户端再访问web服务的话就没问题了，lvs就会根据配置的web节点服务做来回的轮训，完成负载均衡的效果
keepalived来管理部署lvs（工作推荐此方法）
1、修改keepalived的配置文件，添加管理lvs的内容
[root@mysql-server ~]# cat /etc/keepalived/keepalived.conf
global_defs {
notification_email {
396540914@qq.com
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id LVS_01
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
priority 150
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.182.200/24
}
}
virtual_server 192.168.182.200 80 {
delay_loop 6
lb_algo wrr
lb_kind DR
nat_mask 255.255.255.0
persistence_timeout 50
protocol TCP
real_server 192.168.182.142 80 {
weight 1
TCP_CHECK {
connect_timeout 8
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 192.168.182.143 80 {
weight 1
TCP_CHECK {
connect_timeout 8
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
重启keepalived即可，备用的lvs也做同样的配置，还有web服务节点一样也要做lo网卡的vip地址的添加及抑制ARP操作。
这样lvs+keepalived负载均衡+高可用部署完成。。。。
当然lvs客户端也可通过开发shell脚背来设置vip以及抑制ARP的管理脚本：
/etc/init.d/lvsclient {start|stop}
vim lvsclient.sh
#!/bin/bash
# Written by goser
# description: Config realserver lo and apply noarp
VIP=(
192.168.182.200
)
. /etc/init.d/functions
case "$1" in
start)
echo "start LVS of REALServer IP"
for ((i=0; i<`echo ${#VIP[*]}`; i++))
do
interface="lo:`echo ${VIP[$i]}|awk -F . '{print $4}'`"
ip addr add ${VIP[$i]}/32 dev lo label $interface
route add -host ${VIP[$i]} dev $interface
done
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
;;
stop)
for ((i=0; i<`echo ${#VIP[*]}`; i++))
do
interface="lo:`echo ${VIP[$i]}|awk -F . '{print $4}'`"
ip addr del ${VIP[$i]}/32 dev lo label $interface
route del -host ${VIP[$i]} dev $interface
done
echo "STOP LVS of REALServer IP"
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
esac
拷贝此脚本到/etc/init.d/下 并对此目录下新拷贝来的lvsclient添加可执行权限，这样就可以直接使用/etc/init.d/lvsclient start或stop来启动或关闭lvs客户端
浙公网安备 33010602011771号