keepalived+LVS-DR集群

keepalived+LVS-DR集群

1 Keepalived概述

2 Keepalived工作原理

3 Keepalived的作用与构建

4 配置keepalived实现互为主从

1、Keepalived概述

keepalived 是一个类似于 layer3, 4 & 5 交换机制的软件，也就是我们平时说的第 3 层、第 4 层和第 5层交换。 Keepalived 的作用是检测 web 服务器的状态，如果有一台 web 服务器死机，或工作出现故障，Keepalived 将检测到，并将有故障的 web 服务器从系统中剔除，当 web 服务器工作正常后 Keepalived 自动将web 服务器加入到服务器群中，这些工作全部自动完成，不需要人工干涉，需要人工做的只是修复故障的web 服务器。

2、 Keepalived工作原理

Layer3,4&5 工作在 IP/TCP 协议栈的 IP 层， TCP 层，及应用层,。

Layer3： Keepalived 使用 Layer3 的方式工作式时， Keepalived 会定期向服务器群中的服务器发送一个 ICMP 的数据包（既我们平时用的 Ping 程序） , 如果发现某台服务的 IP 地址没有激活，Keepalived 便报告这台服务器失效，并将它从服务器群中剔除，这种情况的典型例子是某台服务器被非法关机。 Layer3 的方式是以服务器的 IP 地址是否有效作为服务器工作正常与否的标准。

Layer4: 主要以 TCP 端口的状态来决定服务器工作正常与否。如 web server 的服务端口一般是80，如果 Keepalived 检测到 80 端口没有启动，则 Keepalived 将把这台服务器从服务器群中删除。

Layer5： Layer5 就是工作在具体的应用层了，比 Layer3,Layer4 要复杂一点，在网络上占用的带宽也要大一些。 Keepalived 将根据用户的设定检查服务器程序的运行是否正常，如果与用户的设定不相符，则 Keepalived 将把服务器从服务器群中剔除。

3 Keepalived的作用与构建

1.管理 VIP，VIP 会在 LVS 之间漂移

2.监控 LVS分发器

运行在主分发的 Keepalived 会以组播的形式向网络中宣告自己，即主分发器还活着，备用节点能收到。当备用节点，在一个时间单位中收不到组播，备用节点会认为主 LVS 挂了，开始接手主分发器工作，把 VIP 配给自己。

VRRP/HSRP

虚拟路由冗余协议(Virtual Router Redundancy Protocol，简称VRRP)是由IETF提出的解决局域网中配置静态网关出现单点失效现象的路由协议。使用组播方式通信。

VRRP是一种路由容错协议，也可以叫做备份路由协议。一个局域网络内的所有主机都设置缺省路由（默认网关），当网内主机发出的目的地址不在本网段时，报文将被通过缺省路由发往外部路由器，从而实现了主机与外部网络的通信。当缺省路由器down掉（即端口关闭）之后，内部主机将无法与外部通信，如果路由器设置了VRRP时，那么这时，虚拟路由将启用备份路由器，从而实现全网通信。

4 配置keepalived实现互为主从

4.1这个原理图

4.2 使用Keepalived构建LVS-DR模式的高可用集群，实验环境如下：

机器名称	机器名称	网关	机器作用
dirctor1	DIP:10.27.17.90/24 VIP:10.27.17.91/24	10.27.17.1	主 LVS
dirctor1	DIP:10.27.17.34/24 VIP:10.27.17.91/24	10.27.17.1	备 LVS
realserver1	10.27.17.92/24	10.27.17.1	RS1 WEB1
realserver2	10.27.17.93/24	10.27.17.1	RS2 WEB2

固化以上按照以上规化进行服务器IP固化。

４.３上传 keeplive,进行安装

[root@director1 ~]# tar -zxvf keepalived-1.2.16.tar.gz

[root@director1 ~]# cd keepalived-1.2.16
[root@director1 keepalived-1.2.16]# yum -y install gcc openssl-devel libnfnetlink-devel

[root@director1 keepalived-1.2.16]# ./configure --prefix=/usr/local/keepalived

Keepalived configuration
------------------------
Keepalived version : 1.2.16
Compiler : gcc
Compiler flags : -g -O2
Extra Lib : -lssl -lcrypto -lcrypt
Use IPVS Framework : Yes
IPVS sync daemon support : Yes
IPVS use libnl : No
fwmark socket support : Yes
Use VRRP Framework : Yes
Use VRRP VMAC : Yes
SNMP support : No
SHA1 support : No
Use Debug flags : No
[root@director1 keepalived-1.2.16]# make && make install

[root@director1 keepalived]# ll
total 0
drwxr-xr-x 2 root root 21 Sep 30 03:22 bin
drwxr-xr-x 5 root root 53 Sep 30 03:22 etc
drwxr-xr-x 2 root root 24 Sep 30 03:22 sbin
drwxr-xr-x 3 root root 17 Sep 30 03:22 share

4.4建立启动脚本

[root@director1 keepalived]# cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/

[root@director1 keepalived]# grep "chkconfig" /usr/local/keepalived/etc/* -R
/usr/local/keepalived/etc/rc.d/init.d/keepalived:# chkconfig: - 21 79
[root@director1 keepalived]# chmod +x /etc/init.d/keepalived

[root@director1 keepalived]# vim /etc/init.d/keepalived
bin/ etc/ sbin/ share/

[root@director1 keepalived]# ln -s /usr/local/keepalived/sbin/keepalived /usr/sbin/
[root@director1 keepalived]# mkdir /etc/keepalived
[root@director1 keepalived]# cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
[root@director1 keepalived]# cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
[root@director1 keepalived]# vim /etc/sysconfig/keepalived

修改为：

KEEPALIVED_OPTIONS="-D -f /etc/keepalived/keepalived.conf"

4.5安装lvs

[root@director1 yum.repos.d]# yum install ipvsadm -y

dirctor2 同上，安装keeplived 和DR

4.6 配置Keepalived+LVS-DR模式

4.6.1 DIRctor1 主节点配置

[root@director1 keepalived]# vim /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {
root@localhost
}
notification_email_from root@localhost
smtp_server localhost
smtp_connect_timeout 30
router_id director1
}
vrrp_instance apache {
state MASTER
interface ens33
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.27.17.91
}
}

virtual_server 10.27.17.91 80 {
delay_loop 6
lb_algo rr
lb_kind DR
nat_mask 255.255.255.0
persistence_timeout 50
protocol TCP

real_server 10.27.17.92 80 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 10.27.17.93 80 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}

[root@director1 keepalived]# systemctl restart keepalived

[root@director1 keepalived]# systemctl enable keepalived

[root@director1 keepalived]# ipvsadm -L -n
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.27.17.91:80 rr persistent 50
-> 10.27.17.92:80 Route 1 0 0
-> 10.27.17.93:80 Route 1 0 0

4.6.2备用节点director2配置

[root@director2 keepalived]# vim /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {
root@localhost
}
notification_email_from root@localhost
smtp_server localhost
smtp_connect_timeout 30
router_id director2
}
vrrp_instance apache {
state BACKUP
interface ens33
virtual_router_id 51
priority 90
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.27.17.91
}
}

virtual_server 10.27.17.91 80 {
delay_loop 6
lb_algo rr
lb_kind DR
nat_mask 255.255.255.0
persistence_timeout 50
protocol TCP

[root@director2 keepalived]# systemctl restart keepalived
[root@director2 keepalived]# systemctl enable keepalived

测试一下

[root@director1 keepalived]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:8d:94:56 brd ff:ff:ff:ff:ff:ff
inet 10.27.17.90/24 brd 10.27.17.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet 10.27.17.91/32 scope global ens33
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe8d:9456/64 scope link noprefixroute
valid_lft forever preferred_lft forever
3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:8d:94:60 brd ff:ff:ff:ff:ff:ff
inet 10.27.17.220/24 brd 10.27.17.255 scope global noprefixroute ens37
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe8d:9460/64 scope link noprefixroute
valid_lft forever preferred_lft forever

[root@director2 keepalived]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:cf:00:0a brd ff:ff:ff:ff:ff:ff
inet 10.27.17.34/24 brd 10.27.17.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fecf:a/64 scope link noprefixroute
valid_lft forever preferred_lft forever

4.7配置RS1

4.2.1建立一个启动脚本启用lvs

[root@realserver1 ~]# vim /etc/init.d/lvsrsdr

#!/bin/bash

VIP=10.27.17.91

source /etc/init.d/functions

case $1 in

start)

echo 'start LVS of Realserver DR'

/sbin/ifconfig lo:1 $VIP broadcast $VIP netmask 255.255.255.255 up

/sbin/route add -host $VIP dev lo:1

echo '1' > /proc/sys/net/ipv4/conf/lo/arp_ignore

echo '2' > /proc/sys/net/ipv4/conf/lo/arp_announce

echo '1' > /proc/sys/net/ipv4/conf/all/arp_ignore

echo '2' > /proc/sys/net/ipv4/conf/all/arp_announce

;;

stop)

/sbin/ifconfig lo:1 down

echo 'Close LVS of Realserver DR'

echo '0' > /proc/sys/net/ipv4/conf/lo/arp_ignore

echo '0' > /proc/sys/net/ipv4/conf/lo/arp_announce

echo '0' > /proc/sys/net/ipv4/conf/all/arp_ignore

echo '0' > /proc/sys/net/ipv4/conf/all/arp_announce

;;

echo "Usage:$0 (start|stop)"

exit 1

esac

[root@realserver1 ~]# chmod +x /etc/init.d/lvsrsdr

[root@realserver1 ~]# /etc/init.d/lvsrsdr start

Reloading systemd: [ OK ]

Starting lvsrsdr (via systemctl): [ OK ]

[root@realserver1 ~]# echo "/etc/init.d/lvsrsdr start" >> /etc/rc.local

安装并启动httpd

[root@realserver1 ~]# yum -y install httpd

[root@realserver1 ~]# echo 10.27.17.92 > /var/www/html/index.html

[root@realserver1 ~]# systemctl restart httpd

realsever2安装同上。

4.8 测试

关闭DIRECTOR1

恢复DIRCTOR1