LVS集群之IP TUN模式以及网站压力测试
今天来看一下LVS的第三种模式IP TUN。
TUN方式，是通过给数据包加上新的IP头部来实现，这个可以跨整个广域网。
环境：
| 主机名 | IP | 系统 | 角色 |
| tiandong63 |
RIP:192.168.199.11、DG:192.168.199.1 VIP:192.168.199.63 |
rhel6.5 | real server1 |
| tiandong64 |
RIP:192.168.199.12、DG:192.168.199.1 VIP:192.168.199.63 |
rhel6.5 | real server2 |
| OpenStack |
DIP:192.168.199.7、DG:192.168.199.1 VIP:192.168.199.63 |
rhel7.4 | 分发器 |
| localhost | 192.168.199.8 | rhel7.4 | 测试机 |
分发器：
1、配置网络：
ens33 192.168.199.7 DIP
ens33:1 192.168.199.63 VIP
[root@openstack ~]#ifconfig ens33:1 192.168.199.63 netmask 255.255.255.0 up
[root@openstack ~]# echo 1 > /proc/sys/net/ipv4/ip_forward 开启路由转发功能
2、配置IP TUN模式
root@openstack ~]#yum install ipvsadm -y
[root@openstack ~]#ipvsadm -A -t 192.168.199.63:80 -s rr
[root@openstack ~]#ipvsadm -a -t 192.168.199.63:80 -r 192.168.199.11:80 -i #-i隧道模式
[root@openstack ~]#ipvsadm -a -t 192.168.199.63:80 -r 192.168.199.12:80 -i
[root@openstack ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.199.63:80 rr
-> 192.168.199.11:80 Tunnel 1 0 0
-> 192.168.199.12:80 Tunnel 1 0 0
Realserver1：tiandong63
1、配置网络：
eth0 RIP：192.168.199.11
[root@tiandong63 ~]# modprobe ipip #在加载好ipip模块后就会有默认的tunl0隧道。
[root@tiandong63 ~]# lsmod |grep ipip
ipip 8371 0
tunnel4 2943 1 ipip
[root@tiandong63 ~]#ifconfig tunl0 192.168.199.63 netmask 255.255.255.255 up
[root@tiandong63 ~]# service network restart
[root@tiandong63 ~]# ifconfig -a 此时就可以看到tunl0的网卡了。
eth0 Link encap:Ethernet HWaddr 00:0C:29:38:0B:14
inet addr:192.168.199.11 Bcast:192.168.199.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe38:b14/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:113954 errors:0 dropped:0 overruns:0 frame:0
TX packets:40448 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:164480499 (156.8 MiB) TX bytes:2794350 (2.6 MiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
tunl0 Link encap:IPIP Tunnel HWaddr
inet addr:192.168.199.63 Mask:255.255.255.255
UP RUNNING NOARP MTU:1480 Metric:1
RX packets:20 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1664 (1.6 KiB) TX bytes:0 (0.0 b)
2、关闭ARP转发：
[root@tiandong63 ~]# vim /etc/sysctl.conf 在最后加上：
net.ipv4.conf.tunl0.arp_ignore = 1
net.ipv4.conf.tunl0.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.tunl0.rp_filter = 0
net.ipv4.conf.all.rp_filter = 0
[root@tiandong63 ~]# sysctl -p 是配置生效
3、配置web服务器
[root@tiandong63 ~]#yum install httpd -y
[root@tiandong63 ~]# echo 'this is 192.168.199.11' > /var/www/html/index.html 写一个测试页面
[root@tiandong63 ~]# /etc/init.d/httpd start
Realserver2：tiandong64
1、配置网络
eth0 RIP：192.168.199.12
[root@tiandong64 ~]# modprobe ipip
[root@tiandong64 ~]# lsmod |grep ipip
ipip 8371 0
tunnel4 2943 1 ipip
[root@tiandong64 ~]#ifconfig tunl0 192.168.199.63 netmask 255.255.255.255 up
[root@tiandong64 ~]# service network restart
[root@tiandong64 ~]# ifconfig -a
eth0 Link encap:Ethernet HWaddr 00:0C:29:C7:20:71
inet addr:192.168.199.12 Bcast:192.168.199.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fec7:2071/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:41108 errors:0 dropped:0 overruns:0 frame:0
TX packets:13848 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:57855802 (55.1 MiB) TX bytes:884097 (863.3 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
tunl0 Link encap:IPIP Tunnel HWaddr
inet addr:192.168.199.63 Mask:255.255.255.255
UP RUNNING NOARP MTU:1480 Metric:1
RX packets:25 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2080 (2.0 KiB) TX bytes:0 (0.0 b)
2、关闭ARP转发：
[root@tiandong64 ~]# vim /etc/sysctl.conf 在最后加上：
net.ipv4.conf.tunl0.arp_ignore = 1
net.ipv4.conf.tunl0.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.tunl0.rp_filter = 0
net.ipv4.conf.all.rp_filter = 0
[root@tiandong64 ~]# sysctl -p 是配置生效
3、配置web服务器
[root@tiandong64 ~]#yum install httpd -y
[root@tiandong64 ~]# echo 'this is 192.168.199.12' > /var/www/html/index.html 写一个测试页面
[root@tiandong64 ~]# /etc/init.d/httpd start
然后进行测试：
[root@localhost ~]#yum install elinks -y
[root@localhost ~]# elinks 192.168.199.63 --dump
this is 192.168.199.11
[root@localhost ~]# elinks 192.168.199.63 --dump
this is 192.168.199.12
[root@localhost ~]# elinks 192.168.199.63 --dump
this is 192.168.199.11
[root@localhost ~]# elinks 192.168.199.63 --dump
this is 192.168.199.12
在分发器上查看： 是平均分配的。
[root@openstack ~]# ipvsadm -ln --stats
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Conns InPkts OutPkts InBytes OutBytes
-> RemoteAddress:Port
TCP 192.168.199.63:80 4 20 0 1664 0
-> 192.168.199.11:80 2 10 0 832 0
-> 192.168.199.12:80 2 10 0 832 0
使用webbench测试网站性能
下载并安装webbench：
[root@tiandong63 ~]# wget http://www.ha97.com/code/webbench-1.5.tar.gz
[root@tiandong63 ~]# tar zxvf webbench-1.5.tar.gz
[root@tiandong63 ~]# cd webbench-1.5
[root@tiandong63 webbench-1.5]# yum install automake autoconf gcc -y
[root@tiandong63 webbench-1.5]# mkdir -p /usr/local/man/man1
[root@tiandong63 webbench-1.5]# make install
install -s webbench /usr/local/bin
install -m 644 webbench.1 /usr/local/man/man1
install -d /usr/local/share/doc/webbench
install -m 644 debian/copyright /usr/local/share/doc/webbench
install -m 644 debian/changelog /usr/local/share/doc/webbench
测试：
一个客户端，持续时间1秒：
-c为客户端数，-t为时间(秒)
[root@tiandong63 webbench-1.5]# webbench -c 1 -t 1 http://192.168.199.11/index.html
Webbench - Simple Web Benchmark 1.5
Copyright (c) Radim Kolar 1997-2004, GPL Open Source Software.
Benchmarking: GET http://192.168.199.11/index.html
1 client, running 1 sec.
Speed=233339 pages/min, 1127520 bytes/sec.
#当只有一个客户端时，一分钟可以响应233329个页面，1秒可以传输1127520字节
Requests: 3889 susceed, 0 failed.
#1个客户端，1秒产生了3889个请求，0个失败。
使用20个客户端并发访问并持续访问10秒
[root@tiandong63 webbench-1.5]# webbench -c 20 -t 10 http://192.168.199.11/index.html
Webbench - Simple Web Benchmark 1.5
Copyright (c) Radim Kolar 1997-2004, GPL Open Source Software.
Benchmarking: GET http://192.168.199.11/index.html
20 clients, running 10 sec.
Speed=329556 pages/min, 1593115 bytes/sec.
Requests: 54926 susceed, 0 failed.
可以同时查看被测试机器的性能：
[root@tiandong63 ~]# vmstat 1 20
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu-----
r b swpd free buff cache si so bi bo in cs us sy id wa st
21 0 0 284568 61060 512612 0 0 23 62 42 46 1 1 97 1 0
4 0 0 282552 61060 513016 0 0 0 0 1005 19827 40 60 0 0 0
4 0 0 279824 61060 513612 0 0 0 0 1003 30257 44 56 0 0 0
4 0 0 277716 61060 514080 0 0 0 0 1004 23301 38 62 0 0 0
4 0 0 276476 61060 514520 0 0 0 0 1002 20532 41 59 0 0 0
19 0 0 274244 61068 514928 0 0 0 20 1003 18902 42 58 0 0 0
4 0 0 272632 61068 515532 0 0 0 0 1003 28352 40 60 0 0 0
5 0 0 272144 61068 516160 0 0 0 0 1004 31111 46 54 0 0 0
4 0 0 271896 61068 516720 0 0 0 0 1002 30103 44 56 0 0 0
4 0 0 273392 61068 517368 0 0 0 0 1003 31856 42 58 0 0 0
