Apache+Tomcat构建Tomcat负载均衡集群
一、环境介绍
二、安装后端服务器
三、安装前端Apache服务
四、配置Apache使用mod_jk模块实现代理及负载均衡
五、配置Apache基于mod_proxy模块实现代理及负载均衡
六、论坛安装
七、安装并配置Keepalived实现前端高可用
一、环境介绍

系统版本:CentOS 6.4_x86_64
Mysql版本:mysql-5.1.66-2.el6_3.x86_64
Apache版本:httpd-2.2.25 点此下载
Keepalived版本:keepalived-1.2.7 点此下载
Tomcat版本:apache-tomcat-7.0.33 点此下载
Jdk版本: jdk-7u40-linux-x64 点此下载
Tomcat-commectors版本:tomcat-connectors-1.2.37 点此下载
论坛版本:JspRun!_6.0.0_GBK 点此下载
环境介绍:
当用户通过 "www.allen.com" 域名访问时,首先是由前端两台Apache代理服务器响应并转发到后端Tomcat服务器上,而在Apache上做的是反向代理负载均衡到后端Tomcat服务器上,前端服务器使用Keepalived做的高可用集群,【Apache1默认为Master,Apache2为Backup】虚拟IP地址为"172.16.14.10"模拟为公网IP;当然这里数据库只有一台主机会成为点故障,这里不要介绍Mysql的高可用,如果有兴趣可以看前面写的博客
提示:这里使用Keepalived做高可用时,前面所有的服务都已完成后,最后实现的对前端服务器做高可用
二、安装后端Tomcat与Mysql服务器
1、在Tomcat1与Tomcat2服务器上安装Tomcat,安装方法相同,这里只介绍一次
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
|
安装JDK# rpm -ivh jdk-7u40-linux-x64.rpm# vim /etc/profile.d/java.shexport JAVA_HOME=/usr/java/latestexport PATH=$JAVA_HOME/bin:$PATH# . /etc/profile.d/java.sh# java -versionjava version "1.7.0_40"Java(TM) SE Runtime Environment (build 1.7.0_40-b43)Java HotSpot(TM) 64-Bit Server VM (build 24.0-b56, mixed mode)------------------------------------------------------------------------安装Tomcat# tar xf apache-tomcat-7.0.33.tar.gz -C /usr/local/# cd /usr/local/# ln -s apache-tomcat-7.0.33 tomcat# vim /etc/profile.d/tomcat.shexport CATALINA_HOME=/usr/local/tomcatexport PATH=$CATALINA_HOME/bin:$PATH# . /etc/profile.d/tomcat.sh# catalina.sh versionUsing CATALINA_BASE: /usr/local/tomcatUsing CATALINA_HOME: /usr/local/tomcatUsing CATALINA_TMPDIR: /usr/local/tomcat/tempUsing JRE_HOME: /usr/java/latestUsing CLASSPATH: /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jarServer version: Apache Tomcat/7.0.33Server built: Nov 18 2012 04:15:21Server number: 7.0.33.0OS Name: LinuxOS Version: 2.6.32-358.el6.x86_64Architecture: amd64JVM Version: 1.7.0_40-b43JVM Vendor: Oracle Corporation |
2、启动Tomcat服务并访问测试,默认访问端口为"8080";如下
|
1
2
3
4
5
6
7
8
9
10
11
|
# catalina.sh startUsing CATALINA_BASE: /usr/local/tomcatUsing CATALINA_HOME: /usr/local/tomcatUsing CATALINA_TMPDIR: /usr/local/tomcat/tempUsing JRE_HOME: /usr/java/latestUsing CLASSPATH: /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar# jps2283 Jps2272 Bootstrap# ss -tanlp | grep 8080LISTEN 0 100 :::8080 :::* users:(("java",2272,40)) |
3、开启Tomcat管理及状态页面
|
1
2
3
4
5
6
|
# vim /usr/local/tomcat/conf/tomcat-users.xml <role rolename="manager-gui"/> <role rolename="admin-gui"/> <user username="allen" password="admin" roles="manager-gui,admin-gui"/># catalina.sh stop# catalina.sh start |
4、访问Tomcat后台管理
5、安装Mysql服务器
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
|
######安装Mysql并创建数据库,授权用户访问新创建的数据库有所有权,方便后面使用[root@Mysql ~]# yum -y install mysql-server[root@Mysql ~]# service mysqld start[root@Mysql ~]# mysqlmysql> create database bbs;mysql> grant all on bbs.* to 'bbsuser'@'172.16.%.%' identified by 'bbspass';mysql> flush privileges;----------------------------------------------------------------------远程连接数据库测试授权用户是否能连接成功# mysql -ubbsuser -pbbspass -h 172.16.14.5 -e 'show databases';+--------------------+| Database |+--------------------+| information_schema || bbs || test |+--------------------+ |
三、安装前端Apache服务
1、在Apache1与Apache2服务器上安装Httpd软件,安装方法相同,这里只介绍一次
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
|
# tar xf httpd-2.2.25.tar.bz2# cd httpd-2.2.25# yum -y install gcc gcc-c++ openssl-devel pcre-devel# ./configure --prefix=/usr/local/apache --sysconfdir=/etc/httpd --enable-modules=most --enable-so --enable-ssl --enable-cgi --enable-rewrite --with-zlib --with-pcre --enable-mpms-shared=all --with-mpm=prefork --enable-proxy --enable-proxy-http --enable-proxy-ajp --enable-proxy-balancer --enable-cache --enable-cache-disk --enable-mem-cache --enable-file-cache --enable-proxy-connect# make && make install# cp build/rpm/httpd.init /etc/init.d/httpd------修改如下内容# vim /etc/init.d/httpdhttpd=${HTTPD-/usr/local/apache/bin/httpd}pidfile=${PIDFILE-/usr/local/apache/logs/${prog}.pid}lockfile=${LOCKFILE-/var/lock/subsys/${prog}}RETVAL=0# check for 1.3 configurationcheck13 () { CONFFILE=/etc/httpd/httpd.conf----------------------------------------------------------------------修改环境变量并把Httpd加入到系统服务# echo "PATH=/usr/local/apache/bin:$PATH" >> /etc/profile.d/http.sh# . /etc/profile.d/http.sh# ln -s /usr/local/apache/include/ /usr/include/httpd# chkconfig --add httpd----------------------------------------------------------------------创建测试页# echo "<h1>APACHE</h1>" > /usr/local/apache/htdocs/index.html |
2、在Apache1与Apache2服务器上安装"tomcat-connectors"
|
1
2
3
4
5
6
7
8
9
10
11
12
13
|
# tar xf tomcat-connectors-1.2.37-src.tar.gz# cd tomcat-connectors-1.2.37-src/native/# ./configure --with-apxs=/usr/local/apache/bin/apxs# make && make install------------------------------------------------------------------------安装完成后会生成如下文件# ls /usr/local/apache/modules/ mod_jk.so------------------------------------------------------------------------启动Httpd服务#service httpd start# ss -tanl |grep 80LISTEN 0 128 :::80 :::* |
3、访问测试Apache是否正常工作

四、配置Apache使用mod_jk模块实现代理及负载均衡
1、修改Apache主配置文件,包含一个文件并创建该文件;实现代理功能
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
|
将下面指令添加到文件末尾即可# vim /etc/httpd/httpd.confInclude /etc/httpd/extra/mod_jk.conf------------------------------------------------------------------------创建文件添加如下内容# vim /etc/httpd/extra/mod_jk.confLoadModule jk_module modules/mod_jk.soJkWorkersFile /etc/httpd/extra/workers.propertiesJkLogFile logs/mod_jk.logJkLogLevel debugJkMount /* TomcatAJkMount /status/ stat1------------------------------------------------------------------------# vim /etc/httpd/extra/workers.propertiesworker.list=TomcatA,stat1worker.TomcatA.port=8009worker.TomcatA.host=172.16.14.3worker.TomcatA.type=ajp13worker.TomcatA.lbfactor=1worker.stat1.type = status----------------------------------------------------------------------# service httpd restart |
2、修改后端Tomcat1服务器,添加一个虚拟主机并提供测试页面
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
|
[root@Tomcat1 ~]# cd /usr/local/tomcat/conf/[root@Tomcat1 conf]# cp server.xml server.xml.bak[root@Tomcat1 conf]# vim server.xml######修改如下内容<Engine name="Catalina" defaultHost="www.allen.com" jvmRoute="TomcatA">######在"Engine"中添加如下内容<Host name="www.allen.com" appBase="/web/allen" unpackWARs="true" autoDeploy="true"> <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs" prefix="allen_access_log." suffix=".txt" pattern="%h %l %u %t "%r" %s %b" /> <Context path="" docBase="/web/allen" /> </Host>----------------------------------------------------------------------######创建网站存放目录并创建测试页[root@Tomcat1 ~]# mkdir -p /web/allen[root@Tomcat1 ~]# vim /web/allen/index.jsp<%@ page language="java" %><html> <head><title>TomcatA</title></head> <body> <h1><font color="red">TomcatA </font></h1> <table align="centre" border="1"> <tr> <td>Session ID</td> <% session.setAttribute("abc","abc"); %> <td><%= session.getId() %></td> </tr> <tr> <td>Created on</td> <td><%= session.getCreationTime() %></td> </tr> </table> </body></html>----------------------------------------------------------------------######重启Tomcat服务并查看是否启动成功[root@Tomcat1 ~]# catalina.sh stop[root@Tomcat1 ~]# catalina.sh configtest[root@Tomcat1 ~]# catalina.sh start[root@Tomcat1 ~]# ss -tanlp | grep javaLISTEN 0 100 :::8080 :::* users:(("java",6323,40))LISTEN 0 1 ::ffff:127.0.0.1:8005 :::* users:(("java",6323,52))LISTEN 0 100 :::8009 :::* users:(("java",6323,41)) |
3、访问Apache1主机,验证是否代理成功

4、修改Apache配置文件(mod_jk.conf、workers.properties);实现负载均衡
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
|
# vim /etc/httpd/extra/mod_jk.confLoadModule jk_module modules/mod_jk.soJkWorkersFile /etc/httpd/extra/workers.propertiesJkLogFile logs/mod_jk.logJkLogLevel debugJkMount /* lbclusterJkMount /status/ stat1------------------------------------------------------------------------# vim /etc/httpd/extra/workers.propertiesworker.list=lbcluster,stat1worker.TomcatA.port=8009worker.TomcatA.host=172.16.14.3worker.TomcatA.type=ajp13worker.TomcatA.lbfactor=1worker.TomcatB.port = 8009worker.TomcatB.host=172.16.14.4worker.TomcatB.type = ajp13worker.TomcatB.lbfactor = 1worker.lbcluster.type = lbworker.lbcluster.sticky_session = 0 #取值为{1|0}1将用户session与后端服务器绑定,0为不绑定,如果支持session复制或session共享可以设置为0worker.lbcluster.balance_workers = TomcatA, TomcatBworker.stat1.type = status----------------------------------------------------------------------# service httpd reload |
5、修改后端Tomcat2服务器,添加一个虚拟主机并提供测试页面
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
|
[root@Tomcat2 ~]# cd /usr/local/tomcat/conf/[root@Tomcat2 conf]# cp server.xml server.xml.bak[root@Tomcat2 conf]# vim server.xml######修改如下内容<Engine name="Catalina" defaultHost="www.allen.com" jvmRoute="TomcatB">######在"Engine"中添加如下内容<Host name="www.allen.com" appBase="/web/allen" unpackWARs="true" autoDeploy="true"> <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs" prefix="allen_access_log." suffix=".txt" pattern="%h %l %u %t "%r" %s %b" /> <Context path="" docBase="/web/allen" /> </Host>----------------------------------------------------------------------######创建网站存放目录并创建测试页[root@Tomcat2 ~]# mkdir -p /web/allen[root@Tomcat2 ~]# vim /web/allen/index.jsp<%@ page language="java" %><html> <head><title>TomcatB</title></head> <body> <h1><font color="blue">TomcatB </font></h1> <table align="centre" border="1"> <tr> <td>Session ID</td> <% session.setAttribute("abc","abc"); %> <td><%= session.getId() %></td> </tr> <tr> <td>Created on</td> <td><%= session.getCreationTime() %></td> </tr> </table> </body></html>----------------------------------------------------------------------######重启Tomcat服务并查看是否启动成功[root@Tomcat2 ~]# catalina.sh stop[root@Tomcat2 ~]# catalina.sh configtest[root@Tomcat2 ~]# catalina.sh start[root@Tomcat2 ~]# ss -tanlp | grep javaLISTEN 0 100 :::8080 :::* users:(("java",6323,40))LISTEN 0 1 ::ffff:127.0.0.1:8005 :::* users:(("java",6323,52))LISTEN 0 100 :::8009 :::* users:(("java",6323,41)) |
6、访问Apache服务器,测试是否实现负载均衡

由上可见,基于Apache的mod_jk模块已成功实现负载均衡功能
五、配置Apache基于mod_proxy模块实现代理及负载均衡
1、修改Apache的主配置文件,包含一个文件并创建该文件;实现代理功能
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
|
# vim /etc/httpd/httpd.conf#Include /etc/httpd/extra/mod_jk.conf #注释此行Include /etc/httpd/extra/mod_proxy.conf------------------------------------------------------------------------# vim /etc/httpd/extra/mod_proxy.confProxyVia OnProxyRequests OffProxyPreserveHost Off<Proxy *> Order allow,deny Allow from all</Proxy> ProxyPass / ajp://172.16.14.3:8009/ ProxyPassReverse / ajp://172.16.14.3:8009/<Location / > Order allow,deny Allow from all</Location>----------------------------------------------------------------------# service httpd reload======================================================================注释: ProxyPass / ajp://172.16.14.3:8009/ #使用的是ajp协议 ProxyPassReverse / ajp://172.16.14.3:8009/可以更改为使用http协议;如下 ProxyPass / http://172.16.14.3:8080/ ProxyPassReverse / http://172.16.14.3:8080/######这里只演示使用ajp协议,如果是Apache与Tomcat结合建议使用ajp协议 |
2、访问Apache1服务器,测试是否代理成功

3、修改Apache配置文件(mod_proxy.conf);实现负载均衡
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
|
# vim /etc/httpd/extra/mod_proxy.confProxyVia OffProxyRequests OffProxyPreserveHost Off<Proxy balancer://allen> BalancerMember ajp://172.16.14.3:8009 loadfactor=1 BalancerMember ajp://172.16.14.4:8009 loadfactor=1 ProxySet lbmethod=bytraffic</Proxy><Location /allen> #设置状态及管理页面 SetHandler balancer-manager Proxypass ! Order allow,deny Allow from all</Location><Proxy *> Order allow,deny Allow from all</Proxy> ProxyPass / balancer://allen/ stickysession=JSESSIONID ProxyPassReverse / balancer://allen/<Location / > Order allow,deny Allow from all</Location>----------------------------------------------------------------------注释: BalancerMember ajp://172.16.14.3:8009 loadfactor=1 #使用ajp协议 BalancerMember ajp://172.16.14.4:8009 loadfactor=1可更改为使用http协议 BalancerMember http://172.16.14.3:8080 loadfactor=1 BalancerMember http://172.16.14.4:8080 loadfactor=1######这里只介绍使用ajp协议,如果有兴趣可以更改一下;loadfactor:权重----------------------------------------------------------------------ProxySet lbmethod=bytraffic #设置调度算法byrequests:即基于权重将统计请求个数进行调度(默认)bytraffic:则执行基于权重的流量计数调度bybusyness:通过考量每个后端服务器的当前负载进行调度----------------------------------------------------------------------# service httpd reload |
4、访问Apache服务器,测试基于mod_proxy模块是否实现负载均衡



六、论坛安装
1、在Tomcat1服务器上安装
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
|
# unzip JspRun\!_6.0.0_GBK.zip# mv /web/allen/index.jsp /web/allen/test.jsp# cp -r upload/* /web/allen/------------------------------------------------------------------------修改论坛数据库连接文件# vim /web/allen/config.propertiesdbhost = 172.16.14.5dbport=3306dbuser = bbsuserdbpw = bbspassdbname = bbs------------------------------------------------------------------------# catalina.sh stop# catalina.sh start |
2、安装论坛程序

3、点击下一步-->我同意-->下一步,进入数据库配置页面

4、点下一步-->(填写管理员密码)下一步-->进入创建数据表阶段-->然后点
-->进入首页

5、将论坛程序拷贝到Tomcat2服务器上一份并访问测试
|
1
2
3
4
5
|
# mv /web/allen/index.jsp /web/allen/test.jsp #把原来的测试页重命名------------------------------------------------------------------------# scp -r 172.16.14.3:/web/allen/* /web/allen/# catalina.sh stop# catalina.sh start |
6、访问Apache服务器,测试是否能正常访问论坛程序,这里就不在做访问测试
七、安装并配置Keepalived
1、将Apache1服务器的Httpd配置文件复制到Apache2服务器
|
1
2
3
4
|
[root@Apache1 ~]# cd /etc/httpd/[root@Apache1 httpd]# scp httpd.conf 172.16.14.2:/etc/httpd/[root@Apache1 httpd]# cd extra/[root@Apache1 extra]# scp mod_jk.conf mod_proxy.conf workers.properties 172.16.14.2:/etc/httpd/extra/ |
2、在Apache1与Apache2服务器上安装Keepalived;这里使用rpm包安装,光盘映像中有
|
1
2
3
|
[root@Apache1 ~]# yum -y install keepalived--------------------------------------------[root@Apache2 ~]# yum -y install keepalived |
3、配置Apache1服务器上的Keepalived
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
|
[root@Apache1 ~]# vim /etc/keepalived/keepalived.conf! Configuration File for keepalivedglobal_defs { notification_email { root@localhost } notification_email_from admin@allen.com smtp_server 172.16.0.0 smtp_connect_timeout 30 router_id LVS_ALLEN}vrrp_script chk_httpd { script "killall -0 httpd" interval 1 weight -2}vrrp_instance httpd_1 { state MASTER interface eth0 virtual_router_id 58 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1058 } virtual_ipaddress { 172.16.14.10 } track_script { chk_httpd }}[root@Apache1 ~]# service keepalived start[root@Apache1 ~]# chkconfig keepalived on[root@Apache1 ~]# chkconfig --list keepalivedkeepalived 0:off 1:off 2:on 3:on 4:on 5:on 6:off[root@Apache1 ~]# ip addr show eth02: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:2c:1a:24 brd ff:ff:ff:ff:ff:ff inet 172.16.14.1/16 brd 172.16.255.255 scope global eth0 inet 172.16.14.10/32 scope global eth0 inet6 fe80::20c:29ff:fe2c:1a24/64 scope link valid_lft forever preferred_lft forever |
4、配置Apache2服务器上的Keepalived
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
|
[root@Apache2 ~]# vim /etc/keepalived/keepalived.conf! Configuration File for keepalivedglobal_defs { notification_email { root@localhost } notification_email_from admin@allen.com smtp_server 172.16.0.0 smtp_connect_timeout 30 router_id LVS_ALLEN}vrrp_script chk_httpd { script "killall -0 httpd" interval 1 weight -2}vrrp_instance httpd_1 { state BACKUP interface eth0 virtual_router_id 58 priority 99 advert_int 1 authentication { auth_type PASS auth_pass 1058 } virtual_ipaddress { 172.16.14.10 } track_script { chk_httpd }}[root@Apache2 ~]# service keepalived start[root@Apache2 ~]# chkconfig --list keepalivedkeepalived 0:off 1:off 2:on 3:on 4:on 5:on 6:off |
5、使用"www.allen.com" 域名访问,验证是否能访问到论坛;由于没有DNS服务器解析,修改了"hosts"文件;如:
|
1
2
3
|
######添加如下内容解析C:\Windows\System32\drivers\etc\hosts172.16.14.10 www.allen.com |
6、模拟前端一台服务器出现故障;查看虚拟IP是否转移
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
|
######停止Apache1服务器上的Httpd服务来模拟故障;并查看IP[root@Apache1 ~]# service httpd stop[root@Apache1 ~]# ip addr show eth02: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:2c:1a:24 brd ff:ff:ff:ff:ff:ff inet 172.16.14.1/16 brd 172.16.255.255 scope global eth0 inet6 fe80::20c:29ff:fe2c:1a24/64 scope link valid_lft forever preferred_lft forever========================================================================######查看Apache2服务器上的IP[root@Apache2 ~]# ip addr show eth02: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:ec:f6:3f brd ff:ff:ff:ff:ff:ff inet 172.16.14.2/16 brd 172.16.255.255 scope global eth0 inet 172.16.14.10/32 scope global eth0 inet6 fe80::20c:29ff:feec:f63f/64 scope link valid_lft forever preferred_lft forever |
7、从上一步可以看出,虚拟IP地址已成功转移,说明还能正常提供服务,可以再次访问测试;如果服务器修复好重新上线,虚拟IP还会转移回去;这里就不再测试
到此,基于Apache做反向代理实现了Tomcat的负载均衡;而Keepalived实现了前端服务器的高可用;关于session保存的问题,方案有很多,比如可以保存到"memcached"、"redis"等,使用哪种方案具体还要根据自己的需求而定...
posted on 2014-01-22 11:50 gentleman_hai 阅读(268) 评论(0) 收藏 举报

浙公网安备 33010602011771号