nginx+tomcat

一、系统规划

前端用nginx作反向代理和处理静态页面(前端负载高可以考虑在中间层添加nginx服务器处理静态页面，图片等，并设置前端只转发请求或者使用四层的lvs)，并通过keepalived保持它的高可用；

以.jsp结尾的动态请求转发至中间层的tomcat集群；

后端数据库使用mysql主从复制，当需要对动态内容加速或者数据库io瓶颈时，考虑memcache/redis缓存或者mysql proxy读写分离；

时间原因，部分服务用yum安装，测试和生产环境会用源码编译精简安装；如下表：

hostname	系统	Ip地址	作用	服务
web1	Centos6.5	10.188.12.200	静态请求和负载均衡	Nginx,keepalived
web2	Centos6.5	10.188.12.201	备用	Nginx，keepalived
		10.188.12.202	对外服务的vip
app1	Centos6.5	10.188.12.203	处理jsp请求	Jdk，tomcat
app2	Centos6.5	10.188.12.204	处理jsp请求	Jdk，tomcat
app...	Centos6.5	按需求添加	处理jsp请求或图片等	Jdk，tomcat
db1	Centos6.5	192.168.77.223	mariadb主库	mariadb-server
db2	Centos6.5	192.168.77.224	mariadb从库	mariadb-server

 

二、系统初始化

1、修改hostname

web1为例，其它类似：

[root@lnmp1 ~]# hostname web1

[root@lnmp1 ~]# sed -i 's/HOSTNAME=lnmp1/HOSTNAME=web1/' /etc/sysconfig/network

2、设置iptables

web1为例，其它类似：

iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT

iptables -A INPUT -p tcp -m tcp --dport 22 -j ACCEPT

iptables -A INPUT -p tcp -m tcp --dport 80 -j ACCEPT

iptables -A INPUT -p tcp -m tcp --dport 443 -j ACCEPT

iptables -A INPUT -p icmp -j ACCEPT

iptables -A INPUT -i lo -j ACCEPT

iptables -A INPUT -j REJECT --reject-with icmp-host-prohibited

iptables -A FORWARD -j REJECT --reject-with icmp-host-prohibited

3、关闭selinux

sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/sysconfig/selinux

setenforce 0

4、安装epel源

yum -y install epel-release.noarch

5、设置linux系统文件句柄数

[root@web1 ~]# ulimit -SHn 65535

[root@web1 ~]# echo -e '* soft   nofile   32768' >> /etc/security/limits.conf

[root@web1 ~]# echo -e '* hard nofile 65536' >> /etc/security/limits.conf

三、安装配置nginx和keepalived

1、安装nginx

groupadd www

useradd -g www -s /sbin/nologin -M -r www

yum -y install wget pcre pcre-devel openssl-devel zlib-devel lrzsz gcc gcc-c++

cd /usr/local/src

wget http://nginx.org/download/nginx-1.12.0.tar.gz

tar zxvf nginx-1.12.0.tar.gz

cd nginx-1.12.0

./configure --user=www --group=www --prefix=/usr/local/nginx --with-http_ssl_module --with-http_gzip_static_module  --with-http_stub_status_module

make -j8

make install

2、配置nginx

vi /usr/local/nginx/nginx.conf

user  www;

worker_processes  1;

error_log  logs/error.log;

 

pid        logs/nginx.pid;

events {

    use epoll;

    worker_connections  1024;

}

http {

    include       mime.types;

    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '

                      '$status $body_bytes_sent "$http_referer" '

                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  logs/access.log  main;

    sendfile        on;

    #tcp_nopush     on;

 

    #keepalive_timeout  0;

    keepalive_timeout  65;

 

    gzip on;

    gzip_min_length 1k;

    gzip_buffers    4 16k;

    gzip_http_version 1.0;

    gzip_comp_level 2;

    gzip_types text/plain application/x-javascripttext/css application/xml;

    gzip_vary on;

 

upstream apps{

iphash;

server 10.188.12.203:8080  max_fails=3  fail_timeout=20s;

server 10.188.12.204:8080  max_fails=3  fail_timeout=20s;

}

    server {

        listen       80;

        server_name  localhost;

        charset utf-8;

        #access_log  logs/host.access.log  main;

        location / {

            root   /root/www/

            index  index.html   index.htm;

        }

 

    location ~ .*.(jsp|do)$ {

            proxy_pass http://apps;  

            proxy_set_header X-Real-IP $remote_addr;

        }

        location ~ .*\.(gif|jpg|png|bmp|swf)$  

        {

            expires 3d;  

        }

       }

    }

}

3、安装配置keepalived

yum -y install keepalived

创建nginx进程监控脚本：

vi /etc/keepalived/check_nginx.sh

#!/bin/bash

counter=$(ps -C nginx --no-heading|wc -l)

if [ "${counter}" = "0" ]; then

    /usr/local/bin/nginx

    sleep 2

    counter=$(ps -C nginx --no-heading|wc -l)

    if [ "${counter}" = "0" ]; then

        /etc/init.d/keepalived stop

    fi

fi

chmod 755 /etc/keepalived/check_nginx.sh

更改keepalived.conf

! Configuration File for keepalived

global_defs {

    notification_email {

        243161146@qq.com

    }

    notification_email_from sh24316@sina.cn

    smtp_server smtp.sina.cn

    smtp_connect_timeout 30

    router_id LVS_DEVEL

}

vrrp_script chk_nginx {

#    script "killall -0 nginx"

    script "/etc/keepalived/check_nginx.sh"

    interval 2

    weight -5

    fall 3 

    rise 2

}

 

vrrp_instance VI_1 {

    state MASTER

    interface eth0

    mcast_src_ip 10.188.12.200

    virtual_router_id 51

    priority 101

    advert_int 2

    authentication {

        auth_type PASS

        auth_pass 1111

    }

    virtual_ipaddress {

        10.188.12.202

    }

    track_script {

       chk_nginx

    }

}

备机上的keepalived.conf只需要更改 state为BACKUP,priority 100, mcast_src_ip 10.188.12.201

四、安装配置tomcat server

1、安装jdk

wget -e 'http_proxy=192.168.77.226:808' -O "jdk-8u121-linux-x64.rpm" http://download.oracle.com/otn/java/jdk/8u121-b13/e9e7ea248e2c4826b92b3f075a80e441/jdk-8u121-linux-x64.rpm?AuthParam=1496053942_30cd718b9bf6c0d4a6bbe34a72e86dd6

(需要先登陆oracle官网，找到相关版本下载链接)

rpm -ivh jdk-8u121-linux-x64.rpm

vi /etc/profile.d/java.sh

#!/bin/bash
JAVA_HOME=/usr/java/jdk1.8.0_121/
PATH=$JAVA_HOME:$PATH
export PATH JAVA_HOME

. /etc/profile.d/java.sh

2、安装配置tomcat

wget https://mirrors.tuna.tsinghua.edu.cn/apache/tomcat/tomcat-8/v8.5.15/bin/apache-tomcat-8.5.15.tar.gz

tar xf apache-tomcat-8.5.15.tar.gz -C /usr/local

ln -sv /usr/local/apache-tomcat-8.5.15 /usr/local/tomcat

vi /etc/profile.d/tomcat.sh

#!/bin/bash
export CATALINA_HOME=/usr/local/tomcat
export PATH=$PATH:$CATALINA_HOME/bin

 .  /etc/profile.d/tomcat.sh

在更改/usr/local/tomcat/conf/server.xml后（比如在Host里修改appBase等）启动tomcat服务：catalina.sh start

3、JVM配置和优化

根据实际情况更改catalina.sh   server.xml         web.xml             tomcat-users.xml

五、安装配置mariadb主从：（使用mariadb10来代替默认的mysql5.1）

1、安装mariadb10

配置yum:

vi /etc/yum.repos.d/mariadb.repo

# http://downloads.mariadb.org/mariadb/repositories/

[mariadb]

name = MariaDB

baseurl = http://yum.mariadb.org/10.1/centos6-amd64

gpgkey=https://yum.mariadb.org/RPM-GPG-KEY-MariaDB

gpgcheck=1

 

yum -y install MariaDB-server

2、配置主从

db1:

cp  /usr/share/mysql/my-huge.cnf  /etc/my.cnf

service mysql restart 　　

mysql_secure_installation

vi /etc/my.cnf　　添加

server-id = 1

log-bin = mysql-bin

binlog_format = row

log-slave-updates

sync_binlog = 1

auto_increment_increment = 2 

auto_increment_offset = 1

登陆mysql并创建复制账号并重置状态：

grant replication slave,replication client on *.* to 'copy'@'192.168.77.224' identified by 'paste';

reset master;
reset slave;

 

db2:

cp  /usr/share/mysql/my-huge.cnf  /etc/my.cnf

service mysql restart 　　

mysql_secure_installation

vi /etc/my.cnf

server-id       = 2

log-bin=mysql-bin

log-slave-updates

relay-log=relay-log-bin

change master to master_host='192.168.77.223',master_user='copy',master_password='paste',master_log_file='mysql-bin.000001',master_log_pos=563;

start slave;
show slave status\G

 

过往配置：

一、安装nginx

yum -y install wget pcre pcre-devel openssl-devel zlib-devel lrzsz gcc gcc-c++

wget http://nginx.org/download/nginx-1.12.0.tar.gz

wget http://nginx.org/download/nginx-1.12.0.tar.gz

groupadd www
useradd -g www -s /sbin/nologin -M -r www

./configure --user=www --group=www --prefix=/usr/local/nginx --with-http_ssl_module --with-http_gzip_static_module --with-http_stub_status_module

make && make install

添加配置文件：

user  www;
worker_processes  1;

error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

pid        logs/nginx.pid;


events {
    use epoll;
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  logs/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;

    gzip on;
    gzip_min_length 1k;
    gzip_buffers    4 16k;
    gzip_http_version 1.0;
    gzip_comp_level 2;
    gzip_types text/plain application/x-javascripttext/css application/xml;
    gzip_vary on;

    server {
        listen       80;
        server_name  www.test.com;

        charset utf-8;

        #access_log  logs/host.access.log  main;

        location / {
            root   /usr/local/tomcat/webapps/ROOT;
            index  index.html index.jsp  index.htm;
        }

    location ~ .*.jsp$ {
            index index.jsp;
            proxy_pass http://127.0.0.1:8080;   
            proxy_redirect off;
            proxy_set_header Host $host;  
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            client_max_body_size 10m;   
            client_body_buffer_size 128k; 
            proxy_connect_timeout 90;   
            proxy_read_timeout 90;      
            proxy_buffer_size 4k;       
            proxy_buffers 6 32k;        
            proxy_busy_buffers_size 64k;
            proxy_temp_file_write_size 64k; 
        }
        location ~ .*\.(gif|jpg|png|bmp|swf)$   
        {
            expires 30d;   
        }
        location ~ .*\.(jsp|js|css)?$
        {
            expires 1d;
        }    
    
       error_page  404              /404.html;

       #redirect server error pages to the static page /50x.html
        
       error_page   500 502 503 504  /50x.html;
       location = /50x.html {
           root   html;
       }

        # proxy the PHP scripts to Apache listening on 127.0.0.1:80
        #
        #location ~ \.php$ {
        #    proxy_pass   http://127.0.0.1;
        #}

        # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
        #
        #location ~ \.php$ {
        #    root           html;
        #    fastcgi_pass   127.0.0.1:9000;
        #    fastcgi_index  index.php;
        #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
        #    include        fastcgi_params;
        #}

        # deny access to .htaccess files, if Apache's document root
        # concurs with nginx's one
        #
        #location ~ /\.ht {
        #    deny  all;
        #}
    }


    # another virtual host using mix of IP-, name-, and port-based configuration
    #
    #server {
    #    listen       8000;
    #    listen       somename:8080;
    #    server_name  somename  alias  another.alias;

    #    location / {
    #        root   html;
    #        index  index.html index.htm;
    #    }
    #}


    # HTTPS server
    #
    #server {
    #    listen       443 ssl;
    #    server_name  localhost;

    #    ssl_certificate      cert.pem;
    #    ssl_certificate_key  cert.key;

    #    ssl_session_cache    shared:SSL:1m;
    #    ssl_session_timeout  5m;

    #    ssl_ciphers  HIGH:!aNULL:!MD5;
    #    ssl_prefer_server_ciphers  on;

    #    location / {
    #        root   html;
    #        index  index.html index.htm;
    #    }
    #}

}

 

添加启动脚本：

#!/bin/bash
# chkconfig: 345 99 20
# description: Nginx servicecontrol script
PROG="/usr/local/nginx/sbin/nginx"
PIDF="/usr/local/nginx/logs/nginx.pid"
case "$1" in
start)
$PROG
echo "Nginx service start success."
;;
stop)
kill -s QUIT $(cat $PIDF)
echo "Nginx service stop success."
;;
restart)
$0 stop
$0 start
;;
reload)
kill -s HUP $(cat $PIDF)
echo "reload Nginx config success."
;;
*)
echo "Usage: $0{start|stop|restart|reload}"
exit 1
esac

chmod +x /etc/init.d/nginx
service nginx restart
chkconfig nginx on

添加nginx环境：

echo -e 'PATH=/usr/local/nginx/sbin:$PATH' >> /etc/profile
source /etc/profile

 

二、安装tomcat

 

tar zxvf apache-tomcat-8.5.15.tar.gz
tar zxvf jdk-8u131-linux-x64.tar.gz

cp /usr/local/src/apache-tomcat-8.5.15 /usr/local/tomcat -rf
cp jdk1.8.0_131  /usr/local/jdk -rf

vi /etc/profile 　　
JAVA_HOME=/usr/local/jdk
PATH=$PATH:$JAVA_HOME/bin
CLASSPATH=$JAVA_HOME/lib:$JAVA_HOME/jre/lib
export JAVA_HOME PATH CLASSPATH
source /etc/profile

/usr/local/tomcat/bin/startup.sh