load balancer docker解决方案

一.gbalancer_keepalived image创建和container创建

1.gbalancer_keepalived Dockerfile:

FROM letv:centos6
MAINTAINER yaofaliang <yaofaliang@letv.com>
RUN rpm -ivh http://pkg-repo.oss.letv.com/pkgs/centos6/letv-release.noarch.rpm
RUN yum install cronie -y
RUN yum install vim -y
RUN yum install gbalancer -y
RUN yum install keepalived -y
EXPOSE 80 8000 2222 3333
USER root
ADD ./init/init.sh /srv/mcluster/init/init.sh
RUN chmod 755 /srv/mcluster/init/init.sh
ENTRYPOINT /srv/mcluster/init/init.sh && /bin/bash

2.在init目录下创建初始化脚本init.sh，内容如下：

cat > /etc/sysconfig/network-scripts/ifcfg-peth0 << EOF
DEVICE=peth0
ONBOOT=yes
BOOTPROTO=static
IPADDR=$IP
NETMASK=$NETMASK
GATEWAY=$GATEWAY
EOF
ifconfig peth0 $IP/16
echo 'set network successfully'
#route
route add default gw 10.154.0.1
route del -net 0.0.0.0 netmask 0.0.0.0 dev eth0

PS：请注意脚本中网卡与宿主机上/etc/sysconfig/docker中网卡的对应关系，如配置文件中是eth0，脚本中请修改为peth0.此外，再注意container 网关的设置。

3.创建gbalancer_keepalived image，在gbalancer_keepalived目录下执行以下命令：

docker build -t "letv/gbalancer_keepalived:0.0.1" .

4.创建gbalancer_keepalived container，一定要注意ip地址是否与线上的有冲突。

docker run -i -t --rm --privileged -n --name "gbalancer_keepalived_admin_01" -v /srv/mcluster/ --env "IP=IP ADDRESS" --env "NETMASK=255.255.0.0" --env "GATEWAY=10.154.0.1" --env "HOSTNAME=gbalancer_keepalived_admin_01" letv/gbalancer_keepalived:0.0.1
docker run -i -t --rm --privileged -n --name "gbalancer_keepalived_admin_02" -v /srv/mcluster/ --env "IP=IP ADDRESS" --env "NETMASK=255.255.0.0" --env "GATEWAY=10.154.0.1" --env "HOSTNAME=gbalancer_keepalived_admin_02" letv/gbalancer_keepalived:0.0.1
 
docker run -i -t --rm --privileged -n --name "gbalancer_keepalived_app_01" -v /srv/mcluster/ --env "IP=IP ADDRESS" --env "NETMASK=255.255.0.0" --env "GATEWAY=10.154.0.1" --env "HOSTNAME=gbalancer_keepalived_app_01" letv/gbalancer_keepalived:0.0.1
docker run -i -t --rm --privileged -n --name "gbalancer_keepalived_app_02" -v /srv/mcluster/ --env "IP=IP ADDRESS" --env "NETMASK=255.255.0.0" --env "GATEWAY=10.154.0.1" --env "HOSTNAME=gbalancer_keepalived_app_02" letv/gbalancer_keepalived:0.0.1
 

二.配置gbalancer+keepalived。可以参考张祥写的文档，比较详细：keepalived+gbalancer 配置文件。下面是结合张祥的文档做的配置，根据需求在其中修改了一些参数：

1.gbalancer的配置。注意：master和backup上gbalancer的配置文件相同。

a.创建gbalancer配置文件的目录

mkdir /etc/gbalancer

b. gbalancer_keepalived_app的gbalancer配置文件：/etc/gbalancer/configuration.json

{
    "Addr": "0.0.0.0",
    "Service": "http",
    "Port": "80",
    "Backend": [
        "10.154.28.164:18001",
    "10.154.28.164:28001"
    ]
}

c. gbalancer_keepalived_admin的gbalancer配置文件：/etc/gbalancer/configuration.json

{
    "Addr": "0.0.0.0",
    "Service": "http",
    "Port": "80",
    "Backend": [
        "10.154.28.164:18000",
    "10.154.28.164:28000"
    ]
}

d. gbalancer_keepalived_app的keepalived master配置文件：

 global_defs {                    
   router_id webportal_keepalived_app_01           
}
vrrp_script chk_port {
    script "sh /etc/keepalived/chk_gbalancer1.sh"
    interval 10
}
vrrp_instance g_01 {
    state MASTER
    interface peth0
    virtual_router_id 101
    priority 150
    advert_int 3
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    track_script {
        chk_port
    }
    virtual_ipaddress {
        vip address
    }
}

e.gbalancer_keepalived_app的keepalived backup配置文件：

global_defs {                    
   router_id webportal_keepalived_app_02    
}
vrrp_script chk_port {
    script "sh /etc/keepalived/chk_gbalancer1.sh"
    interval 10
}
vrrp_instance g_01 {
    state BACKUP
    interface peth0
    virtual_router_id 101
    priority 100
    advert_int 3
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    track_script {
        chk_port
    }
    virtual_ipaddress {
        vip address
    }
}

f. gbalancer_keepalived_admin的keepalived master配置文件：

 global_defs {                    
   router_id webportal_keepalived01
}
vrrp_script chk_port {
    script "sh /etc/keepalived/chk_gbalancer1.sh"
    interval 10
}
vrrp_instance g_01 {
    state MASTER
    interface peth0
    virtual_router_id 101
    priority 150
    advert_int 3
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    track_script {
        chk_port
    }
    virtual_ipaddress {
        此处添加要设定的vip地址
    }
}

g. gbalancer_keepalived_admin的keepalived backup配置文件：

global_defs {                    
   router_id webportal_keepalived02
}
vrrp_script chk_port {
    script "sh /etc/keepalived/chk_gbalancer1.sh"
    interval 10
}
vrrp_instance g_01 {
    state BACKUP
    interface peth0
    virtual_router_id 101
    priority 100
    advert_int 3
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    track_script {
        chk_port
    }
    virtual_ipaddress {
        此处替换成你需要设定的VIP
    }
}

h. 在所有gbalancer_keepalived contaier（包括admin和app的master和backup）的etc/keepalived/目录下创建检查gbalancer的脚本,命名为chk_gbalancer1.sh，内容如下：

#!/bin/sh
#check gbalancer server status
PORT=80
if [ `ss -tlnp|grep -v grep|grep gbalancer |wc -l` -ne 1 ];then
   service gbalancer restart
   sleep 3
   if [ `ss -tlnp|grep -v grep|grep gbalancer|wc -l` -ne 1 ];then
      echo  `date +%Y.%m.%d\ %R:%S\ ` "gbalancer is error" >> /var/log/gbalancer.log
      exit 1
   fi   
fi

i.添加gbalancer的系统启动脚本

#!/bin/bash 
# 
# Description:  This shell script takes care of starting and stopping dennis 
# Hu Dennis created on Sep. 24th, 2010 
# 
# Source function library 
. /etc/init.d/functions 
   
#the service name  for example: dennis 
SNAME=gbalancer
   
#the full path and name of the daemon program 
#Warning: The name of executable file must be identical with service name 
PROG() {
     /usr/bin/gbalancer --config=/etc/gbalancer/configuration.json &>/dev/null &
}
   
kill_gbalancer() {
    kill -9 `ps -ef | grep -v "grep" | grep "/usr/bin/gbalancer --config=/etc/gbalancer/configuration.json" | awk '{print $2}'` &>/dev/null
}
   
# start function 
start() { 
    #check the daemon status first 
    if [ -f /var/lock/subsys/$SNAME ] 
    then 
        echo "$SNAME is already started!" 
        exit 0; 
    else 
        action "Starting $SNAME ..." PROG 
        [ $? -eq 0 ] && touch /var/lock/subsys/$SNAME 
        exit 0; 
    fi 
} 
   
#stop function 
stop() { 
    echo -n "Stopping $SNAME ..." 
    kill_gbalancer
    success
    echo
    rm -rf /var/lock/subsys/$SNAME 
} 
   
case "$1" in 
start) 
  start 
  ;; 
stop) 
  stop 
  ;; 
reload|restart) 
  stop 
  start 
  ;; 
status) 
  status $SNAME 
  ;; 
*) 
  echo $"Usage: $0 {start|stop|restart|status}" 
  exit 1 
esac

g.启动keepalived和gbalancer服务：

service gbalancer start
service keepalived start

k .通过vip访问webportal admin和webportal app查看是否可以提供服务。

posted on 2015-03-03 10:19 漫漫运维路阅读(784) 评论(0) 收藏举报