Nginx 、MySQL、Django 在 Docker-compose 中的部署
概述
本文主要记录了在 Linux 3.10.0-1062.el7.x86_64 下使用 docker-compose 搭建服务的过程，由于公司服务器在内网中，搭建镜像及下载依赖时需要外部代理，如果本机服务无相关限制，可联通外网，配置代理部分忽略即可，其实 firewall 为开启，selinux 为 enforcing 状态。
部署中主要涉及以下内容：
- Nginx 代理 MySQL 和 django 服务以及 https 自签名的部署。
- 使用
gunicorn部署 django 服务。 - MySQL 相关初始化配置，及建权过程。
- 将 .py 文件转为 .pyc 文件
Docker 安装
如果之前安装过的话，需要删除老版本 docker:
sudo yum remove docker \
docker-client \
docker-client-latest \
docker-common \
docker-latest \
docker-latest-logrotate \
docker-logrotate \
docker-engine
安装构建 docker 所需要的工具;
sudo yum install -y yum-utils \
device-mapper-persistent-data \
lvm2
添加 docker 源:
sudo yum-config-manager \
--add-repo \
https://download.docker.com/linux/centos/docker-ce.repo
安装 docker ce:
sudo yum install docker-ce docker-ce-cli containerd.io
配置 docker:
sudo systemctl start docker
sudo systemctl enable docker
安装 docker-compose:
# download
sudo curl -L "https://github.com/docker/compose/releases/download/1.25.5/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
# apply exe permissions
sudo chmod +x /usr/local/bin/docker-compose
# add link
sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
# check
docker-compose --version
如果在安装中下无法下载相应服务，则表明可能需要配置相应代理或开放相关 ACL 配置，关于相关配置过程可参见Centos7 下代理配置。
docker 的代理配置也需要单独做出来，可以参见docker 代理脱坑这篇。
项目结构
下面是 docker-compose 工程构建后完整目录：
├── config
│ ├── auto_start_script.sh
│ ├── mysql
│ │ ├── init
│ │ │ └── init.sql
│ │ ├── my.cnf
│ │ └── my.conf
│ ├── nginx
│ │ ├── conf.d
│ │ │ ├── certs
│ │ │ │ ├── cmi_sdn.crt
│ │ │ │ ├── cmi_sdn.key
│ │ │ │ └── dhparam.pem
│ │ │ ├── ssl.conf
│ │ │ └── ssl.conf.bk
│ │ ├── default.d
│ │ │ └── ssl-redirect.conf
│ │ ├── log
│ │ │ ├── access.log
│ │ │ └── error.log
│ │ └── nginx.conf
│ └── requirements.txt
├── docker-compose.yml
├── Dockerfile
├── db_storage # 创建文件夹就好，用于挂载 mysql 数据
└── project
├── your_project
│ ├── settings.py
│ ├── urls.py
│ └── wsgi.py
├── manage.py
└── static
└── frontend
tree -I '*project*|*frontend*|*db_storage*'
- config 目录：存放 nginx 配置, requirement, mysql 配置
- docker-compose.yml: 用于编排和管理容器的 yaml 文件
- Dockerfile: 用于编写 django 镜像的文件
- project：django 项目代码
Dockerfile 编写
编写 django 的运行环境，如不需要代理可将环境变量删除
FROM python:3.6.8
ENV MY_PROXY_URL="http://173.39.112.117:80"
ENV HTTP_PROXY=$MY_PROXY_URL \
HTTPS_PROXY=$MY_PROXY_URL \
FTP_PROXY=$MY_PROXY_URL \
http_proxy=$MY_PROXY_URL \
https_proxy=$MY_PROXY_URL \
ftp_proxy=$MY_PROXY_URL
WORKDIR /src
COPY /config/requirements.txt /
RUN pip install --no-cache-dir -r /requirements.txt
ENV MY_PROXY_URL=
ENV HTTP_PROXY=$MY_PROXY_URL \
HTTPS_PROXY=$MY_PROXY_URL \
FTP_PROXY=$MY_PROXY_URL \
http_proxy=$MY_PROXY_URL \
https_proxy=$MY_PROXY_URL \
ftp_proxy=$MY_PROXY_URL
创建 config/requirements.txt 文件，并添加所需要的 python 依赖包，部署选用 gunicorn. 文件如下：
Django==2.1.7
djangorestframework==3.9.2
djangorestframework-jwt==1.11.0
mysqlclient==1.4.2
PyJWT==1.7.1
requests==2.21.0
gunicorn==19.9.0
django-role-permissions==2.2.1
aiohttp==3.5.4
apscheduler==3.6.3
networkx==2.4
numpy==1.17.4
Nginx 配置
在 nginx/conf.d/certs 目录下，使用 openssl 进行自签名，生成需要的证书：
sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout example.key -out example.crt;
sudo openssl dhparam -out dhparam.pem 2048
编写 nginx 配置文件 config/nginx/conf.d/ssl.conf：
upstream web_server {
# ip_hash;
server web:8000 fail_timeout=100s;
#server 10.89.200.40:8000 backup;
keepalive 300;
}
upstream topo {
# ip_hash;
server topo:7777 fail_timeout=100s;
#server 10.89.200.40:8000 backup;
keepalive 300;
}
server {
listen 443 http2 ssl;
listen [::]:443 http2 ssl;
server_name server_IP_address;
ssl_certificate /etc/nginx/conf.d/certs/cmi_sdn.crt;
ssl_certificate_key /etc/nginx/conf.d/certs/cmi_sdn.key;
ssl_dhparam /etc/nginx/conf.d/certs/dhparam.pem;
########################################################################
# from https://cipherli.st/ #
# and https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html #
########################################################################
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
ssl_ecdh_curve secp384r1;
ssl_session_cache shared:SSL:10m;
ssl_session_tickets off;
ssl_stapling on;
ssl_stapling_verify on;
resolver 8.8.8.8 8.8.4.4 valid=300s;
resolver_timeout 5s;
# Disable preloading HSTS for now. You can use the commented out header line that includes
# the "preload" directive if you understand the implications.
#add_header Strict-Transport-Security "max-age=63072000; includeSubdomains; preload";
add_header Strict-Transport-Security "max-age=63072000; includeSubdomains";
add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;
##################################
# END https://cipherli.st/ BLOCK #
##################################
location / {
proxy_set_header Host $host:$server_port;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Real-PORT $remote_port;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://web_server/;
proxy_connect_timeout 159s;
proxy_send_timeout 600;
proxy_read_timeout 600;
keepalive_timeout 600s;
}
listen 8000;
server_name localhost;
location /static {
autoindex on;
alias /src/static;
}
}
server {
listen 666 http2 ssl;
listen [::]:666 http2 ssl;
server_name server_IP_address;
ssl_certificate /etc/nginx/conf.d/certs/cmi_sdn.crt;
ssl_certificate_key /etc/nginx/conf.d/certs/cmi_sdn.key;
ssl_dhparam /etc/nginx/conf.d/certs/dhparam.pem;
########################################################################
# from https://cipherli.st/ #
# and https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html #
########################################################################
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
ssl_ecdh_curve secp384r1;
ssl_session_cache shared:SSL:10m;
ssl_session_tickets off;
ssl_stapling on;
ssl_stapling_verify on;
resolver 8.8.8.8 8.8.4.4 valid=300s;
resolver_timeout 5s;
# Disable preloading HSTS for now. You can use the commented out header line that includes
# the "preload" directive if you understand the implications.
#add_header Strict-Transport-Security "max-age=63072000; includeSubdomains; preload";
add_header Strict-Transport-Security "max-age=63072000; includeSubdomains";
#add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;
##################################
# END https://cipherli.st/ BLOCK #
##################################
add_header X-Frame-Options sameorigin always;
location = / {
root /src/frontend/dist;
index index.html index.htm;
}
location ^~ /index.html {
root /src/frontend/dist;
index index.html index.htm;
}
location ^~ /favicon.png {
root /src/frontend/dist;
}
location ^~ /static {
alias /src/frontend/dist/static;
}
location ^~ /topo/get_topology/ {
proxy_set_header Host $host:$server_port;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Real-PORT $remote_port;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://topo;
}
location ^~ /topo/update_topology/ {
proxy_set_header Host $host:$server_port;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Real-PORT $remote_port;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://topo;
}
location ~* /.* {
proxy_set_header Host $host:$server_port;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Real-PORT $remote_port;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://web_server;
proxy_connect_timeout 159s;
proxy_send_timeout 600;
proxy_read_timeout 600;
keepalive_timeout 600s;
}
server_name localhost;
}
注意这里的 web 是 docker-compose 编排中的 django 所在容器名称。
topo 是我这里启用的另一服务，忽略即可。
编写 default.d/ssl-redirect.conf 添加 http 到 https 的自动跳转
return 301 https://$host$request_uri/;
编写 config/nginx/nginx.conf 配置入口文件
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
include /etc/nginx/conf.d/*.conf;
}
stream {
# log_format main 'mysql - $remote_addr - [$time_local] ';
# access_log /var/log/nginx/access.log main;
upstream mysql {
# hash $remote_addr consistent;
server 10.124.207.155:3306;
server 10.124.207.154:3306 backup;
}
server {
listen 3306;#公网机器监听端口
proxy_connect_timeout 10s;
proxy_timeout 800s;
proxy_pass mysql;
}
}
mysql 为 docker-compose 文件下的名称
创建 config/nginx/log/access.log or error.log
MySQL 配置
编写 config/mysql/my.conf
# Remove leading # to turn on a very important data integrity option: logging
# changes to the binary log between backups.
# log_bin
#
# Remove leading # to set options mainly useful for reporting servers.
# The server defaults are faster for transactions and fast SELECTs.
# Adjust sizes as needed, experiment to find the optimal values.
# join_buffer_size = 128M
# sort_buffer_size = 2M
# read_rnd_buffer_size = 2M
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
#server-id=1
log_bin = mysql-bin # 开启二进制log文件
binlog_format = mixed
#relay-log = relay-bin
#relay-log-index = slave-relay-bin.index
#auto-increment-offset = 1 # 从 1 开始
#auto-increment-increment = 2 # 每次增长 2
character-set-server=utf8
[client]
default-character-set=utf8
[mysql]
default-character-set=utf8
[mysqld]
skip-name-resolve
编写初始化 config/mysql/init/init.sql
CREATE USER 'test'@'%';
GRANT ALL PRIVILEGES ON *.* To 'test'@'%' IDENTIFIED BY 'testPass1!';
FLUSH PRIVILEGES;
CREATE DATABASE IF NOT EXISTS test default charset utf8 COLLATE utf8_general_ci;
编写 docker-compose 文件：
version: '3'
services:
mysql:
image: ctg/mysql:5.7.29
container_name: "mysql"
environment:
MYSQL_ROOT_PASSWORD: "MyNewPass1!"
restart: always
ports:
- "3306:3306"
volumes:
- "/etc/localtime:/etc/localtime:ro"
- "./db_storage:/var/lib/mysql"
- "./config/mysql/my.cnf:/etc/my.cnf"
- "./config/mysql/init:/docker-entrypoint-initdb.d/"
networks:
- app_net
proxy_server:
image: ctg/nginx:latest
ports:
- "443:443"
- "3307:3307"
- "444:444"
- "666:666"
volumes:
- "/etc/localtime:/etc/localtime:ro"
- "./project:/src"
- "./config/nginx/conf.d:/etc/nginx/conf.d"
- "./config/nginx/default.d:/etc/nginx/default.d"
- "./config/nginx/log:/var/log/nginx"
- "./config/nginx/nginx.conf:/etc/nginx/nginx.conf"
- "./frontend:/var/frontend"
container_name: "proxy_server"
depends_on:
- web
networks:
- app_net
web:
image: ctg/web:latest
container_name: "ctg_backend"
command: bash -c "python manage.py makemigrations && python manage.py migrate && python manage.py collectstatic --noinput && gunicorn -c gunicorn_config.py ctg.wsgi"
ports:
- "8000:8000"
volumes:
- "/etc/localtime:/etc/localtime:ro"
- "./project:/src"
- "/proc:/hostip/:ro"
restart: always
privileged: true
networks:
- app_net
topo:
image: ctg/topo:latest
container_name: "ctg_topo"
command: bash -c "python /ctg_topo/manage.py runserver 0.0.0.0:7777"
ports:
- "7777:7777"
volumes:
- "/etc/localtime:/etc/localtime:ro"
- "./topo_project/topo_server:/ctg_topo"
restart: always
networks:
- app_net
networks:
app_net:
driver: bridge
ipam:
config:
- subnet: 172.26.0.1/16
由于 nginx 和 django 是分开部署的，记得把 django 层收集的静态文件也给 nginx 拷贝一份，否则会导致 admin 管理界面样式丢失。我这里的 frontend 是又另外的 portal 界面，忽略即可
想着修改 django 项目的 settings.py 文件，添加如下内容：
ALLOW_HOSTS = ['web']
STATIC_ROOT = os.path.join(BASE_DIR, 'static/')
运行
构建镜像：
docker-compose build
如果上述构建成功的话，就可以运行容器了：
docker-compose up -d
如果 docker-compose 的容器是以守护进程运行的话，可以使用下面的命令查看容器的日志：
docker-compose logs -f web
配置开机自启动
# 配置 docker 自启动
systemctl enable docker
配置docker-compose自启动可参考 自启动. 编写 config/auto_start_script.sh 文件。
将 python 转为 pyc 文件
在 django manage.py 同级别执行
# 生成 pyc 文件:
python -m compileall -b .
# 删除 py 文件:
find . -name "*.py" |xargs rm -rf
# 删除 pycache 目录:
find . -name "pycache" |xargs rm -rf
# 最后将 docker-compose.yml 文件中涉及到的 py 文件 换成 pyc.
需要注意的是 gunicorn_config.py 要备份一份，该文件不要转成 pyc.
如果想要实现通过 Jenkins 自动编译加部署，可以参考Jenkins 实现 CI/CD.
参考
how-to-create-a-self-signed-ssl-certificate-for-nginx-on-centos-7
