使用 Jenkins + NetCore + Harbor 自动发布应用到 Docker
使用 Jenkins + NetCore + Harbor 自动发布应用到 Docker
--
防火墙设置
- 
关闭防火墙 systemctl stop firewalld.service setenforce 0 systemctl disable firewalld
- 
安装基础环境 # docker 基础环境 yum install -y yum-utils device-mapper-persistent-data lvm2
安装 Docker
- 
配置宿主机网卡转发 cat <<EOF> /etc/sysctl.d/docker.conf net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 net.ipv4.ip_forward = 1 EOF
- 
使配置生效 sysctl -p /etc/sysctl.d/docker.conf
- 
配置阿里源地址 curl -o /etc/yum.repos.d/Centos-7.repo <http://mirrors.aliyun.com/repo/Centos-7.repo> curl -o /etc/yum.repos.d/docker-ce.repo <http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo> yum clean all && yum makecache
- 
yum 安装 docker # 有安装过先删除 yum remove docker docker-common docker-selinux docker-engine # 查看源中可用版本 yum list docker-ce --showduplicates | sort -r yum install docker-ce-20.10.8 -y
- 
设置开机自启 systemctl enable docker systemctl daemon-reload
- 
启动 docker systemctl start docker
注意:系统自带的 yum 源安装版本为,
1.13.1,后面在 Jenkins 里面挂载宿主机 docker 时会报错,必须升级为新版本 docker
安装 Harbor
- 
安装 docker-compose wget https://github.com/docker/compose/releases/download/1.24.1/docker-compose-Linux-x86_64 mv docker-compose-Linux-x86_64 /usr/local/bin/docker-compose chmod +x /usr/local/bin/docker-compose docker-compose version
- 
下载 wget https://github.com/goharbor/harbor/releases/download/v2.9.1/harbor-offline-installer-v2.9.1.tgz
- 
解压缩 tar xzvf harbor-offline-installer-v2.9.1.tgz -C /home/application cd /home/application/harbor
- 
配置(需要提前准备好域名和证书) cp harbor.yml.tmpl harbor.yml vim harbor.yml # 开起https访问提前准备好域名证书,也可不开启,但是在docker中需要配置`insecure-registries` hostname: harbor.codingbugs.fun port: 443 certificate: /certificate.pem private_key: /certificate.key # 默认帐号admin,修改管理密码 harbor_admin_password: Harbor12345 # 数据持久化 data_volume: /home/applicatoin/harbor/data # 启用外部nginx代理需放开 #external_url: https://harbor.codingbugs.fun:443 # 日志地址 location: /var/log/harbor
- 
修改安装脚本 vim prepare # 54行位置 docker run --rm -v $input_dir:/input \ -v $data_path:/data \ -v $harbor_prepare_path:/compose_location \ -v $config_dir:/config \ -v /home/application/harbor/certs:/hostfs \ --privileged \ goharbor/prepare:v2.9.1 prepare $@
- 
安装 ./prepare ./install.sh注意此处 2.9版本会报找不到证书,解决方案,harbor.yml中的证书地址也需要删除到只保留证书和密钥名称,2.3版本没有发现这个问题
安装 Jenkins
- 
创建持久目录 mkdir -p /home/application/jenkins/jenkins_home chmod 777 /home/application/jenkins/jenkins_home
- 
使用 docker 安装 # 启动jenkins docker run \ --env JAVA_OPTS="-server -Xms1024m -Xmx2048m -XX:PermSize=512m -XX:MaxPermSize=512m" \ --name jenkins \ --privileged=true \ --restart=on-failure \ -itd \ -p 8080:8080 \ -p 50000:50000 \ -e JENKINS_OPTS='--prefix=/jenkins' \ -e TZ='Asia/Shanghai' \ -e JENKINS_ARGS='--prefix=/jenkins' \ -v /home/application/jenkins/jenkins_home:/var/jenkins_home \ -v /etc/localtime:/etc/localtime \ -v /var/run/docker.sock:/var/run/docker.sock \ -v /usr/bin/docker:/usr/bin/docker \ -v /usr/lib64/libltdl.so.7:/usr/lib/x86_64-linux-gnu/libltdl.so.7 \ jenkins/jenkins:2.414-centos7- 注意其它教程版本为 Debian12.2,只支持 NET6 以上版本,官网,如需使用 NET6 以下版本需要使用jenkins/jenkins:2.414-lts-jdk11(不带小版本),本教程使用jenkins/jenkins:2.414-centos7
- 更多版本
- --privileged:使其能操作系统文件
- -v /var/run/docker.sock:/var/run/docker.sock -v /usr/bin/docker:/usr/bin/docker -v /usr/lib64/libltdl.so.7:/usr/lib/x86_64-linux-gnu/libltdl.so.7:使用宿主机的 docker
 
- 注意其它教程版本为 Debian12.2,只支持 NET6 以上版本,官网,如需使用 NET6 以下版本需要使用
- 
登录及配置 - docker logs jenkins找到初始密码
- ip:8080/jenkins登陆系统
- 安装git和pipeline插件
 
安装 NetCore 运行时
- 
进入容器 docker exec -it -uroot jenkins /bin/bash
- 
查看 Debian 版本 cat /etc/debian_version
- 
配置 netcore 源,安装 dotnet # 注意版本号 rpm -Uvh https://packages.microsoft.com/config/rhel/7/packages-microsoft-prod.rpm sudo yum update -y rm packages-microsoft-prod.deb # 安装sdk yum install -y dotnet-sdk-5.0 # 安装运行时,安装aspnetcore-runtime会默认安装netcore-runtime yum install -y aspnetcore-runtime-5.0.x86_64 yum install -y dotnet-runtime-5.x86_64 # 检查 dotnet --list-sdks dotnet --list-runtimes
在 jenkins 容器中安装 python3
- 
进入容器 docker exec -it -uroot jenkins /bin/bash
- 
安装 python3 - 安装依赖环境
 yum -y groupinstall "Development tools" yum -y install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gdbm-devel db4-devel libpcap-devel xz-devel yum install -y libffi-devel zlib1g-dev yum install zlib* -y- 下载安装包
 wget https://www.python.org/ftp/python/3.9.1/Python-3.9.1.tar.xz- 编译安装
 tar -xvf Python-3.9.1.tar.xz mkdir /usr/local/python3 cd Python-3.9.1 ./configure --prefix=/usr/local/python3 --with-ssl make && make install- 创建软连接
 ln -s /usr/local/python3/bin/python3 /usr/local/bin/python3 ln -s /usr/local/python3/bin/pip3 /usr/local/bin/pip3- 验证是否安装成功
 python3 -V pip3 -V
- 
升级 OpenSSL- 检查 Python 的 ssl 模块版本
 # 若低于1.1版本则需升级,否则跳过 python3 -c "import ssl; print(ssl.OPENSSL_VERSION)"- 升级OpenSSL
 wget https://www.openssl.org/source/openssl-1.1.1n.tar.gz --no-check-certificate tar zxf openssl-1.1.1n.tar.gz cd openssl-1.1.1n/ ./config --prefix=/usr/local/openssl make -j && make install # 添加到系统动态库查找路径 export LD_LIBRARY_PATH=/usr/local/openssl/lib:$LD_LIBRARY_PATH source /etc/profile # 重新编译python cd python yum install libffi-devel -y make clean ./configure --prefix=/usr/local/python3 --with-openssl=/usr/local/openssl --with-ssl-default-suites=openssl --with-system-ffi make -j && make install # 检查版本 python3 -c "import ssl; print(ssl.OPENSSL_VERSION)" # 结果应该为:`OpenSSL 1.1.1n 15 Mar 2022` 则升级成功
配置 Pipeline
- 
配置密钥 - 进入路径Manage Jenkins/Credentials/System/Global cerdentials(unrestricted)
- 点击Add Credentials添加 git 仓库和 harbor 凭证
 
- 进入路径
- 
创建 Pipeline 类型 Job pipeline { agent any environment { // 代码仓库配置 GIT_URL = "http://刘春阳@url:10101/r/ContainerTest.git" BRANCH_NAME = "master" // 项目配置 PROJECT_PATH = "./ContainerTest/ContainerTest.csproj" // 镜像仓库配置 HARBOR_PROJECT_NAME = "fineex_test" DOCKER_IMAGE_NAME = "containertest" // 发布服务配置 EnvName = "Master" AppName = "FineEx.Test" // 基础配置(不修改) GIT_CRT_ID = "0ce9e30c-ed14-4945-9444-62c9885594cb" HARBOR_REGISTRY = "harbor.test.com" HARBOR_CRT_ID = 'e50b42a5-cbd6-4f51-bea8-074c8690b912' } stages { stage('=========拉取代码=========') { steps { git branch: "${BRANCH_NAME}", credentialsId: "${GIT_CRT_ID}", url: "${GIT_URL}" script { def commitHash = sh(returnStdout: true, script: 'git rev-parse --short HEAD').trim() env.VERSION_NUMBER = commitHash } } } stage('=========还原代码依赖=========') { steps { sh 'dotnet restore -nowarn:msb3202,nu1503,cs1591 ${PROJECT_PATH} --configfile $JENKINS_HOME/nuget/nuget.config' } } stage('=========编译代码=========') { steps { sh 'dotnet build -nowarn:msb3202,nu1503,cs1591 --no-restore ${PROJECT_PATH}' } } stage('=========发布代码========='){ steps{ script { def publishPath = './JenkinsBuilds/'+env.JOB_NAME+'/'+env.BUILD_NUMBER env.PUBLISH_PATH = publishPath } sh 'dotnet publish ${PROJECT_PATH} -c Release -o ${PUBLISH_PATH}' } } stage('=========构建镜像========='){ steps{ // 切换目录 sh "cd ${PUBLISH_PATH}" sh "ls -l ${PUBLISH_PATH}" sh "pwd" sh "ls -l" // 编译镜像 sh "docker build -t ${HARBOR_REGISTRY}/${HARBOR_PROJECT_NAME}/${DOCKER_IMAGE_NAME}:${VERSION_NUMBER} -f ${PUBLISH_PATH}/Dockerfile ${PUBLISH_PATH}" } } stage('=========推送到镜像仓库========='){ steps{ withCredentials([usernamePassword(credentialsId: "${HARBOR_CRT_ID}", passwordVariable: 'HARBOR_PASSWORD', usernameVariable: 'HARBOR_USERNAME')]) { sh (script: """ # 登录harbor HARBOR_PASSWORD=${HARBOR_PASSWORD} && echo "\$HARBOR_PASSWORD" | docker login ${HARBOR_REGISTRY} -u ${HARBOR_USERNAME} --password-stdin # 推送镜像 docker push ${HARBOR_REGISTRY}/${HARBOR_PROJECT_NAME}/${DOCKER_IMAGE_NAME}:${VERSION_NUMBER} # 删除镜像 docker rmi ${HARBOR_REGISTRY}/${HARBOR_PROJECT_NAME}/${DOCKER_IMAGE_NAME}:${VERSION_NUMBER} """) } } } stage('=========调用发布服务发布应用========='){ steps{ sh 'python3 $JENKINS_HOME/scripts/develop_client.py ${EnvName} ${AppName} ${VERSION_NUMBER}' } } } post{ failure{ echo '执行失败需要发送通知' sh 'python3 $JENKINS_HOME/scripts/sendReport.py ${EnvName} ${AppName} false' } success { echo '执行成功时需要发送通知' sh 'python3 $JENKINS_HOME/scripts/sendReport.py ${EnvName} ${AppName} true' } } }/home/application/jenkins/jenkins_home/nuget/nuget.config内容<?xml version="1.0" encoding="utf-8"?> <configuration> <packageSources> <add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" /> <!-- 内部nuget服务 --> <add key="nuget.fineex" value="http://nuget:8888/Nuget" /> </packageSources> <activePackageSource> <add key="Microsoft and .NET" value="https://www.nuget.org/api/v2/curated-feeds/microsoftdotnet/" /> </activePackageSource> <packageRestore> <add key="enabled" value="True" /> <add key="automatic" value="True" /> </packageRestore> <bindingRedirects> <add key="skip" value="False" /> </bindingRedirects> <packageManagement> <add key="format" value="0" /> <add key="disabled" value="False" /> </packageManagement> </configuration>/home/application/jenkins/jenkins_home/scripts/develop_client.py内容,自定义简单逻辑,仅供参考import pymysql import requests import sys import json def sendPublishRequest(environment, appName, imageVersion): infos = getAppInfo(environment, appName) for info in infos: url = info['connect'] param = { 'environment': environment, 'appName': appName, 'imageVersion': imageVersion } response = requests.get(url=url, params=param) print(json.loads(response.text)['message']) def getAppInfo(environment, appName): sql = f""" select `appid`, `appname`, `connect` from table_release_container where appname='{appName}' and environment = '{environment}' """ try: conn = pymysql.connect(host="host", user="user", passwd="passwd", db="db_name", port=3306) with conn.cursor(cursor=pymysql.cursors.DictCursor) as cursor: cursor.execute(sql) result = cursor.fetchall() cursor.close() conn.close() return result except pymysql.MySQLError as error: cursor.close() conn.close() raise error if __name__ == '__main__': module = sys.modules[__name__] func = getattr(module, 'sendPublishRequest') args = None if len(sys.argv) > 0: args = sys.argv[1:] res = func(*args) print(res)/home/application/jenkins/jenkins_home/scripts/sendReport.py内容,自定义简单逻辑,仅供参考import pymysql import sys import datetime from FineEx.Middleware.Message.pywechat import WorkChatSender def sendMessage(environment, appName, success): info = getAppManager(appName, environment) incharge = info['incharge'].split(',') for item in incharge: if success.lower() == 'true': sendSuccessMessageToChat(appName, environment, item) else: sendFailureMessageToChat(appName, environment, item) def sendSuccessMessageToChat(appName, environment, incharge): corpid = 'corpid' corpsecret = 'corpsecret' agentid = 'agentid' workSender = WorkChatSender(corpid=corpid, corpsecret=corpsecret, agentid=agentid) subject = '应用上线成功' current_time = datetime.datetime.now() current_time_text = current_time.strftime("%Y-%m-%d %H:%M:%S") message = f""" >**发布详情** 系统名称:<font color=\"info\">{appName}</font> 运行环境:<font color=\"info\">{environment}</font> 时 间:<font color=\"info\">{current_time_text}</font> 请尽快安排人员验证! """ workSender.send_markdown(subject + "\n" + message.strip(), touser=incharge) def sendFailureMessageToChat(appName, environment, incharge): corpid = 'corpid' corpsecret = 'corpsecret' agentid = 'agentid' workSender = WorkChatSender(corpid=corpid, corpsecret=corpsecret, agentid=agentid) subject = '应用上线失败' current_time = datetime.datetime.now() current_time_text = current_time.strftime("%Y-%m-%d %H:%M:%S") message = f""" >**发布详情** 系统名称:<font color=\"warning\">{appName}</font> 运行环境:<font color=\"warning\">{environment}</font> 时 间:<font color=\"warning\">{current_time_text}</font> 请尽快安排人员排查! """ workSender.send_markdown(subject + "\n" + message.strip(), touser=incharge) def getAppManager(appName, environment): sql = f""" select `appid`, `appname`, `incharge` from table_release_container where appname='{appName}' and environment = '{environment}' """ try: conn = pymysql.connect(host="host", user="user", passwd="passwd", db="db", port=3306) with conn.cursor(cursor=pymysql.cursors.DictCursor) as cursor: cursor.execute(sql) result = cursor.fetchone() cursor.close() conn.close() return result except pymysql.MySQLError as error: cursor.close() conn.close() raise error if __name__ == '__main__': module = sys.modules[__name__] func = getattr(module, 'sendMessage') args = None if len(sys.argv) > 0: args = sys.argv[1:] res = func(*args) print(res)
- 
编写服务端 from flask import Flask, request, Response import simplejson as json import pymysql import socket import os app = Flask(__name__) @app.route('/publish') def publish(): try: environment = request.args.get('environment') appName = request.args.get('appName') imageVersion = request.args.get('imageVersion') app_config = getAppInfo(environment, appName) for con in app_config: # 生成容器运行命令 cmd = generateCommand(con, con['project'], con['imagename'], imageVersion) # 登录仓库 loginDepository() # 拉取镜像 return_code = imagePull(con['project'], con['imagename'], imageVersion) if return_code == 0: # 停止老容器 stopContainer(appName) # 删除老容器 removeContainer(appName) # 启动新容器 return_code = startContainer(cmd) if return_code == 0: res = { "success": True, "message": f"""{appName}:{imageVersion}在{environment}环境发版成功""" } return Response(json.dumps(res), mimetype='application/json') else: res = { "success": False, "message": f"""{appName}:{imageVersion}在{environment}环境发版失败, 失败编码{return_code}""" } return Response(json.dumps(res), mimetype='application/json') except Exception as e: res = { "success": False, "message": f"""{appName}:{imageVersion}在{environment}环境发版失败,失败信息:{e}""" } return Response(json.dumps(res), mimetype='application/json') def imagePull(projectName, imageName, imageVersion): """ 拉取镜像 :param projectName: 项目名 :param imageName: 镜像名 :param imageVersion: 镜像版本 :return: 是否成功 """ # 修改仓库地址 pull_cmd = f"""docker pull harbor.test.com/{projectName}/{imageName}:{imageVersion}""" result = os.system(pull_cmd) return result def generateCommand(con, projectName, imageName, imageVersion): volumn_mount_sql = f""" select host_path,app_path from table_release_containervolumn where appid ={con['appid']} """ port_map_sql = f""" select host_port,app_port from table_release_containerport where appid = {con['appid']} """ env_parm_sql = f""" select `key`,`value` from table_release_containerenv where appid = {con['appid']} """ try: conn = pymysql.connect(host="host", user="user", passwd="passwd", db="db_name", port=3306) with conn.cursor(cursor=pymysql.cursors.DictCursor) as cursor: # 获取挂载卷配置 cursor.execute(volumn_mount_sql) volumn_mount_result = cursor.fetchall() # 获取端口映射配置 cursor.execute(port_map_sql) port_map_result = cursor.fetchall() # 获取环境变量配置 cursor.execute(env_parm_sql) env_parm_result = cursor.fetchall() volumn_mount_str = " ".join( ["-v " + item['host_path'] + ":" + item['app_path'] for item in volumn_mount_result]) port_map_str = " ".join( ["-p " + str(item['host_port']) + ":" + str(item['app_port']) for item in port_map_result]) env_parm_str = " ".join(["-e " + str(item['key']) + "=" + str(item['value']) for item in env_parm_result]) # 修改仓库地址 command = f"""docker run -{con['mode']} --name {con['appname'].lower()} {port_map_str} {volumn_mount_str} {env_parm_str} harbor.test.com/{projectName}/{imageName}:{imageVersion}""" cursor.close() conn.close() return command except pymysql.MySQLError as error: cursor.close() conn.close() raise error def loginDepository(): # 修改帐号密码 result = os.system("docker login -u User -p Passwor harbor.test.com") print(result) def stopContainer(appName): result = os.system(f"""docker stop {appName.lower()}""") print(result) def removeContainer(appName): result = os.system(f"""docker rm {appName.lower()}""") print(result) def startContainer(cmd): return os.system(cmd) def getLocalIp(): """ 获取本机ip地址 :return: ip地址 """ s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) s.connect(('114.114.114.114', 80)) hostname = s.getsockname()[0] ip_address = socket.gethostbyname(hostname) s.close() return ip_address def getAppInfo(environment, appName): """ 获取应用配置信息 :param environment: 环境 :param appName: 应用名 :return: 应用配置信息 """ ip = getLocalIp() sql = f""" select appid, appname, project, imagename, mode, restart from table_release_container where appname='{appName}' and environment = '{environment}' and ip = '{ip}' """ try: conn = pymysql.connect(host="host", user="user", passwd="passwd", db="db_name", port=3306) with conn.cursor(cursor=pymysql.cursors.DictCursor) as cursor: cursor.execute(sql) result = cursor.fetchall() cursor.close() conn.close() return result except pymysql.MySQLError as error: cursor.close() conn.close() raise error if __name__ == '__main__': app.run(host='0.0.0.0', port=5000, debug=True)用到的 mysql 表结构 CREATE TABLE `table_release_container` ( `appid` int(10) unsigned NOT NULL AUTO_INCREMENT COMMENT '应用id', `appname` varchar(255) NOT NULL COMMENT '应用名称', `project` varchar(255) NOT NULL COMMENT '项目名称(harbor)', `imagename` varchar(255) NOT NULL COMMENT '镜像名称', `type` varchar(255) NOT NULL COMMENT '应用类型 1:站点 2:任务', `environment` varchar(255) NOT NULL COMMENT '环境', `ip` varchar(255) NOT NULL COMMENT '部署的内网ip', `connect` varchar(255) NOT NULL COMMENT '服务暴露的外网ip', `mark` varchar(255) NOT NULL COMMENT '备注', `incharge` varchar(255) NOT NULL COMMENT '应用负责人', `mode` varchar(255) NOT NULL COMMENT '运行模式', `restart` tinyint(4) NOT NULL COMMENT '失败重启', PRIMARY KEY (`appid`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT '应用表' CREATE TABLE `table_release_containerenv` ( `appid` int(11) NOT NULL COMMENT '应用id', `key` varchar(255) NOT NULL COMMENT '环境变量名', `value` varchar(255) NOT NULL COMMENT '环境变量值' ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT '应用运行参数' CREATE TABLE `table_release_containerport` ( `appid` int(11) NOT NULL COMMENT '应用id', `host_port` int(255) NOT NULL COMMENT '宿主机端口', `app_port` int(255) NOT NULL COMMENT '应用端口' ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT '应用映射端口' CREATE TABLE `table_release_containervolumn` ( `appid` int(11) NOT NULL COMMENT '应用id', `host_path` varchar(2000) NOT NULL COMMENT '宿主机目录', `app_path` varchar(2000) NOT NULL COMMENT 'app目录' ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT '应用映射目录'
- 
部署服务端 - 准备 requirements.txt
 aliyun-python-sdk-core==2.14.0 aliyun-python-sdk-dysmsapi==2.1.2 certifi==2023.11.17 cffi==1.15.1 charset-normalizer==2.0.12 click==8.0.4 cryptography==40.0.2 dataclasses==0.8 distlib==0.3.7 docker==5.0.3 Flask==2.0.3 idna==3.6 importlib-metadata==4.8.3 itsdangerous==2.0.1 Jinja2==3.0.3 jmespath==0.10.0 mailutils==0.0.2 MarkupSafe==2.0.1 pycparser==2.21 PyMySQL==1.0.2 requests==2.27.1 simplejson==3.19.2 typing_extensions==4.1.1 urllib3==1.26.18 websocket-client==1.3.1 Werkzeug==2.0.3 zipp==3.6.0- 准备 Dockerfile
 FROM python:3.6-slim-buster LABEL authors="liuchunyang" LABEL email="<liuchunyang@fineex.com>" WORKDIR /flask_app COPY . /flask_app ENV TZ Asia/Shanghai RUN pip install -r ./requirements.txt -i <https://pypi.tuna.tsinghua.edu.cn/simple> EXPOSE 5000 CMD ["python", "/flask_app/develop_server.py"]
- 
打包后运行 docker run -d --net=host --name develop_server -p 5000:5000 -v /var/run/docker.sock:/var/run/docker.sock -v /usr/bin/docker:/usr/bin/docker -v /usr/lib64/libltdl.so.7:/usr/lib/x86_64-linux-gnu/libltdl.so.7 --restart=always --privileged harbor.test.com/fineex_devops/develop_service:1.0.0--net=host使用主机网络,脚本里有逻辑
 -v /var/run/docker.sock:/var/run/docker.sock -v /usr/bin/docker:/usr/bin/docker -v /usr/lib64/libltdl.so.7:/usr/lib/x86_64-linux-gnu/libltdl.so.7使用宿主机 docker
 --privileged使用 root 权限
写在最后
如果Jenkins和应用服务器网络通的情况下,可以直接通过ssh去部署,另外服务端最好加安全验证,这里只是随便搞搞,没写多复杂的逻辑。
如果对你有帮助,且心情愉悦的话,想搞杯速溶咖啡;如果有不理解的地方可以评论或者联系我个人微信
|  |  | 
|---|
 
                     
                    
                 
                    
                
 
                
            
         
         浙公网安备 33010602011771号
浙公网安备 33010602011771号