mysql主从、wordpress、jenkins、dubbo搭建
StatefulSet
StatefulSet本质是Deployment的一种变体，再v1.9版本已成为GA版本，他为了解决有状态服务的问题，他所管理的的Pod拥有固定Pod名称，启停顺序，再sts中，Pod名字被称为网络标识，还必须使用共享存储。 在deployment中，与之对应的服务是service，而在statusfulset中，与之对应的是headlessservice，即无头服务，与service的区别就是它没有clusterIP，解析它的名称时将返回该Headless Service对应的全部Pod的EndPoint列表 sts特点： 1、每个pod分配唯一且固定网络标识 2、给每个pod分配固定且持久化的外部存储 3、对pod进行有序的部署和扩展 4、对pod进行有序的删除和终止 5、对pod进行有序的自动滚动更新 组成部分 1、Headless Service：用来定义Pod网络标识 2、StatusfulSet：定义具体应用，多少Pod副本，为每个pod定义域名 3、volumeClaimTemplates：存储卷申请模板，创建PVC，指定pvc大小将自动创建pvc，且pvc必须由存储类供应
Mysql主从同步(一主两从)
- 小知识
主节点挂了，重启，还能不能和从节点同步数据？
答：可以，因为主从之间没有数据差，从库仍可以到master去同步数据。
主库起来后，就受到slave的同步数据请求后，再从上次备份的位置开始继续备份（master-info日志读取上次同步位置）
--- apiVersion: v1 kind: PersistentVolume metadata: name: mysql-datadir-1 namespace: magedu spec: capacity: storage: 50Gi accessModes: - ReadWriteOnce nfs: path: /data/k8sdata/magedu/mysql-datadir-1 server: 192.168.157.19 --- apiVersion: v1 kind: PersistentVolume metadata: name: mysql-datadir-2 namespace: magedu spec: capacity: storage: 50Gi accessModes: - ReadWriteOnce nfs: path: /data/k8sdata/magedu/mysql-datadir-2 server: 192.168.157.19 --- apiVersion: v1 kind: PersistentVolume metadata: name: mysql-datadir-3 namespace: magedu spec: capacity: storage: 50Gi accessModes: - ReadWriteOnce nfs: path: /data/k8sdata/magedu/mysql-datadir-3 server: 192.168.157.19
创建configmap
apiVersion: v1 kind: ConfigMap metadata: name: mysql namespace: magedu labels: app: mysql data: master.cnf: | # Apply this config only on the master. [mysqld] log-bin log_bin_trust_function_creators=1 lower_case_table_names=1 slave.cnf: | # Apply this config only on slaves. [mysqld] super-read-only log_bin_trust_function_creators=1
创建mysql-service
-
两个service
1、headless-service 无头服务 2、service：selector用于能够提供读服务的mysql pod
# Headless service for stable DNS entries of StatefulSet members. apiVersion: v1 kind: Service metadata: namespace: magedu name: mysql labels: app: mysql spec: ports: - name: mysql port: 3306 clusterIP: None selector: app: mysql --- # Client service for connecting to any MySQL instance for reads. # For writes, you must instead connect to the master: mysql-0.mysql. apiVersion: v1 kind: Service metadata: name: mysql-read namespace: magedu labels: app: mysql spec: ports: - name: mysql port: 3306 selector: app: mysql
创建mysql-sts
apiVersion: apps/v1 kind: StatefulSet metadata: name: mysql namespace: magedu spec: selector: matchLabels: app: mysql serviceName: mysql replicas: 3 template: metadata: labels: app: mysql spec: initContainers: - name: init-mysql image: 192.168.157.101/magedu/mysql:5.7.36 command: - bash - "-c" - | set -ex # Generate mysql server-id from pod ordinal index. [[ `hostname` =~ -([0-9]+)$ ]] || exit 1 ordinal=${BASH_REMATCH[1]} echo [mysqld] > /mnt/conf.d/server-id.cnf # Add an offset to avoid reserved server-id=0 value. echo server-id=$((100 + $ordinal)) >> /mnt/conf.d/server-id.cnf # Copy appropriate conf.d files from config-map to emptyDir. if [[ $ordinal -eq 0 ]]; then cp /mnt/config-map/master.cnf /mnt/conf.d/ else cp /mnt/config-map/slave.cnf /mnt/conf.d/ fi volumeMounts: - name: conf mountPath: /mnt/conf.d - name: config-map mountPath: /mnt/config-map - name: clone-mysql image: 192.168.157.101/magedu/xtrabackup:1.0 command: - bash - "-c" - | set -ex # Skip the clone if data already exists. [[ -d /var/lib/mysql/mysql ]] && exit 0 # Skip the clone on master (ordinal index 0). [[ `hostname` =~ -([0-9]+)$ ]] || exit 1 ordinal=${BASH_REMATCH[1]} [[ $ordinal -eq 0 ]] && exit 0 # Clone data from previous peer. ncat --recv-only mysql-$(($ordinal-1)).mysql 3307 | xbstream -x -C /var/lib/mysql # Prepare the backup. xtrabackup --prepare --target-dir=/var/lib/mysql volumeMounts: - name: data mountPath: /var/lib/mysql subPath: mysql - name: conf mountPath: /etc/mysql/conf.d containers: - name: mysql image: 192.168.157.101/magedu/mysql:5.7.36 env: - name: MYSQL_ALLOW_EMPTY_PASSWORD value: "1" ports: - name: mysql containerPort: 3306 volumeMounts: - name: data mountPath: /var/lib/mysql subPath: mysql - name: conf mountPath: /etc/mysql/conf.d resources: limits: cpu: 1 memory: 1Gi requests: cpu: 500m memory: 1Gi livenessProbe: exec: command: ["mysqladmin", "ping"] initialDelaySeconds: 30 periodSeconds: 10 timeoutSeconds: 5 readinessProbe: exec: # Check we can execute queries over TCP (skip-networking is off). command: ["mysql", "-h", "127.0.0.1", "-e", "SELECT 1"] initialDelaySeconds: 5 periodSeconds: 2 timeoutSeconds: 1 - name: xtrabackup image: 192.168.157.101/magedu/xtrabackup:1.0 ports: - name: xtrabackup containerPort: 3307 command: - bash - "-c" - | set -ex cd /var/lib/mysql # Determine binlog position of cloned data, if any. if [[ -f xtrabackup_slave_info ]]; then # XtraBackup already generated a partial "CHANGE MASTER TO" query # because we're cloning from an existing slave. mv xtrabackup_slave_info change_master_to.sql.in # Ignore xtrabackup_binlog_info in this case (it's useless). rm -f xtrabackup_binlog_info elif [[ -f xtrabackup_binlog_info ]]; then # We're cloning directly from master. Parse binlog position. [[ `cat xtrabackup_binlog_info` =~ ^(.*?)[[:space:]]+(.*?)$ ]] || exit 1 rm xtrabackup_binlog_info echo "CHANGE MASTER TO MASTER_LOG_FILE='${BASH_REMATCH[1]}',\ MASTER_LOG_POS=${BASH_REMATCH[2]}" > change_master_to.sql.in fi # Check if we need to complete a clone by starting replication. if [[ -f change_master_to.sql.in ]]; then echo "Waiting for mysqld to be ready (accepting connections)" until mysql -h 127.0.0.1 -e "SELECT 1"; do sleep 1; done echo "Initializing replication from clone position" # In case of container restart, attempt this at-most-once. mv change_master_to.sql.in change_master_to.sql.orig mysql -h 127.0.0.1 <<EOF $(<change_master_to.sql.orig), MASTER_HOST='mysql-0.mysql', MASTER_USER='root', MASTER_PASSWORD='', MASTER_CONNECT_RETRY=10; START SLAVE; EOF fi # Start a server to send backups when requested by peers. exec ncat --listen --keep-open --send-only --max-conns=1 3307 -c \ "xtrabackup --backup --slave-info --stream=xbstream --host=127.0.0.1 --user=root" volumeMounts: - name: data mountPath: /var/lib/mysql subPath: mysql - name: conf mountPath: /etc/mysql/conf.d resources: limits: cpu: 400m memory: 500Mi requests: cpu: 100m memory: 100Mi volumes: - name: conf emptyDir: {} - name: config-map configMap: name: mysql volumeClaimTemplates: - metadata: name: data spec: accessModes: ["ReadWriteOnce"] resources: requests: storage: 10Gi
-
initcontainers中执行操作解释
- bash - "-c" - | set -ex # 查看pod的名字是否规范，不规范则退出并报错 [[ `hostname` =~ -([0-9]+)$ ]] || exit 1 ordinal=${BASH_REMATCH[1]} ## 将匹配到的hostname内容赋值给ordinal，通过pod名区分去角色 ## 初始化mysql的serverid echo [mysqld] > /mnt/conf.d/server-id.cnf # Add an offset to avoid reserved server-id=0 value. echo server-id=$((100 + $ordinal)) >> /mnt/conf.d/server-id.cnf # Copy appropriate conf.d files from config-map to emptyDir. # master和slave角色的配置文件分pod放置 if [[ $ordinal -eq 0 ]]; then cp /mnt/config-map/master.cnf /mnt/conf.d/ else cp /mnt/config-map/slave.cnf /mnt/conf.d/ fi --------------------------------------- - bash - "-c" - | set -ex # Skip the clone if data already exists. # 若有数据则跳过同步 [[ -d /var/lib/mysql/mysql ]] && exit 0 # Skip the clone on master (ordinal index 0). # 查看当前是哪个角色，若不符合规范则退出 [[ `hostname` =~ -([0-9]+)$ ]] || exit 1 ordinal=${BASH_REMATCH[1]} # 如果为master则退出 [[ $ordinal -eq 0 ]] && exit 0 # Clone data from previous peer. # 从上个对等方克隆数据 ncat --recv-only mysql-$(($ordinal-1)).mysql 3307 | xbstream -x -C /var/lib/mysql # Prepare the backup. xtrabackup --prepare --target-dir=/var/lib/mysql
-
container中的操作
- bash - "-c" - | set -ex cd /var/lib/mysql ## 初始化mysql-slave的change master命令 # Determine binlog position of cloned data, if any. if [[ -f xtrabackup_slave_info ]]; then # XtraBackup already generated a partial "CHANGE MASTER TO" query # because we're cloning from an existing slave. mv xtrabackup_slave_info change_master_to.sql.in # Ignore xtrabackup_binlog_info in this case (it's useless). rm -f xtrabackup_binlog_info elif [[ -f xtrabackup_binlog_info ]]; then # We're cloning directly from master. Parse binlog position. [[ `cat xtrabackup_binlog_info` =~ ^(.*?)[[:space:]]+(.*?)$ ]] || exit 1 rm xtrabackup_binlog_info echo "CHANGE MASTER TO MASTER_LOG_FILE='${BASH_REMATCH[1]}',\ MASTER_LOG_POS=${BASH_REMATCH[2]}" > change_master_to.sql.in fi # Check if we need to complete a clone by starting replication. # 若初始化文件存在则进行下一步，进行slave绑定master的操作 if [[ -f change_master_to.sql.in ]]; then ## 等待mysql服务器连接成功 echo "Waiting for mysqld to be ready (accepting connections)" until mysql -h 127.0.0.1 -e "SELECT 1"; do sleep 1; done echo "Initializing replication from clone position" # In case of container restart, attempt this at-most-once. mv change_master_to.sql.in change_master_to.sql.orig ## 在slave的mysql实例中执行change master操作 mysql -h 127.0.0.1 <<EOF $(<change_master_to.sql.orig), MASTER_HOST='mysql-0.mysql', MASTER_USER='root', MASTER_PASSWORD='', MASTER_CONNECT_RETRY=10; START SLAVE; EOF fi # Start a server to send backups when requested by peers. exec ncat --listen --keep-open --send-only --max-conns=1 3307 -c \ "xtrabackup --backup --slave-info --stream=xbstream --host=127.0.0.1 --user=root"
-
进入master查看,创建数据库
-
进入mysql-1查看状态
-
进入mysql-2查看状态
主从关系正常数据库已同步
#Jenkins Version 2.190.1 FROM 192.168.157.101/magedu/jdk-base:v8.212 MAINTAINER zhangshijie zhangshijie@magedu.net ADD jenkins-2.319.2.war /apps/jenkins/jenkins.war ADD run_jenkins.sh /usr/bin/ EXPOSE 8080 CMD ["/usr/bin/run_jenkins.sh"]
-
run_jenkins.sh
#!/bin/bash cd /apps/jenkins && java -server -Xms1024m -Xmx1024m -Xss512k -jar jenkins.war --webroot=/apps/jenkins/jenkins-data --httpPort=8080
1、 /root/.jenkins 插件、配置文件目录 2、 /apps/jenkins/jenkins-data/ ## 自定义数据目录
--- apiVersion: v1 kind: PersistentVolume metadata: name: jenkins-datadir-pv namespace: magedu spec: capacity: storage: 100Gi accessModes: - ReadWriteOnce nfs: server: 192.168.157.19 path: /data/k8sdata/magedu/jenkins-data --- apiVersion: v1 kind: PersistentVolume metadata: name: jenkins-root-datadir-pv namespace: magedu spec: capacity: storage: 100Gi accessModes: - ReadWriteOnce nfs: server: 192.168.157.19 path: /data/k8sdata/magedu/jenkins-root-data
-
创建pvc
--- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: jenkins-datadir-pvc namespace: magedu spec: volumeName: jenkins-datadir-pv accessModes: - ReadWriteOnce resources: requests: storage: 80Gi --- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: jenkins-root-data-pvc namespace: magedu spec: volumeName: jenkins-root-datadir-pv accessModes: - ReadWriteOnce resources: requests: storage: 80Gi
部署jenkins-deployment
kind: Deployment #apiVersion: extensions/v1beta1 apiVersion: apps/v1 metadata: labels: app: magedu-jenkins name: magedu-jenkins-deployment namespace: magedu spec: replicas: 1 selector: matchLabels: app: magedu-jenkins template: metadata: labels: app: magedu-jenkins spec: containers: - name: magedu-jenkins-container image: 192.168.157.101/magedu/jenkins:v2.319.2 #imagePullPolicy: IfNotPresent imagePullPolicy: Always ports: - containerPort: 8080 protocol: TCP name: http volumeMounts: - mountPath: "/apps/jenkins/jenkins-data/" name: jenkins-datadir-magedu - mountPath: "/root/.jenkins" name: jenkins-root-datadir volumes: - name: jenkins-datadir-magedu persistentVolumeClaim: claimName: jenkins-datadir-pvc - name: jenkins-root-datadir persistentVolumeClaim: claimName: jenkins-root-data-pvc --- kind: Service apiVersion: v1 metadata: labels: app: magedu-jenkins name: magedu-jenkins-service namespace: magedu spec: type: NodePort ports: - name: http port: 80 protocol: TCP targetPort: 8080 nodePort: 38080 selector: app: magedu-jenkins
访问 节点ip:nodePord
-
进入pod获取密码或直接在nfs服务器中获取密码
kubectl exec -itmagedu-jenkins-deployment-6c4c58d697-bgx24 bash
cat /root/.jenkins/secrets/initialAdminPassword
-
安装推荐的插件（有能力可自己选择所需插件）
部署wordpress
构建nginx镜像
- dockerfile
FROM 192.168.157.101/magedu/nginx-base:v1.20.2 ADD nginx.conf /apps/nginx/conf/nginx.conf ADD run_nginx.sh /apps/nginx/sbin/run_nginx.sh RUN mkdir -pv /home/nginx/wordpress RUN useradd -u 2088 nginx && chown nginx.nginx /home/nginx/wordpress/ -R EXPOSE 80 443 CMD ["/apps/nginx/sbin/run_nginx.sh"]
#!/bin/bash #echo "nameserver 10.20.254.254" > /etc/resolv.conf #chown nginx.nginx /home/nginx/wordpress/ -R /usr/sbin/nginx tail -f /etc/hosts
-
nginx.conf
user nginx nginx; worker_processes auto; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; #access_log logs/access.log main; sendfile on; keepalive_timeout 65; client_max_body_size 10M; client_body_buffer_size 16k; client_body_temp_path /apps/nginx/tmp 1 2 2; gzip on; server { listen 80; server_name blogs.magedu.net; location / { root /home/nginx/wordpress; index index.php index.html index.htm; } location ~ \.php$ { root /home/nginx/wordpress; fastcgi_pass 127.0.0.1:9000; fastcgi_index index.php; #fastcgi_param SCRIPT_FILENAME /scripts$fastcgi_script_name; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; include fastcgi_params; } error_page 500 502 503 504 /50x.html; location = /50x.html { root html; } } }
- dockerfile
#PHP Base Image FROM 192.168.157.101/library/magedu-centos-base:7.9.2009 MAINTAINER zhangshijie@magedu.net RUN yum install -y https://mirrors.tuna.tsinghua.edu.cn/remi/enterprise/remi-release-7.rpm && yum install php56-php-fpm php56-php-mysql -y && useradd nginx -u 2088 ADD www.conf /opt/remi/php56/root/etc/php-fpm.d/www.conf #RUN useradd nginx -u 2019 ADD run_php.sh /usr/local/bin/run_php.sh EXPOSE 9000 CMD ["/usr/local/bin/run_php.sh"]
-
/usr/local/bin/run_php.sh
#!/bin/bash #echo "nameserver 10.20.254.254" > /etc/resolv.conf /opt/remi/php56/root/usr/sbin/php-fpm #/opt/remi/php56/root/usr/sbin/php-fpm --nodaemonize tail -f /etc/hosts
在共享文件夹上传wordpress包
- 到wordpress官网下载包，上传到nfs服务器
-
在数据库中创建wordpress数据库
安装时，数据库目录使用mysql-0
采用无头服务的方式去获取mysql-0
mysql-0.mysql.magedu.svc.cluster.local
-
在nfs服务器中创建nginx用户，给共享目录权限（用户id所有服务器保持一致）
useradd nginx -u2088 chown -R 2088.2088 wordpress
部署dubbo
1、将代码压缩包打开
2、修改配置文件(修改为正确的zk地址)
vim dubbo-demo-provider-2.1.5/conf/dubbo.properties dubbo.registry.address=zookeeper://zookeeper1.magedu.svc.cluster.local:2181 | zookeeper://zookeeper2.magedu.svc.cluster.local:2181 | zookeeper://zookeeper3.magedu.svc.cluster.local:2181
3、构建镜像
#Dubbo provider FROM 192.168.157.101/magedu/jdk-base:v8.212 MAINTAINER zhangshijie "zhangshijie@magedu.net" RUN yum install file nc -y RUN mkdir -p /apps/dubbo/provider ADD dubbo-demo-provider-2.1.5/ /apps/dubbo/provider ADD run_java.sh /apps/dubbo/provider/bin RUN useradd nginx -u 2088 && chown nginx.nginx /apps -R RUN chmod a+x /apps/dubbo/provider/bin/*.sh CMD ["/apps/dubbo/provider/bin/run_java.sh"]
4、启动服务
kind: Deployment #apiVersion: extensions/v1beta1 apiVersion: apps/v1 metadata: labels: app: magedu-provider name: magedu-provider-deployment namespace: magedu spec: replicas: 1 selector: matchLabels: app: magedu-provider template: metadata: labels: app: magedu-provider spec: containers: - name: magedu-provider-container image: 192.168.157.101/magedu/dubbo-demo-provider:v1 #command: ["/apps/tomcat/bin/run_tomcat.sh"] #imagePullPolicy: IfNotPresent imagePullPolicy: Always ports: - containerPort: 20880 protocol: TCP name: http --- kind: Service apiVersion: v1 metadata: labels: app: magedu-provider name: magedu-provider-spec namespace: magedu spec: type: NodePort ports: - name: http port: 80 protocol: TCP targetPort: 20880 #nodePort: 30001 selector: app: magedu-provider
1、修改zk地址
vim dubbo-demo-consumer-2.1.5/conf/dubbo.properties dubbo.registry.address=zookeeper://zookeeper1.magedu.svc.cluster.local:2181 | zookeeper://zookeeper2.magedu.svc.cluster.local:2181 | zookeeper://zookeeper3.magedu.svc.cluster.local:2181
2、构建镜像
FROM 192.168.157.101/magedu/jdk-base:v8.212 MAINTAINER zhangshijie "zhangshijie@magedu.net" RUN yum install file -y RUN mkdir -p /apps/dubbo/consumer ADD dubbo-demo-consumer-2.1.5 /apps/dubbo/consumer ADD run_java.sh /apps/dubbo/consumer/bin RUN useradd nginx -u 2088 && chown nginx.nginx /apps -R RUN chmod a+x /apps/dubbo/consumer/bin/*.sh CMD ["/apps/dubbo/consumer/bin/run_java.sh"]
3、部署
kind: Deployment #apiVersion: extensions/v1beta1 apiVersion: apps/v1 metadata: labels: app: magedu-consumer name: magedu-consumer-deployment namespace: magedu spec: replicas: 1 selector: matchLabels: app: magedu-consumer template: metadata: labels: app: magedu-consumer spec: containers: - name: magedu-consumer-container image: 192.168.157.101/magedu/dubbo-demo-consumer:v1 #command: ["/apps/tomcat/bin/run_tomcat.sh"] #imagePullPolicy: IfNotPresent imagePullPolicy: Always ports: - containerPort: 80 protocol: TCP name: http --- kind: Service apiVersion: v1 metadata: labels: app: magedu-consumer name: magedu-consumer-server namespace: magedu spec: type: NodePort ports: - name: http port: 80 protocol: TCP targetPort: 80 #nodePort: 30001 selector: app: magedu-consumer
-
进入消费者pod
tail -f /apps/dubbo/consumer/logs/stdout.log
服务正常
1、修改配置文件
vim dubboadmin/WEB-INF/dubbo.properties dubbo.registry.address=zookeeper://zookeeper1.magedu.svc.cluster.local:2181
2、重新生成war包
zip -r dubboadmin.war dubboadmin
3、构建镜像
#Dubbo dubboadmin #FROM harbor.magedu.local/pub-images/tomcat-base:v8.5.43 FROM 192.168.157.101/magedu/tomcat-base:v8.5.43 MAINTAINER zhangshijie "zhangshijie@magedu.net" RUN yum install unzip -y ADD server.xml /apps/tomcat/conf/server.xml ADD logging.properties /apps/tomcat/conf/logging.properties ADD catalina.sh /apps/tomcat/bin/catalina.sh ADD run_tomcat.sh /apps/tomcat/bin/run_tomcat.sh ADD dubboadmin.war /data/tomcat/webapps/dubboadmin.war RUN cd /data/tomcat/webapps && unzip dubboadmin.war && rm -rf dubboadmin.war && chown -R nginx.nginx /data /apps EXPOSE 8080 8443 CMD ["/apps/tomcat/bin/run_tomcat.sh"]
4、部署
kind: Deployment #apiVersion: extensions/v1beta1 apiVersion: apps/v1 metadata: labels: app: magedu-dubboadmin name: magedu-dubboadmin-deployment namespace: magedu spec: replicas: 1 selector: matchLabels: app: magedu-dubboadmin template: metadata: labels: app: magedu-dubboadmin spec: containers: - name: magedu-dubboadmin-container image: 192.168.157.101/magedu/dubboadmin:v1 #command: ["/apps/tomcat/bin/run_tomcat.sh"] #imagePullPolicy: IfNotPresent imagePullPolicy: Always ports: - containerPort: 8080 protocol: TCP name: http --- kind: Service apiVersion: v1 metadata: labels: app: magedu-dubboadmin name: magedu-dubboadmin-service namespace: magedu spec: type: NodePort ports: - name: http port: 80 protocol: TCP targetPort: 8080 selector: app: magedu-dubboadmin
登陆查看
浙公网安备 33010602011771号