发布系统之发布流程和发布salt相关命令

    1. 发布流程:
        - 代码发布:
            - git
            - saltstack: state推送文件
            - saltstack: state执行命令
        - 静态文件发布
            - git
            - saltstack: state推送文件
        - SQL发布
            - git
            - saltstack: state推送文件
            - saltstack: state执行命令
    
        注意: 
            1. 保留三个版本
            2. SQL发布:数据量比较大时,编写上线SQL时,只对更改的表做操作;+ 回滚的脚本;
            
    2. 发布salt相关命令:
        代码:
            目录:/srv/salt/code_deply/
                  a. 创建 init.sls
                    # 1. 同步文件
                    sync_file:
                      file.recurse:
                        - name: /data/new_files/{{pillar['file_name']}}       # 同步到所有minion
                        - source: salt://s7code/files/{{pillar['file_name']}} # 新拉下来的代码
                        - user: root
                        - makedirs: True
                        - file_mode: 644
                        - dir_mode: 755
                    sync_reboot_file:
                      file.manage:
                        - name: /data/backup/reboot_{{pillar['file_name']}}.py       # 同步到所有minion
                        - source: salt://s7code/files/reboot_{{pillar['file_name']}}.py # 新拉下来的代码
                        - user: root
                        - makedirs: True
                        - file_mode: 644
                    sync_backup_file:
                      file.manage:
                        - name: /data/backup/backup_{{pillar['file_name']}}.py       # 同步到所有minion
                        - source: salt://s7code/files/backup_{{pillar['file_name']}}.py # 新拉下来的代码
                        - user: root
                        - makedirs: True
                        - file_mode: 644
                        
                    # 2. 备份
                    backup_online:
                      cmd.run:
                        - name: mv /data/{{pillar['file_name']}} /data/backup/{{pillar['file_name']}}-4
                        - require:
                            - file: sync_file
                        
                    # 3. 新上线代码
                    apply_file:
                      cmd.run:
                        - name: mv /data/new_files/{{pillar['file_name']}} /data/{{pillar['file_name']}}
                        - require:
                            - cmd: backup_online
                    
                    # 4. 重启程序    
                    reboot_app:
                      file.run:
                        - name: /usr/bin/python /xx/xx/xxpython脚本.py {{pillar['file_name']}} (将现在运行的进程杀掉,重启)
                        
                    # 5. 备份名称重命名
                    rename_backup_file:
                      cmd.run:
                        - name: /usr/bin/python /data/backup/backup_rename.py {{pillar['file_name']}}(找到/data/backup目录下的所有文件夹,)
        
                    ########################## 或 ####################################
                    # 1. 同步文件
                    sync_file:
                      file.recurse:
                        - name: /data/new_files/video       # 同步到所有minion
                        - source: salt://s7code/files/video # 新拉下来的代码
                        - user: root
                        - makedirs: True
                        - file_mode: 644
                        - dir_mode: 755
                        
                    sync_reboot_backup_file:
                      file.manage:
                        - name: /data/backup/reboot_backup.py      
                        - source: salt://s7code/files/reboot_backup.py 
                        - user: root
                        - makedirs: True
                        - file_mode: 644
                        
                    # 2. 备份
                    backup_online:
                      cmd.run:
                        - name: /usr/bin/python /data/backup/reboot_backup.py
                        
            执行: 
                    salt "c1.com" state.sls code_deply
                    
                    问题:
                        salt "c1.com" state.sls code_deply  pillar='{"file_name":"video"}'
                        salt "c1.com" state.sls code_deply  pillar='{"file_name":"api"}'
                        salt "c1.com" state.sls code_deply  pillar='{"file_name":"vue"}'
                        salt "c1.com" state.sls code_deply  pillar='{"file_name":"..."}'
        
        
        vue: 
            目录:/srv/salt/vue_deply/
                ...
                
            执行: 
                    salt "c1.com" state.sls code_deply
                    
                    
        sql:     
            目录:/srv/salt/sql_deply/
                ...        
                
        
            执行: 
                    salt "c1.com" state.sls code_deply

 

 

        1. linux: salt部署命令
            master:
                - selinux关闭
                - 防火墙关闭
                - srv/salt/s7code/ 
                    - init.sls 
                        sync_file:
                          file.recurse:
                            - name: /data/new/{{pillar['appname']}}             # 同步到每个客户端的
                            - source: salt://s7code/files/{{pillar['appname']}} # master上的代码
                            - user: root
                            - makedirs: True
                            - file_mode: 644
                            - dir_mode: 755
                        sync_online_script:
                          file.managed:
                            - name: /data/script/code_publisher.py 
                            - source: salt://s7code/files/code_publisher.py 
                            - user: root
                            - makedirs: True
                            - file_mode: 644

                        run_online_script:
                          cmd.run:
                            - name: /usr/bin/python /data/script/code_publisher.py {{pillar['appname']}}

                        xxxxxxxx:
                          cmd.run:
                            - name: nohup /usr/bin/python run.py >/dev/null 2>&1 &
                            - cwd: /data/{{pillar['appname']}}

                    - files
                        - luffy
                            - run.py 
                                import flask 
                                ...
                        - code_publiser.py 
                            #!/usr/bin/env python
                            # -*- coding:utf-8 -*-
                            """
                            1. salt state进行文件同步
                            2. 线上代码移动到 程序-4 名称
                            3. 新代码移动到线上目录
                            4. 杀掉原进程&重新启动应用程序
                            5. 修改文件名
                            """
                            import os
                            import sys
                            import signal
                            import shutil
                            from subprocess import check_output, CalledProcessError


                            def online_to_backup4(app_name):
                                """
                                线上代码移动到备份4
                                :return:
                                """

                                # 检查是否存在线上代码(新服务器没有)
                                online_path = '/data/%s' % app_name
                                if not os.path.exists(online_path):
                                    return

                                # 创建备份目录
                                backup_path = "/data/backup/%s/" % app_name
                                if not os.path.exists(backup_path):
                                    os.makedirs(backup_path)

                                # 线上代码移动到备份4
                                shutil.move('/data/%s' % app_name, '/data/backup/%s/%s-4' % (app_name, app_name,))


                            def new_to_online(app_name):
                                """
                                新代码移动到线上
                                :return:
                                """
                                # 检查是否有线上目录,没有则创建
                                online_path = "/data"
                                if not os.path.exists(online_path):
                                    os.makedirs(online_path)
                                # 新代码移动到线上
                                os.rename('/data/new/%s' % app_name, '/data/%s' % (app_name,))
                                # shutil.move(('/data/new/%s' % app_name, '/data/%s' % (app_name,))


                            def rename_backup_name(app_name):
                                """
                                备份文件命名:备份-1、备份-2、备份-3
                                :param app_name:
                                :return:
                                """
                                base_path = "/data/backup/%s" % app_name
                                for i in range(1, 5):
                                    # 1/2/3/4
                                    folder_name = "%s-%d" % (app_name, i,)
                                    folder_path = os.path.join(base_path, folder_name)

                                    if not os.path.exists(folder_path):
                                        continue
                                    if i == 1:
                                        # 删除 xx-1
                                        shutil.rmtree(folder_path)
                                        continue
                                    # 2/3 都减1
                                    new_folder_name = "%s-%d" % (app_name, i - 1,)
                                    new_folder_path = os.path.join(base_path, new_folder_name)

                                    shutil.move(folder_path, new_folder_path)


                            def get_pid_list(name):
                                """
                                获取指定进程PID
                                :param name:
                                :return:
                                """
                                try:
                                    output = check_output("pgrep -f '%s'" % (name), shell=True)
                                except CalledProcessError as ex:
                                    output = None
                                if not output:
                                    return []
                                return map(int, output.split())


                            def kill_9(name):
                                """
                                杀死指定进程
                                :param name:
                                :return:
                                """
                                for pid in get_pid_list(name):
                                    try:
                                        os.kill(pid, signal.SIGKILL)
                                    except OSError as e:
                                        pass


                            def run():
                                # luffy
                                app_name = sys.argv[1]

                                # 拷贝
                                online_to_backup4(app_name)
                                new_to_online(app_name)
                                
                                # 删除第一备份
                                rename_backup_name(app_name)
                                
                                # 杀进程
                                kill_9('/usr/bin/python run.py')

                                


                            if __name__ == '__main__':
                                run()

                        
                salt '*' state.sls s7code  pillar='{"appname": "luffy"}'
            minion
                - .....
                /data/new/
                /data/luffy
                /data/backup/luffy/luffy-l
                /data/backup/luffy/luffy-2
                /data/backup/luffy/luffy-3
                    
            
            master上执行:
                salt '*' state.sls s7code  pillar='{"appname": "luffy"}'
                salt '*' state.sls s7code  pillar='{"appname": "luffy"}'
                salt '*' state.sls s7code  pillar='{"appname": "luffy"}'
                salt '*' state.sls s7code  pillar='{"appname": "luffy"}'

 

posted @ 2018-04-10 09:59  九二零  阅读(266)  评论(0编辑  收藏  举报