MySQL集群高可用

MySQL高可用 Galera Cluster

Galera Cluster

Galera Cluster:集成了Galera插件的MySQL集群,是一种新型的,数据不共享的,高度冗余的高可用方案,目前Galera Cluster有两个版本,分别是Percona Xtradb Cluster及MariaDB Cluster,Galera本身是具有多主特性的,即采用multi-master的集群架构,是一个既稳健,又在数据一致性、完整性及高性能方面有出色表现的高可用解决方案

在这里插入图片描述

Galera Cluster特点

• 多主架构:真正的多点读写的集群,在任何时候读写数据,都是最新的

• 同步复制:集群不同节点之间数据同步,没有延迟,在数据库挂掉之后,数据不会丢失

• 并发复制:从节点APPLY数据时,支持并行执行,更好的性能

• 故障切换:在出现数据库故障时,因支持多点写入,切换容易

• 热插拔:在服务期间,如果数据库挂了,只要监控程序发现的够快,不可服务时间就会非常少。在节点故障期间,节点本身对集群的影响非常小

• 自动节点克隆:在新增节点,或者停机维护时,增量数据或者基础数据不需要人工手动备份提供,Galera Cluster会自动拉取在线节点数据,最终集群会变为一致

• 对应用透明:集群的维护,对应用程序是透明的
只支持innodb存储引擎的表

Galera Cluster 缺点

• 由于DDL 需全局验证通过,则集群性能由集群中最差性能节点决定(一般集群节点配置都是一样的)

• 新节点加入或延后较大的节点重新加入需全量拷贝数据(SST,State Snapshot Transfer),作为donor( 贡献者,如: 同步数据时的提供者)的节点在同步过程中无法提供读写

• 只支持innodb存储引擎的表

Galera Cluster工作过程

在这里插入图片描述

Galera Cluster官方文档

http://galeracluster.com/documentation-webpages/galera-documentation.pdf
http://galeracluster.com/documentation-webpages/index.html
https://www.percona.com/doc/percona-xtradb-cluster/LATEST/index.html
https://mariadb.com/kb/en/library/getting-started-with-mariadb-galera-cluster/

Galera Cluster包括两个组件

• Galera replication library (galera-3)
• WSREP:MySQL extended with the Write Set Replication

WSREP复制实现

  • PXC:Percona XtraDB Cluster,是Percona对Galera的实现
    参考仓库:
https://mirrors.tuna.tsinghua.edu.cn/percona/release/$releasever/RPMS/$basearch
  • MariaDB Galera Cluster:
    参考仓库:
https://mirrors.tuna.tsinghua.edu.cn/mariadb/mariadb-5.5.X/yum/centos7-amd64/
注意:两者都需要至少三个节点,不能安装mysql server 或 mariadb-server

PXC 原理

在这里插入图片描述

PXC最常使用如下4个端口号:

• 3306:数据库对外服务的端口号
• 4444:请求SST的端口号
• 4567:组成员之间进行沟通的端口号
• 4568:用于传输IST的端口号

PXC中涉及到的重要概念和核心参数:

  1. 集群中节点的数量:整个集群中节点数量应该控制在最少3个、最多8个的范围内。最少3个节点是为了防止出现脑裂现象,因为只有在2个节点下才会出现此现象。脑裂现象的标志就是输入任何命令,返回的结果都是unknown command。节点在集群中,会因新节点的加入或故障、同步失效等原因发生状态的切换。
  2. 节点状态的变化阶段:
    在这里插入图片描述
• open:节点启动成功,尝试连接到集群时的状态
• primary:节点已处于集群中,在新节点加入并选取donor进行数据同步时的状态
• joiner:节点处于等待接收同步文件时的状态
• joined:节点完成数据同步工作,尝试保持和集群进度一致时的状态
• synced:节点正常提供服务时的状态,表示已经同步完成并和集群进度保持一致
• donor:节点处于为新加入的节点提供全量数据时的状态  
备注:donor节点就是数据的贡献者,如果一个新节点加入集群,此时又需要大量数据的SST数据传输,就有可能因此而拖垮整个集群的性能,所以在生产环境中,如果数据量较小,还可以使用SST全量数据传输,但如果数据量很大就不建议使用这种方式,可以考虑先建立主从关系,然后再加入集群。
  1. 节点的数据传输方式:
• SST:State Snapshot Transfer,全量数据传输
• IST:Incremental State Transfer,增量数据传输 
SST数据传输有xtrabackup、mysqldump和rsync三种方式,而增量数据传输就只有一种方式xtrabackup,但生产环境中一般数据量较小时,可以使用SST全量数据传输,但也只使用xtrabackup方法。
  1. GCache模块:
在PXC中一个特别重要的模块,它的核心功能就是为每个节点缓存当前最新的写集。如果有新节点加入进来,就可以把新数据的增量传递给新节点,而不需要再使用SST传输方式,这样可以让节点更快地加入集群中
  1. 涉及如下参数:
• gcache.size:缓存写集增量信息的大小,它的默认大小是128MB,通过wsrep_provider_options参数设置,建议调整为2GB~4GB范围,足够的空间便于缓存更多的增量信息。
• gcache.mem_size:GCache中内存缓存的大小,适度调大可以提高整个集群的性能
• gcache.page_size:如果内存不够用(GCache不足),就直接将写集写入磁盘文件中

实战案例:Percona XtraDB Cluster(PXC 5.7)

  1. 环境准备

    四台主机:

    pxc1:192.168.39.7
    pxc1:192.168.39.27
    pxc1:192.168.39.37
    pxc4:192.168.39.47

       OS版本

    [root@pxc1 ~]#cat /etc/redhat-release 
    CentOS Linux release 7.6.1810 (Core) 
    关闭防火墙和SELinux,保证时间同步
    注意:如果已经安装MySQL,必须卸载
  2. 安装 Percona XtraDB Cluster 5.7
  • 配置yum源
[root@pxc1 ~]#vim /etc/yum.repos.d/percona_pxc.repo

[percona]
name=percona_repo
baseurl=https://mirrors.tuna.tsinghua.edu.cn/percona/release/$releasever/RPMS/$basearch
enabled=1
gpgcheck=0
  • 其他三台虚拟机也配置yum源
scp /etc/yum.repos.d/percona_pxc.repo 192.168.39.27:/etc/yum.repos.d/
scp /etc/yum.repos.d/percona_pxc.repo 192.168.39.37:/etc/yum.repos.d/
scp /etc/yum.repos.d/percona_pxc.repo 192.168.39.47:/etc/yum.repos.d/
  • 在三个节点都安装好PXC 5.7
[root@pxc1 ~]#yum install Percona-XtraDB-Cluster-57 -y
[root@pxc2 ~]#yum install Percona-XtraDB-Cluster-57 -y
[root@pxc3 ~]#yum install Percona-XtraDB-Cluster-57 -y
[root@pxc4 ~]#yum install Percona-XtraDB-Cluster-57 -y
  1. 在各个节点上分别配置mysql及集群配置文件
/etc/my.cnf为主配置文件,当前版本中,其余的配置文件都放在/etc/percona-xtradb-cluster.conf.d目录里,包括mysqld.cnf,mysqld_safe.cnf,wsrep.cnf 三个文件
#主配置文件不需要修改
[root@pxc1 ~]#cat /etc/my.cnf
#
# The Percona XtraDB Cluster 5.7 configuration file.
#
#
# * IMPORTANT: Additional settings that can override those from this file!
#   The files must end with '.cnf', otherwise they'll be ignored.
#   Please make any edits and changes to the appropriate sectional files
#   included below.
#
!includedir /etc/my.cnf.d/
!includedir /etc/percona-xtradb-cluster.conf.d/

[root@pxc1 ~]#ls /etc/percona-xtradb-cluster.conf.d/  # 查看一下三个文件
mysqld.cnf  mysqld_safe.cnf  wsrep.cnf

#下面配置文件不需要修改
[root@pxc1 ~]#cat /etc/percona-xtradb-cluster.conf.d/mysqld.cnf 
# Template my.cnf for PXC
# Edit to your requirements.
[client]
socket=/var/lib/mysql/mysql.sock

[mysqld]
server-id=1   # 建议各个节点不同
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
log-bin   # 建议启用,非必须项
log_slave_updates
expire_logs_days=7

# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0

#下面配置文件不需要修改
[root@pxc1 ~]#cat /etc/percona-xtradb-cluster.conf.d/mysqld_safe.cnf 
#
# The Percona Server 5.7 configuration file.
#
# One can use all long options that the program supports.
# Run program with --help to get a list of available options and with
# --print-defaults to see which it would actually understand and use.
#
# For explanations see
# http://dev.mysql.com/doc/mysql/en/server-system-variables.html

[mysqld_safe]
pid-file = /var/run/mysqld/mysqld.pid
socket   = /var/lib/mysql/mysql.sock
nice     = 0

#PXC的配置文件必须修改
[mysqld]
wsrep_provider=/usr/lib64/galera3/libgalera_smm.so
wsrep_cluster_address=gcomm://10.0.0.7,10.0.0.17,10.0.0.27 #三个节点的IP
binlog_format=ROW
default_storage_engine=InnoDB
wsrep_slave_threads= 8
wsrep_log_conflicts
innodb_autoinc_lock_mode=2
wsrep_node_address=10.0.0.7 #各个节点,指定自已的IP
wsrep_cluster_name=pxc-cluster   # 集群名称
wsrep_node_name=pxc-cluster-node-1 #各个节点,指定自已节点名称
pxc_strict_mode=ENFORCING
wsrep_sst_method=xtrabackup-v2
wsrep_sst_auth="sstuser:centos7" #取消本行注释,可以改一个自己的密码

# 根据以上修改信息所有主机都修改
[root@pxc2 ~]#vim /etc/percona-xtradb-cluster.conf.d/wsrep.cnf  
[root@pxc3 ~]#vim /etc/percona-xtradb-cluster.conf.d/wsrep.cnf

[root@pxc1 ~]#grep -Ev "^#|^$" /etc/percona-xtradb-cluster.conf.d/wsrep.cnf # 通过这个命令查看配置文件
[mysqld]
wsrep_provider=/usr/lib64/galera3/libgalera_smm.so
wsrep_cluster_address=gcomm://192.168.39.7,192.168.39.27,192.168.39.37
binlog_format=ROW
default_storage_engine=InnoDB
wsrep_slave_threads= 8
wsrep_log_conflicts
innodb_autoinc_lock_mode=2
wsrep_node_address=192.168.39.7
wsrep_cluster_name=pxc-cluster
wsrep_node_name=pxc-cluster-node-1
pxc_strict_mode=ENFORCING
wsrep_sst_method=xtrabackup-v2
wsrep_sst_auth="sstuser:centos7"
注意:尽管Galera Cluster不再需要通过binlog的形式进行同步,但还是建议在配置文件中开启二进制日志功能,原因是后期如果有新节点需要加入,老节点通过SST全量传输的方式向新节点传输数据,很可能会拖垮集群性能,所以让新节点先通过binlog方式完成同步后再加入集群会是一种更好的选择
  1. 配置文件各项配置意义
    在这里插入图片描述
    在这里插入图片描述
[root@pxc1 ~]#ss -ntul
Netid State      Recv-Q Send-Q                                               Local Address:Port                                                              Peer Address:Port              
udp   UNCONN     0      0                                                    192.168.122.1:53                                                                           *:*                  
udp   UNCONN     0      0                                                         *%virbr0:67                                                                           *:*                  
udp   UNCONN     0      0                                                                *:111                                                                          *:*                  
udp   UNCONN     0      0                                                                *:5353                                                                         *:*                  
udp   UNCONN     0      0                                                                *:58153                                                                        *:*                  
udp   UNCONN     0      0                                                                *:812                                                                          *:*                  
udp   UNCONN     0      0                                                               :::111                                                                         :::*                  
udp   UNCONN     0      0                                                               :::812                                                                         :::*                  
tcp   LISTEN     0      128                                                              *:111                                                                          *:*                  
tcp   LISTEN     0      128                                                              *:6000                                                                         *:*                  
tcp   LISTEN     0      5                                                    192.168.122.1:53                                                                           *:*                  
tcp   LISTEN     0      128                                                              *:22                                                                           *:*                  
tcp   LISTEN     0      128                                                      127.0.0.1:631                                                                          *:*                  
tcp   LISTEN     0      128                                                      127.0.0.1:6010                                                                         *:*                  
tcp   LISTEN     0      128                                                             :::111                                                                         :::*                  
tcp   LISTEN     0      128                                                             :::6000                                                                        :::*                  
tcp   LISTEN     0      128                                                             :::22                                                                          :::*                  
tcp   LISTEN     0      128                                                            ::1:631                                                                         :::*                  
tcp   LISTEN     0      128                                                            ::1:6010                                                                        :::*            

#启动第一人节点(第一个节点启动和别的节点启动都不一样)属于创建集群的服务器现在阶段可以理解为主服务器,但是集群做完之后就没有主从之分了都是主
[root@pxc1 ~]#systemctl start mysql@bootstrap.service
[root@pxc1 ~]#ss -ntul  # 以下两个端口启动起来就行了 (3306、4567)
。。。。。。。。。。。。。。。。。。。。。。。
Netid  State      Recv-Q Send-Q Local Address:Port               Peer Address:Port              
udp    UNCONN     0      0      192.168.122.1:53                         *:*                  
udp    UNCONN     0      0       *%virbr0:67                         *:*                  
udp    UNCONN     0      0              *:111                        *:*                  
udp    UNCONN     0      0              *:5353                       *:*                  
udp    UNCONN     0      0              *:58153                      *:*                  
udp    UNCONN     0      0              *:812                        *:*                  
udp    UNCONN     0      0             :::111                       :::*                  
udp    UNCONN     0      0             :::812                       :::*                  
tcp    LISTEN     0      128            *:111                        *:*                  
tcp    LISTEN     0      128            *:6000                       *:*                  
tcp    LISTEN     0      5      192.168.122.1:53                         *:*                  
tcp    LISTEN     0      128            *:22                         *:*                  
tcp    LISTEN     0      128            *:4567                       *:*           
tcp    LISTEN     0      128    127.0.0.1:631                        *:*                  
tcp    LISTEN     0      128    127.0.0.1:6010                       *:*                  
tcp    LISTEN     0      80            :::3306                      :::*           
tcp    LISTEN     0      128           :::111                       :::*                  
tcp    LISTEN     0      128           :::6000                      :::*                  
tcp    LISTEN     0      128           :::22                        :::*                  
tcp    LISTEN     0      128          ::1:631                       :::*                  
tcp    LISTEN     0      128          ::1:6010                      :::*                  
  1. 找到MySQL的临时密码,第一次登录要修改要不执行不了管理操作。
[root@centos7 ~]#grep "password" /var/log/mysqld.log 
2019-12-03T06:13:39.187929Z 1 [Note] A temporary password is generated for root@localhost: a>#;L(Sm4Ln:

[root@centos7 ~]#mysql -uroot -p'a>#;L(Sm4Ln:'   # 需要加引号引起来
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 11
Server version: 5.7.27-30-57-log

Copyright (c) 2009-2019 Percona LLC and/or its affiliates
Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases; # 执行查看库操作提示必须重置密码
ERROR 1820 (HY000): You must reset your password using ALTER USER statement before executing this statement. # 在执行此语句之前,必须使用ALTER USER语句重置密码.

mysql> alter user root@'localhost' identified by 'magedu';  # 修改密码(其他节点不用修改加入集群以后会自动同步密码)
Query OK, 0 rows affected (0.00 sec)
mysql> exit
Bye
[root@centos7 ~]#mysql -uroot -pmagedu  # 修改以后重新登录
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 12
Server version: 5.7.27-30-57-log Percona XtraDB Cluster (GPL), Release rel30, Revision 64987d4, WSREP version 31.39, wsrep_31.39

Copyright (c) 2009-2019 Percona LLC and/or its affiliates
Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>
  1. 查看变量
# 查看写集复制相关变量
mysql> SHOW VARIABLES LIKE 'wsrep%'\G
*************************** 1. row ***************************
Variable_name: wsrep_OSU_method
        Value: TOI
*************************** 2. row ***************************
Variable_name: wsrep_RSU_commit_timeout
        Value: 5000
*************************** 3. row ***************************
Variable_name: wsrep_auto_increment_control
        Value: ON
*************************** 4. row ***************************
Variable_name: wsrep_causal_reads
        Value: OFF
*************************** 5. row ***************************
Variable_name: wsrep_certification_rules
        Value: strict
*************************** 6. row ***************************
Variable_name: wsrep_certify_nonPK
        Value: ON
*************************** 7. row ***************************
Variable_name: wsrep_cluster_address
        Value: gcomm://192.168.39.7,192.168.39.27,192.168.39.37
*************************** 8. row ***************************
Variable_name: wsrep_cluster_name
        Value: pxc-cluster
*************************** 9. row ***************************
Variable_name: wsrep_convert_LOCK_to_trx
        Value: OFF
*************************** 10. row ***************************
Variable_name: wsrep_data_home_dir
        Value: /var/lib/mysql/
*************************** 11. row ***************************
Variable_name: wsrep_dbug_option
        Value: 
*************************** 12. row ***************************
Variable_name: wsrep_debug
        Value: OFF
*************************** 13. row ***************************
Variable_name: wsrep_desync
        Value: OFF
*************************** 14. row ***************************
Variable_name: wsrep_dirty_reads
        Value: OFF
*************************** 15. row ***************************
Variable_name: wsrep_drupal_282555_workaround
        Value: OFF
*************************** 16. row ***************************
Variable_name: wsrep_forced_binlog_format
        Value: NONE
*************************** 17. row ***************************
Variable_name: wsrep_load_data_splitting
        Value: ON
*************************** 18. row ***************************
Variable_name: wsrep_log_conflicts
        Value: ON
*************************** 19. row ***************************
Variable_name: wsrep_max_ws_rows
        Value: 0
*************************** 20. row ***************************
Variable_name: wsrep_max_ws_size
        Value: 2147483647
*************************** 21. row ***************************
Variable_name: wsrep_node_address
        Value: 192.168.39.7
*************************** 22. row ***************************
Variable_name: wsrep_node_incoming_address
        Value: AUTO
*************************** 23. row ***************************
Variable_name: wsrep_node_name
        Value: pxc-cluster-node-1
*************************** 24. row ***************************
Variable_name: wsrep_notify_cmd
        Value: 
*************************** 25. row ***************************
Variable_name: wsrep_on
        Value: ON
*************************** 26. row ***************************
Variable_name: wsrep_preordered
        Value: OFF
*************************** 27. row ***************************
Variable_name: wsrep_provider
        Value: /usr/lib64/galera3/libgalera_smm.so
*************************** 28. row ***************************
Variable_name: wsrep_provider_options
        Value: base_dir = /var/lib/mysql/; base_host = 192.168.39.7; base_port = 4567; cert.log_conflicts = no; cert.optimistic_pa = yes; debug = no; evs.auto_evict = 0; evs.causal_keepalive_period = PT1S; evs.debug_log_mask = 0x1; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.info_log_mask = 0; evs.install_timeout = PT7.5S; evs.join_retrans_period = PT1S; evs.keepalive_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 10; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.use_aggregate = true; evs.user_send_window = 4; evs.version = 0; evs.view_forget_timeout = P1D; gcache.dir = /var/lib/mysql/; gcache.freeze_purge_at_seqno = -1; gcache.keep_pages_count = 0; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.recover = no; gcache.size = 128M; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1; gcs.fc_limit = 100; gcs.fc_master_slave = no;
*************************** 29. row ***************************
Variable_name: wsrep_recover
        Value: OFF
*************************** 30. row ***************************
Variable_name: wsrep_reject_queries
        Value: NONE
*************************** 31. row ***************************
Variable_name: wsrep_replicate_myisam
        Value: OFF
*************************** 32. row ***************************
Variable_name: wsrep_restart_slave
        Value: OFF
*************************** 33. row ***************************
Variable_name: wsrep_retry_autocommit
        Value: 1
*************************** 34. row ***************************
Variable_name: wsrep_slave_FK_checks
        Value: ON
*************************** 35. row ***************************
Variable_name: wsrep_slave_UK_checks
        Value: OFF
*************************** 36. row ***************************
Variable_name: wsrep_slave_threads
        Value: 8
*************************** 37. row ***************************
Variable_name: wsrep_sst_auth
        Value: ********
*************************** 38. row ***************************
Variable_name: wsrep_sst_donor
        Value: 
*************************** 39. row ***************************
Variable_name: wsrep_sst_donor_rejects_queries
        Value: OFF
*************************** 40. row ***************************
Variable_name: wsrep_sst_method
        Value: xtrabackup-v2
*************************** 41. row ***************************
Variable_name: wsrep_sst_receive_address
        Value: AUTO
*************************** 42. row ***************************
Variable_name: wsrep_start_position
        Value: 00000000-0000-0000-0000-000000000000:-1
*************************** 43. row ***************************
Variable_name: wsrep_sync_wait
        Value: 0
43 rows in set (0.01 sec)

# 查看相关状态变量
mysql> SHOW STATUS LIKE 'wsrep%'\G
*************************** 1. row ***************************
Variable_name: wsrep_local_state_uuid
        Value: 10c3398e-1594-11ea-9f84-c2cae1c33555
*************************** 2. row ***************************
Variable_name: wsrep_protocol_version
        Value: 9
*************************** 3. row ***************************
Variable_name: wsrep_last_applied
        Value: 1
*************************** 4. row ***************************
Variable_name: wsrep_last_committed
        Value: 1
*************************** 5. row ***************************
Variable_name: wsrep_replicated
        Value: 1
*************************** 6. row ***************************
Variable_name: wsrep_replicated_bytes
        Value: 232
*************************** 7. row ***************************
Variable_name: wsrep_repl_keys
        Value: 1
*************************** 8. row ***************************
Variable_name: wsrep_repl_keys_bytes
        Value: 32
*************************** 9. row ***************************
Variable_name: wsrep_repl_data_bytes
        Value: 133
*************************** 10. row ***************************
Variable_name: wsrep_repl_other_bytes
        Value: 0
*************************** 11. row ***************************
Variable_name: wsrep_received
        Value: 2
*************************** 12. row ***************************
Variable_name: wsrep_received_bytes
        Value: 154
*************************** 13. row ***************************
Variable_name: wsrep_local_commits
        Value: 0
*************************** 14. row ***************************
Variable_name: wsrep_local_cert_failures
        Value: 0
*************************** 15. row ***************************
Variable_name: wsrep_local_replays
        Value: 0
*************************** 16. row ***************************
Variable_name: wsrep_local_send_queue
        Value: 0
*************************** 17. row ***************************
Variable_name: wsrep_local_send_queue_max
        Value: 1
*************************** 18. row ***************************
Variable_name: wsrep_local_send_queue_min
        Value: 0
*************************** 19. row ***************************
Variable_name: wsrep_local_send_queue_avg
        Value: 0.000000
*************************** 20. row ***************************
Variable_name: wsrep_local_recv_queue
        Value: 0
*************************** 21. row ***************************
Variable_name: wsrep_local_recv_queue_max
        Value: 2
*************************** 22. row ***************************
Variable_name: wsrep_local_recv_queue_min
        Value: 0
*************************** 23. row ***************************
Variable_name: wsrep_local_recv_queue_avg
        Value: 0.500000
*************************** 24. row ***************************
Variable_name: wsrep_local_cached_downto
        Value: 1
*************************** 25. row ***************************
Variable_name: wsrep_flow_control_paused_ns
        Value: 0
*************************** 26. row ***************************
Variable_name: wsrep_flow_control_paused
        Value: 0.000000
*************************** 27. row ***************************
Variable_name: wsrep_flow_control_sent
        Value: 0
*************************** 28. row ***************************
Variable_name: wsrep_flow_control_recv
        Value: 0
*************************** 29. row ***************************
Variable_name: wsrep_flow_control_interval
        Value: [ 100, 100 ]
*************************** 30. row ***************************
Variable_name: wsrep_flow_control_interval_low
        Value: 100
*************************** 31. row ***************************
Variable_name: wsrep_flow_control_interval_high
        Value: 100
*************************** 32. row ***************************
Variable_name: wsrep_flow_control_status
        Value: OFF
*************************** 33. row ***************************
Variable_name: wsrep_cert_deps_distance
        Value: 1.000000
*************************** 34. row ***************************
Variable_name: wsrep_apply_oooe
        Value: 0.000000
*************************** 35. row ***************************
Variable_name: wsrep_apply_oool
        Value: 0.000000
*************************** 36. row ***************************
Variable_name: wsrep_apply_window
        Value: 1.000000
*************************** 37. row ***************************
Variable_name: wsrep_commit_oooe
        Value: 0.000000
*************************** 38. row ***************************
Variable_name: wsrep_commit_oool
        Value: 0.000000
*************************** 39. row ***************************
Variable_name: wsrep_commit_window
        Value: 1.000000
*************************** 40. row ***************************
Variable_name: wsrep_local_state
        Value: 4
*************************** 41. row ***************************
Variable_name: wsrep_local_state_comment
        Value: Synced
*************************** 42. row ***************************
Variable_name: wsrep_cert_index_size
        Value: 1
*************************** 43. row ***************************
Variable_name: wsrep_cert_bucket_count
        Value: 22
*************************** 44. row ***************************
Variable_name: wsrep_gcache_pool_size
        Value: 1592
*************************** 45. row ***************************
Variable_name: wsrep_causal_reads
        Value: 0
*************************** 46. row ***************************
Variable_name: wsrep_cert_interval
        Value: 0.000000
*************************** 47. row ***************************
Variable_name: wsrep_open_transactions
        Value: 0
*************************** 48. row ***************************
Variable_name: wsrep_open_connections
        Value: 0
*************************** 49. row ***************************
Variable_name: wsrep_ist_receive_status
        Value: 
*************************** 50. row ***************************
Variable_name: wsrep_ist_receive_seqno_start
        Value: 0
*************************** 51. row ***************************
Variable_name: wsrep_ist_receive_seqno_current
        Value: 0
*************************** 52. row ***************************
Variable_name: wsrep_ist_receive_seqno_end
        Value: 0
*************************** 53. row ***************************
Variable_name: wsrep_incoming_addresses
        Value: 192.168.39.7:3306
*************************** 54. row ***************************
Variable_name: wsrep_cluster_weight
        Value: 1
*************************** 55. row ***************************
Variable_name: wsrep_desync_count
        Value: 0
*************************** 56. row ***************************
Variable_name: wsrep_evs_delayed
        Value: 
*************************** 57. row ***************************
Variable_name: wsrep_evs_evict_list
        Value: 
*************************** 58. row ***************************
Variable_name: wsrep_evs_repl_latency
        Value: 0/0/0/0/0
*************************** 59. row ***************************
Variable_name: wsrep_evs_state
        Value: OPERATIONAL
*************************** 60. row ***************************
Variable_name: wsrep_gcomm_uuid
        Value: 10c2c674-1594-11ea-83fc-6e0df986585c
*************************** 61. row ***************************
Variable_name: wsrep_cluster_conf_id
        Value: 1
*************************** 62. row ***************************
Variable_name: wsrep_cluster_size
        Value: 1
*************************** 63. row ***************************
Variable_name: wsrep_cluster_state_uuid
        Value: 10c3398e-1594-11ea-9f84-c2cae1c33555
*************************** 64. row ***************************
Variable_name: wsrep_cluster_status
        Value: Primary
*************************** 65. row ***************************
Variable_name: wsrep_connected
        Value: ON
*************************** 66. row ***************************
Variable_name: wsrep_local_bf_aborts
        Value: 0
*************************** 67. row ***************************
Variable_name: wsrep_local_index
        Value: 0
*************************** 68. row ***************************
Variable_name: wsrep_provider_name
        Value: Galera
*************************** 69. row ***************************
Variable_name: wsrep_provider_vendor
        Value: Codership Oy <info@codership.com>
*************************** 70. row ***************************
Variable_name: wsrep_provider_version
        Value: 3.39(rb3295e6)
*************************** 71. row ***************************
Variable_name: wsrep_ready
        Value: ON
71 rows in set (0.00 sec)

#重点内容
mysql> SHOW STATUS like 'wsrep%';
+----------------------------------+--------------------------------------+
| Variable_name                    | Value                                |
+----------------------------------+--------------------------------------+
| wsrep_local_state_uuid           | 10c3398e-1594-11ea-9f84-c2cae1c33555 |
| wsrep_protocol_version           | 9                                    |
| wsrep_last_applied               | 1                                    |
| wsrep_last_committed             | 1                                    |
| wsrep_replicated                 | 1                                    |
| wsrep_replicated_bytes           | 232                                  |
| wsrep_repl_keys                  | 1                                    |
| wsrep_repl_keys_bytes            | 32                                   |
| wsrep_repl_data_bytes            | 133                                  |
| wsrep_repl_other_bytes           | 0                                    |
| wsrep_received                   | 2                                    |
| wsrep_received_bytes             | 154                                  |
| wsrep_local_commits              | 0                                    |
| wsrep_local_cert_failures        | 0                                    |
| wsrep_local_replays              | 0                                    |
| wsrep_local_send_queue           | 0                                    |
| wsrep_local_send_queue_max       | 1                                    |
| wsrep_local_send_queue_min       | 0                                    |
| wsrep_local_send_queue_avg       | 0.000000                             |
| wsrep_local_recv_queue           | 0                                    |
| wsrep_local_recv_queue_max       | 2                                    |
| wsrep_local_recv_queue_min       | 0                                    |
| wsrep_local_recv_queue_avg       | 0.500000                             |
| wsrep_local_cached_downto        | 1                                    |
| wsrep_flow_control_paused_ns     | 0                                    |
| wsrep_flow_control_paused        | 0.000000                             |
| wsrep_flow_control_sent          | 0                                    |
| wsrep_flow_control_recv          | 0                                    |
| wsrep_flow_control_interval      | [ 100, 100 ]                         |
| wsrep_flow_control_interval_low  | 100                                  |
| wsrep_flow_control_interval_high | 100                                  |
| wsrep_flow_control_status        | OFF                                  |
| wsrep_cert_deps_distance         | 1.000000                             |
| wsrep_apply_oooe                 | 0.000000                             |
| wsrep_apply_oool                 | 0.000000                             |
| wsrep_apply_window               | 1.000000                             |
| wsrep_commit_oooe                | 0.000000                             |
| wsrep_commit_oool                | 0.000000                             |
| wsrep_commit_window              | 1.000000                             |
| wsrep_local_state                | 4                                    |
| wsrep_local_state_comment        | Synced                               |
| wsrep_cert_index_size            | 1                                    |
| wsrep_cert_bucket_count          | 22                                   |
| wsrep_gcache_pool_size           | 1592                                 |
| wsrep_causal_reads               | 0                                    |
| wsrep_cert_interval              | 0.000000                             |
| wsrep_open_transactions          | 0                                    |
| wsrep_open_connections           | 0                                    |
| wsrep_ist_receive_status         |                                      |
| wsrep_ist_receive_seqno_start    | 0                                    |
| wsrep_ist_receive_seqno_current  | 0                                    |
| wsrep_ist_receive_seqno_end      | 0                                    |
| wsrep_incoming_addresses         | 192.168.39.7:3306                    |
| wsrep_cluster_weight             | 1                                    |
| wsrep_desync_count               | 0                                    |
| wsrep_evs_delayed                |                                      |
| wsrep_evs_evict_list             |                                      |
| wsrep_evs_repl_latency           | 0/0/0/0/0                            |
| wsrep_evs_state                  | OPERATIONAL                          |
| wsrep_gcomm_uuid                 | 10c2c674-1594-11ea-83fc-6e0df986585c |
| wsrep_cluster_conf_id            | 1                                    |
| wsrep_cluster_size               | 1                                    |
| wsrep_cluster_state_uuid         | 10c3398e-1594-11ea-9f84-c2cae1c33555 |
| wsrep_cluster_status             | Primary                              |
| wsrep_connected                  | ON                                   |
| wsrep_local_bf_aborts            | 0                                    |
| wsrep_local_index                | 0                                    |
| wsrep_provider_name              | Galera                               |
| wsrep_provider_vendor            | Codership Oy <info@codership.com>    |
| wsrep_provider_version           | 3.39(rb3295e6)                       |
| wsrep_ready                      | ON                                   |
+----------------------------------+--------------------------------------+
71 rows in set (0.00 sec)
说明:
wsrep_cluster_size表示,该Galera集群中只有一个节点
wsrep_local_state_comment 状态为Synced(4),表示数据已同步完成(因为是第一个引导节点,无数据需要同步)。 如果状态是Joiner, 意味着 SST 没有完成. 只有所有节点状态是Synced,才可以加新节点
wsrep_cluster_status为Primary,且已经完全连接并准备好

启动PXC集群中其它所有节点

  1. 启动PXC集群中其它所有节点
# 启动前在第一个节点上创建一个授权账户(要不然可能别的节点没有授权数据库会起不来)
mysql> GRANT RELOAD, LOCK TABLES, PROCESS, REPLICATION CLIENT ON *.* TO 'sstuser'@'localhost' identified by 'centos7'; # 这里授权的密码必须和配置文件里设置的密码对应,所有服务器都要一样。
Query OK, 0 rows affected, 1 warning (0.00 sec)

[root@pxc2 etc]#ss -ntl  # 启动前端口情况
State      Recv-Q Send-Q Local Address:Port               Peer Address:Port              
LISTEN     0      128               *:111                           *:*                  
LISTEN     0      128               *:6000                          *:*                  
LISTEN     0      5      192.168.122.1:53                            *:*                  
LISTEN     0      128               *:22                            *:*                  
LISTEN     0      128       127.0.0.1:631                           *:*                  
LISTEN     0      128       127.0.0.1:6010                          *:*                  
LISTEN     0      128              :::111                          :::*                  
LISTEN     0      128              :::6000                         :::*                  
LISTEN     0      128              :::22                           :::*                  
LISTEN     0      128             ::1:631                          :::*                  
LISTEN     0      128             ::1:6010                         :::*    

[root@pxc2 etc]#systemctl start mysql # 启动数据库
[root@pxc2 etc]#ss -ntl   # 启动后 3306、4567 端口都出现了
State       Recv-Q Send-Q                                                 Local Address:Port                                                                Peer Address:Port              
LISTEN      0      128                                                                *:111                                                                            *:*                  
LISTEN      0      128                                                                *:6000                                                                           *:*                  
LISTEN      0      5                                                      192.168.122.1:53                                                                             *:*                  
LISTEN      0      128                                                                *:22                                                                             *:*                  
LISTEN      0      128                                                                *:4567                                                                           *:*                  
LISTEN      0      128                                                        127.0.0.1:631                                                                            *:*                  
LISTEN      0      128                                                        127.0.0.1:6010                                                                           *:*                  
LISTEN      0      128                                                        127.0.0.1:6011                                                                           *:*                  
LISTEN      0      80                                                                :::3306                                                                          :::*                  
LISTEN      0      128                                                               :::111                                                                           :::*                  
LISTEN      0      128                                                               :::6000                                                                          :::*                  
LISTEN      0      128                                                               :::22                                                                            :::*                  
LISTEN      0      128                                                              ::1:631                                                                           :::*                  
LISTEN      0      128                                                              ::1:6010                                                                          :::*                  
LISTEN      0      128                                                              ::1:6011                                                                          :::*         
[root@pxc2 etc]#mysql -uroot -pmagedu  # 登录数据库
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 11
Server version: 5.7.27-30-57-log Percona XtraDB Cluster (GPL), Release rel30, Revision 64987d4, WSREP version 31.39, wsrep_31.39

Copyright (c) 2009-2019 Percona LLC and/or its affiliates
Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> 

# 另一个节点一样启动
[root@pxc3 etc]#systemctl start mysql # 启动数据库
[root@pxc3 ~]#mysql -uroot -pmagedu
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 11
Server version: 5.7.27-30-57-log Percona XtraDB Cluster (GPL), Release rel30, Revision 64987d4, WSREP version 31.39, wsrep_31.39

Copyright (c) 2009-2019 Percona LLC and/or its affiliates
Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> 
  1. 查看集群状态,验证集群是否成功
# 在任意节点,查看集群状态
mysql> SHOW VARIABLES LIKE 'wsrep_node_name';
+-----------------+--------------------+
| Variable_name   | Value              |
+-----------------+--------------------+
| wsrep_node_name | pxc-cluster-node-1 |
+-----------------+--------------------+
1 row in set (0.00 sec)

mysql> show variables like 'wsrep_node_address';
+--------------------+--------------+
| Variable_name      | Value        |
+--------------------+--------------+
| wsrep_node_address | 192.168.39.7 |
+--------------------+--------------+
1 row in set (0.01 sec)

mysql> show variables like 'wsrep_on';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| wsrep_on      | ON    |
+---------------+-------+
1 row in set (0.00 sec)

mysql> show status like 'wsrep_cluster_size';
+--------------------+-------+
| Variable_name      | Value |
+--------------------+-------+
| wsrep_cluster_size | 3     |
+--------------------+-------+
1 row in set (0.00 sec)

# 在任意节点创建数据库测试(pxc2)
mysql> SHOW VARIABLES LIKE 'wsrep_node_address';
+--------------------+---------------+
| Variable_name      | Value         |
+--------------------+---------------+
| wsrep_node_address | 192.168.39.27 |
+--------------------+---------------+
1 row in set (0.00 sec)

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
4 rows in set (0.00 sec)

mysql> create database db1;
Query OK, 1 row affected (0.01 sec)

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| db1                |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
5 rows in set (0.00 sec)

# 在pxc1上查看
mysql> SHOW VARIABLES LIKE 'wsrep_node_address';
+--------------------+--------------+
| Variable_name      | Value        |
+--------------------+--------------+
| wsrep_node_address | 192.168.39.7 |
+--------------------+--------------+
1 row in set (0.00 sec)

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| db1                |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
5 rows in set (0.00 sec)

# 在pxc2上查看  
mysql> SHOW VARIABLES LIKE 'wsrep_node_address';
+--------------------+---------------+
| Variable_name      | Value         |
+--------------------+---------------+
| wsrep_node_address | 192.168.39.37 |
+--------------------+---------------+
1 row in set (0.00 sec)

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| db1                |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
5 rows in set (0.00 sec)
# 两个节点都可以看到同步成功
  1. 测试是否有冲突问题
    在这里插入图片描述
#利用Xshell软件,同时在三个节点数据库,在其中一个节点成功
mysql> create database db2;  # 只有一个节点会成功其他节点报错不创建
Query OK, 1 row affected (0.00 sec)

mysql> create database db2;
ERROR 1007 (HY000): Can't create database 'db2'; database exists 

mysql> create database db2;
ERROR 1007 (HY000): Can't create database 'db2'; database exists
  1. PXC缺点(同时写入数据性能会有影响)
# 使用一个命令循环创建10w条记录
mysql> create table test (id int auto_increment primary key,name char(10));

delimiter ;
Query OK, 0 rows affected (0.01 sec)

mysql> 
mysql> delimiter $$
mysql> 
mysql> create procedure  proc_test() 
    -> begin  
    -> declare i int;
    -> set i = 1; 
    -> while i < 100000 
    -> do  insert into test(name) values (concat('wang',i)); 
    -> set i = i +1; 
    -> end while; 
    -> end$$
Query OK, 0 rows affected (0.01 sec)

mysql> 
mysql> delimiter ;
mysql> call proc_test();
Query OK, 1 row affected (3 min 39.51 sec)   # 用了39s接近40s时间才添加完成

# 使用事务方式添加记录还是很快的
mysql> begin;
Query OK, 0 rows affected (0.00 sec)

mysql> call proc_test();
Query OK, 1 row affected (2.44 sec)

mysql> commit;
Query OK, 0 rows affected (0.25 sec)
  1. 添加记录的话每个节点添加id变化
# 每个节点添加记录可重复。(是自动增长的)
mysql> insert t1(name)values('mage');
Query OK, 1 row affected (0.01 sec)

mysql> select * from t1;
+----+-------+
| id | name  |
+----+-------+
|  1 | mage  |
|  4 | magea |
|  7 | mageb |
| 10 | mage  |
+----+-------+
4 rows in set (0.00 sec)
# 2 节点添加
mysql> insert t1(name)values('mage');
Query OK, 1 row affected (0.00 sec)

mysql> insert t1(name)values('mage');
Query OK, 1 row affected (0.00 sec)

mysql> insert t1(name)values('mage');
Query OK, 1 row affected (0.00 sec)

mysql> select * from t1;
+----+-------+
| id | name  |
+----+-------+
|  1 | mage  |
|  4 | magea |
|  7 | mageb |
| 10 | mage  |
| 11 | mage  |
| 14 | mage  |
| 17 | mage  |
+----+-------+
7 rows in set (0.00 sec)

在PXC集群中加入节点

一个节点加入到Galera集群有两种情况:新节点加入集群、暂时离组的成员再次加入集群
1)新节点加入Galera集群
新节点加入集群时,需要从当前集群中选择一个Donor节点来同步数据,也就是所谓state_snapshot_tranfer(SST)过程。SST同步数据的方式由选项wsrep_sst_method决定,一般选择的是xtrabackup。
必须注意,新节点加入Galera时,会删除新节点上所有已有数据,再通过xtrabackup(假设使用的是该方式)从Donor处完整备份所有数据进行恢复。所以,如果数据量很大,新节点加入过程会很慢。而且,在一个新节点成为Synced状态之前,不要同时加入其它新节点,否则很容易将集群压垮。如果是这种情况,可以考虑使用wsrep_sst_method=rsync来做增量同步,既然是增量同步,最好保证新节点上已经有一部分数据基础,否则和全量同步没什么区别,且这样会对Donor节点加上全局readonly锁。
2)旧节点加入Galera集群
如果旧节点加入Galera集群,说明这个节点在之前已经在Galera集群中呆过,有一部分数据基础,缺少的只是它离开集群时的数据。这时加入集群时,会采用IST(incremental snapshot transfer)传输机制,即使用增量传输。
# 同样先配置yum源
# 在安装数据库
[root@pxc4 ~]#yum install Percona-XtraDB-Cluster-57 -y
# 修改配置文件
[root@pxc4 ~]#grep -Ev "^#|^$" /etc/percona-xtradb-cluster.conf.d/wsrep.cnf
[mysqld]
wsrep_provider=/usr/lib64/galera3/libgalera_smm.so
wsrep_cluster_address=gcomm://192.168.39.7,192.168.39.27,192.168.39.37,192.168.39.47 # 这一行添加上新节点的ip(虽然其他节点配置文件不添加也可以,但是最好都加上,比如说服务器依次关机到新添加的节点(新添加的节点留为最后的引导服务器),但是下次开机别的节点会找不到新添加的服务器,数据库就起不来了)
binlog_format=ROW
default_storage_engine=InnoDB
wsrep_slave_threads= 8
wsrep_log_conflicts
innodb_autoinc_lock_mode=2
wsrep_node_address=192.168.39.47
wsrep_cluster_name=pxc-cluster
wsrep_node_name=pxc-cluster-node-4
pxc_strict_mode=ENFORCING
wsrep_sst_method=xtrabackup-v2
wsrep_sst_auth="sstuser:centos7"

# 查看连接数以变为4个
mysql> show status like 'wsrep_cluster_size'\G
*************************** 1. row ***************************
Variable_name: wsrep_cluster_size
        Value: 4
1 row in set (0.00 sec)

在PXC集群中修复故障节点

  1. 在其他节点停止服务
[root@pxc2 ~]#systemctl stop mysql
[root@pxc2 ~]#systemctl status mysql
● mysql.service - Percona XtraDB Cluster
   Loaded: loaded (/usr/lib/systemd/system/mysql.service; enabled; vendor preset: disabled)
   Active: inactive (dead) since Tue 2019-12-03 16:52:51 CST; 3min 21s ago
  Process: 29259 ExecStopPost=/usr/bin/mysql-systemd stop-post (code=exited, status=0/SUCCESS)
  Process: 29221 ExecStop=/usr/bin/mysql-systemd stop (code=exited, status=0/SUCCESS)
  Process: 26928 ExecStartPost=/usr/bin/mysql-systemd start-post $MAINPID (code=exited, status=0/SUCCESS)
  Process: 26925 ExecStart=/usr/bin/mysqld_safe --basedir=/usr (code=exited, status=0/SUCCESS)
  Process: 26853 ExecStartPre=/usr/bin/mysql-systemd start-pre (code=exited, status=0/SUCCESS)
 Main PID: 26925 (code=exited, status=0/SUCCESS)

Dec 03 15:12:54 pxc2 mysqld_safe[26925]: 2019-12-03T07:12:54.344288Z mysqld_safe Assigning 00000000-0000-0000-0000-000000000000:-1 to wsrep_start_position
Dec 03 15:12:56 pxc2 mysql-systemd[26928]: State transfer in progress, setting sleep higher
Dec 03 15:13:16 pxc2 mysql-systemd[26928]: SUCCESS!
Dec 03 15:13:16 pxc2 systemd[1]: Started Percona XtraDB Cluster.
Dec 03 16:52:39 centos7.localdomain systemd[1]: Stopping Percona XtraDB Cluster...
Dec 03 16:52:39 centos7.localdomain mysql-systemd[29221]: SUCCESS! Stopping Percona XtraDB Cluster......
Dec 03 16:52:51 centos7.localdomain mysqld_safe[26925]: 2019-12-03T08:52:51.720383Z mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
Dec 03 16:52:51 centos7.localdomain mysql-systemd[29259]: WARNING: mysql pid file /var/run/mysqld/mysqld.pid empty or not readable
Dec 03 16:52:51 centos7.localdomain mysql-systemd[29259]: WARNING: mysql may be already dead
Dec 03 16:52:51 centos7.localdomain systemd[1]: Stopped Percona XtraDB Cluster.
  1. 在其它任意节点查看wsrep_cluster_size变量
[root@centos7 ~]#mysql -uroot -pmagedu
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 21
Server version: 5.7.27-30-57-log Percona XtraDB Cluster (GPL), Release rel30, Revision 64987d4, WSREP version 31.39, wsrep_31.39

Copyright (c) 2009-2019 Percona LLC and/or its affiliates
Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show status like 'wsrep_cluster_size'\G
*************************** 1. row ***************************
Variable_name: wsrep_cluster_size
        Value: 3                          # 少了一个节点
1 row in set (0.00 sec)
  1. 添加一个数据库查看同步情况
mysql> create database db4;
Query OK, 1 row affected (0.01 sec)

#在其它任意节点可看到数据已同步
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| db1                |
| db2                |
| db3                |
| db4                |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
8 rows in set (0.00 sec)
# 其他节点正常同步
  1. 恢复服务,数据同步
[root@pxc2 ~]#systemctl start mysql
[root@pxc2 ~]#mysql -uroot -pmagedu
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 11
Server version: 5.7.27-30-57-log Percona XtraDB Cluster (GPL), Release rel30, Revision 64987d4, WSREP version 31.39, wsrep_31.39

Copyright (c) 2009-2019 Percona LLC and/or its affiliates
Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| db1                |
| db2                |
| db3                |
| db4                |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
8 rows in set (0.00 sec)
# 恢复服务之后数据还是能同步的
mysql> SHOW STATUS LIKE 'wsrep_cluster_size';
+--------------------+-------+
| Variable_name      | Value |
+--------------------+-------+
| wsrep_cluster_size | 4     |
+--------------------+-------+
1 row in set (0.00 sec)

实现MariaDB Galera Cluster

#在三个节点上都实现
[root@centos8 ~]#dnf install mariadb-server-galera -y
[root@centos8 ~]#vim /etc/my.cnf.d/galera.cnf  # 修改配置文件
#wsrep_cluster_address="dummy://"
wsrep_cluster_address="gcomm://192.168.39.7,192.168.39.27,192.168.39.37"
#启动第一节点
[root@centos8 ~]#galera_new_cluster
#再启动其它节点
[root@centos8 ~]#systemctl start mariadb
[root@centos8 ~]#ss -ntul
Netid             State              Recv-Q            Send-Q
Local Address:Port                            Peer Address:Port
tcp              LISTEN              0                 128
0.0.0.0:22                                   0.0.0.0:*
tcp              LISTEN              0                 128
0.0.0.0:4567                                 0.0.0.0:*
tcp              LISTEN              0                  80
0.0.0.0:3306                                 0.0.0.0:*
tcp              LISTEN              0                 128
[::]:22                                      [::]:*

[root@centos8 ~]#mysql
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 11
Server version: 10.3.11-MariaDB MariaDB Server
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]>
MariaDB [(none)]> show status like "wsrep_ready";
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| wsrep_ready | ON |
+---------------+-------+
1 row in set (0.001 sec)
MariaDB [(none)]> SHOW STATUS LIKE 'wsrep_cluster_size';
+--------------------+-------+
| Variable_name | Value |
+--------------------+-------+
| wsrep_cluster_size | 3 |
+--------------------+-------+
1 row in set (0.001 sec)
MariaDB [(none)]> SHOW VARIABLES LIKE 'wsrep_%'\G
MariaDB [(none)]> SHOW STATUS LIKE 'wsrep_%';

复制的问题和解决方案

复制的问题和解决方案:
(1) 数据损坏或丢失
    Master: MHA + semi repl
    Slave: 重新复制
(2) 混合使用存储引擎
    MyISAM:不支持事务
    InnoDB: 支持事务
(3) 不惟一的server id
    重新复制
(4) 复制延迟
    需要额外的监控工具的辅助
    一从多主:mariadb10版后支持
    多线程复制:对多个数据库复制

性能衡量指标

数据库服务衡量指标:

Qps:query per second
Tps:transaction per second

压力测试工具

常见MySQl压力测试工具

mysqlslap
Sysbench:功能强大
https://github.com/akopytov/sysbench
tpcc-mysql
MySQL Benchmark Suite
MySQL super-smack
MyBench
MYSQL压力测试

mysqlslap

Mysqlslap:来自于mariadb包,测试的过程默认生成一个mysqlslap的schema,生成测试表t1,查询和插入测试数据,mysqlslap库自动生成,如果已经存在则先删除。用--only-print来打印实际的测试过程,整个测试完成后不会在数据库中留下痕迹。
  • 使用格式:
mysqlslap [options]

常用参数 [options] 说明:

--auto-generate-sql, -a #自动生成测试表和数据,表示用mysqlslap工具自己生成的SQL脚本来测试
并发压力
--auto-generate-sql-load-type=type #测试语句的类型。代表要测试的环境是读操作还是写操作还 
是两者混合的。取值包括:read,key,write,update和mixed(默认)
--auto-generate-sql-add-auto-increment #代表对生成的表自动添加auto_increment列,从 5.1.18版本开始支持
--number-char-cols=N, -x N #自动生成的测试表中包含多少个字符类型的列,默认1
--number-int-cols=N, -y N #自动生成的测试表中包含多少个数字类型的列,默认1
--number-of-queries=N  #总的测试查询次数(并发客户数×每客户查询次数)
--query=name,-q   #使用自定义脚本执行测试,例如可以调用自定义的存储过程或者sql语句来执行测试
--create-schema  #代表自定义的测试库名称,测试的schema
--commint=N #多少条DML后提交一次
--compress, -C #如服务器和客户端都支持压缩,则压缩信息
--concurrency=N, -c N #表示并发量,即模拟多少个客户端同时执行select。可指定多个值,以逗号 
或者--delimiter参数指定值做为分隔符,如:--concurrency=100,200,500
--engine=engine_name, -e engine_name #代表要测试的引擎,可以有多个,用分隔符隔开。例 如:--engines=myisam,innodb
--iterations=N, -i N #测试执行的迭代次数,代表要在不同并发环境下,各自运行测试多少次
--only-print #只打印测试语句而不实际执行。
--detach=N #执行N条语句后断开重连
--debug-info, -T #打印内存和CP

mysqlslap示例

#单线程测试
mysqlslap -a -uroot -pmagedu
#多线程测试。使用--concurrency来模拟并发连接
mysqlslap -a -c 100 -uroot -pmagedu
#迭代测试。用于需要多次执行测试得到平均值
mysqlslap -a -i 10 -uroot -pmagedu
mysqlslap ---auto-generate-sql-add-autoincrement -a
mysqlslap -a --auto-generate-sql-load-type=read
mysqlslap -a --auto-generate-secondary-indexes=3
mysqlslap -a --auto-generate-sql-write-number=1000
mysqlslap --create-schema world -q "select count(*) from City"
mysqlslap -a -e innodb -uroot -pmagedu
mysqlslap -a --number-of-queries=10 -uroot -pmagedu
#测试同时不同的存储引擎的性能进行对比
mysqlslap -a --concurrency=50,100 --number-of-queries 1000 --iterations=5 --
engine=myisam,innodb --debug-info -uroot -pmagedu
#执行一次测试,分别50和100个并发,执行1000次总查询
mysqlslap -a --concurrency=50,100 --number-of-queries 1000 --debug-info -uroot -
pmagedu
#50和100个并发分别得到一次测试结果(Benchmark),并发数越多,执行完所有查询的时间越长。为了准确
起见,可以多迭代测试几次
mysqlslap -a --concurrency=50,100 --number-of-queries 1000 --iterations=5 --
debug-info -uroot -pmagedu

生产环境 my.cnf 配置案例

  • 硬件:内存32G
#打开独立表空间
innodb_file_per_table = 1
#MySQL 服务所允许的同时会话数的上限,经常出现Too Many Connections的错误提示,则需要增大此值
max_connections = 8000
#所有线程所打开表的数量
open_files_limit = 10240
#back_log 是操作系统在监听队列中所能保持的连接数
back_log = 300
#每个客户端连接最大的错误允许数量,当超过该次数,MYSQL服务器将禁止此主机的连接请求,直到MYSQL
服务器重启或通过flush hosts命令清空此主机的相关信息
max_connect_errors = 1000
#每个连接传输数据大小.最大1G,须是1024的倍数,一般设为最大的BLOB的值
max_allowed_packet = 32M
#指定一个请求的最大连接时间
wait_timeout = 10
# 排序缓冲被用来处理类似ORDER BY以及GROUP BY队列所引起的排序
sort_buffer_size = 16M
#不带索引的全表扫描.使用的buffer的最小值
join_buffer_size = 16M
#查询缓冲大小
query_cache_size = 128M
#指定单个查询能够使用的缓冲区大小,缺省为1M
query_cache_limit = 4M
# 设定默认的事务隔离级别
transaction_isolation = REPEATABLE-READ
# 线程使用的堆大小. 此值限制内存中能处理的存储过程的递归深度和SQL语句复杂性,此容量的内存在每次
连接时被预留.
thread_stack = 512K
# 二进制日志功能
log-bin
#二进制日志格式
binlog_format=row
#InnoDB使用一个缓冲池来保存索引和原始数据, 可设置这个变量到物理内存大小的80%
innodb_buffer_pool_size = 24G
#用来同步IO操作的IO线程的数量
innodb_file_io_threads = 4
#在InnoDb核心内的允许线程数量,建议的设置是CPU数量加上磁盘数量的两倍
innodb_thread_concurrency = 16
# 用来缓冲日志数据的缓冲区的大小
innodb_log_buffer_size = 16M
在日志组中每个日志文件的大小
innodb_log_file_size = 512M
# 在日志组中的文件总数
innodb_log_files_in_group = 3
# SQL语句在被回滚前,InnoDB事务等待InnoDB行锁的时间
innodb_lock_wait_timeout = 120
#慢查询时长
long_query_time = 2
#将没有使用索引的查询也记录下来
log-queries-not-using-indexes

MySQL配置最佳实践

高并发大数据的互联网业务,架构设计思路是“解放数据库CPU,将计算转移到服务层”,并发量大的情况下,这些功能很可能将数据库拖死,业务逻辑放到服务层具备更好的扩展性,能够轻易实现“增机器就加性能”
  • 参考:
    阿里巴巴Java开发手册
    58到家数据库30条军规解读
    http://zhuanlan.51cto.com/art/201702/531364.htm在这里插入图片描述
posted @ 2019-12-03 18:00  鱼与于玉  阅读(...)  评论(...编辑  收藏