MySQL5.7 多源复制

1.多源复制介绍

MySQL5.7 开始支持多源复制(Multi-Source Replication)也就是多主一从的复制架构,架构图如下:
MySQL5.7 多源复制

2.为什么要使用多源复制

(1)灾备作用:将各个库汇总在一起,就算是其他库都挂了(整个机房都无法连接了),还有最后一个救命稻草;

(2)备份:直接在这个从库中做备份,不影响线上的数据库;
(3)减少成本:不需要每个库都做一个实例,也减少了 DBA 的维护成本;
(4)数据统计:后期的一些数据统计,需要将所有的库汇总在一起。

二、搭建 MySQL 多源复制

1.实验环境

MySQL5.7 多源复制

多源复制是支持 GTID 和 Binlog + Position ,这篇文档以 GTID 复制来编写

搭建注意事项:
(1)每个实例上 DB 名字不能一样
(2)创建用户或是复制用户时名字不能一样,如果经常遇到一样的账号,或是为了规范创建的账号是一样的,考虑在 slave 的过滤规则把 mysql 复制忽略掉,

stop slave sql_thread; change replication filter Replicate_ignore_DB=(mysql);

2.配置文件

master1:

gtid-mode = on
enforce-gtid-consistency=1

binlog_format = row
server-id = 1
log-bin = /data/mysql/mysql3306/logs/mysql-bin
skip_slave_start = 1

master2:

gtid-mode = on
enforce-gtid-consistency=1

binlog_format = row
server-id = 2
log-bin = /data/mysql/mysql3306/logs/mysql-bin
skip_slave_start = 1

slave:

gtid-mode = on
enforce-gtid-consistency=1

binlog_format = row
server-id =3
log-bin = /data/mysql/mysql3307/logs/mysql-bin
skip_slave_start = 1
# multi-source-replication  # 使用多源复制,一些信息只能存放在 table 里面,不能存放到文件里面;
master-info-repository=TABLE
relay-log-info-repository=TABLE
replicate_ignore_db=mysql # 忽略 mysql 库同步

3.启动数据库并确认 server_uuid

启动数据库:

# /etc/init.d/mysql start 

确认 server-uuid
master1:

root@localhost [unixfbi1]>show global variables like '%uuid%';       
+---------------+--------------------------------------+
| Variable_name | Value                                |
+---------------+--------------------------------------+
| server_uuid   | b5a3240c-8946-11e7-bf07-d067e528dfb8 |
+---------------+--------------------------------------+
1 row in set (0.00 sec)

master2:

root@localhost [unixfbi2]>show global variables like '%uuid%';
+---------------+--------------------------------------+
| Variable_name | Value                                |
+---------------+--------------------------------------+
| server_uuid   | 3229c868-9d0f-11e7-9053-7845c401c236 |
+---------------+--------------------------------------+
1 row in set (0.01 sec)

slave:

root@localhost [(none)]>show global variables like '%uuid%';
+---------------+--------------------------------------+
| Variable_name | Value                                |
+---------------+--------------------------------------+
| server_uuid   | b4300f44-86e3-11e7-868a-2c27d72e9d73 |
+---------------+--------------------------------------+
1 row in set (0.00 sec)

在复制环境或者多源复制环境中每个数据库的 server_uuid 不要相同,否则无法同步。

4.创建复制账号

master1:

mysql> grant replication slave,replication client on *.* to 'repl'@'192.168.199.%'  identified  by  'unixfbi';
mysql> flush privileges;

master2:

mysql> grant replication slave,replication client on *.* to 'repl'@'192.168.199.%'  identified  by  'unixfbi';
mysql> flush privileges;

5.从库启动复制

我们这里都是新环境搭建的复制,所以没有 mysqldump 备份原来的数据,如果是旧环境中配置多源复制,就需要使用 mysqldump 备份数据并导入到 slave 库中。

# mysqldump -uuser -p --single-transaction –master-data=2 --add-drop-database -B db1 > master1_db1.sql

GTID

mysql> change master  to master_host='192.168.199.230',master_port=3306,master_user='repl',master_password='unixfbi',master_auto_position=1 for channel 'master-1';

mysql> change master to master_host='192.168.199.231',master_port=3306,master_user='repl',master_password='unixfbi',master_auto_position=1 for channel 'master-2';

mysql> start slave;  
# 也可以 start slave for channel 'master-1';  启动单个 channel 的复制。

Binlog + Position
如果是 Binlog + Postion 方式搭建的多源复制,请使用欧冠如下命令

mysql> change master  to master_host='192.168.199.230',master_port=3306,master_user='repl',master_password='unixfbi',master_log_file='mysql-bin.000001' ,master_log_pos=154  for channel 'master-1';

mysql> change master  to master_host='192.168.199.231',master_port=3306,master_user='repl',master_password='unixfbi',master_log_file='mysql-bin.000001' ,master_log_pos=154  for channel 'master-2';

mysql> start slave;

6.查看多源复制状态

查看所有多源 channel 复制状态

root@localhost [(none)]>show slave status\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 192.168.199.230
                  Master_User: repl
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000001
          Read_Master_Log_Pos: 154
               Relay_Log_File: relay-bin-master@002d1.000002
                Relay_Log_Pos: 367
        Relay_Master_Log_File: mysql-bin.000001
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB: 
          Replicate_Ignore_DB: mysql
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 154
              Relay_Log_Space: 581
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 1
                  Master_UUID: b5a3240c-8946-11e7-bf07-d067e528dfb8
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: 
            Executed_Gtid_Set: 
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: master-1
           Master_TLS_Version: 
*************************** 2. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 192.168.199.231
                  Master_User: repl
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000003
          Read_Master_Log_Pos: 154
               Relay_Log_File: relay-bin-master@002d2.000002
                Relay_Log_Pos: 367
        Relay_Master_Log_File: mysql-bin.000003
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB: 
          Replicate_Ignore_DB: mysql
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 154
              Relay_Log_Space: 581
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 2
                  Master_UUID: 3229c868-9d0f-11e7-9053-7845c401c236
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: 
            Executed_Gtid_Set: 
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: master-2
           Master_TLS_Version: 
2 rows in set (0.00 sec)

7.多源复制校验

master1 创建 unixfbi1 库和 tb1 表

root@localhost [(none)]>create   database unixfbi1;
Query OK, 1 row affected (0.00 sec)

root@localhost [(none)]>use unixfbi1;
Database changed
root@localhost [unixfbi1]>create table tb1(id int);
Query OK, 0 rows affected (0.18 sec)

root@localhost [unixfbi1]>insert into tb1(id) values(1),(2);
Query OK, 2 rows affected (0.01 sec)
Records: 2  Duplicates: 0  Warnings: 0

master2 创建 unixfbi2 库和 tb2 表

root@localhost [(none)]>create database unixfbi2;
Query OK, 1 row affected (0.00 sec)

root@localhost [(none)]>use unixfbi2;
Database changed
root@localhost [unixfbi2]>create table tb2(id int);
Query OK, 0 rows affected (0.21 sec)

root@localhost [unixfbi2]>insert into tb2(id) values(3),(4);
Query OK, 2 rows affected (0.01 sec)
Records: 2  Duplicates: 0  Warnings: 0

再次查看从库状态:

root@localhost [(none)]>show slave status\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 192.168.199.230
                  Master_User: repl
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000001
          Read_Master_Log_Pos: 763
               Relay_Log_File: relay-bin-master@002d1.000002
                Relay_Log_Pos: 976
        Relay_Master_Log_File: mysql-bin.000001
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB: 
          Replicate_Ignore_DB: mysql
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 763
              Relay_Log_Space: 1190
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 1
                  Master_UUID: b5a3240c-8946-11e7-bf07-d067e528dfb8
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: b5a3240c-8946-11e7-bf07-d067e528dfb8:1-3
            Executed_Gtid_Set: 3229c868-9d0f-11e7-9053-7845c401c236:1-4,
b5a3240c-8946-11e7-bf07-d067e528dfb8:1-3
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: master-1
           Master_TLS_Version: 
*************************** 2. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 192.168.199.231
                  Master_User: repl
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000003
          Read_Master_Log_Pos: 934
               Relay_Log_File: relay-bin-master@002d2.000002
                Relay_Log_Pos: 1147
        Relay_Master_Log_File: mysql-bin.000003
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB: 
          Replicate_Ignore_DB: mysql
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 934
              Relay_Log_Space: 1361
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 2
                  Master_UUID: 3229c868-9d0f-11e7-9053-7845c401c236
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: 3229c868-9d0f-11e7-9053-7845c401c236:1-4
            Executed_Gtid_Set: 3229c868-9d0f-11e7-9053-7845c401c236:1-4,
b5a3240c-8946-11e7-bf07-d067e528dfb8:1-3
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: master-2
           Master_TLS_Version: 
2 rows in set (0.00 sec)

到 slave 上查看数据是否正常复制过来

root@localhost [(none)]>show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |
| unixfbi1           |
| unixfbi2           |
+--------------------+
6 rows in set (0.00 sec)

root@localhost [(none)]>select * from unixfbi1.tb1;
+------+
| id   |
+------+
|    1 |
|    2 |
+------+
2 rows in set (0.00 sec)

root@localhost [(none)]>select * from unixfbi2.tb2;
+------+
| id   |
+------+
|    3 |
|    4 |
+------+
2 rows in set (0.00 sec)

到这里,我们已经成功实现了 MySQL5.7 多源复制搭建;

8.管理命令

(1)开启多源复制

start  slave [thread_type] [for channel...]

start slave 后面什么也不加,表示同时开启多个多源复制;

(2)查看多源复制状态

show  slave status [for channel....]

(3)查看多源复制配置

select * from performance_schema.replication_connection_configuration\G

root@localhost [(none)]>select * from performance_schema.replication_connection_configuration\G
*************************** 1. row ***************************
                 CHANNEL_NAME: master-1
                         HOST: 192.168.199.230
                         PORT: 3306
                         USER: repl
            NETWORK_INTERFACE: 
                AUTO_POSITION: 1
                  SSL_ALLOWED: NO
                  SSL_CA_FILE: 
                  SSL_CA_PATH: 
              SSL_CERTIFICATE: 
                   SSL_CIPHER: 
                      SSL_KEY: 
SSL_VERIFY_SERVER_CERTIFICATE: NO
                 SSL_CRL_FILE: 
                 SSL_CRL_PATH: 
    CONNECTION_RETRY_INTERVAL: 60
       CONNECTION_RETRY_COUNT: 86400
           HEARTBEAT_INTERVAL: 30.000
                  TLS_VERSION: 
*************************** 2. row ***************************
                 CHANNEL_NAME: master-2
                         HOST: 192.168.199.231
                         PORT: 3306
                         USER: repl
            NETWORK_INTERFACE: 
                AUTO_POSITION: 1
                  SSL_ALLOWED: NO
                  SSL_CA_FILE: 
                  SSL_CA_PATH: 
              SSL_CERTIFICATE: 
                   SSL_CIPHER: 
                      SSL_KEY: 
SSL_VERIFY_SERVER_CERTIFICATE: NO
                 SSL_CRL_FILE: 
                 SSL_CRL_PATH: 
    CONNECTION_RETRY_INTERVAL: 60
       CONNECTION_RETRY_COUNT: 86400
           HEARTBEAT_INTERVAL: 30.000
                  TLS_VERSION: 
2 rows in set (0.00 sec)

(4)查看多源复制连接状态

root@localhost [(none)]>select * from performance_schema.replication_connection_status\G
*************************** 1. row ***************************
             CHANNEL_NAME: master-1
               GROUP_NAME: 
              SOURCE_UUID: 
                THREAD_ID: NULL
            SERVICE_STATE: OFF
COUNT_RECEIVED_HEARTBEATS: 0
 LAST_HEARTBEAT_TIMESTAMP: 0000-00-00 00:00:00
 RECEIVED_TRANSACTION_SET: 
        LAST_ERROR_NUMBER: 0
       LAST_ERROR_MESSAGE: 
     LAST_ERROR_TIMESTAMP: 0000-00-00 00:00:00
*************************** 2. row ***************************
             CHANNEL_NAME: master-2
               GROUP_NAME: 
              SOURCE_UUID: 
                THREAD_ID: NULL
            SERVICE_STATE: OFF
COUNT_RECEIVED_HEARTBEATS: 0
 LAST_HEARTBEAT_TIMESTAMP: 0000-00-00 00:00:00
 RECEIVED_TRANSACTION_SET: 
        LAST_ERROR_NUMBER: 0
       LAST_ERROR_MESSAGE: 
     LAST_ERROR_TIMESTAMP: 0000-00-00 00:00:00
2 rows in set (0.00 sec)

三、多源复制常见问题解决方法

1.复制出错的解决方法

(1)单个复制解决方法:

传统 Binlog + pos :单个复制

stop  slave sql_thread;
set global sql_slave_skip_counter=1;
start slave sql_thread;

GTID :单个复制

stop  slave sql_thread;
set  gtid_next = 'uuid:N';
begin;commit;
set gtid_next='automatic';
start slave sql_thread;

uuid:N 为 show slave status\G 查看到的 Retrieved_Gtid_Set 的 Gtid 值 3229c868-9d0f-11e7-9053-7845c401c236:1-1920,如果查看到的 Executed_Gtid_Set 为:3229c868-9d0f-11e7-9053-7845c401c236:1-8,那么 uuid:N 就要设置为 3229c868-9d0f-11e7-9053-7845c401c236:9
注意:看到的 Retrieved_Gtid_Set 收到的 3229c868-9d0f-11e7-9053-7845c401c236 必须和 Executed_Gtid_Set 3229c868-9d0f-11e7-9053-7845c401c236 这两个 uuid 值一样 才能确认是一个事务;一定要确认是同一个事务;

(2)多源复制解决方法:

多源 binlog+pos:

stop  slave sql_thread for channel 'master-1';
set global sql_slave_skip_counter=1;
start slave sql_thread for channlel 'master-1';

GTID: 多源复制

stop  slave sql_thread   for channel 'master-1';
set  gtid_next = 'uuid:N';
begin;commit;
set gtid_next='automatic';
start slave sql_thread for channel 'master-1';

uuid:N 为 show slave status\G 查看到的 Retrieved_Gtid_Set 的 Gtid 值 3229c868-9d0f-11e7-9053-7845c401c236:1-1920,如果查看到 Executed_Gtid_Set 的为:3229c868-9d0f-11e7-9053-7845c401c236:1-8,那么 uuid:N 就要设置为 3229c868-9d0f-11e7-9053-7845c401c236:9
注意:看到的 Retrieved_Gtid_Set 收到的 3229c868-9d0f-11e7-9053-7845c401c236 必须和 Executed_Gtid_Set 3229c868-9d0f-11e7-9053-7845c401c236 这两个 uuid 值一样 才能确认是一个事务;一定要确认是同一个事务;

2.复制账号重复问题

方法一:

set  sql_log_bin=0;
grant all privileges on ....;
set sql_log_bin=1;

创建用户时,不管是否使用 use mysql;新创建的用户都不会复制到从库;
方法二:

stop slave sql_thread; change replication filter Replicate_ignore_DB=(mysql);

创建用户时,先执行 use mysql; 然后再 create user XXX; 否则新创建的用户还是会复制到从库;

四、遇到的问题

因 Master_UUID 值相同,导致其中一个 master 无法同步;

root@localhost [(none)]>show slave status\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 192.168.199.230
                  Master_User: repl
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000001
          Read_Master_Log_Pos: 154
               Relay_Log_File: relay-bin-master@002d1.000002
                Relay_Log_Pos: 367
        Relay_Master_Log_File: mysql-bin.000001
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              ................
             Master_Server_Id: 1
                  Master_UUID: b5a3240c-8946-11e7-bf07-d067e528dfb8
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  ...................
                 Channel_Name: master-1
           Master_TLS_Version: 
*************************** 2. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 192.168.199.231
                  Master_User: repl
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000002
          Read_Master_Log_Pos: 154
               Relay_Log_File: relay-bin-master@002d2.000002
                Relay_Log_Pos: 367
        Relay_Master_Log_File: mysql-bin.000002
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
             ..........................
             Master_Server_Id: 2
                  Master_UUID: b5a3240c-8946-11e7-bf07-d067e528dfb8
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  ..........................
                 Channel_Name: master-2
           Master_TLS_Version: 
2 rows in set (0.00 sec)

解决方法:
把其中一个 master 的 auto.cnf 文件删掉,重启一下 mysql 即可生成新的 server-uuid

# rm -f /data/mysql/mysql3306/data/auto.cnf 
# /etc/init.d/mysql restart

参考文档

https://dev.mysql.com/doc/refman/5.7/en/replication-multi-source-overview.html
https://dev.mysql.com/doc/refman/5.7/en/replication-multi-source-tutorials.html
https://dev.mysql.com/doc/refman/5.7/en/change-master-to.html
http://www.linuxidc.com/Linux/2017-05/143929.htm

posted @ 2018-12-18 16:12  workdsz  阅读(432)  评论(0)    收藏  举报