【Canal】Canal Adapter 部署与使用

Canal部署

　　【Canal】Canal快速入门

 Canal Adapter部署

1、下载

　　下载地址：https://github.com/alibaba/canal/releases，本例下载的是1.1.7版本，1.1.6版本使用jdk8启动有点问题

2、修改配置

　　修改conf/bootstrap.yml配置，将其中内容注释掉

# canal:
#   manager:
#     jdbc:
#       url: jdbc:mysql://127.0.0.1:3306/canal_manager?useUnicode=true&characterEncoding=UTF-8
#       username: canal
#       password: canal

 

 　　修改conf/application.yml配置

server:
  port: 8081
spring:
  jackson:
    date-format: yyyy-MM-dd HH:mm:ss
    time-zone: UTC
    default-property-inclusion: non_null

canal.conf:
  mode: tcp #tcp kafka rocketMQ rabbitMQ
  flatMessage: true
  zookeeperHosts:
  syncBatchSize: 1000
  retries: -1
  timeout:
  accessKey:
  secretKey:
  consumerProperties:
    # canal tcp consumer
    canal.tcp.server.host: 127.0.0.1:11111
    canal.tcp.zookeeper.hosts:
    canal.tcp.batch.size: 500
    canal.tcp.username: admin
    canal.tcp.password: 4ACFE3202A5FF5CF467898FC58AAB1D615029441
    # # kafka consumer
    # kafka.bootstrap.servers: 127.0.0.1:9092
    # kafka.enable.auto.commit: false
    # kafka.auto.commit.interval.ms: 1000
    # kafka.auto.offset.reset: latest
    # kafka.request.timeout.ms: 40000
    # kafka.session.timeout.ms: 30000
    # kafka.isolation.level: read_committed
    # kafka.max.poll.records: 1000
    # # rocketMQ consumer
    # rocketmq.namespace:
    # rocketmq.namesrv.addr: 127.0.0.1:9876
    # rocketmq.batch.size: 1000
    # rocketmq.enable.message.trace: false
    # rocketmq.customized.trace.topic:
    # rocketmq.access.channel:
    # rocketmq.subscribe.filter:
    # # rabbitMQ consumer
    # rabbitmq.host:
    # rabbitmq.virtual.host:
    # rabbitmq.username:
    # rabbitmq.password:
    # rabbitmq.resource.ownerId:

  srcDataSources:
    defaultDS:
      url: jdbc:mysql://127.0.01:3306/origin_mysql?characterEncoding=UTF-8
      username: root
      password: 123456
  canalAdapters:
  - instance: example # canal instance Name or mq topic name
    groups:
    - groupId: g1
      outerAdapters:
      - name: logger
      - name: rdb
        key: to_mysql
        properties:
          jdbc.driverClassName: com.mysql.cj.jdbc.Driver
          jdbc.url: jdbc:mysql://127.0.01:3306/target_mysql?characterEncoding=UTF-8
          jdbc.username: root
          jdbc.password: 123456
          druid.stat.enable: false
          druid.stat.slowSqlMillis: 1000
#      - name: rdb
#        key: oracle1
#        properties:
#          jdbc.driverClassName: oracle.jdbc.OracleDriver
#          jdbc.url: jdbc:oracle:thin:@localhost:49161:XE
#          jdbc.username: mytest
#          jdbc.password: m121212
#      - name: rdb
#        key: postgres1
#        properties:
#          jdbc.driverClassName: org.postgresql.Driver
#          jdbc.url: jdbc:postgresql://localhost:5432/postgres
#          jdbc.username: postgres
#          jdbc.password: 121212
#          threads: 1
#          commitSize: 3000
#      - name: hbase
#        properties:
#          hbase.zookeeper.quorum: 127.0.0.1
#          hbase.zookeeper.property.clientPort: 2181
#          zookeeper.znode.parent: /hbase
#      - name: es
#        hosts: 127.0.0.1:9300 # 127.0.0.1:9200 for rest mode
#        properties:
#          mode: transport # or rest
#          # security.auth: test:123456 #  only used for rest mode
#          cluster.name: elasticsearch
#      - name: kudu
#        key: kudu
#        properties:
#          kudu.master.address: 127.0.0.1 # ',' split multi address
#      - name: phoenix
#        key: phoenix
#        properties:
#          jdbc.driverClassName: org.apache.phoenix.jdbc.PhoenixDriver
#          jdbc.url: jdbc:phoenix:127.0.0.1:2181:/hbase/db
#          jdbc.username:
#          jdbc.password:

  　　修改conf/rdb/mytest_user.yml配置

dataSourceKey: defaultDS
destination: example
groupId: g1
outerAdapterKey: to_mysql
concurrent: true
dbMapping:
  database: origin_mysql
  table: user
  targetTable: user
  targetPk:
    id: id
#  mapAll: true
  targetColumns:
    id: id
    name: name
    remarks: remarks
  # etlCondition: "where delete_flag = 0"
  commitBatch: 300 # 批量提交的大小



## Mirror schema synchronize config
#dataSourceKey: defaultDS
#destination: example
#groupId: g1
#outerAdapterKey: mysql1
#concurrent: true
#dbMapping:
#  mirrorDb: true
#  database: mytest

 Canal Adapter使用

　　启动命令：./bin/startup.bat

　　修改原表user的数据，即可同步到目标表中

参考：https://juejin.cn/post/7120135900385771556

参考：https://blog.csdn.net/steve_frank/article/details/144262012