消息队列 -- Apache RocketMQ 概述

1.官方资料

1.1 官方文档详解 (推荐阅读所有相关 markdown & 源码)

https://github.com/apache/rocketmq/tree/master/docs/cn

 

2.安装指导

2.1 可参考下述官方文档(单master, 多master, 多maseter多slave-异步复制, 多master多slave-同步复制)

 https://github.com/apache/rocketmq/blob/master/docs/cn/operation.md 

 

 

broker.conf 常见配置项:

brokerClusterName=rocketmq-cluster
brokerName=broker-a
brokerId=0
#这个很有讲究 如果是正式环境 这里一定要填写内网地址（安全）
#当有多个网卡时, 需要注意一定要配置上, 否则注册到namesrv的可能是其中一个网卡对应的IP, producer/consumer无法与之建立连接
brokerIP1=192.168.0.156
#凌晨4点删除commitlog
deleteWhen=04
fileReservedTime=48
brokerRole=ASYNC_MASTER
flushDiskType=ASYNC_FLUSH
# 内网的(阿里云有内网IP和外网IP)
namesrvAddr=192.168.0.156:9876;192.168.0.156:9877
autoCreateTopicEnable=true
#Broker 对外服务的监听端口,
listenPort=10911
#Broker角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=ASYNC_MASTER
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH

 

更多配置项参考:

org.apache.rocketmq.store.config.MessageStoreConfig
org.apache.rocketmq.common.BrokerConfig
等等各种Config类, 注意标注了@ImportantField的属性

 

2.2 docker-compose 搭建2m模式

2.2.1 配置文件

2.2.1.1 /home/rocketmq/broker-a/ 文件夹下创建文件: broker-a.conf

 

brokerClusterName=rocketmq-cluster
brokerName=broker-a
brokerId=0
#这个很有讲究 如果是正式环境 这里一定要填写内网地址（安全）
#如果是用于测试或者本地这里建议要填外网地址，因为你的本地代码是无法连接到阿里云内网，只能连接外网。
brokerIP1=192.168.0.156
deleteWhen=04
fileReservedTime=48
brokerRole=ASYNC_MASTER
flushDiskType=ASYNC_FLUSH
# 内网的(阿里云有内网IP和外网IP)
namesrvAddr=192.168.0.156:9876;192.168.0.156:9877
autoCreateTopicEnable=true
#Broker 对外服务的监听端口,
listenPort=10911
#Broker角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=ASYNC_MASTER
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH

 

2.2.1.2 /home/rocketmq/broker-b/ 文件夹下创建文件: broker-b.conf

brokerClusterName = rocketmq-cluster
brokerName = broker-b
brokerId = 0
brokerIP1 = 192.168.0.156
deleteWhen = 04
fileReservedTime = 48
brokerRole = ASYNC_MASTER
flushDiskType = ASYNC_FLUSH
# 内网的(阿里云有内网IP和外网IP)
namesrvAddr=192.168.0.156:9876;192.168.0.156:9877
autoCreateTopicEnable=true
#Broker 对外服务的监听端口,
listenPort = 10909
#Broker角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=ASYNC_MASTER
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH

 

2.2.1.3 docker-compose.yml

version: '3.5'
services:
  rmqnamesrv-a:
    image: rocketmqinc/rocketmq:4.4.0
    container_name: rmqnamesrv-a
    ports:
      - 9876:9876
    volumes:
      - /opt/rocketmq/logs/nameserver-a:/opt/logs
      - /opt/rocketmq/store/nameserver-a:/opt/store
    command: sh mqnamesrv
    networks:
        rmq:
          aliases:
            - rmqnamesrv-a

  rmqnamesrv-b:
    image: rocketmqinc/rocketmq:4.4.0
    container_name: rmqnamesrv-b
    ports:
      - 9877:9876
    volumes:
      - /opt/rocketmq/logs/nameserver-b:/opt/logs
      - /opt/rocketmq/store/nameserver-b:/opt/store
    command: sh mqnamesrv
    networks:
        rmq:
          aliases:
            - rmqnamesrv-b

  rmqbroker-a:
    image: rocketmqinc/rocketmq:4.4.0
    container_name: rmqbroker-a
    ports:
      - 10911:10911
    #解决 docker-compose 的 volume 映射发生 Permission denied 问题
    privileged: true
    volumes:
      - /opt/rocketmq/logs/broker-a:/opt/logs
      - /opt/rocketmq/store/broker-a:/opt/store
      - /home/rocketmq/broker-a/broker-a.conf:/opt/rocketmq-4.4.0/conf/broker.conf 
    environment:
        TZ: Asia/Shanghai
        NAMESRV_ADDR: "rmqnamesrv-a:9876"
        JAVA_OPTS: " -Duser.home=/opt"
        JAVA_OPT_EXT: "-server -Xms256m -Xmx256m -Xmn256m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m"
    command: sh mqbroker -n 192.168.0.156:9876;192.168.0.156:9877 -c /opt/rocketmq-4.4.0/conf/broker.conf autoCreateTopicEnable=true &
    depends_on:
      - rmqnamesrv-a
      - rmqnamesrv-b
    links:
      - rmqnamesrv-a:rmqnamesrv-a
      - rmqnamesrv-b:rmqnamesrv-b
    networks:
      rmq:
        aliases:
          - rmqbroker-a

  rmqbroker-b:
    image: rocketmqinc/rocketmq:4.4.0
    container_name: rmqbroker-b
    ports:
      - 10909:10909
    #解决 docker-compose 的 volume 映射发生 Permission denied 问题
    privileged: true
    volumes:
      - /opt/rocketmq/logs/broker-b:/opt/logs
      - /opt/rocketmq/store/broker-b:/opt/store
      - /home/rocketmq/broker-b/broker-b.conf:/opt/rocketmq-4.4.0/conf/broker.conf 
    environment:
        TZ: Asia/Shanghai
        NAMESRV_ADDR: "rmqnamesrv-b:9876"
        JAVA_OPTS: " -Duser.home=/opt"
        JAVA_OPT_EXT: "-server -Xms256m -Xmx256m -Xmn256m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m"
    command: sh mqbroker -n 192.168.0.156:9876;192.168.0.156:9877 -c /opt/rocketmq-4.4.0/conf/broker.conf autoCreateTopicEnable=true &
    depends_on:
      - rmqnamesrv-a
      - rmqnamesrv-b
    links:
      - rmqnamesrv-a:rmqnamesrv-a
      - rmqnamesrv-b:rmqnamesrv-b
    networks:
      rmq:
        aliases:
          - rmqbroker-b
  rmqconsole:
    image: styletang/rocketmq-console-ng
    container_name: rmqconsole
    # 注意, 这里端口映射时, 要是8080.
    ports:
      - 9001:8080
    environment:
        JAVA_OPTS: "-Drocketmq.namesrv.addr=rmqnamesrv-a:9876;rmqnamesrv-b:9877 -Dcom.rocketmq.sendMessageWithVIPChannel=false"
    depends_on:
      - rmqnamesrv-a
      - rmqnamesrv-b
    networks:
      rmq:
        aliases:
          - rmqconsole
networks:
  rmq:
    name: rmq
    driver: bridge

 

2.2.2 启动

#执行前, 先确保
#开启docker挂载权限: chmod a+rw /var/run/docker.sock
#开启了可执行权限: chmod +x /usr/local/bin/docker-compose
docker-compose -f docker-compose.yml up -d

 

2.2.3 启动成功后如下图

 

 

2.2.4 从管理台查看RocketMQ 情况

 

 

3.ACL

org.apache.rocketmq.acl.plain.PlainPermissionManager#validate

 

 

 

参考资源