RacketMQ

RocketMQ

1.MQ简介

  • MQ(Message Queue)消息队列,是一种用来保存消息数据的队列
    • 队列:数据结构的一种,特征为"先进先出"

2.MQ作用

  1. 应用解耦(异步的消息发送)

  2. 快速应用变更维护(异步的消息发送)

  3. 流量削锋(削峰填谷)(异步的消息发送)

如果mysql的qps是1000,但是突然来了4000个请求,mysql就顶不住了,但是有了mq,我们就可以把这4000个请求都放到mq中,把这4000qps分为四次发给mysql

3.MQ的优缺点

优点

  1. 应用解耦(异步的消息发送)
  2. 快速应用变更维护(异步的消息发送)
  3. 流量削锋(削峰填谷)(异步的消息发送)

缺点

  1. 系统可用性降低: 集群
  2. 系统复杂度提高:(程序员提升水平)
  3. 异步消息机制(都有解决方案)
    消息顺序性
    消息丢失
    消息一致性
    消息重复使用

4.常见的MQ产品

ActiveMQ:java语言实现,万级数据吞吐量,处理速度ms级,主从架构,成熟度高
RabbitMQ :erlang语言实现,万级数据吞吐量,处理速度us级,主从架构,
RocketMQ :java语言实现,十万级数据吞吐量,处理速度ms级,分布式架构,功能强大,扩展性强
kafka :scala语言实现,十万级数据吞吐量,处理速度ms级,分布式架构,功能较少,应用于大数据较多

RocketMQ阿里开源的一款非常优秀中间件产品,脱胎于阿里的另一款队列技术MetaQ,后捐赠给Apache基金会作为一款孵化技术,仅仅经历了一年多的时间就成为Apache基金会的顶级项目。并且它现在已经在阿里内部被广泛的应用,并且经受住了多次双十一的这种极致场景的压力(2017年的双十一,RocketMQ流转的消息量达到了万亿级,峰值TPS达到5600万)
解决所有缺点

5.安装

jdk

1)解压 jdk
tar -zxvf jdk-8u171-linux-x64.tar.gz
2)配置环境变量
>vim /etc/profile
export JAVA_HOME=/opt/jdk1.8.0_171
export PATH=$PATH:${JAVA_HOME}/bin
3)重新加载配置
>source /etc/profile
>java -version

错误解决

如果安装完毕 jdk 后  java -version 看到的是 openjdk(需要删除)
因为 操作系统默认已经安装了 opendjdk,
# 查看
rpm -qa | grep java
# 删除(把上一个命令看到的所有的jdk文件 用 如下命令删除)
rpm -e --nodeps java-1.8.0-openjdk-1.8.0.232.b09-0.el7_7.x86_64
rpm -e --nodeps java-1.8.0-openjdk-headless-1.8.0.232.b09-0.el7_7.x86_64
rpm -e --nodeps java-1.7.0-openjdk-headless-1.7.0.241-2.6.20.0.el7_7.x86_64
rmp -e --nodeps java-1.7.0-openjdk-1.7.0.241-2.6.20.0.el7_7.x86_64
rpm -e --nodeps java-1.7.0-openjdk-1.7.0.241-2.6.20.0.el7_7.x86_64

rocketMQ

# 解压
unzip rocketmq-all-4.5.2-bin-release.zip
# 修改目录名称
mv rocketmq-all-4.5.2-bin-release rocketmq
# 调整启动内存 为128m
runserver.sh
runbroker.sh

如果和后天的课程(docker 一起 需要修改)

conf/broker.conf

brokerClusterName = DefaultCluster
brokerName = broker-a
brokerId = 0
deleteWhen = 04
fileReservedTime = 48
brokerRole = ASYNC_MASTER
flushDiskType = ASYNC_FLUS
# 解决和docker 冲突的
brokerIP1=192.168.31.80
namesrvAddr=192.168.31.80:9876

启动

#启动nameserv
sh mqnamesrv
# 启动mq  服务  -n 指定 nameserv 的地址(bin)
sh mqbroker -n localhost:9876  -c ../conf/broker.conf
# 关闭防火墙
systemctl stop firewalld.service 

测试

export NAMESRV_ADDR=localhost:9876
bin 目录
sh tools.sh org.apache.rocketmq.example.quickstart.Producer
sh tools.sh org.apache.rocketmq.example.quickstart.Consumer

6.消息发送

环境搭建

<dependency>
     <groupId>org.apache.rocketmq</groupId>
     <artifactId>rocketmq-client</artifactId>
     <version>4.5.2</version>
</dependency>

消息

1.单生产者单消费者消息发送(OneToOne)

生产者,产生消息

package com.itheima.base;

import org.apache.rocketmq.client.producer.DefaultMQProducer;
import org.apache.rocketmq.client.producer.SendResult;
import org.apache.rocketmq.common.message.Message;

/**
 * @Author: hepeng
 * @Date: 2022/2/21 22:24
 * 生产者产生消息
 */
public class Producer {
    public static void main(String[] args) throws Exception {
        //1.创建一个发送消息的对象
        DefaultMQProducer producer = new DefaultMQProducer("group1");
        //2.设定发送的命名服务器地址
        producer.setNamesrvAddr("192.168.200.128:9876");
        //3.1 启动发送的服务
        producer.start();
        //4.创建要发送的消息对象,指定topic,指定内容body
        Message msg = new Message("topic1","hello rocketmq".getBytes("UTF-8"));
        //3.2 发送消息
        SendResult result = producer.send(msg);
        System.out.println("返回结果"+result);
        //返回结果
        //5.关闭连接
        producer.shutdown();
    }
}
SendResult [sendStatus=SEND_OK, msgId=A9FE7C2907F418B4AAC26BE8D6350000, offsetMsgId=AC11000100002A9F000000000002DF62, messageQueue=MessageQueue[topic=topic1, brokerName=localhost.localdomain, queueId=2], queueOffset=12]

消费者,接收消息

package com.itheima.base;

import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyContext;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyStatus;
import org.apache.rocketmq.client.consumer.listener.MessageListenerConcurrently;
import org.apache.rocketmq.common.message.MessageExt;

import java.util.List;

/**
 * @Author: hepeng
 * @Date: 2022/2/21 22:24
 * 消费者,消费消息
 */
public class Consumer {
    public static void main(String[] args) throws Exception{
        //1.创建一个接收消息的对象
        DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("group1");
        //2.设定接收的命名服务器地址
        consumer.setNamesrvAddr("192.168.200.128:9876");
        //3.设置接收消息对应的topic,订阅消息,对应的sub标签为任意
        consumer.subscribe("topic1","*");
        //4.开启监听,用于接收消息
        consumer.registerMessageListener(new MessageListenerConcurrently() {
            public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> list,
                ConsumeConcurrentlyContext consumeConcurrentlyContext) {
                //遍历消息
                for (MessageExt msg : list) {
                    System.out.println("收到消息"+msg);
                    System.out.println("消息"+new String(msg.getBody()));
                }
                return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
            }
        });
        //5.启动接收消息的服务
        consumer.start();
        System.out.println("接收消息服务已开启运行");

    }
}
接收消息服务已开启运行
收到消息MessageExt [queueId=1, storeSize=164, queueOffset=15, sysFlag=0, bornTimestamp=1645456838142, bornHost=/192.168.200.1:18813, storeTimestamp=1645456838144, storeHost=/172.17.0.1:10911, msgId=AC11000100002A9F000000000002EAE9, commitLogOffset=191209, bodyCRC=328254045, reconsumeTimes=0, preparedTransactionOffset=0, toString()=Message{topic='topic1', flag=0, properties={MIN_OFFSET=0, MAX_OFFSET=16, CONSUME_START_TIME=1645456846037, UNIQ_KEY=A9FE7C29442C18B4AAC26C0181FE0006, WAIT=true}, body=[104, 101, 108, 108, 111, 32, 114, 111, 99, 107, 101, 116, 109, 113, 55], transactionId='null'}]
消息hello rocketmq

2.单生产者对多消费者(OneToMany)

单生产者

package com.itheima.one2many;

import org.apache.rocketmq.client.producer.DefaultMQProducer;
import org.apache.rocketmq.client.producer.SendResult;
import org.apache.rocketmq.common.message.Message;

/**
 * @Author: hepeng
 * @Date: 2022/2/21 22:24
 * 生产者产生消息
 */
public class Producer {
    public static void main(String[] args) throws Exception {
        //1.创建一个发送消息的对象
        DefaultMQProducer producer = new DefaultMQProducer("group1");
        //2.设定发送的命名服务器地址
        producer.setNamesrvAddr("192.168.200.128:9876");
        //3.1 启动发送的服务
        producer.start();
        //4.创建要发送的消息对象,指定topic,指定内容body
        for (int i = 1; i <= 10; i++) {
            Message msg = new Message("topic1",("hello rocketmq"+ i).getBytes("UTF-8"));
            //3.2 发送消息
            SendResult result = producer.send(msg);
            System.out.println("返回结果"+result);
        }
        //5.关闭连接
        producer.shutdown();
    }
}

多消费者

package com.itheima.one2many;

import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyContext;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyStatus;
import org.apache.rocketmq.client.consumer.listener.MessageListenerConcurrently;
import org.apache.rocketmq.common.message.MessageExt;
import org.apache.rocketmq.common.protocol.heartbeat.MessageModel;

import java.util.List;

/**
 * @Author: hepeng
 * @Date: 2022/2/21 22:24
 * 消费者,消费消息
 */
public class Consumer {
    public static void main(String[] args) throws Exception{
        //1.创建一个接收消息的对象
        DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("group1");
        //2.设定接收的命名服务器地址
        consumer.setNamesrvAddr("192.168.200.128:9876");
        //3.设置接收消息对应的topic,订阅消息,对应的sub标签为任意
        consumer.subscribe("topic1","*");
        //设置当前消费者的消费模式(默认为负载均衡)
        //consumer.setMessageModel(MessageModel.CLUSTERING);
        //设置当前消费者的消费模式为广播模式:所有客户端接收的消息都一样
        consumer.setMessageModel(MessageModel.BROADCASTING);
        //4.开启监听,用于接收消息
        consumer.registerMessageListener(new MessageListenerConcurrently() {
            public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> list,
                ConsumeConcurrentlyContext consumeConcurrentlyContext) {
                //遍历消息
                for (MessageExt msg : list) {
                    // System.out.println("收到消息"+msg);
                    System.out.println("消息"+new String(msg.getBody()));
                }
                return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
            }
        });
        //5.启动接收消息的服务
        consumer.start();
        System.out.println("接收消息服务已开启运行");

    }
}

3.多生产者多消费者消息发送(ManyToMany)

生产者运行多次

/**
 * @Author: hepeng
 * @Date: 2022/2/21 22:24
 * 生产者产生消息
 */
public class Producer {
    public static void main(String[] args) throws Exception {
        DefaultMQProducer producer = new DefaultMQProducer("group1");
        producer.setNamesrvAddr("192.168.200.128:9876");
        producer.start();
        for (int i = 1; i <= 10; i++) {
            Message msg = new Message("topic1",("生产者1:"+"hello rocketmq"+ i).getBytes("UTF-8"));
            SendResult result = producer.send(msg);
            System.out.println("返回结果"+result);
        }
        producer.shutdown();
    }
}

消费者

/**
 * @Author: hepeng
 * @Date: 2022/2/21 22:24
 * 消费者,消费消息
 */
public class Consumer {
    public static void main(String[] args) throws Exception{
        DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("group1");
        consumer.setNamesrvAddr("192.168.200.128:9876");
        consumer.subscribe("topic1","*");
        consumer.registerMessageListener(new MessageListenerConcurrently() {
            public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> list,
                ConsumeConcurrentlyContext consumeConcurrentlyContext) {
                for (MessageExt msg : list) {
                    System.out.println("消息"+new String(msg.getBody()));
                }
                return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
            }
        });
        consumer.start();
        System.out.println("接收消息服务已开启运行");
    }
}

4. 消息分类

  • 同步消息

    特征:即时性强,重要的消息,且必须有回执的消息,例如短信,通知(转账成功)

    SendResult result = producer.send(msg);
    
  • 异步消息

    特征:即时性较弱,但需要有回执的消息,例如订单中的某些信息

    public class Producer {
        public static void main(String[] args) throws Exception {
            DefaultMQProducer producer = new DefaultMQProducer("group1");
            producer.setNamesrvAddr("192.168.200.128:9876");
            producer.start();
            for (int i = 1; i <= 10; i++) {
                //异步消息发送
                Message msg = new Message("topic2",("异步消息:"+"hello rocketmq"+ i).getBytes("UTF-8"));
                producer.send(msg, new SendCallback() {
                    //表示成功返回结果
                    public void onSuccess(SendResult sendResult) {
                        System.out.println(sendResult);
                    }
                    //表示消息发送失败
                    public void onException(Throwable throwable) {
                        System.out.println(throwable);
                    }
                });
            }
            //添加一个休眠操作,确保异步消息能够成功回调
            TimeUnit.SECONDS.sleep(10);
            producer.shutdown();
        }
    }
    
  • 单向消息

    特征:不需要有回执的消息,例如日志类消息

    producer.sendOneway(msg);
    

5. 延时消息

立刻发送, 只是 告诉MQ ,消息隐藏一段时间再暴露

应用场景 
   下订单时 网mq  发一个取消订单消息 (订单号 30分钟演示)
   30分钟后,消费者能看到这个消息,开始处理取消订单(如果没付费)
  • 消息发送时并不是直接到达消息服务器,而是根据设定的等待时间到达,起到延时到达的缓冲效果
public static void main(String[] args) throws Exception {
        DefaultMQProducer producer = new DefaultMQProducer("group1");
        producer.setNamesrvAddr("192.168.31.80:9876");
        producer.start();
        for (int i = 1; i <= 5; i++) {
            Message msg = new Message("topic3",("非延时消息:hello rocketmq "+i).getBytes("UTF-8"));
           // 30秒后再发送,而是先发送,但是通知mq , 30s 才对外暴露数据
            //设置当前消息的延时效果(比如订单,下订单后,20分钟后,决定这个订单是否删除,)
            msg.setDelayTimeLevel(3);
            // 1s 5s 10s 30s 1m 2m 3m 4m 5m 6m 7m 8m 9m 10m 20m 30m 1h 2h
            SendResult result = producer.send(msg);
            System.out.println("返回结果:"+result);
        }
        producer.shutdown();
    }
  • 目前支持的消息时间
    • 秒级:1,5,10,30
    • 分级:1-10,20,30
    • 时级1,2
    • 1s 5s 10s 30s 1m 2m 3m 4m 5m 6m 7m 8m 9m 10m 20m 30m 1h 2h

6. 批量消息

public class Producer {
    public static void main(String[] args) throws Exception {
        DefaultMQProducer producer = new DefaultMQProducer("group1");
        producer.setNamesrvAddr("192.168.200.128:9876");
        producer.start();

        //创建一个集合,保存多个消息
        List<Message> list = new ArrayList<Message>();

        Message msg1 = new Message("topic1",("批量消息:"+"hello rocketmq").getBytes("UTF-8"));
        Message msg2 = new Message("topic1",("批量消息:"+"hello rocketmq").getBytes("UTF-8"));
        Message msg3 = new Message("topic1",("批量消息:"+"hello rocketmq").getBytes("UTF-8"));

        list.add(msg1);
        list.add(msg2);
        list.add(msg3);

        SendResult result = producer.send(list);
        System.out.println("返回结果"+result);
        producer.shutdown();
    }
}

注意:

消息内容总长度不超过4M
消息内容总长度包含如下:
topic(字符串字节数)
body (字节数组长度)
消息追加的属性(key与value对应字符串字节数)
日志(固定20字节)

7. Tag

发送者

public static void main(String[] args) throws Exception {
    DefaultMQProducer producer = new DefaultMQProducer("group1");
    producer.setNamesrvAddr("192.168.31.80:9876");
    producer.start();

    //创建消息的时候除了制定topic,还可以指定tag
    Message msg = new Message("topic6","tag2",("消息过滤按照tag:hello rocketmq 2").getBytes("UTF-8"));

    SendResult send = producer.send(msg);
    System.out.println(send);

    producer.shutdown();
}

消费者

* 代表任意tag
"tag1 || tag2" 代表两个  tag  那个都行

//接收消息的时候,除了制定topic,还可以指定接收的tag,*代表任意tag
consumer.subscribe("topic6","tag1 || tag2");
posted @ 2022-02-22 00:18  正在努力的澎澎  阅读(127)  评论(0)    收藏  举报