• 博客园logo
  • 会员
  • 众包
  • 新闻
  • 博问
  • 闪存
  • 赞助商
  • HarmonyOS
  • Chat2DB
    • 搜索
      所有博客
    • 搜索
      当前博客
  • 写随笔 我的博客 短消息 简洁模式
    用户头像
    我的博客 我的园子 账号设置 会员中心 简洁模式 ... 退出登录
    注册 登录
孙龙 程序员
少时总觉为人易,华年方知立业难
博客园    首页    新随笔    联系   管理    订阅  订阅
RabbitMq使用详解
php实现rabbitmq,golang实现rabbitmq;RabbitMq使用说明,rabbitmq消息队列安装,启动,消息幂等性,confirm消息确认机制,rabbitmq事务使用,return消息机制,消息端的手工ACK和NACK与重回队列,消息TTL过期时间;死信队列;消费端的限流;消息持久化:

说明:

几个概念说明:
Broker:简单来说就是消息队列服务器实体。
  Exchange:消息交换机,它指定消息按什么规则,路由到哪个队列。
  Queue:消息队列载体,每个消息都会被投入到一个或多个队列。
  Binding:绑定,它的作用就是把exchange和queue按照路由规则绑定起来。
  Routing Key:路由关键字,exchange根据这个关键字进行消息投递。
  vhost:虚拟主机,一个broker里可以开设多个vhost,用作不同用户的权限分离。
  producer:消息生产者,就是投递消息的程序。
  consumer:消息消费者,就是接受消息的程序。
  channel:消息通道,在客户端的每个连接里,可建立多个channel,每个channel代表一个会话任务。
消息队列的使用过程大概如下:
(1)客户端连接到消息队列服务器,打开一个channel。
  (2)客户端声明一个exchange,并设置相关属性。
  (3)客户端声明一个queue,并设置相关属性。
  (4)客户端使用routing key,在exchange和queue之间建立好绑定关系。
  (5)客户端投递消息到exchange。
exchange接收到消息后,就根据消息的key和已经设置的binding,进行消息路由,将消息投递到一个或多个队列里。
exchange也有几个类型,完全根据key进行投递的叫做Direct交换机,例如,绑定时设置了routing key为”abc”,那么客户端提交的消息,只有设置了key为”abc”的才会投递到队列。
对key进行模式匹配后进行投递的叫做Topic交换机,符号”#”匹配一个或多个词,符号”
*”匹配正好一个词。例如”abc.#”匹配”abc.def.ghi”,”abc.*”只匹配”abc.def”。
还有一种不需要key的,叫做Fanout交换机,它采取广播模式,一个消息进来时,投递到与该交换机绑定的所有队列。 RabbitMQ支持消息的持久化,也就是数据写在磁盘上,为了数据安全考虑,我想大多数用户都会选择持久化。消息队列持久化包括3个部分:   (
1)exchange持久化,在声明时指定durable => 1   (2)queue持久化,在声明时指定durable => 1   (3)消息持久化,在投递时指定delivery_mode => 2(1是非持久化) 如果exchange和queue都是持久化的,那么它们之间的binding也是持久化的。如果exchange和queue两者之间有一个持久化,一个非持久化,就不允许建立绑定。

 

安装:

下载:
wget www.rabbitmq.com/releases/erlang/erlang-18.3-1.el7.centos.x86_64.rpm
wget http://repo.iotti.biz/CentOS/7/x86_64/socat-1.7.3.2-5.el7.lux.x86_64.rpm
wget www.rabbitmq.com/releases/rabbitmq-server/v3.6.5/rabbitmq-server-3.6.5-1.noarch.rpm

安装:
rpm -ivh  erlang-18.3-1.el7.centos.x86_64.rpm 
rpm -ivh  rabbitmq-server-3.6.5-1.noarch.rpm
rpm -ivh  socat-1.7.3.2-1.1.el7.x86_64.rpm

配置文件:
vim /usr/lib/rabbitmq/lib/rabbitmq_server-3.6.5/ebin/rabbit.app
比如修改密码、配置等等,例如:loopback_users 中的 <<"guest">>,只保留guest
服务启动和停止:
启动 rabbitmq-server start &
停止 rabbitmqctl app_stop

rabbitmqctl start_app
rabbitmq-plugins enable rabbitmq_management
rabbitmqctl stop

管理插件:rabbitmq-plugins enable rabbitmq_management
访问地址:http://192.168.11.76:15672/

 

 本地测试环境移除所有数据:

rabbitmqtl reset 移除所有数据 要在 rabbitmqctl stop_app之后使用

 

[root@localhost ~]# rabbitmq
rabbitmqctl       rabbitmq-plugins  rabbitmq-server

 

[root@localhost soft]# rabbitmqctl list_queues
Listing queues ...
task_queue123   0

[root@localhost soft]# rabbitmqctl list_vhosts
Listing vhosts ...
/
/sunlong

  

[root@localhost soft]# rabbitmqctl list_exchanges
Listing exchanges ...
amq.fanout      fanout
amq.match       headers
        direct
amq.rabbitmq.trace      topic
amq.headers     headers
amq.direct      direct
amq.rabbitmq.log        topic
amq.topic       topic

  

  

 

 1 [root@localhost soft]# rabbitmqctl status
 2 Status of node rabbit@localhost ...
 3 [{pid,2521},
 4  {running_applications,
 5      [{rabbitmq_management,"RabbitMQ Management Console","3.6.5"},
 6       {rabbitmq_management_agent,"RabbitMQ Management Agent","3.6.5"},
 7       {rabbit,"RabbitMQ","3.6.5"},
 8       {os_mon,"CPO  CXC 138 46","2.4"},
 9       {rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.6.5"},
10       {webmachine,"webmachine","1.10.3"},
11       {mochiweb,"MochiMedia Web Server","2.13.1"},
12       {compiler,"ERTS  CXC 138 10","6.0.3"},
13       {ssl,"Erlang/OTP SSL application","7.3"},
14       {public_key,"Public key infrastructure","1.1.1"},
15       {ranch,"Socket acceptor pool for TCP protocols.","1.2.1"},
16       {amqp_client,"RabbitMQ AMQP Client","3.6.5"},
17       {asn1,"The Erlang ASN1 compiler version 4.0.2","4.0.2"},
18       {inets,"INETS  CXC 138 49","6.2"},
19       {rabbit_common,[],"3.6.5"},
20       {xmerl,"XML parser","1.3.10"},
21       {mnesia,"MNESIA  CXC 138 12","4.13.3"},
22       {syntax_tools,"Syntax tools","1.7"},
23       {crypto,"CRYPTO","3.6.3"},
24       {sasl,"SASL  CXC 138 11","2.7"},
25       {stdlib,"ERTS  CXC 138 10","2.8"},
26       {kernel,"ERTS  CXC 138 10","4.2"}]},
27  {os,{unix,linux}},
28  {erlang_version,
29      "Erlang/OTP 18 [erts-7.3] [source] [64-bit] [async-threads:64] [hipe] [kernel-poll:true]\n"},
30  {memory,
31      [{total,59824960},
32       {connection_readers,0},
33       {connection_writers,0},
34       {connection_channels,0},
35       {connection_other,2680},
36       {queue_procs,19328},
37       {queue_slave_procs,0},
38       {plugins,1009776},
39       {other_proc,18123920},
40       {mnesia,70096},
41       {mgmt_db,750320},
42       {msg_index,43856},
43       {other_ets,1399424},
44       {binary,28720},
45       {code,27824046},
46       {atom,1000601},
47       {other_system,9552193}]},
48  {alarms,[]},
49  {listeners,[{clustering,25672,"::"},{amqp,5672,"::"}]},
50  {vm_memory_high_watermark,0.4},
51  {vm_memory_limit,768196608},
52  {disk_free_limit,50000000},
53  {disk_free,16176914432},
54  {file_descriptors,
55      [{total_limit,65435},
56       {total_used,2},
57       {sockets_limit,58889},
58       {sockets_used,0}]},
59  {processes,[{limit,1048576},{used,229}]},
60  {run_queue,0},
61  {uptime,95},
62  {kernel,{net_ticktime,60}}]
status

 

rabbitmq php扩展amqp安装

①安装rabbitmq-c-0.7.1

没有安装就会提示上面的错误
下载地址:https://github.com/alanxz/rabbitmq-c
我选择的是最新版本0.7.1

wget https://github.com/alanxz/rabbitmq-c/releases/download/v0.7.1/rabbitmq-c-0.7.1.tar.gz
tar zxf rabbitmq-c-0.7.1.tar.gz

cd rabbitmq-c-0.7.1
./configure --prefix=/usr/local/rabbitmq-c-0.7.1
make && make install

 

②安装amqp

下载地址https://pecl.php.net/package/amqp
我选择的是1.6.1

wget https://pecl.php.net/get/amqp-1.6.1.tgz
tar zxf amqp-1.6.1.tgz
cd amqp-1.6.1

/usr/local/php/bin/phpize

./configure --with-php-config=/usr/local/php/bin/php-config --with-amqp --with-librabbitmq-dir=/usr/local/rabbitmq-c-0.7.1

注意:这里的/usr/local/rabbitmq-c-0.7.1要跟上面rabbitmq-c安装的地址一样

make && make install

③添加php模块

vi /usr/local/php/etc/php.ini

最后添加一行

extension = /usr/local/php/lib/php/extensions/no-debug-non-zts-20100525/amqp.so

重启php

 

 

<?php
require_once __DIR__ . '/../vendor/autoload.php';
use PhpAmqpLib\Connection\AMQPStreamConnection;

$connection = new AMQPStreamConnection('localhost', 5672, 'guest', 'guest');
$channel = $connection->channel();
//声明队列
$channel->queue_declare('hello', false, false, false, false);

echo " [*] Waiting for messages. To exit press CTRL+C\n";

$callback = function ($msg) {
    echo ' [x] Received ', $msg->body, "\n";
};
//监听队列
$channel->basic_consume('hello', '', false, true, false, false, $callback);

while (count($channel->callbacks)) {
    $channel->wait();
}
receive

 

<?php
require_once __DIR__ . '/../vendor/autoload.php';
use PhpAmqpLib\Connection\AMQPStreamConnection;
use PhpAmqpLib\Message\AMQPMessage;

//获取连接
$connection = new AMQPStreamConnection('localhost', 5672, 'guest', 'guest');
//从连接中创建通道
$channel = $connection->channel();
//声明队列
$channel->queue_declare('hello', false, false, false, false);

$msg = new AMQPMessage('Hello World!');
$channel->basic_publish($msg, '', 'hello');

echo " [x] Sent 'Hello World!'\n";

$channel->close();
$connection->close();
send.php

 

channel->basic_consume("TestQueue", "", false, false, false, false, $callback);

  

顺序参数名默认值作用
1 queue   消息要取得消息的队列名
2 consumer_tag   消费者标签
3 no_local false 这个功能属于AMQP的标准,但是rabbitMQ并没有做实现.
4 no_ack false 收到消息后,是否不需要回复确认即被认为被消费
5 exclusive false 排他消费者,即这个队列只能由一个消费者消费.适用于任务不允许进行并发处理的情况下.比如系统对接
6 nowait false 不返回执行结果,但是如果排他开启的话,则必须需要等待结果的,如果两个一起开就会报错
7 callback null 回调函数
8 ticket null  
9 arguments null  

 

queue_declare(
        $queue = '',
        $passive = false,
        $durable = false,//是否持久化
        $exclusive = false,//独占 保证顺序消费
        $auto_delete = true,//如果和交换机没有绑定关系 则自动删除
        $nowait = false,
        $arguments = array(),
        $ticket = null
)

  

 

 

<?php
require_once __DIR__ . '/../vendor/autoload.php';
use PhpAmqpLib\Connection\AMQPStreamConnection;
use PhpAmqpLib\Message\AMQPMessage;

//获取连接
$connection = new AMQPStreamConnection('localhost', 5672, 'guest', 'guest');
//从连接中创建通道
$channel = $connection->channel();
//声明队列
$channel->queue_declare('task_queue123', false, true, false, false);

for($i=0;$i<=50;$i++){
    $data = implode(' ', array_slice($argv, 1));
    if (empty($data)) {
        $data = "Hello World!  ".$i;
    }
    $msg = new AMQPMessage($data);
    $channel->basic_publish($msg, '', 'task_queue123');
}




$channel->close();
$connection->close();
new_task

 

<?php
require_once __DIR__ . '/../vendor/autoload.php';
use PhpAmqpLib\Connection\AMQPStreamConnection;

$connection = new AMQPStreamConnection('localhost', 5672, 'guest', 'guest');
$channel = $connection->channel();
//声明队列
$channel->queue_declare('task_queue123', false, true, false, false);

echo " [*] Waiting for messages. To exit press CTRL+C\n";

$callback = function ($msg) {
    echo ' [x] Received ', $msg->body, "\n";
//    sleep(2);
    //手动确认
    $msg->delivery_info['channel']->basic_ack($msg->delivery_info['delivery_tag']);
    echo " [x] Done\n";
};
$channel->basic_qos(null, 1, null);
//手动确认 第四个参数为true
$channel->basic_consume('task_queue123', '', false, false, false, false, $callback);

while (count($channel->callbacks)) {
    $channel->wait();
}
work

 

<?php
require_once __DIR__ . '/../vendor/autoload.php';
use PhpAmqpLib\Connection\AMQPStreamConnection;

$connection = new AMQPStreamConnection('localhost', 5672, 'guest', 'guest');
$channel = $connection->channel();
//声明队列
$channel->queue_declare('task_queue123', false, true, false, false);

echo " [*] Waiting for messages. To exit press CTRL+C\n";

$callback = function ($msg) {
    echo ' [x] Received ', $msg->body, "\n";
    sleep(1);
    echo " [x] Done\n";
};
$channel->basic_qos(null, 1, null);
//自动确认 第四个参数为true
$channel->basic_consume('task_queue123', '', false, true, false, false, $callback);

while (count($channel->callbacks)) {
    $channel->wait();
}
work2

 

 

<?php
require_once __DIR__ . '/../vendor/autoload.php';
use PhpAmqpLib\Connection\AMQPStreamConnection;
use PhpAmqpLib\Message\AMQPMessage;

//获取连接
$connection = new AMQPStreamConnection('localhost', 5672, 'guest', 'guest');
//从连接中创建通道
$channel = $connection->channel();
$channel->exchange_declare('logs', 'fanout', false, false, false);

for($i=0;$i<=50;$i++){
    $data = implode(' ', array_slice($argv, 1));
    if (empty($data)) {
        $data = " hi ,Hello World!  ".$i;
    }
    $msg = new AMQPMessage($data);
    $channel->basic_publish($msg, 'logs');
}




$channel->close();
$connection->close();
new_task
<?php
require_once __DIR__ . '/../vendor/autoload.php';
use PhpAmqpLib\Connection\AMQPStreamConnection;

$connection = new AMQPStreamConnection('localhost', 5672, 'guest', 'guest');
$channel = $connection->channel();
$channel->exchange_declare('logs', 'fanout', false, false, false);
list($queue_name, ,) = $channel->queue_declare("", false, false, true, false);
$channel->queue_bind($queue_name, 'logs');
//声明队列

echo " [*] Waiting for messages. To exit press CTRL+C\n";

$callback = function ($msg) {
    echo ' [x] Received ', $msg->body, "\n";
    echo " [x] Done\n";
};
$channel->basic_qos(null, 1, null);
//手动确认 第四个参数为true
$channel->basic_consume($queue_name, '', false, true, false, false, $callback);

while (count($channel->callbacks)) {
    $channel->wait();
}
work
<?php
require_once __DIR__ . '/../vendor/autoload.php';
use PhpAmqpLib\Connection\AMQPStreamConnection;

$connection = new AMQPStreamConnection('localhost', 5672, 'guest', 'guest');
$channel = $connection->channel();
$channel->exchange_declare('logs', 'fanout', false, false, false);
list($queue_name, ,) = $channel->queue_declare("", false, false, true, false);
$channel->queue_bind($queue_name, 'logs');

echo " [*] Waiting for messages. To exit press CTRL+C\n";

$callback = function ($msg) {
    echo ' [x] Received ', $msg->body, "\n";
//    sleep(2);
    //手动确认
    echo " [x] Done\n";
};
$channel->basic_qos(null, 1, null);
//手动确认 第四个参数为true
$channel->basic_consume($queue_name, '', false, true, false, false, $callback);

while (count($channel->callbacks)) {
    $channel->wait();
}
work2

 

<?php
require_once __DIR__ . '/../vendor/autoload.php';
use PhpAmqpLib\Connection\AMQPStreamConnection;
use PhpAmqpLib\Message\AMQPMessage;

//获取连接
$connection = new AMQPStreamConnection('localhost', 5672, 'guest', 'guest');
//从连接中创建通道
$channel = $connection->channel();
$channel->exchange_declare('direct_logs', 'direct', false, false, false);

for($i=0;$i<=50;$i++){
    $data = implode(' ', array_slice($argv, 1));
    if (empty($data)) {
        $data = " hi ,Hello World!  ".$i;
    }
    $msg = new AMQPMessage($data);
    $channel->basic_publish($msg, 'direct_logs','info');
}




$channel->close();
$connection->close();
task
<?php
require_once __DIR__ . '/../vendor/autoload.php';
use PhpAmqpLib\Connection\AMQPStreamConnection;

$connection = new AMQPStreamConnection('localhost', 5672, 'guest', 'guest');
$channel = $connection->channel();
$channel->exchange_declare('direct_logs', 'direct', false, false, false);
list($queue_name, ,) = $channel->queue_declare("", false, false, true, false);
$channel->queue_bind($queue_name, 'direct_logs','info');


echo " [*] Waiting for messages. To exit press CTRL+C\n";

$callback = function ($msg) {
    echo ' [x] Received ', $msg->body, "\n";
    echo " [x] Done\n";
};
$channel->basic_qos(null, 1, null);
//手动确认 第四个参数为true
$channel->basic_consume($queue_name, '', false, true, false, false, $callback);

while (count($channel->callbacks)) {
    $channel->wait();
}
work
<?php
require_once __DIR__ . '/../vendor/autoload.php';
use PhpAmqpLib\Connection\AMQPStreamConnection;

$connection = new AMQPStreamConnection('localhost', 5672, 'guest', 'guest');
$channel = $connection->channel();
$channel->exchange_declare('direct_logs', 'direct', false, false, false);
list($queue_name, ,) = $channel->queue_declare("", false, false, true, false);
$channel->queue_bind($queue_name, 'direct_logs','info.hello');


echo " [*] Waiting for messages. To exit press CTRL+C\n";

$callback = function ($msg) {
    echo ' [x] Received ', $msg->body, "\n";
    echo " [x] Done\n";
};
$channel->basic_qos(null, 1, null);
//手动确认 第四个参数为true
$channel->basic_consume($queue_name, '', false, true, false, false, $callback);

while (count($channel->callbacks)) {
    $channel->wait();
}
work2

 

<?php
require_once __DIR__ . '/../vendor/autoload.php';
use PhpAmqpLib\Connection\AMQPStreamConnection;
use PhpAmqpLib\Message\AMQPMessage;

//获取连接
$connection = new AMQPStreamConnection('localhost', 5672, 'guest', 'guest');
//从连接中创建通道
$channel = $connection->channel();
$channel->exchange_declare('topic_logs', 'topic', false, false, false);

for($i=0;$i<=5;$i++){
    $data = implode(' ', array_slice($argv, 1));
    if (empty($data)) {
        $data = " info.hello  ".$i;
    }
    $msg = new AMQPMessage($data);
    $channel->basic_publish($msg, 'topic_logs','info.hello');
}

for($i=0;$i<=5;$i++){
    $data = implode(' ', array_slice($argv, 1));
    if (empty($data)) {
        $data = " info.helloword  ".$i;
    }
    $msg = new AMQPMessage($data);
    $channel->basic_publish($msg, 'topic_logs','info.helloword');
}




$channel->close();
$connection->close();
task
<?php
require_once __DIR__ . '/../vendor/autoload.php';
use PhpAmqpLib\Connection\AMQPStreamConnection;

$connection = new AMQPStreamConnection('localhost', 5672, 'guest', 'guest');
$channel = $connection->channel();
$channel->exchange_declare('topic_logs', 'topic', false, false, false);
list($queue_name, ,) = $channel->queue_declare("", false, false, true, false);
$channel->queue_bind($queue_name, 'topic_logs','info.#');


echo " [*] Waiting for messages. To exit press CTRL+C\n";

$callback = function ($msg) {
    echo ' [x] Received ', $msg->body, "\n";
    echo " [x] Done\n";
};
$channel->basic_qos(null, 1, null);
//手动确认 第四个参数为true
$channel->basic_consume($queue_name, '', false, true, false, false, $callback);

while (count($channel->callbacks)) {
    $channel->wait();
}
work
<?php
require_once __DIR__ . '/../vendor/autoload.php';
use PhpAmqpLib\Connection\AMQPStreamConnection;

$connection = new AMQPStreamConnection('localhost', 5672, 'guest', 'guest');
$channel = $connection->channel();
$channel->exchange_declare('topic_logs', 'topic', false, false, false);
list($queue_name, ,) = $channel->queue_declare("", false, false, true, false);
$channel->queue_bind($queue_name, 'topic_logs','info.hello');


echo " [*] Waiting for messages. To exit press CTRL+C\n";

$callback = function ($msg) {
    echo ' [x] Received ', $msg->body, "\n";
    echo " [x] Done\n";
};
$channel->basic_qos(null, 1, null);
//手动确认 第四个参数为true
$channel->basic_consume($queue_name, '', false, true, false, false, $callback);

while (count($channel->callbacks)) {
    $channel->wait();
}
work2

 

消息的可靠性投递方案:

1,中小型公司

1,消息入库 status=0,2,生产端发送消息到booker,3,ack异步确认 修改db消息状态

如果异步确认出现rpc中断 闪断现象 可以用定时任务弥补,查找状态为0的 5分钟还没消费的,重新发送

 

 

2,大型公司 高并发

 

可靠性投递下订单后发送消息1,接着发送延时消息2 (补偿消息),消息1处理后入库(mysql),处理消息2判断db是否有这条消息 是否已经处理ok,如果

消息1处理失败,则rpc调用,重新请求service

 

 什么是幂等性?

  可能你要对一件事情执行一个操作,要操作一百次 一千次,最终我们执行结果要相同的,那样就是一个幂等性。

比如我们执行一条sql,无论我执行多少次 这个结果都是相同的,这个就是幂等性的保障。

高并发情况下,大量消息 投递 和  消费,可能会出现重复投递,网络原因导致闪断,导致booker重发消息,这个时候我们不去做幂等就会出现重复消费

,我们消息做幂等,就为了让消息不被重复消费,即使我们收到很多同样的消息,我们也只会对这条消息消费一次,可能我们代码会跑多次,但是我们消息

只会执行这一步操作。

指纹码:时间搓或者外部 内部的唯一码;

利用hash算法 分库分表,分摊流量压力

 

 

 

sexex  

如果要入库的话,怎么保证数据一致性? redis有事务,mysql也有事务 怎么去处理?会存在redis写成功,mysql写失败。。。。。。思考。。。。

 

 

 

 

 利用加version版本号 来仿造乐观锁,当第二个人进来后 version=1 已经查询不到了,保证数据唯一

 

boolean setSuccess = redis.setnx(request.serializeToString(),"");//原子操作
if(setSuccess){
  doBusiness(); //执行业务
}else{
  doNothing(); //什么都不做
}

  

 

1,conrim机制:

 

正常情况下,如果消息经过交换器进入队列就可以完成消息的持久化,但如果消息在没有到达broker之前出现意外,那就造成消息丢失,有没有办法可以解决这个问题?

RabbitMQ有两种方式来解决这个问题:

  1. 通过AMQP提供的事务机制实现;
  2. 使用发送者确认模式实现;

一、事务使用

事务的实现主要是对信道(Channel)的设置,主要的方法有三个:

  1. channel.txSelect()声明启动事务模式;

  2. channel.txComment()提交事务;

  3. channel.txRollback()回滚事务;

从上面的可以看出事务都是以tx开头的,tx应该是transaction extend(事务扩展模块)的缩写,如果有准确的解释欢迎在博客下留言。

我们来看具体的代码实现:

 

// 创建连接
ConnectionFactory factory = new ConnectionFactory();
factory.setUsername(config.UserName);
factory.setPassword(config.Password);
factory.setVirtualHost(config.VHost);
factory.setHost(config.Host);
factory.setPort(config.Port);   
Connection conn = factory.newConnection();
// 创建信道
Channel channel = conn.createChannel();
// 声明队列
channel.queueDeclare(_queueName, true, false, false, null);
String message = String.format("时间 => %s", new Date().getTime());
try {
    channel.txSelect(); // 声明事务
    // 发送消息
    channel.basicPublish("", _queueName, MessageProperties.PERSISTENT_TEXT_PLAIN, message.getBytes("UTF-8"));
    channel.txCommit(); // 提交事务
} catch (Exception e) {
    channel.txRollback();
} finally {
    channel.close();
    conn.close();
}
View Code

 

注意:用户需把config.xx配置成自己Rabbit的信息。

从上面的代码我们可以看出,在发送消息之前的代码和之前介绍的都是一样的,只是在发送消息之前,需要声明channel为事务模式,提交或者回滚事务即可。

了解了事务的实现之后,那么事务究竟是怎么执行的,让我们来使用wireshark抓个包看看,如图所示:

输入ip.addr==rabbitip && amqp查看客户端和rabbit之间的通讯,可以看到交互流程:

  • 客户端发送给服务器Tx.Select(开启事务模式)
  • 服务器端返回Tx.Select-Ok(开启事务模式ok)
  • 推送消息
  • 客户端发送给事务提交Tx.Commit
  • 服务器端返回Tx.Commit-Ok

以上就完成了事务的交互流程,如果其中任意一个环节出现问题,就会抛出IoException移除,这样用户就可以拦截异常进行事务回滚,或决定要不要重复消息。

那么,既然已经有事务了,没什么还要使用发送方确认模式呢,原因是因为事务的性能是非常差的。事务性能测试:

事务模式,结果如下:

  • 事务模式,发送1w条数据,执行花费时间:14197s
  • 事务模式,发送1w条数据,执行花费时间:13597s
  • 事务模式,发送1w条数据,执行花费时间:14216s

非事务模式,结果如下:

  • 非事务模式,发送1w条数据,执行花费时间:101s
  • 非事务模式,发送1w条数据,执行花费时间:77s
  • 非事务模式,发送1w条数据,执行花费时间:106s

从上面可以看出,非事务模式的性能是事务模式的性能高149倍,我的电脑测试是这样的结果,不同的电脑配置略有差异,但结论是一样的,事务模式的性能要差很多,那有没有既能保证消息的

可靠性又能兼顾性能的解决方案呢?那就是接下来要讲的Confirm发送方确认模式。

扩展知识

我们知道,消费者可以使用消息自动或手动发送来确认消费消息,那如果我们在消费者模式中使用事务(当然如果使用了手动确认消息,完全用不到事务的),会发生什么呢?

消费者模式使用事务

假设消费者模式中使用了事务,并且在消息确认之后进行了事务回滚,那么RabbitMQ会产生什么样的变化?

结果分为两种情况:

  1. autoAck=false手动应对的时候是支持事务的,也就是说即使你已经手动确认了消息已经收到了,但在确认消息会等事务的返回解决之后,在做决定是确认消息还是重新放回队列,如果你
  2. 手动确认现在之后,又回滚了事务,那么已事务回滚为主,此条消息会重新放回队列;
  3. autoAck=true如果自定确认为true的情况是不支持事务的,也就是说你即使在收到消息之后在回滚事务也是于事无补的,队列已经把消息移除了;

2、Confirm发送方确认模式

 

Confirm发送方确认模式使用和事务类似,也是通过设置Channel进行发送方确认的。

Confirm的三种实现方式:

方式一:channel.waitForConfirms()普通发送方确认模式;

方式二:channel.waitForConfirmsOrDie()批量确认模式;

方式三:channel.addConfirmListener()异步监听发送方确认模式;

方式一:普通Confirm模式

// 创建连接
ConnectionFactory factory = new ConnectionFactory();
factory.setUsername(config.UserName);
factory.setPassword(config.Password);
factory.setVirtualHost(config.VHost);
factory.setHost(config.Host);
factory.setPort(config.Port);
Connection conn = factory.newConnection();
// 创建信道
Channel channel = conn.createChannel();
// 声明队列
channel.queueDeclare(config.QueueName, false, false, false, null);
// 开启发送方确认模式
channel.confirmSelect();
String message = String.format("时间 => %s", new Date().getTime());
channel.basicPublish("", config.QueueName, null, message.getBytes("UTF-8"));
if (channel.waitForConfirms()) {
    System.out.println("消息发送成功" );
}
View Code

看代码可以知道,我们只需要在推送消息之前,channel.confirmSelect()声明开启发送方确认模式,再使用channel.waitForConfirms()等待消息被服务器确认即可。

方式二:批量Confirm模式

// 创建连接
ConnectionFactory factory = new ConnectionFactory();
factory.setUsername(config.UserName);
factory.setPassword(config.Password);
factory.setVirtualHost(config.VHost);
factory.setHost(config.Host);
factory.setPort(config.Port);
Connection conn = factory.newConnection();
// 创建信道
Channel channel = conn.createChannel();
// 声明队列
channel.queueDeclare(config.QueueName, false, false, false, null);
// 开启发送方确认模式
channel.confirmSelect();
for (int i = 0; i < 10; i++) {
    String message = String.format("时间 => %s", new Date().getTime());
    channel.basicPublish("", config.QueueName, null, message.getBytes("UTF-8"));
}
channel.waitForConfirmsOrDie(); //直到所有信息都发布,只要有一个未确认就会IOException
System.out.println("全部执行完成");
View Code

以上代码可以看出来channel.waitForConfirmsOrDie(),使用同步方式等所有的消息发送之后才会执行后面代码,只要有一个消息未被确认就会抛出IOException异常。

方式三:异步Confirm模式

// 创建连接
ConnectionFactory factory = new ConnectionFactory();
factory.setUsername(config.UserName);
factory.setPassword(config.Password);
factory.setVirtualHost(config.VHost);
factory.setHost(config.Host);
factory.setPort(config.Port);
Connection conn = factory.newConnection();
// 创建信道
Channel channel = conn.createChannel();
// 声明队列
channel.queueDeclare(config.QueueName, false, false, false, null);
// 开启发送方确认模式
channel.confirmSelect();
for (int i = 0; i < 10; i++) {
    String message = String.format("时间 => %s", new Date().getTime());
    channel.basicPublish("", config.QueueName, null, message.getBytes("UTF-8"));
}
//异步监听确认和未确认的消息
channel.addConfirmListener(new ConfirmListener() {
    @Override
    public void handleNack(long deliveryTag, boolean multiple) throws IOException {
        System.out.println("未确认消息,标识:" + deliveryTag);
    }
    @Override
    public void handleAck(long deliveryTag, boolean multiple) throws IOException {
        System.out.println(String.format("已确认消息,标识:%d,多个消息:%b", deliveryTag, multiple));
    }
});
View Code

异步模式的优点,就是执行效率高,不需要等待消息执行完,只需要监听消息即可,以上异步返回的信息如下:

以看出,代码是异步执行的,消息确认有可能是批量确认的,是否批量确认在于返回的multiple的参数,此参数为bool值,如果true表示批量执行了deliveryTag这个值以前的所有消息,如果为false的话表示单条确认。

Confirm性能测试

测试前提:与事务一样,我们发送1w条消息。

方式一:Confirm普通模式

  • 执行花费时间:2253s
  • 执行花费时间:2018s
  • 执行花费时间:2043s

方式二:Confirm批量模式

  • 执行花费时间:1576s
  • 执行花费时间:1400s
  • 执行花费时间:1374s

方式三:Confirm异步监听方式

  • 执行花费时间:1498s
  • 执行花费时间:1368s
  • 执行花费时间:1363s

总结

综合总体测试情况来看:Confirm批量确定和Confirm异步模式性能相差不大,Confirm模式要比事务快10倍左右。

 

phpcode:

 1 <?php
 2 require_once __DIR__ . '/../vendor/autoload.php';
 3 use PhpAmqpLib\Connection\AMQPStreamConnection;
 4 use PhpAmqpLib\Message\AMQPMessage;
 5 
 6 
 7 $queue = "test_confirm";
 8 $exchange = "test_confirm";
 9 //获取连接
10 $connection = new AMQPStreamConnection('localhost', 5672, 'guest', 'guest');
11 //从连接中创建通道
12 $channel = $connection->channel();
13 
14 
15 $channel->queue_declare($queue, false, true, false, false);
16 $channel->exchange_declare($exchange, 'direct', false, true, false);
17 $channel->queue_bind($queue, $exchange);
18 
19 //在channel上开启确认模式
20 $channel->confirm_select();
21 
22 
23 $channel->set_ack_handler(
24     function (AMQPMessage $message) {
25         echo "Message acked with content " . $message->body . PHP_EOL;
26     }
27 );
28 $channel->set_nack_handler(
29     function (AMQPMessage $message) {
30         echo "Message nacked with content " . $message->body . PHP_EOL;
31     }
32 );
33 
34 
35 $channel->wait_for_pending_acks();
36 
37 
38 
39 for($i=0;$i<=3;$i++){
40     $data = implode(' ', array_slice($argv, 1));
41     if (empty($data)) {
42         $data = " info.hello  ".$i;
43     }
44     $msg = new AMQPMessage($data);
45     $channel->basic_publish($msg, $exchange,'info.hello');
46 }
47 
48 $channel->wait_for_pending_acks();
49 
50 //$channel->set_return_listener(function(){
51 //    echo 5;
52 //});
53 
54 
55 $channel->close();
56 $connection->close();
task

 

 

 1 <?php
 2 require_once __DIR__ . '/../vendor/autoload.php';
 3 use PhpAmqpLib\Connection\AMQPStreamConnection;
 4 
 5 
 6 $queue = "test_confirm";
 7 $exchange = "test_confirm";
 8 
 9 $connection = new AMQPStreamConnection('localhost', 5672, 'guest', 'guest');
10 $channel = $connection->channel();
11 $channel->queue_declare($queue, false, true, false, false);
12 $channel->exchange_declare($exchange, 'direct', false, true, false);
13 
14 $channel->queue_bind($queue, $exchange,'info.hello');
15 
16 
17 echo " [*] Waiting for messages. To exit press CTRL+C\n";
18 
19 $callback = function ($msg) {
20     echo ' [x] Received ', $msg->body, "\n";
21     echo " [x] Done\n";
22 };
23 //$channel->basic_qos(null, 1, null);
24 
25 
26 //自动签收 第四个参数为true
27 $channel->basic_consume($queue, '', false, true, false, false, $callback);
28 
29 while (count($channel->callbacks)) {
30     $channel->wait();
31 }
woker

 

 

生产者confirm确认消息模式,消费者自动签收

[root@localhost Conifrm]# php7 ./worker.php
 [*] Waiting for messages. To exit press CTRL+C
 [x] Received  info.hello  0
 [x] Done
 [x] Received  info.hello  1
 [x] Done
 [x] Received  info.hello  2
 [x] Done
 [x] Received  info.hello  3
 [x] Done

[root@localhost Conifrm]# php7 ./task.php
Message acked with content  info.hello  0
Message acked with content  info.hello  1
Message acked with content  info.hello  2
Message acked with content  info.hello  3

  

 什么时候回触发NACK呢?

  比如磁盘写满了,比如mq出现异常,queue容量到达上限;

  或者消费者发送ack过程中出现网络闪断,需要通过定时任务抓取中间状态,重发或者补偿

3.return消息机制

return Listener用于处理一些不可路由的消息!

对一些不可达的消息第一时间进行监听

生产者通过指定一个exchange 和 routingkey  把消息送达到某个队列中去,然后消费者监听队列,进行消费处理。但是在某些情况下,如果我们在发送消息时,当前的exchange 不存在或者指定的routingkey路由不到,这个时候如果要监听这种不可达的消息,就要使用 return Listener。流程图如下所示

 

实现return消息机制

在基础API中有一个关键的配置项 Mandatory:如果为true,则监听器会接收到路由不可达的消息,然后进行后续处理,如果为false,则broker端自动删除该消息。

 

 

 

 

phpcode:

 1 <?php
 2 require_once __DIR__ . '/../vendor/autoload.php';
 3 use PhpAmqpLib\Connection\AMQPStreamConnection;
 4 use PhpAmqpLib\Message\AMQPMessage;
 5 
 6 
 7 $queue = "test_return";
 8 $exchange = "test_return";
 9 //获取连接
10 $connection = new AMQPStreamConnection('localhost', 5672, 'guest', 'guest');
11 //从连接中创建通道
12 $channel = $connection->channel();
13 
14 
15 $channel->queue_declare($queue, false, true, false, false);
16 $channel->exchange_declare($exchange, 'topic', false, true, false);
17 $channel->queue_bind($queue, $exchange);
18 
19 //在channel上开启确认模式
20 $channel->confirm_select();
21 
22 
23 $channel->set_ack_handler(
24     function (AMQPMessage $message) {
25         echo "Message acked with content " . $message->body . PHP_EOL;
26     }
27 );
28 $channel->set_nack_handler(
29     function (AMQPMessage $message) {
30         echo "Message nacked with content " . $message->body . PHP_EOL;
31     }
32 );
33 
34 
35 $channel->wait_for_pending_acks();
36 
37 
38 
39 $wait = true;
40 $returnListener = function (
41     $replyCode,
42     $replyText,
43     $exchange,
44     $routingKey,
45     $message
46 ) use ($wait) {
47     $GLOBALS['wait'] = false;
48     echo "return: ",
49     "replyCode:".$replyCode, "\n",
50     "replyText:".$replyText, "\n",
51     "exchange:".$exchange, "\n",
52     "routingKey:".$routingKey, "\n",
53     "message:".$message->body, "\n";
54 };
55 $channel->set_return_listener($returnListener);
56 
57 for($i=0;$i<=3;$i++){
58     $data = implode(' ', array_slice($argv, 1));
59     if (empty($data)) {
60         $data = " msg info.hello  ".$i;
61     }
62     $msg = new AMQPMessage($data);
63     # 默认Mandatory=false 则会监听路由不可达消息  booker会自动删除该消息
64 //    $channel->basic_publish($msg, $exchange,'info.hello');
65 
66     # Mandatory=false 则会监听路由不可达消息
67     $channel->basic_publish($msg, $exchange,'aa.bb',true);
68 }
69 
70 $channel->wait_for_pending_acks();
71 
72 //$channel->set_return_listener(function(){
73 //    echo 5;
74 //});
75 
76 
77 
78 while ($wait) {
79     $channel->wait();
80 }
81 
82 $channel->close();
83 $connection->close();
task

 

 

<?php
require_once __DIR__ . '/../vendor/autoload.php';
use PhpAmqpLib\Connection\AMQPStreamConnection;


$queue = "test_return";
$exchange = "test_return";

$connection = new AMQPStreamConnection('localhost', 5672, 'guest', 'guest');
$channel = $connection->channel();
$channel->queue_declare($queue, false, true, false, false);
$channel->exchange_declare($exchange, 'topic', false, true, false);

$channel->queue_bind($queue, $exchange,'info.#');


echo " [*] Waiting for messages. To exit press CTRL+C\n";

$callback = function ($msg) {
    echo ' [x] Received ', $msg->body, "\n";
    echo " [x] Done\n";
};
//$channel->basic_qos(null, 1, null);


//自动签收 第四个参数为true
$channel->basic_consume($queue, '', false, true, false, false, $callback);

while (count($channel->callbacks)) {
    $channel->wait();
}
worker

 

 

 task:

[root@localhost return]# php7 ./task.php
Message acked with content  msg info.hello  0
Message acked with content  msg info.hello  1
Message acked with content  msg info.hello  2
Message acked with content  msg info.hello  3
return: replyCode:312
replyText:NO_ROUTE
exchange:test_return
routingKey:aa.bb
message: msg info.hello  0

  

 

 4,消息端的手工ACK和NACK与重回队列

 

 

很多地方都没有说清楚怎么去手动ack,其实手动ack就是在当前channel里面调用basicAsk的方法,并传入当前消息的tagId就可以了。

 

注意如果抛异常或unack(并且requeue为true),消息会一直重新入队列,一不小心就会xxxxx一大堆消息不断重复~。

 

//消息的标识,false只确认当前一个消息收到,true确认所有consumer获得的消息 (正常消费)

channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);

 

//ack返回false,并重新回到队列,api里面解释得很清楚  (本地异常)

channel.basicNack(message.getMessageProperties().getDeliveryTag(), false, true);

 

//拒绝消息

channel.basicReject(message.getMessageProperties().getDeliveryTag(), true);

最后一个参数为true就重回队列

 

phpcode:

 

 1 <?php
 2 require_once __DIR__ . '/../vendor/autoload.php';
 3 use PhpAmqpLib\Connection\AMQPStreamConnection;
 4 use PhpAmqpLib\Message\AMQPMessage;
 5 
 6 
 7 $queue = "test_ack_queue";
 8 $exchange = "test_ack_queue";
 9 //获取连接
10 $connection = new AMQPStreamConnection('localhost', 5672, 'guest', 'guest');
11 //从连接中创建通道
12 $channel = $connection->channel();
13 
14 
15 $channel->queue_declare($queue, false, true, false, false);
16 $channel->exchange_declare($exchange, 'topic', false, true, false);
17 $channel->queue_bind($queue, $exchange);
18 
19 //在channel上开启确认模式
20 $channel->confirm_select();
21 
22 
23 $channel->set_ack_handler(
24     function (AMQPMessage $message) {
25         echo "Message acked with content " . $message->body . PHP_EOL;
26     }
27 );
28 $channel->set_nack_handler(
29     function (AMQPMessage $message) {
30         echo "Message nacked with content " . $message->body . PHP_EOL;
31     }
32 );
33 
34 
35 $channel->wait_for_pending_acks();
36 
37 
38 
39 $wait = true;
40 $returnListener = function ($replyCode, $replyText, $exchange, $routingKey, $message) use ($wait) {
41     $GLOBALS['wait'] = false;
42     echo "return: ",
43     "replyCode:".$replyCode, "\n",
44     "replyText:".$replyText, "\n",
45     "exchange:".$exchange, "\n",
46     "routingKey:".$routingKey, "\n",
47     "message:".$message->body, "\n";
48 };
49 //路由不可达  监听
50 $channel->set_return_listener($returnListener);
51 
52 for($i=0;$i<=0;$i++){
53     $data = implode(' ', array_slice($argv, 1));
54     if (empty($data)) {
55         $data = " msg info.hello  ".$i;
56     }
57     $msg = new AMQPMessage($data);
58 
59 
60     $headers = new \PhpAmqpLib\Wire\AMQPTable(array(
61         'x-foo'=>'bar',
62         'table'=>['figuf', 'ghf'=>5, 5=>675],
63 //        'ack_type' => 'good',
64         'date' => new DateTime(),
65     ));
66 
67     $headers->set('shortshort', -5, \PhpAmqpLib\Wire\AMQPTable::T_INT_SHORTSHORT);
68     $headers->set('short', -1024, \PhpAmqpLib\Wire\AMQPTable::T_INT_SHORT);
69     $headers->set('ack_type','good1');
70 
71 //    var_dump($headers->getNativeData());
72     echo PHP_EOL;
73     $msg->set('application_headers', $headers);
74 
75     # 默认Mandatory=false 则会监听路由不可达消息  booker会自动删除该消息
76     $channel->basic_publish($msg, $exchange,'info.hello',true);
77 }
78 
79 $channel->wait_for_pending_acks();
80 
81 
82 
83 
84 
85 while ($wait) {
86     $channel->wait();
87 }
88 
89 $channel->close();
90 $connection->close();
task

 

 

 1 <?php
 2 require_once __DIR__ . '/../vendor/autoload.php';
 3 use PhpAmqpLib\Connection\AMQPStreamConnection;
 4 use \PhpAmqpLib\Message\AMQPMessage;
 5 
 6 $queue = "test_ack_queue";
 7 $exchange = "test_ack_queue";
 8 
 9 $connection = new AMQPStreamConnection('localhost', 5672, 'guest', 'guest');
10 $channel = $connection->channel();
11 $channel->queue_declare($queue, false, true, false, false);
12 $channel->exchange_declare($exchange, 'topic', false, true, false);
13 
14 $channel->queue_bind($queue, $exchange,'info.#');
15 
16 
17 echo " [*] Waiting for messages. To exit press CTRL+C\n";
18 
19 $callback = function ($msg) {
20     echo ' [x] Received ', $msg->body, "\n";
21     echo " [x] Done\n";
22 };
23 
24 $callback = function (AMQPMessage $message) {
25     echo PHP_EOL . ' [x] ', $message->delivery_info['routing_key'], ':', $message->body, "\n";
26     echo 'Message headers follows' . PHP_EOL;
27     $header = $message->get('application_headers')->getNativeData();
28 //    var_dump($header);
29     echo PHP_EOL;
30 
31 
32     if ($header['ack_type'] == 'good') {
33         $message->delivery_info['channel']->basic_ack($message->delivery_info['delivery_tag']);
34     } else {
35         sleep(2);
36         //第二个参数,批量确认,第三个参数:重回队列
37         $message->delivery_info['channel']->basic_nack($message->delivery_info['delivery_tag'],false,true);
38     }
39 
40     // Send a message with the string "quit" to cancel the consumer.
41     if ($message->body === 'quit') {
42         $message->delivery_info['channel']->basic_cancel($message->delivery_info['consumer_tag']);
43     }
44 
45 };
46 
47 //限流每次只接一个消息
48 $channel->basic_qos(null, 1, null);
49 
50 //自动签收 第四个参数为true  no_ack=false
51 $channel->basic_consume($queue, '', false, false, false, false, $callback);
52 
53 while (count($channel->callbacks)) {
54     $channel->wait();
55 }
worker

 

 

配注:

生产者设置头部  $headers->set('ack_type','good1');

 消费端通过判断头部变量ack_type 

确认签收:

$message->delivery_info['channel']->basic_ack($message->delivery_info['delivery_tag']);

  



不签收重回队列:
sleep(2);
        //第二个参数,批量确认,第三个参数:重回队列
        $message->delivery_info['channel']->basic_nack($message->delivery_info['delivery_tag'],false,true);

  

打印结果:

[root@localhost ack]# php7 ./task.php

Message acked with content  msg info.hello  0

  

消费端没没隔2秒会消费一次,因为没签收 也设置了重回队列

[root@localhost ack]# php7 ./worker.php
 [*] Waiting for messages. To exit press CTRL+C

 [x] info.hello: msg info.hello  0
Message headers follows


 [x] info.hello: msg info.hello  0
Message headers follows


 [x] info.hello: msg info.hello  0
Message headers follows


 [x] info.hello: msg info.hello  0
Message headers follows

  

 

5,TTL

两种TTL:如果两种都设置了,按照TTL小的那个处理

1.通过队列属性设置,通过队列发送出去的消息都遵循这个TTL;

2.通过对消息本身单独设置

第一种,一旦消息过期直接丢弃;第二种,即使消息过期,也不一定立刻丢弃,因为只有当消息被投递的时候,才能判断该消息是否过期。

设置队列TTL:

如果不设置TTL,则表示此消息不会过期,如果将TTL设置为0,则表示除非此时可以直接将消息投递到消费者,否则该消息立即被丢弃。

                ConnectionFactory factory = new ConnectionFactory();
                factory.HostName = IP_ADDRESS;
                factory.Port = PORT;
                factory.UserName = USER_NAME;
                factory.Password = PASSWORD;
                con = factory.CreateConnection();
                channel = con.CreateModel();
                channel.ExchangeDeclare(EXCHANGE_NAME, "topic", true, false, null);
 
                Dictionary<string, object> agres = new Dictionary<string, object>();
                //消息TTL01.Queue设置
                agres.Add("x-message-ttl", 6000);
                //队列TTL设置:1800000ms
                agres.Add("x-expires", 1800000);
                channel.QueueDeclare(QUEUE_NAME, true, false, false, agres);
 
 
                channel.QueueBind(QUEUE_NAME, EXCHANGE_NAME, BINDING_KEY, null);//channel.ExchangeBind()
                string message = "Hello Word!";
                var body = Encoding.UTF8.GetBytes(message);
                var properties = channel.CreateBasicProperties();
                properties.Persistent = true;
View Code

设置消息TTL:

     ConnectionFactory factory = new ConnectionFactory();
                factory.HostName = IP_ADDRESS;
                factory.Port = PORT;
                factory.UserName = USER_NAME;
                factory.Password = PASSWORD;
                con = factory.CreateConnection();
                channel = con.CreateModel();
                channel.ExchangeDeclare(EXCHANGE_NAME, "topic", true, false, null);
 
                //Dictionary<string, object> agres = new Dictionary<string, object>();
                ////消息TTL01.Queue设置
                //agres.Add("x-message-ttl", 6000);
                ////队列TTL设置:1800000ms
                //agres.Add("x-expires", 1800000);
                //channel.QueueDeclare(QUEUE_NAME, true, false, false, agres);
 
 
                channel.QueueBind(QUEUE_NAME, EXCHANGE_NAME, BINDING_KEY, null);//channel.ExchangeBind()
                string message = "Hello Word!";
                var body = Encoding.UTF8.GetBytes(message);
                var properties = channel.CreateBasicProperties();
                properties.Persistent = true;
 
                //消息TTL02.每条消息设置设置,如果两个都设置了,则按照TTL小的那个
                properties.Expiration = "6000";//TTL = 6000ms
                channel.BasicPublish(EXCHANGE_NAME, ROUTING_KEY, properties, body);
View Code

 

6,死信队列

 DLX,Dead-Letter-Exchange

利用DLX,当消息在一个队列中变成死信(dead message)之后,它能被重新publish到另一个Exchange,这个Exchange就是Dlx

一 进入死信队列(进入死信的三种方式)

1.消息被拒绝(basic.reject or basic.nack)并且requeue=false

2.消息TTL过期过期时间

3.队列达到最大长度

DLX也是一下正常的Exchange同一般的Exchange没有区别,它能在任何的队列上被指定,实际上就是设置某个队列的属性,当这个队列中有死信时,RabbitMQ就会自动的将这个消息重新发布到设置的Exchange中去,进而被路由到另一个队列, publish可以监听这个队列中消息做相应的处理, 这个特性可以弥补R abbitMQ 3.0.0以前支持的immediate参数中的向publish确认的功能。

 

 

如果消息在队列中到达TTL,将被丢弃。这时候,消息变成死信(dead letter).过期是导致死信的原因之一,在RabbitMQ中,以下情况都会产生死信:

  • 消息过期
  • 消息被消费着拒绝(reject/nack),并且设置requeue参数为false
  • 队列到达最大长度

消息在队列中变成死信默认将被丢弃,为了处理死信,可以使用死信交换器(DLX)。
死信交换器可以认为是队列的备胎,当队列中产生死信时,死信被发送到死信交换器,由死信交换器重新路由到与之绑定的队列,这些队列被成为死信队列。
声明队列时,可以通过x-dead-letter-exchange参数设置该队列的死信交换器,也可以通过policy方式设定队列的死信交换器。



 

 

Map<String,Object> params = new HashMap<String, Object>();
params.put("x-dead-letter-exchange","dlx-exchange");
channel.queueDeclare("myqueue",false,false,false,params); 

  

这样,当myquue队列中产生死信时,死信将被发送到dlx-exchange交换器,与它重新路由。

消息到路由键是后生产者发送是设置到,在死信被发送到死信交换器时,我们有机会修改消息到路由键。在声明队列是,指定x-dead-letter-routing-key参数即可。

params.put("x-dead-letter-routing-key","deadKey");

  

这样,当死信被发送到死信交换器时,它到路由键变为deadKey,后续在死信交换器中将根据该路由键进行路由。通过这种在队列上为死信统一更新路由键到方式,使得在某些
情况下可以统一将死信路由到指定队列,方便对死信统一处理。

 

 7,消费端的限流

假如单个生产者一分钟生产几百条数据,但是单个消费端一分钟只能消费60条数据,这个时候生产端和消费端肯定是不平衡的;高并发情况下,生产端我们没法做限制,消费端肯定要做消峰,

这个时候我们消费端就要做限流操作,目的就是为了让消费端更稳定,不然超出了最大负载 可能会导致消费端的资源耗尽。

在我们mq中有两种签收模式,自动,手动;在高并发情况下,我们一定不能设置为自动签收,在工作没有人会自动签收;
签收设置为手工确定,只要你消息没有被确定前,是不会有新的消息到达consume端的,这是mq的机制,这也是给消费者减压减负;

 

 

消费者端:
BasicQos(
prefetchSize:0   #消息大小限制,一般为0,不限制
perfetchCount:1 #一次最多处理多少消息,一般设置为1。会告诉mq不要同时给一个消费者推送多余N个消息,即一旦有N个消息还没有ack
则该consumer将block掉,直到有消息ack
简单来说:就是不要一次给我太多消息 我扛不住
global:true/false # 是否将上面设置应用于channel,就是上面限制是channel级别还是consumer级别  限流策略在什么时候应用,true:channel通道上限制    false:消费者上做限制
)
注意:prefetchSize和global这两项,mq还没有实现,暂时不研究,prefetch_count在no_ask=false情况下生效,即在自动应答的情况下这两个值不生效

  

 消费者添加:

$channel->basic_qos(null, 1, null);

 

 1 <?php
 2 require_once __DIR__ . '/../vendor/autoload.php';
 3 use PhpAmqpLib\Connection\AMQPStreamConnection;
 4 use PhpAmqpLib\Message\AMQPMessage;
 5 
 6 
 7 $queue = "qos_queue";
 8 $exchange = "qos_queue";
 9 //获取连接
10 $connection = new AMQPStreamConnection('localhost', 5672, 'guest', 'guest');
11 //从连接中创建通道
12 $channel = $connection->channel();
13 
14 
15 $channel->queue_declare($queue, false, true, false, false);
16 $channel->exchange_declare($exchange, 'topic', false, true, false);
17 $channel->queue_bind($queue, $exchange);
18 
19 //在channel上开启确认模式
20 $channel->confirm_select();
21 
22 
23 $channel->set_ack_handler(
24     function (AMQPMessage $message) {
25         echo "Message acked with content " . $message->body . PHP_EOL;
26     }
27 );
28 $channel->set_nack_handler(
29     function (AMQPMessage $message) {
30         echo "Message nacked with content " . $message->body . PHP_EOL;
31     }
32 );
33 
34 
35 $channel->wait_for_pending_acks();
36 
37 
38 
39 $wait = true;
40 $returnListener = function (
41     $replyCode,
42     $replyText,
43     $exchange,
44     $routingKey,
45     $message
46 ) use ($wait) {
47     $GLOBALS['wait'] = false;
48     echo "return: ",
49     "replyCode:".$replyCode, "\n",
50     "replyText:".$replyText, "\n",
51     "exchange:".$exchange, "\n",
52     "routingKey:".$routingKey, "\n",
53     "message:".$message->body, "\n";
54 };
55 $channel->set_return_listener($returnListener);
56 
57 for($i=0;$i<=3;$i++){
58     $data = implode(' ', array_slice($argv, 1));
59     if (empty($data)) {
60         $data = " msg info.hello  ".$i;
61     }
62     $msg = new AMQPMessage($data);
63     # 默认Mandatory=false 则会监听路由不可达消息  booker会自动删除该消息
64     $channel->basic_publish($msg, $exchange,'info.hello',true);
65 
66     # Mandatory=false 则会监听路由不可达消息
67 //    $channel->basic_publish($msg, $exchange,'aa.bb',true);
68 }
69 
70 $channel->wait_for_pending_acks();
71 
72 //$channel->set_return_listener(function(){
73 //    echo 5;
74 //});
75 
76 
77 
78 while ($wait) {
79     $channel->wait();
80 }
81 
82 $channel->close();
83 $connection->close();
task

 

  

 

 1 <?php
 2 require_once __DIR__ . '/../vendor/autoload.php';
 3 use PhpAmqpLib\Connection\AMQPStreamConnection;
 4 
 5 
 6 $queue = "qos_queue";
 7 $exchange = "qos_queue";
 8 
 9 $connection = new AMQPStreamConnection('localhost', 5672, 'guest', 'guest');
10 $channel = $connection->channel();
11 $channel->queue_declare($queue, false, true, false, false);
12 $channel->exchange_declare($exchange, 'topic', false, true, false);
13 
14 $channel->queue_bind($queue, $exchange,'info.#');
15 
16 
17 echo " [*] Waiting for messages. To exit press CTRL+C\n";
18 
19 $callback = function ($msg) {
20     echo ' [x] Received ', $msg->body, "\n";
21     echo " [x] Done\n";
22 };
23 $channel->basic_qos(null, 1, null);
24 
25 //自动签收 第四个参数为true
26 $channel->basic_consume($queue, '', false, true, false, false, $callback);
27 
28 while (count($channel->callbacks)) {
29     $channel->wait();
30 }
worker

 

 

消息持久化:

如果我们希望即使在RabbitMQ服务重启的情况下,也不会丢失消息,我们可以将Queue与Message都设置为可持久化的(durable),这样可以保证绝大部分情况下我们的RabbitMQ消息不会丢失。当然还是会有一些小概率事件会导致消息丢失。

持久化队列:

我们就hello队列持久化

  在声明队列名称时,持久化队列,生产端和消费端都要

channel.queue_declare(queue='hello', durable=True)

 

我们重复上面的操作,但是给hello队列做持久化,而hello1不做,并重启rabbitmq

可以看到重启后,hello队列还在,hello1队列消失了,但是原本hello中的一条消息也没有保存下来。所以在这边我们仅仅做到了消息队列的持久化,还没有做消息持久化。

消息持久化:

我们刚才实现了在rabbitmq崩溃的情况下,就队列本身保存下来,重启后队列还在。接下来我们要将消息也保存下来,即消息的持久化

channel.basic_publish(exchange='',
                      routing_key='hello',
                      body='hello',
                      properties=pika.BasicProperties(
                          delivery_mode=2,  # make message persistent
                      ))
 
# 增加properties,这个properties 就是消费端 callback函数中的properties
# delivery_mode = 2  持久化消息

 

  1. 队列持久化需要在声明队列时添加参数 durable=True,这样在rabbitmq崩溃时也能保存队列
  2. 仅仅使用durable=True ,只能持久化队列,不能持久化消息
  3. 消息持久化需要在消息生成时,添加参数 properties=pika.BasicProperties(delivery_mode=2)

 

 

更多demo参考

https://github.com/sunlongv520/php-amqplib/tree/master/demo

 

 消息尝试机制

golang使用rabbitmq监听消息 并实现rbmq断开断线重连

 

 

 

本文来自博客园,作者:孙龙-程序员,转载请注明原文链接:https://www.cnblogs.com/sunlong88/articles/9742020.html

posted on 2018-10-04 12:44  孙龙-程序员  阅读(563)  评论(0)    收藏  举报
刷新页面返回顶部
博客园  ©  2004-2025
浙公网安备 33010602011771号 浙ICP备2021040463号-3