[TOC]
一、简介 :smiley:
RabbitMQ是实现了高级消息队列协议(AMQP)的开源消息代理软件(亦称面向消息的中间件)。RabbitMQ服务器是用Erlang语言编写的,而集群和故障转移是构建在开放电信平台框架上的。
二、安装 :rocket:
三、RabbitMQ的作用 :bullettrain_side:
1.异步处理消息
场景: 将连续的业务流程拆分成并行的业务操作,减低并发情况下的延迟。
2.系统解耦
场景: 针对存在多个子系统业务关联的情况下,通过消息队列实现解耦,实现个子系统的独立性和业务正确性,在多需求和版本以及接口的变更下,只跟消息对列产生业务关联,减低各子系统的强依赖,提高整体系统的健壮性。
3.流量削峰
场景:流量削峰在秒杀活动中应用广泛,在高并发的场景下,过高的请求并发到后端容易导致应用或数据库宕机,因此,采用消息队列的方式,当消息超过队列的最大长度时,采用丢弃后排队的方式,保障后台服务正常运行.
业务数据量处理的基本准则:读多写少用缓存,写多读少用缓冲(队列)
4.异步通信
通过把把消息发送给消息中间件,消息中间件并不立即处理它,后续在慢慢处理。
四、 RabbitMQ的特点 :artificial_satellite:
-
可靠性:RabbitMQ使用一些机制来保证可靠性,如持久化、传输确认及发布确认等。
-
灵活的路由:在消息进入队列之前,通过交换器来路由消息。对于典型的路由功能,RabbitMQ己经提供了一些内置的交换器来实现。针对更复杂的路由功能,可以将多个交换器绑定在一起,也可以通过插件机制来实现自己的交换器。
-
扩展性:多个RabbitMQ节点可以组成一个集群,也可以根据实际业务情况动态地扩展集群中节点。
-
高可用性:队列可以在集群中的机器上设置镜像,使得在部分节点出现问题的情况下队仍然可用。
-
多种协议:RabbitMQ除了原生支持AMQP协议,还支持STOMP,MQTT等多种消息中间件协议。
-
多语言客户端:RabbitMQ几乎支持所有常用语言,比如Jav a、Python、Ruby、PHP、C#、JavaScript等。
-
管理界面:RabbitMQ提供了一个易用的用户界面,使得用户可以监控和管理消息、集群中的节点等。
-
插件机制:RabbitMQ提供了许多插件,以实现从多方面进行扩展,当然也可以编写自己的插件。
五、RabbitMQ消息模型 :bowing_man:
POM单项目组成
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.mmr</groupId>
<artifactId>myrabbitmq</artifactId>
<version>0.0.1-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>com.rabbitmq</groupId>
<artifactId>amqp-client</artifactId>
<version>4.0.2</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.10</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.5</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
</dependency>
<dependency>
<groupId>org.springframework.amqp</groupId>
<artifactId>spring-rabbit</artifactId>
<version>1.7.5.RELEASE</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>
</plugins>
</build>
</project>
MQ 连接工具类:
public class ConnectionUtils {
/**
* 获取MQ的连接
* @return
* @throws TimeoutException
* @throws IOException
*/
public static Connection getConnection() throws IOException, TimeoutException{
//定义一个连接工厂
ConnectionFactory factory =new ConnectionFactory();
//设置服务地址
factory.setHost("127.0.0.1");
//AMQP 5672
factory.setPort(5672);
//vhost
factory.setVirtualHost("/vhost_mmr");
//用户名
factory.setUsername("user_mmr");
//密码
factory.setPassword("123");
return factory.newConnection();
}
}
1.简单队列模型
队列模型如下图:
P:消息的生产者 C:消息的消费者 红色:队列
生产者将消息发送到队列,消费者从队列中获取消息。
队列的不足:耦合性高 生产消费一一对应(如果有多个消费者想都消费这个消息,就不行了) 队列名称变更时需要同时更改
发送端:
public class Send {
private static final String QUEUE_NAME="test_simple_queue";
public static void main(String[] args) throws IOException, TimeoutException {
//获取一个连接
Connection connection = ConnectionUtils.getConnection();
//从连接中获取一个通道
Channel channel = connection.createChannel();
//创建队列声明
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
String msg="hello simple !";
//第一个参数是exchangeName(默认情况下代理服务器端是存在一个""名字的exchange的,
//因此如果不创建exchange的话我们可以直接将该参数设置成"",如果创建了exchange的话
//我们需要将该参数设置成创建的exchange的名字),第二个参数是路由键
channel.basicPublish("", QUEUE_NAME, null, msg.getBytes());
System.out.println("--send msg:"+msg);
channel.close();
connection.close();
}
}
接收端:
public class Recv {
private static final String QUEUE_NAME = "test_simple_queue";
@SuppressWarnings("deprecation")
public static void main(String[] args) throws IOException,
TimeoutException, ShutdownSignalException,
ConsumerCancelledException, InterruptedException {
// 获取连接
Connection connection = ConnectionUtils.getConnection();
// 创建频道
Channel channel = connection.createChannel();
//队列声明
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
//定义消费者
DefaultConsumer consumer = new DefaultConsumer(channel){
//获取到达消息
@Override
public void handleDelivery(String consumerTag, Envelope envelope,
BasicProperties properties, byte[] body) throws IOException {
String msg=new String(body,"utf-8");
System.out.println("new api recv:"+msg);
}
};
//监听队列
channel.basicConsume(QUEUE_NAME, true,consumer);
}
private static void oldapi() throws IOException, TimeoutException,
InterruptedException {
// 获取连接
Connection connection = ConnectionUtils.getConnection();
// 创建频道
Channel channel = connection.createChannel();
// 定义队列的消费者
QueueingConsumer consumer = new QueueingConsumer(channel);
// 监听队列
channel.basicConsume(QUEUE_NAME, true, consumer);
while (true) {
Delivery delivery = consumer.nextDelivery();
String msgString = new String(delivery.getBody());
System.out.println("[recv] msg:" + msgString);
}
}
}
2.workQueues 工作队列
队列模型如下图:
特点: 在同一个队列上创建多个消费者,让他们相互竞争,这样消费者就可以同时处理多条消息了
优点:可以轻易的并行工作。如果我们积压了好多工作,我们可以通过增加工作者(消费者) 来解决这一问题,使得系统的伸缩性更加容易。
1.轮询分发
生产者:
public class Send {
/* |---C1
* P-----Queue----|
* ` |---C2
*
*/
private static final String QUEUE_NAME="test_work_queue";
public static void main(String[] args) throws IOException, TimeoutException, InterruptedException {
//获取连接
Connection connection = ConnectionUtils.getConnection();
//获取channel
Channel channel = connection.createChannel();
//声明队列
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
for (int i = 0; i <50; i++) {
String msg="hello "+i;
System.out.println("[WQ ]send:"+msg);
channel.basicPublish("", QUEUE_NAME, null, msg.getBytes());
Thread.sleep(i*20);
}
channel.close();
connection.close();
}
}
消费者1:
public class Recv1 {
private static final String QUEUE_NAME="test_work_queue";
public static void main(String[] args) throws IOException, TimeoutException {
//获取连接
Connection connection = ConnectionUtils.getConnection();
//获取channel
Channel channel = connection.createChannel();
//声明队列
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
//定义一个消费者
Consumer consumer=new DefaultConsumer(channel){
//消息到达 触发这个方法
@Override
public void handleDelivery(String consumerTag, Envelope envelope,
BasicProperties properties, byte[] body) throws IOException {
String msg=new String(body,"utf-8");
System.out.println("[1] Recv msg:"+msg);
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}finally{
System.out.println("[1] done ");
}
}
};
boolean autoAck=true;
channel.basicConsume(QUEUE_NAME,autoAck , consumer);
}
}
消费 2:
public class Recv2 {
private static final String QUEUE_NAME="test_work_queue";
public static void main(String[] args) throws IOException, TimeoutException {
//获取连接
Connection connection = ConnectionUtils.getConnection();
//获取channel
Channel channel = connection.createChannel();
//声明队列
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
//定义一个消费者
Consumer consumer=new DefaultConsumer(channel){
//消息到达 触发这个方法
@Override
public void handleDelivery(String consumerTag, Envelope envelope,
BasicProperties properties, byte[] body) throws IOException {
String msg=new String(body,"utf-8");
System.out.println("[2] Recv msg:"+msg);
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}finally{
System.out.println("[2] done ");
}
}
};
boolean autoAck=true;
channel.basicConsume(QUEUE_NAME,autoAck , consumer);
}
}
结果:
-
消费者 1 和消费者 2 获取到的消息内容是不同的,同一个消息只能被一个消费者获取
-
消费者 1 和消费者 2 货到的消息数量是一样的 一个奇数一个偶数
-
这种方式叫做轮询分发 结果就是不管谁忙或清闲,都不会给谁多一个任务或少一个任务,任务总是你一个我一个 的分
2.公平分发模型
模型名称:公平分发 workFair
- 使用公平分发,必须关闭自动应答,改为手动应答。
- 使用 basicQos( prefetchCount = 1)方法,来限制 RabbitMQ 只发不超过 1 条的消息给同 一个消费者
生产者:
public class Send {
/* |---C1
* P-----Queue----|
* ` |---C2
*
*/
private static final String QUEUE_NAME="test_work_queue";
public static void main(String[] args) throws IOException, TimeoutException, InterruptedException {
//获取连接
Connection connection = ConnectionUtils.getConnection();
//获取channel
Channel channel = connection.createChannel();
//声明队列
channel.queueDeclare(QUEUE_NAME, true, false, false, null);
/**
* 每个消费者 发送确认消息之前,消息队列不发送下一个消息到消费者,一次只处理一个消息
*
* 限制发送给同一个消费者 不得超过一条消息
*/
int prefetchCount=1;
channel.basicQos(prefetchCount);
for (int i = 0; i <50; i++) {
String msg="hello "+i;
System.out.println("[WQ ]send:"+msg);
channel.basicPublish("", QUEUE_NAME, null, msg.getBytes());
Thread.sleep(i*5);
}
channel.close();
connection.close();
}
}
消费者1:
public class Recv1 {
private static final String QUEUE_NAME="test_work_queue";
public static void main(String[] args) throws IOException, TimeoutException {
//获取连接
Connection connection = ConnectionUtils.getConnection();
//获取channel
final Channel channel = connection.createChannel();
//声明队列
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
channel.basicQos(1);//保证一次只分发一个
//定义一个消费者
Consumer consumer=new DefaultConsumer(channel){
//消息到达 触发这个方法
@Override
public void handleDelivery(String consumerTag, Envelope envelope,
BasicProperties properties, byte[] body) throws IOException {
String msg=new String(body,"utf-8");
System.out.println("[1] Recv msg:"+msg);
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}finally{
System.out.println("[1] done ");
channel.basicAck(envelope.getDeliveryTag(), false);
}
}
};
boolean autoAck=false;//自动应答 false
channel.basicConsume(QUEUE_NAME,autoAck , consumer);
}
}
消费者2 :
public class Recv2 {
private static final String QUEUE_NAME="test_work_queue";
public static void main(String[] args) throws IOException, TimeoutException {
//获取连接
Connection connection = ConnectionUtils.getConnection();
//获取channel
final Channel channel = connection.createChannel();
//声明队列
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
channel.basicQos(1);//保证一次只分发一个
//定义一个消费者
Consumer consumer=new DefaultConsumer(channel){
//消息到达 触发这个方法
@Override
public void handleDelivery(String consumerTag, Envelope envelope,
BasicProperties properties, byte[] body) throws IOException {
String msg=new String(body,"utf-8");
System.out.println("[2] Recv msg:"+msg);
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}finally{
System.out.println("[2] done ");
//手动回执
channel.basicAck(envelope.getDeliveryTag(), false);
}
}
};
boolean autoAck=false;
channel.basicConsume(QUEUE_NAME,autoAck , consumer);
}
}
结果: 消费者 1 速度大于消费者 2
3.发布订阅模型 Publish/Subscribe
队列模型图如下:
模型特点:
-
一个消息 能被多个消费者消费
-
不处理路由键。你只需要将队列绑定到交换机上。发送消息到交换机都会被转发到与该交换机绑定的所有队列
类似微信订阅号 发布文章消息 就可以广播给所有的接收者。(订阅者)
流程:
- 1 个生产者,多个消费者
- 2、每一个消费者都有自己的一个队列
- 3、生产者没有将消息直接发送到队列,而是发送到了交换机(转发器)
- 4、每个队列都要绑定到交换机
- 5、生产者发送的消息,经过交换机,到达队列,实现,一个消息被多个消费者获取的目的
生产者:
public class Send {
private static final String EXCHANGE_NAME="test_exchange_fanout";
public static void main(String[] args) throws IOException, TimeoutException {
Connection connection = ConnectionUtils.getConnection();
Channel channel = connection.createChannel();
//声明交换机
channel.exchangeDeclare(EXCHANGE_NAME, "fanout");//分发
//发送消息
String msg="hello ps";
channel.basicPublish(EXCHANGE_NAME, "", null, msg.getBytes());
System.out.println("Send :"+msg);
channel.close();
connection.close();
}
}
消费者1:
public class Recv1 {
private static final String QUEUE_NAME="test_queue_fanout_email";
private static final String EXCHANGE_NAME="test_exchange_fanout";
public static void main(String[] args) throws IOException, TimeoutException {
Connection connection = ConnectionUtils.getConnection();
final Channel channel = connection.createChannel();
//队列声明
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
//绑定队列到交换机 转发器
channel.queueBind(QUEUE_NAME, EXCHANGE_NAME, "");
channel.basicQos(1);//保证一次只分发一个
//定义一个消费者
Consumer consumer=new DefaultConsumer(channel){
//消息到达 触发这个方法
@Override
public void handleDelivery(String consumerTag, Envelope envelope,
BasicProperties properties, byte[] body) throws IOException {
String msg=new String(body,"utf-8");
System.out.println("[1] Recv msg:"+msg);
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}finally{
System.out.println("[1] done ");
channel.basicAck(envelope.getDeliveryTag(), false);
}
}
};
boolean autoAck=false;//自动应答 false
channel.basicConsume(QUEUE_NAME,autoAck , consumer);
}
}
消费者2:
public class Recv2 {
private static final String QUEUE_NAME="test_queue_fanout_sms";
private static final String EXCHANGE_NAME="test_exchange_fanout";
public static void main(String[] args) throws IOException, TimeoutException {
Connection connection = ConnectionUtils.getConnection();
final Channel channel = connection.createChannel();
//队列声明
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
//绑定队列到交换机 转发器
channel.queueBind(QUEUE_NAME, EXCHANGE_NAME, "");
channel.basicQos(1);//保证一次只分发一个
//定义一个消费者
Consumer consumer=new DefaultConsumer(channel){
//消息到达 触发这个方法
@Override
public void handleDelivery(String consumerTag, Envelope envelope,
BasicProperties properties, byte[] body) throws IOException {
String msg=new String(body,"utf-8");
System.out.println("[2] Recv msg:"+msg);
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}finally{
System.out.println("[2] done ");
channel.basicAck(envelope.getDeliveryTag(), false);
}
}
};
boolean autoAck=false;//自动应答 false
channel.basicConsume(QUEUE_NAME,autoAck , consumer);
}
}
TIPS:消息发送到了一个没有绑定队列的交换机时,消息就会丢失!
4.路由模型Routing
队列模型:
特点:
-
根据匹配的路由键进行消息的分发和消费,exchange交换机根据路由键分发到特定的队列。
-
处理路由键。 需要将一个队列绑定到交换机上,要求该消息与一个特定的路由键完全匹配。这是一个完整的匹配。如果一个队列 绑定到该交换机上要求路由键 “dog”,则只有被标记为“dog”的消息才被转发,不会转发 dog.puppy,也不会转发 dog.guard,只会转发 dog。
流程:
-
一个生产者,发送消息
-
每个消费者,都有一个独立的队列
-
消息发送到交换机,交换机发送到每个队列
-
根据key,是否相等,来接收消息
生产 者代码:
public class Send {
private static final String EXCHANGE_NAME="test_exchange_direct";
public static void main(String[] args) throws IOException, TimeoutException {
Connection connection = ConnectionUtils.getConnection();
Channel channel = connection.createChannel();
//exchange
channel.exchangeDeclare(EXCHANGE_NAME, "direct");
String msg="hello direct!";
String routingKey="info";
channel.basicPublish(EXCHANGE_NAME, routingKey, null, msg.getBytes());
System.out.println("send "+msg);
channel.close();
connection.close();
}
}
消费者1:
public class Recv1 {
private static final String EXCHANGE_NAME = "test_exchange_direct";
private static final String QUEUE_NAME = "test_queue_direct_1";
public static void main(String[] args) throws IOException, TimeoutException {
Connection connection = ConnectionUtils.getConnection();
final Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
channel.queueBind(QUEUE_NAME, EXCHANGE_NAME, "error");
channel.basicQos(1);
//定义一个消费者
Consumer consumer=new DefaultConsumer(channel){
//消息到达 触发这个方法
@Override
public void handleDelivery(String consumerTag, Envelope envelope,
BasicProperties properties, byte[] body) throws IOException {
String msg=new String(body,"utf-8");
System.out.println("[1] Recv msg:"+msg);
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}finally{
System.out.println("[1] done ");
channel.basicAck(envelope.getDeliveryTag(), false);
}
}
};
boolean autoAck=false;//自动应答 false
channel.basicConsume(QUEUE_NAME,autoAck , consumer);
}
}
消费 2:
public class Recv2 {
private static final String EXCHANGE_NAME = "test_exchange_direct";
private static final String QUEUE_NAME = "test_queue_direct_2";
public static void main(String[] args) throws IOException, TimeoutException {
Connection connection = ConnectionUtils.getConnection();
final Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
channel.queueBind(QUEUE_NAME, EXCHANGE_NAME, "error");
channel.queueBind(QUEUE_NAME, EXCHANGE_NAME, "info");
channel.queueBind(QUEUE_NAME, EXCHANGE_NAME, "warning");
channel.basicQos(1);
//定义一个消费者
Consumer consumer=new DefaultConsumer(channel){
//消息到达 触发这个方法
@Override
public void handleDelivery(String consumerTag, Envelope envelope,
BasicProperties properties, byte[] body) throws IOException {
String msg=new String(body,"utf-8");
System.out.println("[2] Recv msg:"+msg);
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}finally{
System.out.println("[2] done ");
channel.basicAck(envelope.getDeliveryTag(), false);
}
}
};
boolean autoAck=false;//自动应答 false
channel.basicConsume(QUEUE_NAME,autoAck , consumer);
}
}
消费者1 只消费routing Key=error的消息,而消费者2 消费 routinKey 为error ,info,waring的消息,因此运行后,消费者1 收不到消息,消费者收到消息。
5.主题模式Topic
队列模型:
特点:
- 将路由键和某模式进行匹配。
- 此时队列需要绑定要一个模式上。符号“#”匹配一个或多个词,符号“*”匹配一个词。因此“audit.#”能够匹配到 “audit.irs.corporate”,但是“audit.*” 只会匹配到“audit.irs”。
生产者代码:
public class Send {
private static final String EXCHANGE_NAME = "test_exchange_topic";
public static void main(String[] args) throws IOException, TimeoutException {
Connection connection = ConnectionUtils.getConnection();
Channel channel = connection.createChannel();
//exchange
channel.exchangeDeclare(EXCHANGE_NAME, "topic");
String msgString="商品....";
channel.basicPublish(EXCHANGE_NAME, "goods.delete", null, msgString.getBytes());
System.out.println("---send "+msgString);
channel.close();
connection.close();
}
}
消费者1:
public class Recv1 {
private static final String EXCHANGE_NAME = "test_exchange_topic";
private static final String QUEUE_NAME = "test_queue_topic_1";
public static void main(String[] args) throws IOException, TimeoutException {
Connection connection = ConnectionUtils.getConnection();
final Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
channel.queueBind(QUEUE_NAME, EXCHANGE_NAME, "goods.add");
channel.queueBind(QUEUE_NAME, EXCHANGE_NAME, "goods.update");
channel.basicQos(1);
//定义一个消费者
Consumer consumer=new DefaultConsumer(channel){
//消息到达 触发这个方法
@Override
public void handleDelivery(String consumerTag, Envelope envelope,
BasicProperties properties, byte[] body) throws IOException {
String msg=new String(body,"utf-8");
System.out.println("[1] Recv msg:"+msg);
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}finally{
System.out.println("[1] done ");
channel.basicAck(envelope.getDeliveryTag(), false);
}
}
};
boolean autoAck=false;//自动应答 false
channel.basicConsume(QUEUE_NAME,autoAck , consumer);
}
}
消费者2:
public class Recv2 {
private static final String EXCHANGE_NAME = "test_exchange_topic";
private static final String QUEUE_NAME = "test_queue_topic_2";
public static void main(String[] args) throws IOException, TimeoutException {
Connection connection = ConnectionUtils.getConnection();
final Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
channel.queueBind(QUEUE_NAME, EXCHANGE_NAME, "goods.#");
channel.basicQos(1);
//定义一个消费者
Consumer consumer=new DefaultConsumer(channel){
//消息到达 触发这个方法
@Override
public void handleDelivery(String consumerTag, Envelope envelope,
BasicProperties properties, byte[] body) throws IOException {
String msg=new String(body,"utf-8");
System.out.println("[2] Recv msg:"+msg);
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}finally{
System.out.println("[2] done ");
channel.basicAck(envelope.getDeliveryTag(), false);
}
}
};
boolean autoAck=false;//自动应答 false
channel.basicConsume(QUEUE_NAME,autoAck , consumer);
}
}
六、消息应答与消息持久化:information_desk_person:
- mandatory 当mandatory标志位设置为true时,如果exchange根据自身类型和消息routeKey无法找到一个符合条件的queue,那么会调用basic.return方法将消息返回给生产者(Basic.Return + Content-Header + Content-Body);当mandatory设置为false时,出现上述情形broker会直接将消息扔掉。
1.消息应答 Message acknowledgment
boolean autoAck = false;
channel.basicConsume(QUEUE_NAME, autoAck, consumer);
-
boolean autoAck = true;(自动确认模式)一旦 RabbitMQ 将消息分发给了消费者,就会从内存中删除。 在这种情况下,如果杀死正在执行任务的消费者,会丢失正在处理的消息,也会丢失已经分发给这个消 费者但尚未处理的消息。
-
boolean autoAck = false; (手动确认模式) 我们不想丢失任何任务,如果有一个消费者挂掉了,那么 我们应该将分发给它的任务交付给另一个消费者去处理。 为了确保消息不会丢失,RabbitMQ 支持消 息应答。消费者发送一个消息应答,告诉 RabbitMQ 这个消息已经接收并且处理完毕了。RabbitMQ 可 以删除它了。
-
消息应答是默认打开的。也就是 boolean autoAck =false;
2.消息持久化
若实现消息的彻底持久化,需要实现以下三点:
- 队列的持久化
- 消息体的持久化
- exchange 交换机的持久化
1.队列的持久化
boolean durable = true;
channel.queueDeclare("test_queue_work", durable, false, false, null);
2.消息的持久化
//持久化消息
Message message = MessageBuilder.withBody(requestJSON.getBytes("UTF-8")).build();
message.getMessageProperties().setDeliveryMode(MessageDeliveryMode.PERSISTENT);
rabbitTemplate.convertAndSend("xxxExchange", "xxxRouteKey", message);
- deliveryMode=1代表不持久化,deliveryMode=2代表持久化
3.交换机exchange持久化
channel.exchangeDeclare(exchangeName, “direct/topic/header/fanout”, true);
- 在申明exchange时,最后一个参数设置为true.
七、RabbitMQ消息确认机制:tangerine:
消息到达服务器之前丢失,那么持久化也不能解决此问题,因为消息根本就没有到达 Rabbit 服务器,因此引入事务的机制来确保消息消费的可靠性。
RabbitMQ 为我们提供了两种方式:
- 通过 AMQP 事务机制实现,这也是 AMQP 协议层面提供的解决方案
- 通过将 channel 设置成 confirm 模式来实现
1.事务机制
RabbitMQ 中与事务机制有关的方法有三个:txSelect(), txCommit()以及 txRollback(), txSelect 用于将当前 channel 设 置成 transaction 模式,txCommit 用于提交事务,txRollback 用于回滚事务,在通过 txSelect 开启事务之后,我们便 可以发布消息给 broker 代理服务器了,如果 txCommit 提交成功了,则消息一定到达了 broker 了,如果在 txCommit 执行之前 broker 异常崩溃或者由于其他原因抛出异常,这个时候我们便可以捕获异常通过 txRollback 回滚事务了。
代码:
# 在消息发送端开启事务和进行事务回滚
channel.txSelect();
channel.basicPublish("", QUEUE_NAME, null, msg.getBytes());
channel.txCommit();
特点:此种模式还是很耗时的,采用这种方式 降低了 Rabbitmq 的消息吞吐量
2.confirm模式
生产者将信道设置成 confirm 模式,一旦信道进入 confirm 模式,所有在该信道上面发布的消息都会被指派一个唯
一的 ID(从 1 开始),一旦消息被投递到所有匹配的队列之后,broker 就会发送一个确认给生产者(包含消息的唯一
ID),这就使得生产者知道消息已经正确到达目的队列了,如果消息和队列是可持久化的,那么确认消息会将消息写
入磁盘之后发出,broker 回传给生产者的确认消息中 deliver-tag 域包含了确认消息的序列号,此外 broker 也可以设
置 basic.ack 的 multiple 域,表示到这个序列号之前的所有消息都已经得到了处理
confirm 模式最大的好处在于他是异步的,一旦发布一条消息,生产者应用程序就可以在等信道返回确认的同时继
续发送下一条消息,当消息最终得到确认之后,生产者应用便可以通过回调方法来处理该确认消息,如果
RabbitMQ 因为自身内部错误导致消息丢失,就会发送一条 nack 消息,生产者应用程序同样可以在回调方法中处
理该 nack 消息。
1.开启confirm
TIPS:
已经在 transaction 事务模式的 channel 是不能再设置成 confirm 模式的,即这两种模式是不能共存的。
//生产者通过调用channel的confirmSelect方法将channel设置为confirm模式
channel.confirmSelect();
流程:
-
普通 confirm 模式:每发送一条消息后,调用 waitForConfirms()方法,等待服务器端 confirm。实际上是一种串行 confirm 了。
-
- 批量 confirm 模式:每发送一批消息后,调用 waitForConfirms()方法,等待服务器端 confirm。
-
- 异步 confirm 模式:提供一个回调方法,服务端 confirm 了一条或者多条消息后 Client 端会回 调这个方法。
普通confirm:
public class Send1 {
private static final String QUEUE_NAME="test_queue_confirm1";
public static void main(String[] args) throws IOException, TimeoutException, InterruptedException {
Connection connection = ConnectionUtils.getConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME,false,false,false,null);
//生产者调用confirmSelect 将channel设置为confirm模式 注意
channel.confirmSelect();
String msgString="hello confirm message!";
channel.basicPublish("", QUEUE_NAME, null,msgString.getBytes());
if(!channel.waitForConfirms()){
System.out.println("message send failed");
}else {
System.out.println("message send ok");
}
channel.close();
connection.close();
}
}
批量confirm:
public class Send2 {
private static final String QUEUE_NAME="test_queue_confirm1";
public static void main(String[] args) throws IOException, TimeoutException, InterruptedException {
Connection connection = ConnectionUtils.getConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME,false,false,false,null);
//生产者调用confirmSelect 将channel设置为confirm模式 注意
channel.confirmSelect();
String msgString="hello confirm message batch!";
//批量发送
for (int i = 0; i < 10; i++) {
channel.basicPublish("", QUEUE_NAME, null,msgString.getBytes());
}
//确认
if(!channel.waitForConfirms()){
System.out.println("message send failed");
}else {
System.out.println("message send ok");
}
channel.close();
connection.close();
}
}
异步confirm:
public class Send3 {
private static final String QUEUE_NAME="test_queue_confirm3";
public static void main(String[] args) throws IOException, TimeoutException {
Connection connection = ConnectionUtils.getConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME,false,false,false,null);
//生产者调用confirmSelect 将channel设置为confirm模式 注意
channel.confirmSelect();
//未确认的消息标识
final SortedSet<Long> confirmSet=Collections.synchronizedSortedSet(new TreeSet<Long>());
//通道添加监听
channel.addConfirmListener(new ConfirmListener() {
//没有问题的handleAck
public void handleAck(long deliveryTag, boolean multiple)
throws IOException {
if(multiple){
System.out.println("----handleAck----multiple");
confirmSet.headSet(deliveryTag+1).clear();
}else{
System.out.println("----handleAck-----multiple false");
confirmSet.remove(deliveryTag);
}
}
//handleNack
public void handleNack(long deliveryTag, boolean multiple)
throws IOException {
if(multiple){
System.out.println("---handleNack------multiple");
confirmSet.headSet(deliveryTag+1).clear();
}else{
System.out.println("--handleNack-------multiple false");
confirmSet.remove(deliveryTag);
}
}
});
String msgStr="ssssss";
while(true){
long seqNo = channel.getNextPublishSeqNo();
channel.basicPublish("", QUEUE_NAME, null, msgStr.getBytes());
confirmSet.add(seqNo);
}
}
}
消息消费:
public class Recv {
private static final String QUEUE_NAME="test_queue_confirm3";
public static void main(String[] args) throws IOException, TimeoutException {
Connection connection = ConnectionUtils.getConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME,false,false,false,null);
channel.basicConsume(QUEUE_NAME, true,new DefaultConsumer(channel){
@Override
public void handleDelivery(String consumerTag, Envelope envelope,
BasicProperties properties, byte[] body) throws IOException {
System.out.println("recv[confirm] msg:"+new String(body,"utf-8"));
}
});
}
}
八、SpringMVC集成RabbitMQ:baby_chick:
1.maven 依赖
<dependency>
<groupId>org.springframework.amqp</groupId>
<artifactId>spring-rabbit</artifactId>
<version>2.2.2.RELEASE</version>
</dependency>
<dependency>
<groupId>com.rabbitmq</groupId>
<artifactId>amqp-client</artifactId>
<version>5.8.0</version>
</dependency>
2.配置spring-rabbitmq.xml 文件
<!-- 1.定义RabbitMQ的连接工厂 -->
<rabbit:connection-factory
id="connectionFactory"
host="192.168.92.25" port="5675" username="xxx" password="xxx"
publisher-confirms="true" virtual-host="/xx_mq"/>
<!-- 2.定义Rabbit模板,指定连接工厂以及定义 direct-exchange -->
<rabbit:template id="rabbitTemplate" connection-factory="connectionFactory" mandatory="true"/>
<!-- MQ的管理,包括队列、交换器 声明等 -->
<rabbit:admin connection-factory="connectionFactory"/>
<!-- 声明队列 direct -->
<rabbit:queue name="direct_report" auto-declare="true" durable="true" auto-delete="false"/>
<rabbit:queue name="direct_index" auto-declare="true" durable="true" auto-delete="false"/>
<!-- 声明队列 fanout -->
<rabbit:queue name="queueNumFirst" auto-declare="true" durable="true" auto-delete="false"/>
<rabbit:queue name="queueNumFSecond" auto-declare="true" durable="true" auto-delete="false"/>
<!-- 声明队列 topic -->
<rabbit:queue name="topInfo" auto-declare="true" durable="true" auto-delete="false"/>
<rabbit:queue name="topDelete" auto-declare="true" durable="true" auto-delete="false"/>
<rabbit:queue name="topUpdate" auto-declare="true" durable="true" auto-delete="false"/>
<!-- 定义交换器路由模式 direct,自动声明 -->
<rabbit:direct-exchange name="directExchange" id="testDirectExchange" durable="true">
<rabbit:bindings>
<rabbit:binding queue="direct_report" key="dircet_report_key"/>
<rabbit:binding queue="direct_index" key="dircet_index_key"/>
</rabbit:bindings>
</rabbit:direct-exchange>
<!-- 定义交换器路由模式 fanout,自动声明 -->
<rabbit:fanout-exchange name="fanoutExchange" id="testFanoutExchange" durable="true">
<rabbit:bindings>
<rabbit:binding queue="queueNumFirst"/>
<rabbit:binding queue="queueNumFSecond"/>
</rabbit:bindings>
</rabbit:fanout-exchange>
<!-- 定义交换器路由模式 topic,自动声明 -->
<rabbit:topic-exchange name="topicExchange" id="testTopicExchange" durable="true">
<rabbit:bindings>
<rabbit:binding pattern="goods.#" queue="topInfo"/>
<rabbit:binding pattern="goods.update" queue="topUpdate"/>
<rabbit:binding pattern="goods.delete" queue="topDelete"/>
</rabbit:bindings>
</rabbit:topic-exchange>
<!-- direct消费者 -->
<bean name="directHander01" class="com.gsww.yxy.mq.consumer.DirectConsumerOne"/>
<bean name="directHander02" class="com.gsww.yxy.mq.consumer.DirectConsumerTwo"/>
<!--fanout 消费者-->
<bean name="fanOutHander01" class="com.gsww.yxy.mq.consumer.FanOutConsumerOne"/>
<bean name="fanOutHander02" class="com.gsww.yxy.mq.consumer.FanOutConsumerTwo"/>
<!--topic 消费者-->
<bean name="topHander01" class="com.gsww.yxy.mq.consumer.TopicConsumerOne"/>
<bean name="topHander02" class="com.gsww.yxy.mq.consumer.TopicConsumerTwo"/>
<bean name="topHander03" class="com.gsww.yxy.mq.consumer.TopicConsumerThree"/>
<!-- 定义消费者监听队列 -->
<rabbit:listener-container acknowledge="manual" auto-declare="true" connection-factory="connectionFactory">
<!--监听direct队列-->
<rabbit:listener ref="directHander01" queues="direct_report"/>
<rabbit:listener ref="directHander02" queues="direct_index"/>
<!--监听fanout队列-->
<rabbit:listener ref="fanOutHander01" queues="queueNumFirst"/>
<rabbit:listener ref="fanOutHander02" queues="queueNumFSecond"/>
<!--监听topic队列-->
<rabbit:listener ref="topHander01" queues="topInfo"/>
<rabbit:listener ref="topHander02" queues="topUpdate"/>
<rabbit:listener ref="topHander03" queues="topDelete"/>
</rabbit:listener-container>
项目结构如下:
-
发送消息:
@RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = {"/rabbitmq/application-rabbitmq.xml"}) public class RabbitSenderTest { @Autowired private RabbitTemplate rabbitTemplate; //测试Direct模式 @Test public void testSendDirectMsg()throws Exception{ String exchangeName="directExchange"; String info="Direct COnsum01"; String routeKey="dircet_report_key"; Message message = MessageBuilder.withBody(info.getBytes("UTF-8")).build(); //持久化消息 message.getMessageProperties().setDeliveryMode(MessageDeliveryMode.PERSISTENT); rabbitTemplate.send(exchangeName,routeKey,message); //send2 String exchangeNames="directExchange"; String infos="Direct COnsum02"; String routeKeys="dircet_index_key"; Message messages = MessageBuilder.withBody(infos.getBytes("UTF-8")).build(); //持久化消息 message.getMessageProperties().setDeliveryMode(MessageDeliveryMode.PERSISTENT); rabbitTemplate.send(exchangeNames,routeKeys,messages); } //测试fanout模式 @Test public void testFanout()throws Exception{ String exchangeNames="fanoutExchange"; String infos="这是群发消息"; Message messages = MessageBuilder.withBody(infos.getBytes("UTF-8")).build(); //持久化消息 messages.getMessageProperties().setDeliveryMode(MessageDeliveryMode.PERSISTENT); rabbitTemplate.send(exchangeNames,null,messages); } //测试Topic模式 @Test public void testTopic()throws Exception{ String exchangeNames="topicExchange"; //info String infos="这是topic主题消息info"; String routeKey="goods.info"; Message messages = MessageBuilder.withBody(infos.getBytes("UTF-8")).build(); messages.getMessageProperties().setDeliveryMode(MessageDeliveryMode.PERSISTENT); rabbitTemplate.send(exchangeNames,routeKey,messages); //update String info="这是topic主题消息update"; String routeKeys="goods.update"; Message message= MessageBuilder.withBody(info.getBytes("UTF-8")).build(); message.getMessageProperties().setDeliveryMode(MessageDeliveryMode.PERSISTENT); rabbitTemplate.send(exchangeNames,routeKeys,message); //delete String deleteInfo="这是topic主题消息delete"; String deleteKey="goods.delete"; Message deletResult= MessageBuilder.withBody(deleteInfo.getBytes("UTF-8")).build(); deletResult.getMessageProperties().setDeliveryMode(MessageDeliveryMode.PERSISTENT); rabbitTemplate.send(exchangeNames,deleteKey,deletResult); } }
- 接收消息
Direct消费者1:
public class DirectConsumerOne implements ChannelAwareMessageListener { private Logger logger = LoggerFactory.getLogger(DirectConsumerOne.class); @Override public void onMessage(Message message, Channel channel) throws Exception { channel.basicQos(1); String jsonInfo = new String(message.getBody(), "UTF-8"); logger.info("DiercetConsumerOne:" + jsonInfo); Thread.sleep(5000); channel.basicAck(message.getMessageProperties().getDeliveryTag(), false); } }
Fanout消费者1:
public class FanOutConsumerOne implements ChannelAwareMessageListener {
private Logger logger = LoggerFactory.getLogger(FanOutConsumerOne.class);
@Override
public void onMessage(Message message, Channel channel) throws Exception {
String jsonInfo = new String(message.getBody(), "UTF-8");
logger.info("FanOutConsumerOne:" + jsonInfo);
channel.basicAck(message.getMessageProperties().getDeliveryTag(),false);
}
}
Topic消费者1:
public class TopicConsumerOne implements ChannelAwareMessageListener {
private Logger logger = LoggerFactory.getLogger(TopicConsumerOne.class);
@Override
public void onMessage(Message message, Channel channel) throws Exception {
String jsonInfo = new String(message.getBody(), "UTF-8");
logger.info("匹配goods开头所有信息:" + jsonInfo);
channel.basicAck(message.getMessageProperties().getDeliveryTag(),false);
}
}
九、SpringBoot集成RabbitMQ :motor_scooter:
1.maven依赖引入
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.amqp</groupId>
<artifactId>spring-rabbit-test</artifactId>
<scope>test</scope>
</dependency>
2.application.yml配置
spring:
rabbitmq:
host: 10.18.33.xxx
port: 5672
username: yxy
password: xxx
virtual-host: /test_mq
publisher-confirms: true
publisher-returns: true
template:
mandatory: true
listener:
simple:
acknowledge-mode: manual
prefetch: 1
3.申明队列/交换机
FanoutCOnfig, 配置发布订阅模式的消息类型
/**
* 发布/订阅模式配置类 Publish/Subscribe
*/
@Configuration
public class FanoutConfig {
//定义fanout交换机名称
private String EXCHANGE_NAME = "textExchange";
//xxxx队列名称
private String FANOUT_SPLITSTUDY_QUEUE = "test_splitStudy";
//xxxx队列
private String FANOUT_SYNCSTUDY_QUEUE = "test_syncStudy";
// xxxx推送队列
private String FANOUT_PUSHYJPT_QUEUE = "test_yjptStudy";
//xxx推送队列
private String FANOUT_AISTUDY_QUEUE = "test_aiStudy";
@Bean
public Queue splitStudyQueue() {
return new Queue(FANOUT_SPLITSTUDY_QUEUE, true);
}
@Bean
public Queue syncStudyQueue() {
return new Queue(FANOUT_SYNCSTUDY_QUEUE, true);
}
@Bean
public Queue yjptStudyQueue() {
return new Queue(FANOUT_PUSHYJPT_QUEUE, true);
}
@Bean
public Queue aiStudyQueue() {
return new Queue(FANOUT_AISTUDY_QUEUE, true);
}
//配置交换器
@Bean
FanoutExchange fanoutExchange() {
return new FanoutExchange(EXCHANGE_NAME, true, false);
}
// 绑定队列到交换器
@Bean
Binding bindingFanoutExchangeQueue(Queue splitStudyQueue, FanoutExchange fanoutExchange) {
return BindingBuilder.bind(splitStudyQueue).to(fanoutExchange);
}
// 绑定队列到交换器
@Bean
Binding bindingFanoutExchangeQueue2(Queue syncStudyQueue, FanoutExchange fanoutExchange) {
return BindingBuilder.bind(syncStudyQueue).to(fanoutExchange);
}
// 绑定队列到交换器
@Bean
Binding bindingFanoutExchangeQueue3(Queue yjptStudyQueue, FanoutExchange fanoutExchange) {
return BindingBuilder.bind(yjptStudyQueue).to(fanoutExchange);
}
// 绑定队列到交换器
@Bean
Binding bindingFanoutExchangeQueue4(Queue aiStudyQueue, FanoutExchange fanoutExchange) {
return BindingBuilder.bind(aiStudyQueue).to(fanoutExchange);
}
}
申明一个路由Direct交换配置模型
@Configuration
public class DirectConfig {
//定义Direct交换机名称
private String EXCHANGE_NAME = "dcmExchange";
//定义通知队列和路由名称
private String DIRECT_DCMDOWN_QUEUE = "dcm_down";
@Bean
public Queue dcmDownQueue() {
return new Queue(DIRECT_DCMDOWN_QUEUE, true);
}
//配置交换器
@Bean
DirectExchange directExchange() {
return new DirectExchange(EXCHANGE_NAME, true, false);
}
// 绑定队列到交换器
@Bean
Binding bindingDirectExchangeQueue(@Qualifier("dcmDownQueue") Queue dcmDownQueue, DirectExchange directExchange) {
return BindingBuilder.bind(dcmDownQueue).to(directExchange).with(DIRECT_DCMDOWN_QUEUE);
}
}
4.实现消息消费
@Component
@Slf4j
public class FanoutAiStudy {
@Autowired
private HosCodeSwitchService hosCodeSwitchService;
@Autowired
private AiReportService aiReportService;
/**
* 消息队列任务处理
*
* @param message
* @param channel
* @throws IOException
*/
@RabbitListener(queues = "film_aiStudy")
@RabbitHandler
public void onMessage(Message message, Channel channel) throws IOException {
try {
channel.basicQos(1);
String messageInfo = new String(message.getBody(), "UTF-8");
//业务处理逻辑
//消息消费成功,手动确认消息
channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);
} catch (IOException e) {
log.error("处理xxxxx:", e);
//消息消费失败,重回队列
channel.basicNack(message.getMessageProperties().getDeliveryTag(), false, true);
}
}
}
5.发送广播消息
string message="这是测试消息";
Message messageInfo = MessageBuilder.withBody(message.getBytes("UTF-8")).build();
messageInfo.getMessageProperties().setDeliveryMode(MessageDeliveryMode.PERSISTENT);
//fanout模式
rabbitTemplate.send(exchangeName, null, messageInfo);
6. rabbitmq实现延迟消息业务逻辑
-
前置条件: rabbitmq 下载启用了延时队列插件
rabbitmq_delayed_message_exchange
-
注意插件版本需要跟rabbitmq版本相匹配
Exchange和Queue配置
import org.springframework.amqp.core.Binding;
import org.springframework.amqp.core.BindingBuilder;
import org.springframework.amqp.core.CustomExchange;
import org.springframework.amqp.core.Queue;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import java.util.HashMap;
import java.util.Map;
/**
* 延迟消息处理机制
*
* @author zhangyongliang
* @create 2019-10-28 16:56
**/
@Configuration
public class DelayedConfig {
public final static String DELAY_QUEUE_NAME = "delayed.goods.order";
public final static String DELAY_EXCHANGE_NAME = "delayedec";
@Bean
public Queue queue() {
return new Queue(DelayedConfig.DELAY_QUEUE_NAME,true);
}
// 配置默认的交换机
@Bean
CustomExchange customExchange() {
Map<String, Object> args = new HashMap<>();
args.put("x-delayed-type", "direct");
//参数二为类型:必须是x-delayed-message
return new CustomExchange(DelayedConfig.DELAY_EXCHANGE_NAME, "x-delayed-message", true, false, args);
}
// 绑定队列到交换器
@Bean
Binding binding(Queue queue, CustomExchange exchange) {
return BindingBuilder.bind(queue).to(exchange).with(DelayedConfig.DELAY_QUEUE_NAME).noargs();
}
这里要特别注意的是,使用的是CustomExchange,不是DirectExchange,另外CustomExchange的类型必须是x-delayed-message。
实现消息发送
public void sendDelayMsg(String msg) {
System.out.println("发送时间:" + DateUtil.now());
rabbitmqTemplate.convertAndSend(DelayedConfig.DELAY_EXCHANGE_NAME, DelayedConfig.DELAY_QUEUE_NAME, msg, message -> {
message.getMessageProperties().setHeader("x-delay", 3000);
return message;
});
}
注意在发送的时候,必须加上一个header x-delay
在这里我设置的延迟时间是3秒。
消费者消费
@Component
@RabbitListener(queues = "delayed.goods.order")
public class DelayedReceiver {
@RabbitHandler
public void process(String msg) {
System.out.println("接收时间:" + DateUtil.now());
System.out.println("消息内容:" + msg);
}
}
发送端控制台打印:
发送时间:2023-03-06 17:01:44
消费端控制台打印:
接收时间:2023-03-06 17:01:47
消息内容:I am a delay MessageInfo