拉勾教育学习-笔记分享の"斡旋"消息中间件 II (RabbitMQ)

401 阅读16分钟

二、RabbitMQ

part 1 - RabbitMQ架构与实战

「RabbitMQ介绍、概念、基本架构」

RabbitMQ,俗称 “兔子MQ”(可见其轻巧,敏捷),是目前非常热门的一款开源消息中间件,不管是互联网行业还是传统行业都广泛使用(最早是为了解决电信行业系统之间的可靠通信而设计)。

官方社区有很多功能强大的插件可供使用

  • 特点
    • 高可靠性、易扩展、高可用、功能丰富
    • 支持大多数(甚至冷门)的编程语言客户端
    • RabbitMQ遵循AMQP协议,自身采用Erlang(一种由爱立信开发的通用面向并发编程的语言)编写
    • 也支持MQTT等其他协议
  • 逻辑架构
  • Exchange类型:
    • Fanout:会把所有发送到该交换器的消息路由到所有与该交换器绑定的队列中
    • Direct : 会把消息路由到那些BindingKey和RoutingKey完全匹配的队列中
    • Topic : 在direct匹配规则上进行了扩展;将消息路由到BindingKey和RoutingKey相匹配的队列中,这里的匹配规则稍微不同,它约定:
      • BindingKey和RoutingKey一样都是由"."分隔的字符串;
        BindingKey中可以存在两种特殊字符 “*”“#”,用于模糊匹配,其中 "*" 用于匹配一个单词,"#" 用于匹配多个单词(可以是0个)
    • Headers : 不依赖于路由键的匹配规则来路由信息,而是根据发送的消息内容中的headers属性进行匹配。
      在绑定队列和交换器时指定一组键值对,当发送的消息到交换器时,RabbitMQ会获取到该消息的headers,对比其中的键值对是否完全匹配队列和交换器绑定时指定的键值对,如果匹配,消息就会路由到该队列。性能很差,不实用
  • 数据存储
    • 存储机制:RabbitMQ消息有两种类型(都会被写入磁盘):
      • 持久化消息:持久化消息在到达队列时写入磁盘,同时会内存中保存一份备份,当内存吃紧时,消息从内存中清除。这会提高一定的性能
      • 非持久化消息:非持久化消息一般只存于内存中,当内存压力大时数据刷盘处理,以节省内存空间
    • 存储层两个部分
      • 队列索引 rabbit_queue_index
      • 消息存储 rabbit_msg_store

「安装和部署RabbitMQ」

RabbitMQ 的安装部署首先需要安装 Erlang(基于Erlang 的 VM 运行)

  1. 安装依赖:yum install socat -y
  2. 安装Erlang:
    • wget https://packages.erlang-solutions.com/erlang-solutions-1.0-1.noarch.rpm 下载(至当前目录)
    • sudo yum install erlang 安装
    • erl -version 检验
    • whereis erlang 查看安装路径
  3. 安装RabbitMQ
    • wget https://github.com/rabbitmq/rabbitmq-server/releases/download/v3.8.7/rabbitmq-server-3.8.7-1.el6.noarch.rpm 下载(至当前目录)笔者当前是最后一版支持Erlang的版本,读者可根据需要前去官网查询
    • rpm –import https://www.rabbitmq.com/rabbitmq-release-signing-key.asc 导入GPG密钥
    • rpm -Uvh rabbitmq-server-3.8.7-1.el6.noarch.rpm 安装RPM包
  4. 启动RabbitMQ
    • systemctl start/enable rabbitmq-server 启动(/自动启动)rabbitMQ服务器进程
    • systemctl status rabbitmq-server 查看rabbitMQ信息
  5. 启动rabbitMQ Web管理控制台
    • rabbitmq-plugins enable rabbitmq_management 启动控制台
    • chown -R rabbitmq:rabbitmq /var/lib/rabbitmq/ 将RabbitMQ文件的所有权提供给RabbitMQ用户
    • 为RabbitMQ Web管理控制台创建管理用户:
      # 将管理员更改为管理员用户的首选用户名。 确保将StrongPassword更改为非常强大的密码
      rabbitmqctl add_user admin StrongPassword
      rabbitmqctl set_user_tags admin administrator
      rabbitmqctl set_permissions -p / admin “.*” “.*” “.*”
      

      用户的标签和权限:

      • (None): 没有访问Management插件的权利
      • management: 可以使用消息协议做任何操作的权限
        • 可以使用AMQP协议登录的虚拟主机的权限
        • 查看它们能登录的所有虚拟主机中所有队列、交换器和绑定的权限
        • 查看和关闭它们自己的通道和连接的权限
        • 查看它们能访问的虚拟主机中的全局统计信息,包括其他用户的活动
      • policymaker: 所有management标签可以做的
        • 在它们能通过AMQP协议登录的虚拟主机上,查看、创建和删除策略以及虚拟主机参数的权限
      • monitoring: 所有management能做的
        • 列出所有的虚拟主机,包括列出不能使用消息协议访问的虚拟主机的权限
        • 查看其他用户连接和通道的权限
        • 查看节点级别的数据如内存使用和集群的权限
        • 查看真正的全局所有虚拟主机统计数据的权限
      • administrator: 所有policymaker和monitoring能做的
        • 创建删除虚拟主机的权限
        • 查看、创建和删除用户的权限
        • 查看、创建和删除权限的权限
        • 关闭其他用户连接的权限
  6. 访问控制台
    image.png image.png

「RabbitMQ常用操作命令」

# 前台启动 Erlang VM & RabbitMQ
rabbitmq-server

# 后台启动 Erlang VM & RabbitMQ (detached : /dɪˈtætʃt/  adj. 单独的;冷漠的;分离的;)
rabbitmq-server -detached

# 停止 Erlang VM & RabbitMQ
rabbitmqctl stop

# 查看所有队列
rabbitmqctl list_queues

# 查看所有虚拟机
rabbitmqctl list_vhosts

# 在 Erlang VM 运行的情况下 启动/停止 RabbitMQ 应用
rabbitmqctl start_app/stop_app

# 查看所有节点状态
rabbitmqctl status

# 查看所有可用的插件
rabbitmq-plugins list

# 启动/停用 插件
rabbitmq-plugins enable/disable <PLUGIN-NAME>

# 添加用户
rabbitmqctl add_user USERNAME PASSWORD

# 列出所有用户
rabbitmqctl list_users

# 删除用户
rabbitctl delete_user USERNAME

# 清除用户权限
rabbitmqctl clear_permission -p vhostpath USERNAME

# 列出用户权限
rabbitmqctl list_user_permissions USERNAME

# 修改密码
rabbitmqctl change_password USERNAME NEW_PASSWORD

# 设置用户权限
rabbitmqctl set_permissions -p vhostpath USERNAME ".*" ".*" ".*"

# 创建虚拟主机
rabbitmqctl add_vhost vhostpath

# 列出所有虚拟主机
rabbitmqctl list_vhosts

# 列出虚拟主机上的所有权限
rabbitmqctl list_permissions -p vhostpath

# 删除虚拟主机
rabbitmqctl delete_vhost vhost vhostpath

# 移除所有数据 (要在 rabbitemqctl stop_app 之后使用)
rabbitmqctl reset

「RabbitMQ工作流程」

  • 生产者发送消息的流程
    1. 生产者连接 RabbitMQ,建立TCP连接(Connection),开启通道(Channel)
    2. 生产者声明一个Exchange(交换器),设置相关属性(如 交换器类型、是否持久化等)
    3. 生产者声明一个 消息队列 并设置相关属性(如 是否排他、是否持久化、是否自动删除等)
    4. 生产者通过 bindingkey(绑定Key) 将交换器和队列绑定
    5. 生产者发送消息至 RabbitMQ Broker,其中包含 routingkey(路由键)、交换器等信息
    6. 相应的交换器根据接收到的 routingkey 查找相匹配的队列
    7. ==> 如果找到,则将生产者传输过来的消息存入响应的队列
    8. ==> 如果没找到,则根据生产者配置的属性选择 丢弃 or 退还生产者
    9. 关闭信道
    10. 关闭连接
  • 消费者接收消息的过程
    1. 消费者连接到 RabbitMQ Broker,建立TCP连接(Connection),开启通道(Channel)
    2. 消费者向 RabbitMQ Broker 请求消费队列,设置相关信息
    3. 等待 RabbitMQ Broker 回应并投递对应队列的消息,消费者接收消费
    4. 消费者 确认(Ack) 接收到的消息
    5. RabbitMQ 从队列中删除已被确认的消息
    6. 关闭信道
    7. 关闭连接

image.png 该图为AMQP的协议模式,可将其中的 AMQP Broker 替换为 RabbitMQ Broker 理解RabbitMQ的工作流程

【案例展示】

-Producer- 生产者
public class HelloWorldSender {
    
    private static String QUEUE_NAME = "hello.queue";
    private static String EXCHANGE_NAME = "my.exchange";
    private static String BINDING_KEY = "hello.bind";
    
    public static void main(String[] args) throws IOException, TimeoutException {
        
        ConnectionFactory factory = new ConnectionFactory();
        factory.setHost("106.75.60.49"); // 设置主机名
        factory.setVirtualHost("/"); // 设置虚拟主机名称  '/' 在url中的转义字符 %2f
        factory.setUsername("admin");
        factory.setPassword("123456");
        factory.setPort(5672); // AMQP 协议的默认端口
        
        /* 借助 TWR 创建连接和通道(try块结束会自动关闭) */
        try(Connection conn = factory.newConnection();
            Channel channel = conn.createChannel()){
    
            // 声明一个消息队列 ==》 s:消息队列名称 / b:是否持久化 / b1:是否排他 / b2:是否自动删除 / map:属性参数集合
            channel.queueDeclare(QUEUE_NAME, false, false, true, null);
            // 声明一个交换器 ==》 s:交换器名称 / builtinExchangeType:交换器类型(枚举可选) /  b:是否持久化 / b1:是否自动删除 / map:属性参数集合
            channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.DIRECT, false, false, null);
            // 声明一个 QueueBind,将队列和交换器绑定
            channel.queueBind(QUEUE_NAME, EXCHANGE_NAME, BINDING_KEY);
            // 发送消息
            channel.basicPublish(EXCHANGE_NAME, BINDING_KEY, null, "Hello RabbitMQ !".getBytes());
            
        } catch (Exception e) {
            e.printStackTrace();
        }
        
    }

}

image.png

-C1- 消费者(主动拉消息模式)
public class HelloWorldConsumer {
    
    private static String QUEUE_NAME = "hello.queue";
    
    public static void main(String[] args) throws NoSuchAlgorithmException, KeyManagementException, URISyntaxException {
        ConnectionFactory factory = new ConnectionFactory();
        factory.setUri("amqp://admin:123456@106.75.60.49:5672/%2f");
    
        try(Connection conn = factory.newConnection();
            Channel channel = conn.createChannel()){
    
            // 拉消息模式: 指定从哪个消费者消费消息,true:是否自动确认
            final GetResponse getResponse = channel.basicGet(QUEUE_NAME, true);
            // 获取消息实体
            final byte[] body = getResponse.getBody();
            System.out.println(new String(body));
    
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
    
}

image.png

image.png

-C2- 消费者(被动监听消息模式)
public class HelloWorldConsumer2 {
    
    private static String QUEUE_NAME = "hello.queue";
    
    public static void main(String[] args) throws NoSuchAlgorithmException, KeyManagementException, URISyntaxException {
        ConnectionFactory factory = new ConnectionFactory();
        factory.setUri("amqp://admin:123456@106.75.60.49:5672/%2f");
        
        try(Connection conn = factory.newConnection();
            Channel channel = conn.createChannel()){
    
            // 确保MQ中有该队列,如果没有则创建
            channel.queueDeclare(QUEUE_NAME, false, false, true, null);
    
            DeliverCallback callback = (consumerTag, message) -> {
                System.out.println(new String(message.getBody()));
            };
            // 监听消息,一旦有消息推送过来,就调用第一个lambda表达式
            channel.basicConsume(QUEUE_NAME, callback, (consumerTag) -> {});
    
            System.out.println("通道和连接关闭后,使用的exchange和queue会被自动删除");
            
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
    
}

image.png

「RabbitMQ工作模式」

生产者和消费者,需要与RabbitMQ Broker 建立TCP连接,也就是Connection 。一旦TCP 连接建立起来,客户端紧接着创建一个AMQP 信道(Channel),每个信道都会被指派一个唯一的ID。信道是建立在Connection 之上的虚拟连接, RabbitMQ 处理的每条AMQP 指令都是通过信道完成的。

image.png

为什么不直接使用TCP连接,而是使用信道?
==> RabbitMQ 采用类似NIO的做法,复用TCP 连接,减少性能开销,便于管理。

△ Work Queue

生产者发消息,启动多个消费者实例来消费消息,每个消费者仅消费部分信息,可达到负载均衡的效果

-Client- 消费者
public class Consumer {
    
    private static String QUEUE_NAME = "my.queue";
    
    public static void main(String[] args) throws NoSuchAlgorithmException, KeyManagementException, URISyntaxException {
        ConnectionFactory factory = new ConnectionFactory();
        factory.setUri("amqp://admin:123456@117.50.40.96:5672/%2f");
    
        try(Connection conn = factory.newConnection();
            Channel channel = conn.createChannel()){
        
            // 确保MQ中有该队列,如果没有则创建
            channel.queueDeclare(QUEUE_NAME, true, false, false, null);
        
            channel.basicConsume(QUEUE_NAME,
                    (s, delivery) -> System.out.println("推送来的消息" + new String(delivery.getBody(), "utf-8")),
                    s -> System.out.println("Cancel : " + s));
        
            while (true) {} // 循环监控效果
            
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
    
}

image.png

image.png

-Producer- 生产者
public class Producer {
    
    private static String QUEUE_NAME = "hello.queue";
    private static String EXCHANGE_NAME = "my.exchange";
    private static String BINDING_KEY = "hello.bind";
    
    public static void main(String[] args) throws NoSuchAlgorithmException, KeyManagementException, URISyntaxException, IOException, TimeoutException {
        ConnectionFactory factory = new ConnectionFactory();
        factory.setUri("amqp://admin:123456@117.50.40.96:5672/%2f");
    
        try(Connection conn = factory.newConnection();
            Channel channel = conn.createChannel()){
        
            // 声明一个消息队列 ==》 s:消息队列名称 / b:是否持久化 / b1:是否排他 / b2:是否自动删除 / map:属性参数集合
            channel.queueDeclare(QUEUE_NAME, false, false, true, null);
            // 声明一个交换器 ==》 s:交换器名称 / builtinExchangeType:交换器类型(枚举可选) /  b:是否持久化 / b1:是否自动删除 / map:属性参数集合
            channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.DIRECT, false, false, null);
            channel.exchangeBind(EXCHANGE_NAME, QUEUE_NAME, BINDING_KEY);
            // 连续生产15次消息
            for (int i = 0; i < 15; i++) {
                channel.basicPublish(EXCHANGE_NAME, QUEUE_NAME, null, ("工作队列" + i).getBytes("utf-8"));
            }
        
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
    
}

启动生产者进程后,消费者轮询式消费:

image.png

△ 发布订阅模式

使用 fanout(扇出)类型交换器,忽略 routingKey
每个消费者定义一个队列并绑定到同一个 Exchange,然后各自消费到完整的消息。

见名知意:风扇型的交换器,将受到的所有消息“”到它所知道的所有队列上去...

image.png

-Client1- 消费者1
public class OneConsumer {
    
    private static String FAN_NAME = "my.fan";
    
    public static void main(String[] args) throws NoSuchAlgorithmException, KeyManagementException, URISyntaxException {
        ConnectionFactory factory = new ConnectionFactory();
        factory.setUri("amqp://admin:123456@117.50.40.96:5672/%2f");
    
        try(Connection conn = factory.newConnection();
            Channel channel = conn.createChannel()){
            
            // 声明临时队列,队列的名字由RabbitMQ自动生成
            final String queueName = channel.queueDeclare().getQueue();
            System.out.println("生成的临时队列的名字为:" + queueName);
        
            channel.exchangeDeclare(FAN_NAME, BuiltinExchangeType.FANOUT, true, false, null);
            // fanout类型的交换器绑定不需要 routing key
            channel.queueBind(queueName, FAN_NAME, "");
    
            channel.basicConsume(queueName, (consumerTag, message) -> {
                System.out.println("One   " + new String(message.getBody(), "utf-8"));
            }, consumerTag -> {});
            
            while (true) {} // 循环监控效果
        
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
    
}
-Client2- 消费者2
public class TwoConsumer {
    
    private static String FAN_NAME = "my.fan";
    
    public static void main(String[] args) throws NoSuchAlgorithmException, KeyManagementException, URISyntaxException {
        ConnectionFactory factory = new ConnectionFactory();
        factory.setUri("amqp://admin:123456@117.50.40.96:5672/%2f");
    
        try(Connection conn = factory.newConnection();
            Channel channel = conn.createChannel()){
            
            // 声明临时队列,队列的名字由RabbitMQ自动生成
            final String queueName = channel.queueDeclare().getQueue();
            System.out.println("生成的临时队列的名字为:" + queueName);
        
            channel.exchangeDeclare(FAN_NAME, BuiltinExchangeType.FANOUT, true, false, null);
            // fanout类型的交换器绑定不需要 routing key
            channel.queueBind(queueName, FAN_NAME, "");
    
            channel.basicConsume(queueName, (consumerTag, message) -> {
                System.out.println("Two   " + new String(message.getBody(), "utf-8"));
            }, consumerTag -> {});
            
            while (true) {} // 循环监控效果
        
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
    
}
-Client3- 消费者3
public class ThreeConsumer {
    
    private static String FAN_NAME = "my.fan";
    
    public static void main(String[] args) throws NoSuchAlgorithmException, KeyManagementException, URISyntaxException {
        ConnectionFactory factory = new ConnectionFactory();
        factory.setUri("amqp://admin:123456@117.50.40.96:5672/%2f");
    
        try(Connection conn = factory.newConnection();
            Channel channel = conn.createChannel()){
            
            // 声明临时队列,队列的名字由RabbitMQ自动生成
            final String queueName = channel.queueDeclare().getQueue();
            System.out.println("生成的临时队列的名字为:" + queueName);
        
            channel.exchangeDeclare(FAN_NAME, BuiltinExchangeType.FANOUT, true, false, null);
            // fanout类型的交换器绑定不需要 routing key
            channel.queueBind(queueName, FAN_NAME, "");
    
            channel.basicConsume(queueName, (consumerTag, message) -> {
                System.out.println("Three   " + new String(message.getBody(), "utf-8"));
            }, consumerTag -> {});
            
            while (true) {} // 循环监控效果
        
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
    
}
-Producer- 生产者
public class FanoutProducer {
    
    private static String FAN_NAME = "my.fan";
    
    public static void main(String[] args) throws NoSuchAlgorithmException, KeyManagementException, URISyntaxException {
    
        ConnectionFactory factory = new ConnectionFactory();
        factory.setUri("amqp://admin:123456@117.50.40.96:5672/%2f");
    
        try(Connection conn = factory.newConnection();
            Channel channel = conn.createChannel()){
        
            // 声明一个fanout交换器 ==》 s:交换器名称 / builtinExchangeType:交换器类型(枚举可选) /  b:是否持久化 / b1:是否自动删除 / map:属性参数集合
            channel.exchangeDeclare(FAN_NAME, BuiltinExchangeType.FANOUT, true, false, null);
            // 连续生产15次消息
            for (int i = 0; i < 20; i++) {
                // fanout类型的交换器无需指定路由键
                channel.basicPublish(FAN_NAME, "", null, ("Fan-" + i).getBytes("utf-8"));
            }
        
        } catch (Exception e) {
            e.printStackTrace();
        }
        
    }
    
}

先启动三个消费者,然后启动生产者:

image.png

三个消费者消费了同样的消息

△ 路由模式

fanout模式的交换器实现了将消息散步给多个消费者,但有些情形下, 我们只想让接收者接收到部分消息,如:将关键信息记录到log文件,同时在控制台正常打印所有的日志消息

direct交换器的核心就是:只要消息的routingKey 和 bindingKey对应,消息就会退给指定的队列。

-Client1- Error级别日志消费者
public class ErrorConsumer {
    
    private static String QUEUE_NAME = "error.queue";
    private static String EXCHANGE_NAME = "log-route.exchange";
    
    public static void main(String[] args) throws NoSuchAlgorithmException, KeyManagementException, URISyntaxException {
        ConnectionFactory factory = new ConnectionFactory();
        factory.setUri("amqp://admin:123456@117.50.40.96:5672/%2f");
    
        try(Connection conn = factory.newConnection();
            Channel channel = conn.createChannel()){
    
            channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.DIRECT, false, false, null);
            // 确保MQ中有该队列,如果没有则创建
            channel.queueDeclare(QUEUE_NAME, true, false, false, null);
            channel.queueBind(QUEUE_NAME, EXCHANGE_NAME, "ERROR");
            
            channel.basicConsume(QUEUE_NAME,
                    (s, delivery) -> System.out.println("ErrorConsumer收到的消息:" + new String(delivery.getBody(), "utf-8")),
                    s -> System.out.println("Cancel : " + s));
        
            while (true) {} // 循环监控效果
        
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
    
}
-Client2- Warn级别日志消费者
public class WarnConsumer {
    
    private static String QUEUE_NAME = "warn.queue";
    private static String EXCHANGE_NAME = "log-route.exchange";
    
    public static void main(String[] args) throws NoSuchAlgorithmException, KeyManagementException, URISyntaxException {
        ConnectionFactory factory = new ConnectionFactory();
        factory.setUri("amqp://admin:123456@117.50.40.96:5672/%2f");
    
        try(Connection conn = factory.newConnection();
            Channel channel = conn.createChannel()){
    
            channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.DIRECT, false, false, null);
            // 确保MQ中有该队列,如果没有则创建
            channel.queueDeclare(QUEUE_NAME, true, false, false, null);
            channel.queueBind(QUEUE_NAME, EXCHANGE_NAME, "WARN");
            
            channel.basicConsume(QUEUE_NAME,
                    (s, delivery) -> System.out.println("WarnConsumer收到的消息:" + new String(delivery.getBody(), "utf-8")),
                    s -> System.out.println("Cancel : " + s));
        
            while (true) {} // 循环监控效果
        
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
    
}
-Client3- Fatal级别日志消费者
public class FatalConsumer {
    
    private static String QUEUE_NAME = "fatal.queue";
    private static String EXCHANGE_NAME = "log-route.exchange";
    
    public static void main(String[] args) throws NoSuchAlgorithmException, KeyManagementException, URISyntaxException {
        ConnectionFactory factory = new ConnectionFactory();
        factory.setUri("amqp://admin:123456@117.50.40.96:5672/%2f");
    
        try(Connection conn = factory.newConnection();
            Channel channel = conn.createChannel()){
    
            channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.DIRECT, false, false, null);
            // 确保MQ中有该队列,如果没有则创建
            channel.queueDeclare(QUEUE_NAME, true, false, false, null);
            channel.queueBind(QUEUE_NAME, EXCHANGE_NAME, "FATAL");
            
            channel.basicConsume(QUEUE_NAME,
                    (s, delivery) -> System.out.println("FatalConsumer收到的消息:" + new String(delivery.getBody(), "utf-8")),
                    s -> System.out.println("Cancel : " + s));
        
            while (true) {} // 循环监控效果
        
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
    
}
-Producer- 日志生产者(实际场景:汇总了系统所有日志文件的集中站)
public class RouteProducer {
    
    private static String EXCHANGE_NAME = "log-route.exchange";
    
    // 日志级别
    private static String[] LOG_LEVEL = {"ERROR", "FATAL", "WARN"};
    // 随机分配器
    private static Random random = new Random();
    
    public static void main(String[] args) throws NoSuchAlgorithmException, KeyManagementException, URISyntaxException {
    
        ConnectionFactory factory = new ConnectionFactory();
        factory.setUri("amqp://admin:123456@117.50.40.96:5672/%2f");
    
        try(Connection conn = factory.newConnection();
            Channel channel = conn.createChannel()){
    
            // 声明direct类型的交换器,交换器和消息队列的绑定不需要在这里处理
            channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.DIRECT, false, false, null);
    
            for (int i = 0; i < 100; i++) {
                String level = LOG_LEVEL[random.nextInt(100) % LOG_LEVEL.length];
                channel.basicPublish(EXCHANGE_NAME, level, null, ("这是【" + level + "】的消息").getBytes());
            }
        
        } catch (Exception e) {
            e.printStackTrace();
        }
        
    }
    
}

启动三个消费者后查看到:

image.png

启动生产者将log全部输送到交换器:

image.png

△ 主题模式

使用 topic 类型的交换器,队列绑定到交换器、 bindingKey 时使用通配符,交换器将消息路由转发到具体队列时会根据消息 routingKey 模糊匹配,比较灵活

要想 topic 类型的交换器, routingKey 就不能随便写了,它必须得是点分单词。
单词可以随便写,生产中一般使用消息的特征。
如:“stock.usd.nyse”,“nyse.vmw”,“quick.orange.rabbit”等。该点分单词字符串最长255字节

  1. * (star)匹配一个单词
  2. # 匹配0到多个单词

image.png

如上图消息发送的时候指定的 routingKey 包含了三个词,两个点:<speed>.<color>.<species>
分别代表:速度.颜色.物种
Q1 绑定了 *.orange.* ==> 绑定颜色为orange颜色的动物的消息
Q2 绑定了 *.*.rabbit + slow.# ==> 绑定所有兔子+所有慢速的动物
(绑定失败的消息则丢弃)

如果在 topic 类型的交换器中 bindingKey 使用 # ,则就是 fanout 类型交换器的行为。
如果在 topic 类型的交换器中 bindingKey 中不使用 *# ,则就是 direct 类型交换器的行为。

实例代码
-Producer- 主题生产者
public class TopicProducer {
    
    private static String EXCHANGE_NAME = "topic.exchange";
    private static final String[] SPEED = {"slow", "quick", "normal"};
    private static final String[] COLOR = {"black", "orange", "red", "yellow", "blue", "white"};
    private static final String[] SPECIES = {"dog", "cat", "rabbit", "bear", "horse"};
    private static final Random RANDOM = new Random();
    
    public static void main(String[] args) throws NoSuchAlgorithmException, KeyManagementException, URISyntaxException {
    
        ConnectionFactory factory = new ConnectionFactory();
        factory.setUri("amqp://admin:123456@117.50.40.96:5672/%2f");
    
        try(Connection conn = factory.newConnection();
            Channel channel = conn.createChannel()){
        
            // 声明一个交换器 ==》 s:交换器名称 / builtinExchangeType:交换器类型(枚举可选) /  b:是否持久化 / b1:是否自动删除 / map:属性参数集合
            channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.TOPIC, true, false, null);
            
            String speed;
            String color;
            String species;
            String routingKey;
            
            // 连续生产15次消息
            for (int i = 0; i < 20; i++) {
                speed = SPEED[RANDOM.nextInt(SPEED.length)];
                color = COLOR[RANDOM.nextInt(SPEED.length)];
                species = SPECIES[RANDOM.nextInt(SPEED.length)];
                routingKey = speed + "." + color +  "." + species;
                channel.basicPublish(EXCHANGE_NAME, routingKey, null,
                        (speed + "-" + color +  "-" + species).getBytes("utf-8"));
            }
        
        } catch (Exception e) {
            e.printStackTrace();
        }
    
    }
}
-Client1- 兔子消费者
public class RabbitConsumer {
    
    private static String EXCHANGE_NAME = "topic.exchange";
    
    public static void main(String[] args) throws NoSuchAlgorithmException, KeyManagementException, URISyntaxException {
    
        ConnectionFactory factory = new ConnectionFactory();
        factory.setUri("amqp://admin:123456@117.50.40.96:5672/%2f");
    
        try(Connection conn = factory.newConnection();
            Channel channel = conn.createChannel()){
    
            // 临时队列,返回值是服务器为该队列生成的名称
            String queue = channel.queueDeclare().getQueue();
            channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.TOPIC, true, false, null);
            channel.queueBind(queue, EXCHANGE_NAME, "*.*.rabbit"); // 匹配所有rabbit
        
            channel.basicConsume(queue,
                    (s, delivery) -> System.out.println("*.*.rabbit 匹配到的消息:" + new String(delivery.getBody(), "utf-8")),
                    s -> System.out.println("Cancel : " + s));
        
            while (true) {} // 循环监控效果
        
        } catch (Exception e) {
            e.printStackTrace();
        }
        
    }
}
-Client2- 橘色消费者
public class OrangeConsumer {
    
    private static String EXCHANGE_NAME = "topic.exchange";
    
    public static void main(String[] args) throws NoSuchAlgorithmException, KeyManagementException, URISyntaxException {
    
        ConnectionFactory factory = new ConnectionFactory();
        factory.setUri("amqp://admin:123456@117.50.40.96:5672/%2f");
    
        try(Connection conn = factory.newConnection();
            Channel channel = conn.createChannel()){
    
            // 临时队列,返回值是服务器为该队列生成的名称
            String queue = channel.queueDeclare().getQueue();
            channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.TOPIC, true, false, null);
            channel.queueBind(queue, EXCHANGE_NAME, "*.rabbit.*"); // 匹配所有orange
        
            channel.basicConsume(queue,
                    (s, delivery) -> System.out.println("*.orange.* 匹配到的消息:" + new String(delivery.getBody(), "utf-8")),
                    s -> System.out.println("Cancel : " + s));
        
            while (true) {} // 循环监控效果
        
        } catch (Exception e) {
            e.printStackTrace();
        }
        
    }
}
-Client3- 慢速消费者
public class SlowConsumer {
    
    private static String EXCHANGE_NAME = "topic.exchange";
    
    public static void main(String[] args) throws NoSuchAlgorithmException, KeyManagementException, URISyntaxException {
    
        ConnectionFactory factory = new ConnectionFactory();
        factory.setUri("amqp://admin:123456@117.50.40.96:5672/%2f");
    
        try(Connection conn = factory.newConnection();
            Channel channel = conn.createChannel()){
    
            // 临时队列,返回值是服务器为该队列生成的名称
            String queue = channel.queueDeclare().getQueue();
            channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.TOPIC, true, false, null);
            channel.queueBind(queue, EXCHANGE_NAME, "slow.#"); // 匹配所有slow
        
            channel.basicConsume(queue,
                    (s, delivery) -> System.out.println("slow.# 匹配到的消息:" + new String(delivery.getBody(), "utf-8")),
                    s -> System.out.println("Cancel : " + s));
        
            while (true) {} // 循环监控效果
        
        } catch (Exception e) {
            e.printStackTrace();
        }
        
    }
}

先运行三个消费者,然后执行生产者:

image.png

「Spring整合RabbitMQ」

spring-amqp是对AMQP概念的一些抽象,spring-rabbit是对RabbitMQ操作的封装实现

核心类:

  1. RabbitAdmin: 完成对Exchange,Queue,Binding的操作,在容器中管理了 RabbitAdmin 类的时候,可以对Exchange,Queue,Binding进行自动声明
  2. RabbitTemplate:发送和接收消息的工具类
  3. SimpleMessageListenerContainer:消费消息的容器

目前比较新的一些项目都会选择基于注解方式,而比较老的一些项目可能还是基于配置文件的

△ 基于配置文件的整合
-Pom- pom配置
<dependencies>
        <dependency>
            <groupId>org.springframework.amqp</groupId>
            <artifactId>spring-rabbit</artifactId>
            <version>2.2.7.RELEASE</version>
        </dependency>
</dependencies>
-Xml- xml配置
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:rabbit="http://www.springframework.org/schema/rabbit"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/rabbit
        http://www.springframework.org/schema/rabbit/spring-rabbit.xsd">

    <!--创建连接工厂-->
    <rabbit:connection-factory id="connectionFactory" host="117.50.40.96" virtual-host="/"
                               username="admin" password="123456" port="5672" />

    <!--用于自动向RabbitMQ声明队列、交换器、绑定等操作的工具类-->
    <rabbit:admin id="rabbitAdmin" connection-factory="connectionFactory" />
    <!--用于简化操作的模板类-->
    <rabbit:template id="rabbitTemplate" connection-factory="connectionFactory" />

    <!--声明一个消息队列-->
    <rabbit:queue id="Q1" name="spring.queue" durable="false" exclusive="false" auto-delete="false" />

    <!--声明Direct交换器-->
    <rabbit:direct-exchange id="directExchange" name="spring.exchange" durable="false" auto-delete="false" >
        <rabbit:bindings>
            <!-- key : 绑定键 -->
            <!-- queue : 将该交换器绑定到哪个队列(此处使用队列的id,而非名称) -->
            <rabbit:binding queue="Q1" key="spring.routing"/>
        </rabbit:bindings>
    </rabbit:direct-exchange>

</beans>
-Producer- 生产者
public class ProducerApp {
    
    private static final String DEFAULT_SPRING_EXCHANGE = "spring.exchange";
    private static final String DEFAULT_SPRING_ROUTING_KEY = "spring.routing";
    
    public static void main(String[] args) throws UnsupportedEncodingException {
        // 获得解析xml文件后的容器
        ClassPathXmlApplicationContext applicationContext =
                new ClassPathXmlApplicationContext("spring-rabbit.xml");
        // 获取RabbitTemplate类
        RabbitTemplate template = applicationContext.getBean(RabbitTemplate.class);
        // 准备好Message
        Message msg;
        // 创建builder并设置好初始编码和HTTP文件扩展名
        MessagePropertiesBuilder builder = MessagePropertiesBuilder.newInstance();
        builder.setContentEncoding("gbk");
        builder.setContentType(MessageProperties.CONTENT_TYPE_TEXT_PLAIN);
        // 推送1000条消息
        // withBody() 设置消息体内容流
        // andProperties() 注入来自builder的属性配置
        // build() 正式构建出message
        for (int i = 0; i < 1000; i++) {
            msg = MessageBuilder.withBody(("No." + i).getBytes("gbk"))
                    .andProperties(builder.build())
                    .build();
            /* 通过 template 发送消息到符合路由键的交换器 */
            template.send(DEFAULT_SPRING_EXCHANGE, DEFAULT_SPRING_ROUTING_KEY, msg);
        }
        // 关闭容器
        applicationContext.close();
    }
    
}

image.png

-Client- 消费者
public class ConsumerApp {
    
    private static final String DEFAULT_SPRING_QUEUE = "spring.queue";
    
    public static void main(String[] args) throws UnsupportedEncodingException {
        // 获得解析xml文件后的容器
        ClassPathXmlApplicationContext applicationContext =
                new ClassPathXmlApplicationContext("spring-rabbit.xml");
        // 获取RabbitTemplate类
        RabbitTemplate template = applicationContext.getBean(RabbitTemplate.class);
        // 准备接收消息用的message
        Message msg = template.receive(DEFAULT_SPRING_QUEUE);
        // 拉取消息并打印
        while (msg != null || !StringUtils.isEmpty(msg.getBody())) {
            msg = template.receive(DEFAULT_SPRING_QUEUE);
            System.out.println(new String(msg.getBody(), msg.getMessageProperties().getContentEncoding()));
        }
        // 关闭容器
        applicationContext.close();
    }
    
}

image.png

-Listener- 监听器
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:rabbit="http://www.springframework.org/schema/rabbit"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/rabbit
        http://www.springframework.org/schema/rabbit/spring-rabbit.xsd">

    <!--创建连接工厂-->
    <rabbit:connection-factory id="connectionFactory" host="117.50.40.96" virtual-host="/"
                               username="admin" password="wjl19951020" port="5672" />

    <!--用于自动向RabbitMQ声明队列、交换器、绑定等操作的工具类-->
    <rabbit:admin id="rabbitAdmin" connection-factory="connectionFactory" />
    <!--用于简化操作的模板类-->
    <rabbit:template id="rabbitTemplate" connection-factory="connectionFactory" />

    <!--声明一个消息队列-->
    <rabbit:queue id="Q1" name="spring.queue" durable="false" exclusive="false" auto-delete="false" />

    <!--声明监听器-->
    <bean id="messageListener" class="com.archie.MyMessageListener"/>
    <!-- 监听器绑定队列 -->
    <rabbit:listener-container connection-factory="connectionFactory">
        <rabbit:listener queues="Q1" ref="messageListener" />
    </rabbit:listener-container>

</beans>
@Component
public class MyMessageListener implements ChannelAwareMessageListener {
    
    @Override
    public void onMessage(Message msg, Channel channel) throws UnsupportedEncodingException {
        System.out.println(new String(msg.getBody(), msg.getMessageProperties().getContentEncoding()));
    }
    
}
public class ListenerApp {
    public static void main(String[] args) {
        // 直接启动spring注入bean即可,上一个类中的onMessage()方法会自动启动
        new ClassPathXmlApplicationContext("spring-rabbit.xml");
    }
}
△ 基于注解的整合
-Xml- pom配置
<dependencies>
        <dependency>
            <groupId>org.springframework.amqp</groupId>
            <artifactId>spring-rabbit</artifactId>
            <version>2.2.7.RELEASE</version>
        </dependency>
</dependencies>
-Config- 配置类
@Configuration
public class RabbitConfig {
    
    private static final String DEFAULT_SPRING_FANOUT_EXCHANGE = "spring.fanout.exchange";
    private static final String DEFAULT_SPRING_ROUTING_KEY = "spring.annotation.routing";
    private static final String DEFAULT_SPRING_QUEUE = "spring.annotation.queue";
    
    // 连接工厂
    @Bean
    public ConnectionFactory connectionFactory() {
        return new CachingConnectionFactory(URI.create("amqp://admin:123456@117.50.40.96:5672/%2f"));
    }
    
    // RabbitTemplate
    @Bean
    @Autowired
    public RabbitTemplate rabbitTemplate(ConnectionFactory factory) {
        return new RabbitTemplate(factory);
    }
    
    // RabbitAdmin
    @Bean
    @Autowired
    public RabbitAdmin rabbitAdmin(ConnectionFactory factory) {
        return new RabbitAdmin(factory);
    }
    
    // Queue
    @Bean
    public Queue queue() {
        return QueueBuilder.nonDurable(DEFAULT_SPRING_QUEUE).build();
    }
    
    // Exchange
    @Bean
    public Exchange exchange() {
        return new FanoutExchange(DEFAULT_SPRING_FANOUT_EXCHANGE, false, false, null);
    }
    
    // Binding
    @Bean
    @Autowired
    public Binding binding(Queue queue, Exchange exchange) {
        // 创建一个绑定,不指定绑定的参数
        return BindingBuilder.bind(queue).to(exchange).with(DEFAULT_SPRING_ROUTING_KEY).noargs();
    }
}
-Producer- 生产者
public class ProducerApplication {
    
    private static final String DEFAULT_SPRING_FANOUT_EXCHANGE = "spring.fanout.exchange";
    private static final String DEFAULT_SPRING_ROUTING_KEY = "spring.annotation.routing";
    
    public static void main(String[] args) throws UnsupportedEncodingException {
        // 获得解析注解后的容器
        AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext(RabbitConfig.class);
        // 获取RabbitTemplate类
        RabbitTemplate template = context.getBean(RabbitTemplate.class);
        // 通过builder创建消息属性集合类(用于配置消息的传输属性)
        MessageProperties messageProperties = MessagePropertiesBuilder
                .newInstance()
                .setContentType(MessageProperties.CONTENT_TYPE_TEXT_PLAIN)
                .setContentEncoding("gbk")
                .setHeader("myKey", "myValue")
                .build();
        // 准备好Message
        Message msg;
        for (int i = 0; i < 1000; i++) {
            msg = MessageBuilder.withBody(("No." + i).getBytes("gbk"))
                    .andProperties(messageProperties)
                    .build();
            /* 通过 template 发送消息到符合路由键的交换器 */
            template.send(DEFAULT_SPRING_FANOUT_EXCHANGE, DEFAULT_SPRING_ROUTING_KEY, msg);
    
            context.close();
        }
    }
}
-Client- 消费者
public class ConsumerApplication {
    
    private static final String DEFAULT_SPRING_QUEUE = "spring.annotation.queue";
    
    public static void main(String[] args) throws UnsupportedEncodingException {
        // 获得解析注解后的容器
        AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext(RabbitConfig.class);
        // 获取RabbitTemplate类
        RabbitTemplate template = context.getBean(RabbitTemplate.class);
        // 接收消息
        Message message = template.receive(DEFAULT_SPRING_QUEUE);
        // 拉取消息并连续打印
        while (message != null || !StringUtils.isEmpty(message.getBody())) {
            message = template.receive(DEFAULT_SPRING_QUEUE);
            System.out.println(new String(message.getBody(), message.getMessageProperties().getContentEncoding()));
        }
        // 关闭spring的上下文
        context.close();
    }

}

效果同配置文件模式类样,读者可自行实现

「SpingBoot整合RabbitMQ」

-Dependency- 依赖添加
<!-- 声明springboot版本 -->
<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>2.2.8.RELEASE</version>
    <relativePath/> <!-- lookup parent from repository -->
</parent>

<!-- 具体依赖(读者可根据需要增删) -->
<dependencies>
    <!-- springboot整合rabbitmq核心依赖 -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-amqp</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-test</artifactId>
        <scope>test</scope>
        <exclusions>
            <exclusion>
                <groupId>org.junit.vintage</groupId>
                <artifactId>junit-vintage-engine</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <groupId>org.springframework.amqp</groupId>
        <artifactId>spring-rabbit-test</artifactId>
        <scope>test</scope>
    </dependency>
</dependencies>
-Properties- 服务信息配置
spring.application.name=springboot_rabbitmq
spring.rabbitmq.host=117.50.40.96
spring.rabbitmq.virtual-host=/
spring.rabbitmq.username=admin
spring.rabbitmq.password=123456
spring.rabbitmq.port=5672
-App- 启动入口
@SpringBootApplication
public class SpringbootRabbitMQApplication {
    
    public static void main(String[] args) {
        SpringApplication.run(SpringbootRabbitMQApplication.class, args);
    }
    
}
-config- bean配置管理
@Configuration
public class RabbitConfig {
    
    private static final String MY_QUEUE = "springboot.queue"; // 使用的队列queue
    private static final String MY_EXCHANGE = "springboot.exchange"; // 使用的交换器exchange
    private static final String MY_BINDING = "springboot.binding"; // 使用的绑定(路由)键binding
    
    @Bean
    public Queue myQueue() {
        return new Queue(MY_QUEUE, false, false, false, null);
    }
    
    @Bean
    public Exchange myExchange() {
        return new TopicExchange(MY_EXCHANGE, false, false, null);
    }
    
    @Bean
    public Binding binding() {
        return new Binding(MY_QUEUE, Binding.DestinationType.QUEUE, MY_EXCHANGE, MY_BINDING, null);
    }
    
}
-controller- 表现层实现
@Controller
public class DemoController {
    
    private static final String MY_EXCHANGE = "springboot.exchange";
    private static final String MY_ROUTING_KEY = "springboot.routing";
    
    @Autowired
    private AmqpTemplate rabbitTemplate;
    
    @RequestMapping("/rabbit/{clientMessage}")
    public ResponseEntity<String> sendMessage(@PathVariable String clientMessage) throws UnsupportedEncodingException {
        // 配置好消息属性集合-->MessageProperties
        MessageProperties messageProperties = MessagePropertiesBuilder.newInstance().
                setContentType(MessageProperties.CONTENT_TYPE_TEXT_PLAIN)
                .setContentEncoding("utf-8")
                .setHeader("headerKey", "headerValue")
                .build();
        // 结合客户端传来的消息并根据自主配置的消息属性创建Rabbit传输用的message
        Message message = MessageBuilder.withBody(clientMessage.getBytes("utf-8"))
                .andProperties(messageProperties)
                .build();
        // 发送消息到 AMQP Broker 中
        rabbitTemplate.send(MY_EXCHANGE, MY_ROUTING_KEY, message);
        // 使用spring web模块提供的http组件返回消息给客户
        return new ResponseEntity(message, HttpStatus.OK);
    }
    
}