Java-第十九部分-消息队列-RocketMQ消息

99永远差一分
233 阅读4分钟

消息队列全文

消息属性

生产

  • 消息生产组，对于普通消息意义不大；用于配合事务的回查机制
DefaultMQProducer producer = new DefaultMQProducer("group_test");
  • 设置每一个主题在每一个Broker队列数量，默认4个，创建时有效
producer.setDefaultTopicQueueNums(8);
  • 发送消息的默认超时时间，默认3s
producer.setSendMsgTimeout(1000 * 4);
  • 消息超过阈值，启动压缩，默认4k
producer.setCompressMsgBodyOverHowmuch(1024 * 4);
  • 同步发送失败的重试次数，默认2次
producer.setRetryTimesWhenSendFailed(2);
  • 异步发送失败的重试次数，默认2次
producer.setRetryTimesWhenSendAsyncFailed(2);
  • 消息重试时选择另一个Broker发送，默认为false
producer.setRetryAnotherBrokerWhenNotStoreOK(true);
  • 允许发送的最大信息长度，默认4M
producer.setMaxMessageSize(1024 * 1024 * 4);
  • 设置nameserver的地址
producer.setNamesrvAddr("127.0.0.1:9876");

消费

  • 指定消费者组，集群消费组，每一个组均摊消费同一个主题的消息
DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("sqlFilter");
  • 消费模式，默认就是集群
consumer.setMessageModel(MessageModel.CLUSTERING);
  • 指定消费开始的偏移量，上次消费偏移量/最大偏移量/最小偏移量/启动时间戳
consumer.setConsumeFromWhere(ConsumeFromWhere.CONSUME_FROM_LAST_OFFSET);
  • 消费者最小偏移数量，默认20
consumer.setConsumeThreadMin(20);
  • 消费者最大偏移数量，默认20
consumer.setConsumeThreadMax(20);
  • 推模式的任务间隔，推模式是一种不断轮询调取的封装
consumer.setPullInterval(0);
  • 推模式下任务拉取一批的条数，默认32条
consumer.setPullBatchSize(32);
  • 消息重试次数，-1代表16，超过代表死信消息
consumer.setMaxReconsumeTimes(-1);
  • 消息消费超时时间，阻塞当前线程的最大时间，分钟为单位
consumer.setConsumeTimeout(15);
  • 消费者的主题下有多少消息队列
Set<MessageQueue> queues = consumer.fetchSubscribeMessageQueues("Topic");

普通消息

  • maven依赖
<dependency>
    <groupId>org.apache.rocketmq</groupId>
    <artifactId>rocketmq-client</artifactId>
    <version>4.8.0</version>
</dependency>

消息发送

同步消息发送

DefaultMQProducer producer = new DefaultMQProducer("group_test");
// 设置nameserver的地址
producer.setNamesrvAddr("127.0.0.1:9876");

//启动producer
producer.start();
for (int i = 0; i < 10; i++) {
    Message msg = new Message("TopicTest",
            "TagA",
            ("hello rocketMQ " + i).getBytes(RemotingHelper.DEFAULT_CHARSET));
    SendResult result = producer.send(msg);
    System.out.println(result);
}
// 即时关闭
producer.shutdown();

异步发送

DefaultMQProducer producer = new DefaultMQProducer("group_test");
// 设置nameserver的地址
producer.setNamesrvAddr("127.0.0.1:9876");
//启动producer
producer.start();
for (int i = 0; i < 10; i++) {
    Message msg = new Message("TopicTest",
            "TagB",
            ("hello rocketMQ " + i).getBytes(RemotingHelper.DEFAULT_CHARSET));
    // 传入异步回调接口
    producer.send(msg, new SendCallback() {
        // 发送成功
        @Override
        public void onSuccess(SendResult sendResult) {
            System.out.println(sendResult);
        }
        // 发送失败
        @Override
        public void onException(Throwable throwable) {
            throwable.printStackTrace();
        }
    });
}
Thread.sleep(10000);
// 即时关闭
producer.shutdown();

单向发送

DefaultMQProducer producer = new DefaultMQProducer("group_test");
// 设置nameserver的地址
producer.setNamesrvAddr("127.0.0.1:9876");
//启动producer
producer.start();
for (int i = 0; i < 10; i++) {
    Message msg = new Message("TopicTest",
            "TagC",
            ("hello rocketMQ " + i).getBytes(RemotingHelper.DEFAULT_CHARSET));
    // 得不到响应结果
    producer.sendOneway(msg);
}
// 即时关闭
producer.shutdown();

消息消费

  • 集群消费，一个消费组，均摊消费一个topic的消息
  1. 消息不回重复
  2. 消费进度保存在broker
  • 广播消费，一个消费组中每一个消费者，会将所有消息都消费一遍
  1. 消息重复，不支持顺序消息，重置消费点
  2. 消费进度，保存在消费者
  3. 只会处理消费者启动时获取的最新消息

集群消费

DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("group_consumer");
// 指定注册中心
consumer.setNamesrvAddr("127.0.0.1:9876");
// 订阅Topic 指定tag
consumer.subscribe("TopicTest", "TagC");
// 集群模式消费，默认就是集群
consumer.setMessageModel(MessageModel.CLUSTERING);
// 注册回调处理函数
consumer.registerMessageListener(new MessageListenerConcurrently() {
    @Override
    public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> list, ConsumeConcurrentlyContext consumeConcurrentlyContext) {
        try {
        for (MessageExt msg : list) {
            String topic = msg.getTopic();
            String msgBody = new String(msg.getBody(), "utf-8");
            String tags = msg.getTags();
            System.out.println("topic: " + topic + " body: " + msgBody + " tags: " + tags);
        }
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
            return ConsumeConcurrentlyStatus.RECONSUME_LATER;
        }
        return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
    }
});
consumer.start();
// 关闭
//consumer.shutdown();

广播消费

DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("group_consumer");
// 指定注册中心
consumer.setNamesrvAddr("127.0.0.1:9876");
// 订阅Topic 指定tag
consumer.subscribe("TopicTest", "TagB");
// 区别就是消费模式
consumer.setMessageModel(MessageModel.BROADCASTING);
// 注册回调处理函数
consumer.registerMessageListener(new MessageListenerConcurrently() {
    @Override
    public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> list, ConsumeConcurrentlyContext consumeConcurrentlyContext) {
        try {
            for (MessageExt msg : list) {
                String topic = msg.getTopic();
                String msgBody = new String(msg.getBody(), "utf-8");
                String tags = msg.getTags();
                System.out.println("topic: " + topic + " body: " + msgBody + " tags: " + tags);
            }
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
            return ConsumeConcurrentlyStatus.RECONSUME_LATER;
        }
        return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
    }
});
consumer.start();
// 关闭
//consumer.shutdown();

顺序消息

生产

全局顺序消息

  • 先进先出，发送消息的时候，指定队列
SendResult result = producer.send(msg, new MessageQueueSelector() {
    @Override
    public MessageQueue select(List<MessageQueue> list, Message message, Object o) {
        return list.get((Integer) o);
    }
}, 1);

部分顺序消息

  • 不同的msg放入不同的queue，发送时，实现MessageQueueSelector接口
DefaultMQProducer producer = new DefaultMQProducer("group_test");
// 设置nameserver的地址
producer.setNamesrvAddr("172.20.10.12:9876");
//启动producer
producer.start();
List<Integer> orders = new ArrayList<>();
for (int i = 0; i < 10; i++) {
    orders.add(i);
}
for (int i = 0; i < orders.size(); i++) {
    Message msg = new Message("OrderTest",
            "TagA",
            ("order " + i).getBytes(RemotingHelper.DEFAULT_CHARSET));
    SendResult result = producer.send(msg, new MessageQueueSelector() {
        @Override
        public MessageQueue select(List<MessageQueue> list, Message message, Object o) {
            return list.get((Integer) o % list.size());
        }
    }, i);
    System.out.println(result.getMessageQueue().getQueueId() + " - " + i);
}
// 即时关闭
producer.shutdown();

不同Queue的消费

  • 顺序消费要实现MessageListenerOrderly接口，并且异常返回等一会在处理，否则会影响顺序
DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("group_consumer");
// 指定注册中心
consumer.setNamesrvAddr("127.0.0.1:9876");
// 消费的起始偏移
consumer.setConsumeFromWhere(ConsumeFromWhere.CONSUME_FROM_LAST_OFFSET);
// 订阅Topic 指定tag
consumer.subscribe("OrderTest", "*");
// 注册回调处理函数
consumer.registerMessageListener(new MessageListenerOrderly() {
    Random r = new Random();
    @Override
    public ConsumeOrderlyStatus consumeMessage(List<MessageExt> list, ConsumeOrderlyContext consumeOrderlyContext) {
        consumeOrderlyContext.setAutoCommit(true);
        for (MessageExt msg : list) {
            System.out.println("consumerThread " + Thread.currentThread().getName() +
                                " queueId " + msg.getQueueId() + " content " + new String(msg.getBody()));
        }
        try {
            TimeUnit.MILLISECONDS.sleep(r.nextInt(300));
        } catch (InterruptedException e) {
            e.printStackTrace();
            // 如果顺序消息的处理，不能返回重试，要等一会，在进行处理
            // 重试会影响顺序
            return ConsumeOrderlyStatus.SUSPEND_CURRENT_QUEUE_A_MOMENT;
        }
        return ConsumeOrderlyStatus.SUCCESS;
    }
});
consumer.start();

延时消息

  • 延迟一段时间再发给消费者
  • 限时订单，没有及时支付的订单，释放掉

生产

DefaultMQProducer producer = new DefaultMQProducer("group_test");
// 设置nameserver的地址
producer.setNamesrvAddr("127.0.0.1:9876");
//启动producer
producer.start();
for (int i = 0; i < 10; i++) {
    Message msg = new Message("ScheduledTopic", ("Hello scheduled message " + i).getBytes(StandardCharsets.UTF_8));
    // 18个等级 1s 5s 10s 30s 1m 2m 3m 4m 5m 6m 7m 8m 9m 10m 20m 30m 1h 2h
    msg.setDelayTimeLevel(2);
    producer.send(msg);
}
producer.shutdown();

消费

DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("group_consumer");
// 指定注册中心
consumer.setNamesrvAddr("127.0.0.1:9876");
// 订阅Topic 指定tag
consumer.subscribe("ScheduledTopic", "*");
consumer.registerMessageListener(new MessageListenerConcurrently() {
    @Override
    public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> list, ConsumeConcurrentlyContext consumeConcurrentlyContext) {
        for (MessageExt msg : list) {
            System.out.println("msgId: " + msg.getMsgId() + " time: " +
                    (msg.getStoreTimestamp() - msg.getBornTimestamp()));
        }
        return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
    }
});
consumer.start();

批量消息

  • 一批消息大小不能超过4m，大的批量消息需要进行拆分

普通生产

DefaultMQProducer producer = new DefaultMQProducer("group_test");
// 设置nameserver的地址
producer.setNamesrvAddr("127.0.0.1:9876");
//启动producer
producer.start();
List<Message> msgs = new ArrayList<>();
for (int i = 0; i < 10; i++) {
    msgs.add(new Message("BatchTopic", "TagD", i + "", ("Hello " + i).getBytes(StandardCharsets.UTF_8)));
}
producer.send(msgs);
producer.shutdown();

消费

DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("group_consumer");
// 指定注册中心
consumer.setNamesrvAddr("127.0.0.1:9876");
// 订阅Topic 指定tag
consumer.subscribe("BatchTopic", "*");
consumer.registerMessageListener(new MessageListenerConcurrently() {
    @Override
    public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> list, ConsumeConcurrentlyContext consumeConcurrentlyContext) {
        for (MessageExt msg : list) {
            System.out.println("msgId: " + msg.getMsgId() + " content: " +
                    new String(msg.getBody()));
        }
        return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
    }
});
consumer.start();

过滤消息

  • Tag过滤，支持正则表达式

消费者订阅时，指定tag即可

  • SQL过滤

broker要配置enablePropertyFilter=true

生产

DefaultMQProducer producer = new DefaultMQProducer("group_test");
// 设置nameserver的地址
producer.setNamesrvAddr("127.0.0.1:9876");
//启动producer
producer.start();
String[] tags = new String[]{"TagA", "TagB", "TagC"};
for (int i = 0; i < 10; i++) {
    Message msg = new Message("SQLFilterTopic", tags[i % 3],
            ("Hello " + i).getBytes(StandardCharsets.UTF_8));
    msg.putUserProperty("a", String.valueOf(i));
    SendResult res = producer.send(msg);
    System.out.println(res);
}
producer.shutdown();

消费

DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("sqlFilter");
// 指定注册中心
consumer.setNamesrvAddr("127.0.0.1:9876");
consumer.setConsumeFromWhere(ConsumeFromWhere.CONSUME_FROM_LAST_OFFSET);
// 订阅Topic 指定tag
consumer.subscribe("SQLFilterTopic",
        MessageSelector.bySql("TAGS is not null and TAGS in ('TagA', 'TagB') and" +
                "(a is not null and a between 0 and 3)"));
consumer.registerMessageListener(new MessageListenerConcurrently() {
    @Override
    public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> list, ConsumeConcurrentlyContext consumeConcurrentlyContext) {
        for (MessageExt msg : list) {
            String topic = msg.getTopic();
            String body = new String(msg.getBody());
            String pro = msg.getProperty("a");
            String tags = msg.getTags();
            System.out.println("topic: " + topic +
                            "\nbody: " + body +
                            "\npro: " + pro +
                            "\ntags: " + tags);
        }
        return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
    }
});
consumer.start();
avatar
文章
阅读
粉丝