我正在参加「掘金·启航计划」
1.何为延迟队列
顾名思义,延迟队列就是进入该队列的消息会被延迟消费的队列。而一般的队列,消息一旦入队了之后就会被消费者马上消费。 延迟队列多用于需要延迟工作的场景。最常见的是以下两种场景:
1. 延迟消费
用户生成订单之后,需要过一段时间校验订单的支付状态,如果订单仍未支付则需要及时地关闭订单。用户注册成功之后,需要过一段时间比如一周后校验用户的使用情况,如果发现用户活跃度较低,则发送邮件或者短信来提醒用户使用。
1.2 延迟重试
比如消费者从队列里消费消息时失败了,但是想要延迟一段时间后自动重试。 如果不使用延迟队列,那么我们只能通过一个轮询扫描程序去完成。这种方案既不优雅,也不方便做成统一的服务便于开发人员使用。但是使用延迟队列的话,我们就可以轻而易举地完成。
2 RocketMq 对延迟队列的支持
其实rocketMq是支持延迟消息队列的。我们后端一般会使用到Springboot 来集成RocketMq 来处理数据。其中官方会有RocketMQTemplate , 其中的sysend 的方法有一个入参 dealyLevel 是代表延时级别的。
这里我们先不对他进行讨论, 他这个是基于生产者发送方来延时的。 现在我们要做的是类似于
@RocketMQMessageListener, 做为消费方, 接收方来做的。
3 设计思想和架构
1, 要实现自定义注解,里面包含rocketMq 的地址, 分组信息, topic 信息等。 这些信息有的默认使用的是配置文件的信息。
2, 在使用SpringBean 的后置处理器,对 自定义注解 进行拦截 封装成消息发送生产者对象。 在监听器里面使用哪个thread.sleep() 来获取注解里面的 延迟时长。在调用methon.invoke 的方法。直接返回结果。
3, 消息发送对象,要实现 InitializingBean , DisposableBean 来重新启动和销毁的方法。
4, 在使用springboot 的spi 把自定义的后置处理器的配置类 写入spring.factories 里面。
4,具体的代码
4.1 自定义注解
package com.ducheng.rocket.annotation;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface DelayRocketMQListener {
/**
* RocketMQ topic
* @return
*/
String topic();
/**
* Tag
* @return
*/
String tag() default "*";
/**
* 延迟的时长
* @return
*/
int delayTime() default 0;
/**
* nameServer, 支持两种模式,模式是从配置文件获取。
* @return
*/
String nameServer() default "${rocketmq.name-server}";
/**
* 消费者组信息, 支持两种模式, m默认是从配置文件获取。
* @return
*/
String consumerGroup() default "${rocketmq.producer.group}" ;
}
复制代码
4.2 构建消费者对象
package com.ducheng.rocket.delay;
import com.ducheng.rocket.annotation.DelayRocketMQListener;
import com.fasterxml.jackson.databind.ObjectMapper;
import lombok.extern.slf4j.Slf4j;
import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer;
import org.apache.rocketmq.client.consumer.listener.*;
import org.apache.rocketmq.common.message.MessageExt;
import org.apache.rocketmq.spring.core.RocketMQListener;
import org.springframework.beans.factory.DisposableBean;
import org.springframework.beans.factory.InitializingBean;
import org.springframework.core.env.Environment;
import org.springframework.util.StringUtils;
import java.lang.reflect.Method;
import java.nio.charset.Charset;
import java.util.List;
import java.util.Objects;
@Slf4j
public class DelayConsumerContainer implements InitializingBean , DisposableBean {
private final DelayRocketMQListener delayBasedRocketMQ;
private final Environment environment;
private Method method;
private Object bean;
private static final String charset = "UTF-8";
private final Class aClass;
public DelayConsumerContainer(DelayRocketMQListener delayBasedRocketMQ, Environment environment, Method method, Object bean, Class aClass) {
this.delayBasedRocketMQ = delayBasedRocketMQ;
this.environment = environment;
this.method = method;
this.bean = bean;
this.aClass = aClass;
}
protected DefaultMQPushConsumer createConsumer() throws Exception {
// 构建 DefaultMQPushConsumer
DefaultMQPushConsumer consumer = new DefaultMQPushConsumer();
String consumerGroup = resolve(this.delayBasedRocketMQ.consumerGroup());
String nameServerAddress = resolve(this.delayBasedRocketMQ.nameServer());
consumer.setConsumerGroup(consumerGroup);
consumer.setNamesrvAddr(nameServerAddress);
// 订阅 topic
//String topic = resolve(this.delayBasedRocketMQ.topic());
String topic = this.delayBasedRocketMQ.topic();
//String tag = resolve(this.delayBasedRocketMQ.tag());
String tag = this.delayBasedRocketMQ.tag();
consumer.subscribe(topic, tag);
// 增加监听器
consumer.setMessageListener(new DefaultMessageListener(delayBasedRocketMQ.delayTime()));
//consumer.setMessageModel();
log.info("success to subscribe {}, topic {}, tag {}, group {}", nameServerAddress, topic, tag, consumerGroup);
return consumer;
}
@Override
public void afterPropertiesSet() throws Exception {
DefaultMQPushConsumer consumer = createConsumer();
consumer.start();
}
@Override
public void destroy() throws Exception {
createConsumer().shutdown();
}
private class DefaultMessageListener implements MessageListenerConcurrently {
private Integer delayTime;
public Integer getDelayTime() {
return delayTime;
}
public void setDelayTime(Integer delayTime) {
this.delayTime = delayTime;
}
public DefaultMessageListener(Integer delayTime) {
this.delayTime = delayTime;
}
@SuppressWarnings("unchecked")
@Override
public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext context) {
for (MessageExt messageExt : msgs) {
log.debug("received msg: {}", messageExt);
try {
long now = System.currentTimeMillis();
Thread.sleep(delayTime);
Object doConvertMessage = doConvertMessage(messageExt);
method.invoke(bean,doConvertMessage);
long costTime = System.currentTimeMillis() - now;
log.debug("consume {} cost: {} ms", messageExt.getMsgId(), costTime);
} catch (Exception e) {
log.warn("consume message failed. messageExt:{}", messageExt, e);
//context.setDelayLevelWhenNextConsume(delayLevelWhenNextConsume);
return ConsumeConcurrentlyStatus.RECONSUME_LATER;
}
}
return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
}
}
@SuppressWarnings("unchecked")
private Object doConvertMessage(MessageExt messageExt) {
if (Objects.equals(this.aClass, MessageExt.class)) {
return messageExt;
} else {
String str = new String(messageExt.getBody(), Charset.forName(charset));
if (Objects.equals(this.aClass, String.class)) {
return str;
} else {
// If msgType not string, use objectMapper change it.
try {
ObjectMapper objectMapper = new ObjectMapper();
return objectMapper.readValue(str, this.aClass);
} catch (Exception e) {
log.info("convert failed. str:{}, msgType:{}", str, this.aClass);
throw new RuntimeException("cannot convert message to " + this.aClass, e);
}
}
}
}
protected String resolve(String value) {
if (StringUtils.hasText(value)) {
return this.environment.resolvePlaceholders(value);
}
return value;
}
}
复制代码
4.3 自定义注册器
package com.ducheng.rocket.delay;
import com.ducheng.rocket.annotation.DelayRocketMQListener;
import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.reflect.MethodUtils;
import org.apache.rocketmq.spring.core.RocketMQListener;
import org.springframework.aop.support.AopUtils;
import org.springframework.beans.BeansException;
import org.springframework.beans.factory.config.BeanPostProcessor;
import org.springframework.core.annotation.AnnotatedElementUtils;
import org.springframework.core.env.Environment;
import java.lang.reflect.Method;
import java.util.List;
@Slf4j
public class DelayConsumerContainerRegistry implements BeanPostProcessor {
private final Environment environment;
public DelayConsumerContainerRegistry(Environment environment) {
this.environment = environment;
}
@SneakyThrows
@Override
public Object postProcessAfterInitialization(Object proxy, String beanName) throws BeansException {
// 1. 获取 @DelayRocketMQListener 注解方法
Class targetCls = AopUtils.getTargetClass(proxy);
List<Method> methodsListWithAnnotation = MethodUtils.getMethodsListWithAnnotation(targetCls, DelayRocketMQListener.class);
// 2. 为每个 @DelayRocketMQListener 注解方法 注册 DelayConsumerContainer
for(Method method : methodsListWithAnnotation){
DelayRocketMQListener annotation = AnnotatedElementUtils.findMergedAnnotation(method, DelayRocketMQListener.class);
//获取真实的代理对象
//Object bean = AopProxyUtils.getSingletonTarget(proxy);
Class<?> parameterType = method.getParameterTypes()[0];
DelayConsumerContainer delayConsumerContainer =
new DelayConsumerContainer(
annotation,
environment,
method,
proxy,
parameterType);
delayConsumerContainer.afterPropertiesSet();
}
return proxy;
}
}
复制代码
4.4 自定义配置类
@Configuration
// 这里一定要记住,一定要在RocketMq 自动装配完成之后
@AutoConfigureAfter(RocketMQAutoConfiguration.class)
public class DelayBasedRocketMQAutoConfiguration {
@Autowired
private Environment environment;
@Bean
public DelayConsumerContainerRegistry delayConsumerContainerRegistry(){
return new DelayConsumerContainerRegistry(environment);
}
}
复制代码
4.5 使用spi
org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
com.ducheng.rocket.config.DelayBasedRocketMQAutoConfiguration
复制代码
5. 编写测试代码
5.1 生产者
@Autowired
private RocketMQTemplate rocketMQTemplate;
@GetMapping("/index")
public String index() {
rocketMQTemplate.convertAndSend("test","测试");
log.info("开始发送信息:{}",new Date().toString());
return "ok";
}
复制代码
5.2 消费者
@Component
@Slf4j
public class TestDemo {
@DelayRocketMQListener(topic = "test",delayTime = 5000)
//@RocketMQMessageListener()
public String message(String message) {
log.info("返回的参数:{}",message);
return message;
}
}
复制代码
6.测试结果
37 秒-32 秒 五秒钟, 完美。