延时消息的简单实现(Redis过期回调+定时任务)

WinchellGao
1,464 阅读3分钟

Redis实现延时消息存在的问题

redis消息一旦发送到客户端，不管客户端又没有接收到，redis都会将它丢弃，造成redis实现这种场景的可靠性低的原因。
解决思路:缓存到redis同时入库，开启一个定时任务去处理那些过期而未完成的消息。这样机能保证大部分消息都能按照我们预期的那样去处理，也能保证那些小部分消息能够在我们容忍的时间内通过定时任务去执行。

实现思路

  1. 采用redis过期key回调+定时任务
  2. redis配置notify-keyspace-events Ex 打开，创建一个Listener类继KeyExpirationEventMessageListener类，监听过期key。
  3. 定义一个公共接口，不同类别的消息实现该接口，Listener采用策略模式，根据key值获取不同类去执行,执行完成更新库中该消息状态。
  4. Key值的组成=前缀+业务类型+事件id(雪花算法生成)+具体业务id(userid/orderNo) value值无实际意义可以任意值
  5. 定义一个定时任务，定时拉取表中数据，获取已经过期且未执行的消息，执行该消息。
  6. 发送消息到redis时徐需要入库.

实现逻辑

  1. redis配置如下
@Configuration
public class RedisConfig {


    /**
     * redis 乱码问题
     *
     * @param redisTemplate
     * @return
     */
    @Bean
    public RedisTemplate initRedisTemple(RedisTemplate redisTemplate) {
        redisTemplate.setKeySerializer(new StringRedisSerializer());
        redisTemplate.setValueSerializer(new GenericJackson2JsonRedisSerializer());
        redisTemplate.setHashKeySerializer(new StringRedisSerializer());
        redisTemplate.setHashValueSerializer(new GenericJackson2JsonRedisSerializer());
        return redisTemplate;
    }

    /**
     * redis 消息订阅(监听)者容器
     * <p>
     * {过期监听 1. redis配置打开 notify-keyspace-events Ex 2. 监听的类继承KeyExpirationEventMessageListener}
     *
     * @param redisConnectionFactory
     * @return
     */
    @Bean
    public RedisMessageListenerContainer redisMessageListenerContainer(RedisConnectionFactory redisConnectionFactory) {
        RedisMessageListenerContainer messageListenerContainer = new RedisMessageListenerContainer();
        messageListenerContainer.setConnectionFactory(redisConnectionFactory);
        return messageListenerContainer;
    }
}
  1. RedisExpiredKeyListener为统一处理逻辑，具体某个类型的key失效回调的处理需要实现IRedisExpiredKeyEventHandler接口，这里采用了类似于策略模式，就不需要写if去判断key。
@Component
public class RedisExpiredKeyListener extends KeyExpirationEventMessageListener {

    private static final Logger log = LoggerFactory.getLogger(RedisExpiredKeyListener.class);


    @Autowired
    private UserService userService;

    public RedisExpiredKeyListener(RedisMessageListenerContainer redisMessageListenerContainer) {
        super(redisMessageListenerContainer);
    }

    @Override
    public void onMessage(Message message, byte[] pattern) {
        String expiredKey = new String(message.getBody());
        log.info("redisExKeyOnMessage|redis过期key监听start|{}", expiredKey);
        String[] keys = expiredKey.split(":");
        String key = keys[0] + ":" + keys[1] + ":";
        String eventId = keys[2];
        String value = keys[3];
        Map<String, IRedisExpiredKeyEventHandler> handlers = ApplicationContextHelper.getBeansOfType(IRedisExpiredKeyEventHandler.class);
        handlers.values().forEach(hander -> {
            if (hander.type().equals(key)) {
                hander.onMessage(value);
                RedisExpiredEvent redisExpiredEvent = new RedisExpiredEvent();
                redisExpiredEvent.setEventId(eventId);
                redisExpiredEvent.setState(1);
                userService.updateRedisExpiredEvent(redisExpiredEvent);
            }
        });
        log.info("redisExKeyOnMessage|redis过期key监听end|{}", expiredKey);

    }
}
  1. IRedisExpiredKeyEventHandler
public interface IRedisExpiredKeyEventHandler {

    String type();

    void onMessage(String value);
}

4.处理具体消息的业务逻辑

@Component
public class UserIdKeyExpiredEventHandler implements IRedisExpiredKeyEventHandler {

    @Override
    public String type() {
        return Constant.EX_USER_ID;
    }

    @Override
    public void onMessage(String value) {
        System.out.println("execute -> UserIdKeyExListener-> " + Constant.EX_USER_ID + " " + value);
    }
}

@Component
public class OrderNoKeyExpiredEventHandler implements IRedisExpiredKeyEventHandler {


    @Override
    public String type() {
        return Constant.EX_ORDER_ID;
    }

    @Override
    public void onMessage(String value) {
        System.out.println("execute -> OrderNoKeyExListener-> " + Constant.EX_ORDER_ID + " " + value);
    }
}

4.1 实体类 unionKey值的组成=前缀+业务类型+事件id(雪花算法生成)+具体业务id(userid/orderNo)

@Data
public class RedisExpiredEvent {

    private Long id;

    private String eventId;

    private String eventKey;

    private String eventValue;

    private Integer state;

    private Date expiredTime;

    private Date createTime;

    private Date updateTime;

    private String createId;

    private String updateId;

    public String getUnionKey() {
        return eventKey + eventId + ":" + eventValue;
    }

}

4.2 定义key值(IRedisExpiredKeyEventHandler)的类型。

public interface Constant {

    String EX_KEY_CALLBACK_PREFIX = "EX_KEY_CALLBACK:";

    String EX_USER_ID = EX_KEY_CALLBACK_PREFIX + "EX_USER_ID:";
    String EX_ORDER_ID = EX_KEY_CALLBACK_PREFIX + "EX_ORDER_ID:";

}

5.测试接口，入库(消息持久化，防止因为客户端原因造成的消息丢失)与缓存到redis

  @RequestMapping(method = RequestMethod.GET, path = "/redis-expiration")
    public Resp redisKeyExpiration() {
        SnowflakeIdWorker snowflakeIdWorker = new SnowflakeIdWorker(10, 1, 1);
        RedisExpiredEvent event = new RedisExpiredEvent();
        event.setEventKey(Constant.EX_USER_ID);
        event.setEventId(snowflakeIdWorker.nextId()+"");
        event.setEventValue("10086");
        event.setState(0);
        event.setExpiredTime(addDate(Calendar.MINUTE,3));
        userService.insertRedisExpiredEvent(event);
//        redisTemplate.opsForValue().set(event.getUnionKey(), "", 3, TimeUnit.MINUTES);


        RedisExpiredEvent orderIdEvent = new RedisExpiredEvent();
        orderIdEvent.setEventKey(Constant.EX_ORDER_ID);
        orderIdEvent.setEventValue("202206221008611");
        orderIdEvent.setState(0);
        orderIdEvent.setEventId(snowflakeIdWorker.nextId()+"");
        orderIdEvent.setExpiredTime(addDate(Calendar.MINUTE, 3));
        userService.insertRedisExpiredEvent(orderIdEvent);
        redisTemplate.opsForValue().set(orderIdEvent.getUnionKey(), "", 3, TimeUnit.MINUTES);

        return Resp.ok();
    }

    private Date addDate(int field, int amount) {
        Calendar calendar = Calendar.getInstance();
        calendar.setTime(new Date());
        calendar.add(field, amount);
        return calendar.getTime();
    }

6.定时任务5分钟跑一次，执行已过期而未执行的消息。

@Component
public class RedisExpiredKeyJob {

    private static final Logger log = LoggerFactory.getLogger(RedisExpiredKeyJob.class);

    @Autowired
    private UserService userService;

    @Scheduled(cron = "0 0/5 * * * ?")
    public void execute(){
        List<RedisExpiredEvent> list = userService.getExpiredEvent();
        if (list == null || list.isEmpty()){
            return;
        }
        log.info("RedisExpiredKeyJob|redisKey过期事件定时任务start|{}",list);
        Map<String, IRedisExpiredKeyEventHandler> events = ApplicationContextHelper.getBeansOfType(IRedisExpiredKeyEventHandler.class);
        for (RedisExpiredEvent event : list) {
            events.values().forEach(handler ->{
                if (event.getEventKey().equals(handler.type())){
                    handler.onMessage(event.getEventValue());
                    event.setState(1);
                    userService.updateRedisExpiredEvent(event);
                }
            });
        }
        log.info("RedisExpiredKeyJob|redisKey过期事件定时任务end|{}",list);
    }
}

7.查询过期消息时会获取过期30分钟以内的，这里 -1min种是防止redis过期回调与定时任务同时执行。

@Select("SELECT * FROM redis_expired_event WHERE expired_time <=  DATE_SUB(CURRENT_TIME,INTERVAL 1 MINUTE) AND expired_time >=  DATE_SUB(CURRENT_TIME,INTERVAL 31 MINUTE) AND state <= 0")
List<RedisExpiredEvent> queryRedisExpireEvents();

@Insert("INSERT INTO redis_expired_event(event_id,event_key,event_value,state,expired_time,create_time,update_time) " +
        "VALUES (#{eventId},#{eventKey},#{eventValue},#{state},#{expiredTime},CURRENT_TIMESTAMP,CURRENT_TIMESTAMP)")
Integer insertRedisExiredEvent(RedisExpiredEvent redisExpiredEvent);

@Insert("UPDATE  redis_expired_event SET state = #{state},update_time = CURRENT_TIMESTAMP WHERE event_id = #{eventId}")
Integer updateRedisExiredEvent(RedisExpiredEvent redisExpiredEvent);

结尾

以上方式基本能满足延时消息的发送，但需要注意的问题是如果同时一时间有大量的key过期会给redis与程序造成很大的压力，还有就是消息重试与异常处理这里并没有重试。

avatar
文章
阅读
粉丝