GenericJackson2JsonRedisSerializer序列化

0 bug

RedisTemplate中 value 的序列化，比较常用的是JdkSerializationRedisSerializer和GenericJackson2JsonRedisSerializer。

前者需要对象实现java.io.Serializable接口，并且序列化后结果较大，不易阅读，所以更推荐采用GenericJackson2JsonRedisSerializer方式。

然后，就报错了：

Caused by: com.fasterxml.jackson.databind.exc.InvalidTypeIdException: Missing type id when trying to resolve subtype of [simple type, class java.lang.Object]: missing type id property '@class'
 at ...

代码如下：

@RestController
@RequestMapping("/json")
public class ObjectNodeController {
    @Autowired
    @Qualifier("reactiveRedisTemplate")
    private ReactiveRedisTemplate reactiveRedisTemplate;

    @GetMapping("/getjson")
    public Mono<ObjectNode> findByName(@RequestParam("id") String id) {
        ReactiveValueOperations<String, ObjectNode> operations = reactiveRedisTemplate.opsForValue();
        return operations.get(id);
    }

    @PostMapping("/storejson")
    public Mono<Boolean> storeUser(@RequestBody ObjectNode node) {
        ReactiveValueOperations<String, ObjectNode> operations = reactiveRedisTemplate.opsForValue();
        return operations.set(node.get("id").asText(), node, Duration.ofHours(24));
    }
}

先通过 /storejson 接口在 redis 中保存一个 json，然后通过 /getjson 获取时报错。

从错误日志来看，是由于反序列化时缺少 @class 字段；查看 redis：

127.0.0.1:6379> get 100
"{\"id\":\"100\",\"name\":\"aaarrr\"}"

果然如此。但众所周知，GenericJackson2JsonRedisSerializer和Jackson2JsonRedisSerializer的主要区别就在于，前者会在序列化时插入 class 的信息，这样在反序列化时就知道该怎么处理。但现在显然是碰到了特殊情况。

所以我们通过阅读源码，来一探究竟。

版本：group: 'org.springframework.boot', name: 'spring-boot-starter-data-redis', version: '2.3.3.RELEASE'。

1 Jackson 序列化

/** GenericJackson2JsonRedisSerializer **/
private final ObjectMapper mapper;

public byte[] serialize(@Nullable Object source) throws SerializationException {

    if (source == null) {
        return SerializationUtils.EMPTY_ARRAY;
    }

    try {
        // 内部有个ObjectMapper对象，序列化实际是调用om.writeValueAsBytes方法
        return mapper.writeValueAsBytes(source);  //主线
    } catch (JsonProcessingException e) {
        throw new SerializationException("Could not write JSON: " + e.getMessage(), e);
    }
}

/** ObjectMapper **/
public byte[] writeValueAsBytes(Object value)
        throws JsonProcessingException {
    ByteArrayBuilder bb = new ByteArrayBuilder(_jsonFactory._getBufferRecycler());
    try {
        _writeValueAndClose(createGenerator(bb, JsonEncoding.UTF8), value);  //主线
    } catch (JsonProcessingException e) { 
        throw e;
    } catch (IOException e) { 
        throw JsonMappingException.fromUnexpectedIOE(e);
    }
    byte[] result = bb.toByteArray();
    bb.release();
    return result;
}

protected final void _writeValueAndClose(JsonGenerator g, Object value)
        throws IOException {
    SerializationConfig cfg = getSerializationConfig();
    if (cfg.isEnabled(SerializationFeature.CLOSE_CLOSEABLE) && (value instanceof Closeable)) {
        _writeCloseable(g, value, cfg);  //序列化后调用value的close方法
        return;
    }
    try {
        _serializerProvider(cfg).serializeValue(g, value);  //主线
    } catch (Exception e) {
        ClassUtil.closeOnFailAndThrowAsIOE(g, e);
        return;
    }
    g.close();
}

这样一路跟着主线走下去，最终来到了 DefaultSerializerProvider的serializeValue方法。

/** DefaultSerializerProvider **/
public void serializeValue(JsonGenerator gen, Object value) throws IOException {
	...
    final Class<?> cls = value.getClass();
    final JsonSerializer<Object> ser = findTypedValueSerializer(cls, true, null);
    ...
    _serialize(gen, value, ser);
}

private final void _serialize(JsonGenerator gen, Object value,
            JsonSerializer<Object> ser) throws IOException {
    try {
        ser.serialize(value, gen, this);
    } catch (Exception e) {
        throw _wrapAsIOE(gen, e);
    }
}

这个方法主要干了两个事情：1、根据 value 的类型，找到合适的JsonSerializer；2、调用JsonSerializer的 serialize 方法，进行序列化。

findTypedValueSerializer

/** SerializerProvider **/
public JsonSerializer<Object> findTypedValueSerializer(Class<?> valueType,
            boolean cache, BeanProperty property)
        throws JsonMappingException
{
    // 先在本地缓存中找，找到了直接返回
    JsonSerializer<Object> ser = _knownSerializers.typedValueSerializer(valueType);
    if (ser != null) {
        return ser;
    }
    ser = _serializerCache.typedValueSerializer(valueType);
    if (ser != null) {
        return ser;
    }

    // 如果没找到
    ser = findValueSerializer(valueType, property);  //主线
    TypeSerializer typeSer = _serializerFactory.createTypeSerializer(_config,
            _config.constructType(valueType));
    if (typeSer != null) {
        typeSer = typeSer.forProperty(property);
        ser = new TypeWrappedSerializer(typeSer, ser);
    }
    if (cache) {
        _serializerCache.addTypedSerializer(valueType, ser);
    }
    return ser;
}

public JsonSerializer<Object> findValueSerializer(Class<?> valueType, BeanProperty property)
    throws JsonMappingException
{
    // 还是在缓存中找
    JsonSerializer<Object> ser = _knownSerializers.untypedValueSerializer(valueType);
    if (ser == null) {
        ser = _serializerCache.untypedValueSerializer(valueType);
        if (ser == null) {
            ser = _serializerCache.untypedValueSerializer(_config.constructType(valueType));
            if (ser == null) {
                ser = _createAndCacheUntypedSerializer(valueType);  //主线
                if (ser == null) {
                    ser = getUnknownTypeSerializer(valueType);
                    if (CACHE_UNKNOWN_MAPPINGS) {
                        _serializerCache.addAndResolveNonTypedSerializer(valueType, ser, this);
                    }
                    return ser;
                }
            }
        }
    }
    return (JsonSerializer<Object>) handleSecondaryContextualization(ser, property);
}

// 最终调用的是这个方法
protected JsonSerializer<Object> _createUntypedSerializer(JavaType type)
    throws JsonMappingException
{
    return (JsonSerializer<Object>)_serializerFactory.createSerializer(this, type);
}

默认的_serializerFactory是BeanSerializerFactory类型 ：

/** ObjectMapper **/
public ObjectMapper(JsonFactory jf, DefaultSerializerProvider sp, DefaultDeserializationContext dc)
{
    ...
    // Default serializer factory is stateless, can just assign
    _serializerFactory = BeanSerializerFactory.instance;
}

/** BeanSerializerFactory **/
public JsonSerializer<Object> createSerializer(SerializerProvider prov,
        JavaType origType)
    throws JsonMappingException
{
    // Very first thing, let's check if there is explicit serializer annotation:
    final SerializationConfig config = prov.getConfig();
    BeanDescription beanDesc = config.introspect(origType);
    JsonSerializer<?> ser = findSerializerFromAnnotation(prov, beanDesc.getClassInfo());
    if (ser != null) {
        return (JsonSerializer<Object>) ser;
    }
    boolean staticTyping;
    // Next: we may have annotations that further indicate actual type to use (a super type)
    final AnnotationIntrospector intr = config.getAnnotationIntrospector();
    JavaType type;

    if (intr == null) {
        type = origType;
    } else {
        try {
            type = intr.refineSerializationType(config, beanDesc.getClassInfo(), origType);
        } catch (JsonMappingException e) {
            return prov.reportBadTypeDefinition(beanDesc, e.getMessage());
        }
    }
    if (type == origType) { // no changes, won't force static typing
        staticTyping = false;
    } else { // changes; assume static typing; plus, need to re-introspect if class differs
        staticTyping = true;
        if (!type.hasRawClass(origType.getRawClass())) {
            beanDesc = config.introspect(type);
        }
    }
    // Slight detour: do we have a Converter to consider?
    Converter<Object,Object> conv = beanDesc.findSerializationConverter();
    if (conv == null) { // no, simple
        return (JsonSerializer<Object>) _createSerializer2(prov, type, beanDesc, staticTyping);
    }
    JavaType delegateType = conv.getOutputType(prov.getTypeFactory());
    
    // One more twist, as per [databind#288]; probably need to get new BeanDesc
    if (!delegateType.hasRawClass(type.getRawClass())) {
        beanDesc = config.introspect(delegateType);
        // [#359]: explicitly check (again) for @JsonSerializer...
        ser = findSerializerFromAnnotation(prov, beanDesc.getClassInfo());
    }
    // [databind#731]: Should skip if nominally java.lang.Object
    if (ser == null && !delegateType.isJavaLangObject()) {
        ser = _createSerializer2(prov, delegateType, beanDesc, true);
    }
    return new StdDelegatingSerializer(conv, delegateType, ser);
}

查找JsonSerializer的过程极其复杂，有很多针对 Issue 的特殊处理，就不一一分析了。大概思路是：在SerializerProvider中有个本地缓存，记录着class -> JsonSerializer的对应关系，如果查不到，那么根据复杂的匹配规则生成一个对应的JsonSerializer对象，并保存到缓存中；之后有相同的类需要序列化时，可以直接返回对应的JsonSerializer，大幅提升性能。

JsonSerializer

JsonSerializer，负责实际的序列化工作。

BeanSerializer

以常见的 BeanSerializer为例：

/** BeanSerializerBase **/
public void serializeWithType(Object bean, JsonGenerator gen,
        SerializerProvider provider, TypeSerializer typeSer)
    throws IOException
{
    if (_objectIdWriter != null) {
        gen.setCurrentValue(bean); // [databind#631]
        _serializeWithObjectId(bean, gen, provider, typeSer);
        return;
    }

    gen.setCurrentValue(bean); // [databind#631]
    WritableTypeId typeIdDef = _typeIdDef(typeSer, bean, JsonToken.START_OBJECT);
    typeSer.writeTypePrefix(gen, typeIdDef);
    if (_propertyFilterId != null) {
        serializeFieldsFiltered(bean, gen, provider);
    } else {
        serializeFields(bean, gen, provider);
    }
    typeSer.writeTypeSuffix(gen, typeIdDef);
}

WritableTypeId 里面包含了序列化后的信息，但是还没有写入到 gen 中。

/** TypeSerializer **/
public WritableTypeId writeTypePrefix(WritableTypeId typeIdDef) throws IOException
{
    ...
    Inclusion incl = typeIdDef.include;
    switch (incl) {
		...
        case METADATA_PROPERTY:
            writeStartObject(typeIdDef.forValue);
            writeStringField(typeIdDef.asProperty, idStr);
            return typeIdDef;
		...
    }
    ...
}

实际写入时，会根据WritableTypeId.Inclusion的类型，比如METADATA_PROPERTY类型的话，会把 id 属性也带上，通常是 @class:xxxpackage.yyyclass。

ObjectNode

如果是一个ObjectNode对象，那么最终序列化时调用的是ObjectNode类的serialize(..)方法，

/** ObjectNode **/
public void serialize(JsonGenerator g, SerializerProvider provider)
    throws IOException
{
    @SuppressWarnings("deprecation")
    boolean trimEmptyArray = (provider != null) &&
            !provider.isEnabled(SerializationFeature.WRITE_EMPTY_JSON_ARRAYS);
    g.writeStartObject(this); // {
    for (Map.Entry<String, JsonNode> en : _children.entrySet()) {
        BaseJsonNode value = (BaseJsonNode) en.getValue();
        if (trimEmptyArray && value.isArray() && value.isEmpty(provider)) {
            continue;
        }
        g.writeFieldName(en.getKey()); // 写key
        value.serialize(g, provider);  // 写value
    }
    g.writeEndObject();  // }
}

可以看到，其方法内部没有写入 @class 相关信息，这也解释了上面 bug 出现的原因。

2 处理办法

• 逃避

  既然GenericJackson2JsonRedisSerializer有问题，那我采用JdkSerializationRedisSerializer就好了，省事；

  或者既然知道默认的 ObjectNode 序列化后没有 @class 的信息，那就不用 ObjectNode 了，比如换成 Map 即可。

• 死磕

  耿直 boy，不想动GenericJackson2JsonRedisSerializer和 ObjectNode，怎么解决呢？

  最简单的，针对ObjectNode对象的序列化，不调用其内部的serialize方法即可。

  自定义一个模块，对ObjectNode进行特殊处理，代码如下：

  public class CustomModule extends SimpleModule {
      public CustomModule() {
          super(PackageVersion.VERSION);
          addSerializer(ObjectNode.class, CustomSerializer.INSTANCE);
      }
  
      static class CustomSerializer extends JsonSerializer {
          public static final CustomSerializer INSTANCE = new CustomSerializer();
  
          @Override
          public void serialize(Object value, JsonGenerator gen, SerializerProvider serializers) throws IOException {
              if (value instanceof ObjectNode) {
                  gen.writeStartObject(this);
                  gen.writeStringField("@class", value.getClass().getName());  //加上@class信息
                  Iterator<Map.Entry<String, JsonNode>> iterator = ((ObjectNode) value).fields();
                  while (iterator.hasNext()) {
                      Map.Entry<String, JsonNode> field = iterator.next();
                      JsonNode v = field.getValue();
                      if (v.isArray() && v.isEmpty(serializers)) {
                          continue;
                      }
                      gen.writeFieldName(field.getKey());
                      v.serialize(gen, serializers);
                  }
                  gen.writeEndObject();
              }
          }
      }
  }
  

  然后注册该模块到ObjectMapper即可。

  @Configuration
  public class RedisConfig {
      @Bean
      public ReactiveRedisTemplate<String, Object> reactiveRedisTemplate(ReactiveRedisConnectionFactory connectionFactory) {
          ObjectMapper om = new ObjectMapper();
          GenericJackson2JsonRedisSerializer.registerNullValueSerializer(om, null);
          om.activateDefaultTyping(om.getPolymorphicTypeValidator(), ObjectMapper.DefaultTyping.NON_FINAL, JsonTypeInfo.As.PROPERTY);
          om.registerModule(new CustomModule());  //注册自定义模块
  
          StringRedisSerializer stringSerializer = new StringRedisSerializer();
          GenericJackson2JsonRedisSerializer valueSerializer = new GenericJackson2JsonRedisSerializer(om);
          RedisSerializationContext<String, Object> redisSerializationContext = RedisSerializationContext.<String, Object>newSerializationContext()
                  .key(stringSerializer)
                  .value(valueSerializer)
                  .hashKey(stringSerializer)
                  .hashValue(valueSerializer)
                  .build();
          return new ReactiveRedisTemplate<>(connectionFactory, redisSerializationContext);
      }
  }
  

3 递归序列化

之前总结了，序列化的过程是：首先找到该对象对应的JsonSerializer，然后调用其序列化方法。对于复杂对象来说，对象的属性也可能是各种类型，因此需要递归调用其属性对应的序列化方法。

比如对于类：

class User {
    private String username;
    private Integer status;
}

序列化 username 和 status 时，需要分别调用StringSerializer和NumberSerializer对应的序列化方法。

考虑以下类，序列化后是什么结果？

class Nest {
    private String u;
    private Nest n;

    public Nest() {
        this.n = this;
    }

    public String getU() {
        return u;
    }

    public void setU(String u) {
        this.u = u;
    }

    public Nest getN() {
        return n;
    }

    public void setN(Nest n) {
        this.n = n;
    }
}

结果是会报错：

SerializationException: Could not write JSON: Direct self-reference leading to cycle (through reference chain: me.pggsnap.demos.redis.controller.UserController$Nest["n"]);

字段引用自身，会导致递归序列化，最终栈溢出。为了防止这种情况，默认配置为FAIL_ON_SELF_REFERENCES(true)，直接抛出异常。

om.disable(SerializationFeature.FAIL_ON_SELF_REFERENCES);

更改默认配置后测试，果然报栈溢出异常。

SerializationException: Could not write JSON: Infinite recursion (StackOverflowError)