JSON schema springboot 的集成和使用

643 阅读4分钟

继上篇文章(What is the JSON schema)介绍了项目引入 JSON Schema 的背景和对其简略的说明。

当时调研的时候,在网上发现很少有后端对 JSON Schema 使用的博客的分享。本篇文章分享下后端项目( SpringBoot )对 JSON Schema 的集成,算是填补一下这块的空白吧。😁(哈哈)

校验引擎(Validator)实现的选择

首先,在编写好 JSON Schema 的文档后,需要选用解析/校验引擎的实现( Implementations )。

Java 语言的Validator实现库

Java 语言的 Validator 有如下的开源实现库可用:

  • Vert.x Json Schema

    • Supports:  2020-12, 2019-09 draft-07, -04
    • License:  Apache License, Version 2.0
    • Notes:  Validator for Eclipse Vert.x project JSON types. Includes custom keywords support, custom dialect support, asynchronous validation
    • Information last updated:  2022-08-31
  • jsonschemafriend

    • Supports:  2020-12, 2019-09 draft-07, -06, -04, -03
    • License:  Apache License 2.0
    • Information last updated:  2022-08-31
  • networknt/json-schema-validator

    • Supports:  2020-12, 2019-09 draft-07, -06, -04
    • Compliance:  This implementation documents that you must set handleNullableField to false to produce specification-compliant behavior.
    • License:  Apache License 2.0
    • Notes:  Support OpenAPI 3.0 with Jackson parser
    • Information last updated:  2022-08-31
  • Snow

    • Supports:  2019-09 draft-07, -06
    • License:  GNU Affero General Public License v3.0
    • Notes:  Uses Maven for the project and Gson under the hood.
    • Information last updated:  2022-08-31
  • everit-org/json-schema

    • Supports:  draft-07, -06, -04
    • License:  Apache License 2.0
    • Information last updated:  2022-08-31
  • Justify

    • Supports:  draft-07, -06, -04
    • License:  Apache License 2.0
    • Information last updated:  2022-08-31

Java 语言的 Validator 实现库的对比和选择

其中,主要考虑版社区活跃度(包括 stars 数)、对 specification 版本支持的丰富度、解析日志的友好性以及性能等几个方面来进行选择。

标题stars (更新于2023.3.9)版本支持日志
github.com/networknt/j…552020-12, 2019-09 draft-07, -06, -04
github.com/jimblackler…302020-12, 2019-09 draft-07, -06, -04, -03--
github.com/eclipse-ver…5782020-12, 2019-09 draft-07, -04---

另外,根据 networknt/json-schema-validator 提供的性能测试数据,其性能远远优于其他开源实现。

综上,我最终选择了 networknt/json-schema-validator来作为后端集成的 JSON schema 校验框架。

项目的集成

集成步骤如下:

  1. 将写好的 schema 文件放到 resource 的文件夹下面,以./resource/json/UTPSchema为例: image.png
  2. 将对应的依赖引入项目中,以Gradle为例:
 implementation group: "com.networknt", name: "json-schema-validator", version: "1.0.52"
  1. 定义Config类:
@Configuration
public class JsonSchemaConfig {
    @Value(value = "${utp.schema.path:json/UTPSchema.json}")
    String schemaJsonPath;

    @Bean
    public JsonSchema getSchema() throws IOException {
        // With automatic version detection
        JsonNode schemaNode = JsonSchemaUtils.getJsonNodeFromClasspath(schemaJsonPath);
        return JsonSchemaUtils.getJsonSchemaFromJsonNodeAutomaticVersion(schemaNode);
    }
}

public final class JsonSchemaUtils {

    private JsonSchemaUtils() {
    }

    private static final ObjectMapper MAPPER = new ObjectMapper();


    public static JsonNode getJsonNodeFromClasspath(String name) throws IOException {
        JsonNode jsonNode;
        try (InputStream is1 = Thread.currentThread().getContextClassLoader().getResourceAsStream(name)) {
            jsonNode = MAPPER.readTree(is1);
        }
        return jsonNode;
    }

    public static JsonNode getJsonNodeFromStringContent(String content) throws IOException {
        return MAPPER.readTree(content);
    }

    public static JsonNode getJsonNodeFromUrl(String url) throws IOException {
        return MAPPER.readTree(new URL(url));
    }

    // Automatically detect version for given JsonNode
    public static JsonSchema getJsonSchemaFromJsonNodeAutomaticVersion(JsonNode jsonNode) {
        JsonSchemaFactory factory = JsonSchemaFactory.getInstance(SpecVersionDetector.detect(jsonNode));
        return factory.getSchema(jsonNode);
    }
    
    public static Map<String, List<String>> getGroupedMessage(Set<ValidationMessage> errors) {
        Map<String, List<String>> collect = new TreeMap<>();
        errors.stream()
            .collect(Collectors.groupingBy(ValidationMessage::getPath))
            .forEach((path, validationMessages) -> {
                List<String> messageList =
                    validationMessages.stream().
                        map(ValidationMessage::getMessage)
                        .collect(Collectors.toList());
                collect.put(path, messageList);
            });
        return collect;
    }

}

  1. 在业务逻辑中使用
@Slf4j
@Component
@AllArgsConstructor
public class JsonSchemaService {
    private JsonSchema jsonSchema;

    public void validateByJsonSchema(JsonNode jsonNode) {
        if (ObjectUtils.isEmpty(jsonNode)) {
            throw new UtpValidationException(ResultStatus.Product.NO_REQUIRED_PARAM);
        }
        Set<ValidationMessage> errors = getValidationMessages(jsonNode);
        if (!CollectionUtils.isEmpty(errors)) {
            Map<String, List<String>> groupedMessage = JsonSchemaUtils.getGroupedMessage(errors);
            throw new UtpValidationException(ResultCode.Product.INVALID_PARAMETER, JsonUtils.toJson(groupedMessage));
        }
    }

    public Set<ValidationMessage> getValidationMessages(JsonNode jsonNode) {
        return jsonSchema.validate(jsonNode);
    }

}
  1. 测试用例
class JsonSchemaServiceTest extends Specification {
    JsonSchemaService jsonSchemaService;
    static final String UNHAPPY_PATH_PREFIX =  "json/unhappy/"
    static final String HAPPY_PATH_PREFIX =  "json/happy/"
    static final String SCHEMA_PATH =  "json/UTPSchema.json"

    public static JsonNode getUnhappyTestJsonNode(String fileName){
        return JsonSchemaUtils.getJsonNodeFromClasspath(UNHAPPY_PATH_PREFIX + fileName + ".json")
    }

    def setup() {
        //init schema With automatic version detection
        JsonNode schemaNode = JsonSchemaUtils.getJsonNodeFromClasspath(SCHEMA_PATH)
        jsonSchemaService = new JsonSchemaService(JsonSchemaUtils.getJsonSchemaFromJsonNodeAutomaticVersion(schemaNode))
    }

    @Unroll
    def "happyPath"() {
        given:
        JsonNode node = JsonSchemaUtils.getJsonNodeFromClasspath(HAPPY_PATH_PREFIX+ fileName)

        when:
        jsonSchemaService.validateByJsonSchema(node)

        then:
        notThrown(UtpValidationException.class)

        where:
        fileName << ["text.json", "number.json", "boolean.json", "enum.json", "arrayWithBasic.json",
                         "arrayWithStruct.json", "struct.json", "bitwise.json"]
    }

    def "numberUnhappyTest"() {
        given:
        JsonNode node = getUnhappyTestJsonNode("numberUnhappy")

        expect:
        Set<ValidationMessage> errors = jsonSchemaService.getValidationMessages(node)
        errors.size() == 5
        errors[0].getMessage() == "\$.profile.sourceProfiles[0].sourceModelIdentifierVersion: is missing but it is required"
        errors[1].getMessage() == "\$.properties[0].specs.min: is missing but it is required"
        errors[2].getMessage() == "\$.properties[0].specs.step: does not match the regex pattern " +
                "^(\\+|\\-)?(\\d+)(\\.\\d+)?\$"
        errors[3].getMessage() == "\$.properties[1].specs.step: does not match the regex pattern" +
                " ^(\\+|\\-)?(\\d+)(\\.\\d+)?\$"
        errors[4].getMessage() == "\$.properties[2].specs.step: does not match the regex " +
                "pattern ^(\\+|\\-)?(\\d+)(\\.\\d+)?\$"
    }

    def "textUnhappyTest"() {
        given:
        JsonNode node = getUnhappyTestJsonNode("textUnhappy")

        expect:
        Set<ValidationMessage> errors = jsonSchemaService.getValidationMessages(node)
        errors.size() == 6
        errors[0].getMessage() == "\$.profile.sourceProfiles[1].sourceModelIdentifier: is missing but it is required"
        errors[1].getMessage() == "\$.properties[1].specs.length: does not match the regex pattern ^[1-9]\\d*\$"
        errors[2].getMessage() ==  "\$.properties[1].specs.length: should be valid to any of the schemas integer"
        errors[3].getMessage() == "\$.properties[2].sourceProperties[1].sourceDataType: does not have a value in the " +
                "enumeration [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]"
        errors[4].getMessage() == "\$.properties[2].specs.length: does not match the regex pattern ^[1-9]\\d*\$"
        errors[5].getMessage() == "\$.properties[2].specs.length: should be valid to any of the schemas integer"
    }

    def "booleanUnhappyTest"() {
        given:
        JsonNode node = getUnhappyTestJsonNode("booleanUnhappy")

        expect:
        Set<ValidationMessage> errors = jsonSchemaService.getValidationMessages(node)
        errors.size() == 1
        errors[0].getMessage() == "\$.properties[0].items[0].sourceItems[0].position: is missing but it is required"
    }

    def "enumUnhappyTest"() {
        given:
        JsonNode node = getUnhappyTestJsonNode("enumUnhappy")

        expect:
        Set<ValidationMessage> errors = jsonSchemaService.getValidationMessages(node)
        errors.size() == 7
        errors[0].getMessage() == "\$.profile.modelIdentifierDisplayName: is missing but it is required"
        errors[1].getMessage() == "\$.properties[0].sourceProperties[0].sourcePlatform: does not have a" +
                " value in the enumeration [0, 1]"
        errors[2].getMessage() == "\$.properties[0].readWriteMode: does not have a value in the enumeration [1, 2]"
        errors[3].getMessage() == "\$.properties[0].items[1].value: is missing but it is required"
        errors[4].getMessage() == "\$.properties[0].items[2].dataType: does not have a value in the enumeration [1, 2]"
        errors[5].getMessage() == "\$.properties[0].items[2].sourceItems[0].sourceValue: should be valid to any of the schemas integer"
        errors[6].getMessage() == "\$.properties[0].items[2].sourceItems[0].sourceValue: should be valid to any of the schemas string"
    }

    def "structUnhappyTest"() {
        given:
        JsonNode node = getUnhappyTestJsonNode("structUnhappy")

        expect:
        Set<ValidationMessage> errors = jsonSchemaService.getValidationMessages(node)
        errors.size() == 5
        errors[0].getMessage() == "\$.properties[0].sourceProperties[0].sourceDataType: is missing but it is required"
        errors[1].getMessage() == "\$.properties[0].readWriteMode: does not have a value in the enumeration [1, 2]"
        errors[2].getMessage() == "\$.properties[0].items[0].identifier: is missing but it is required"
        errors[3].getMessage() == "\$.properties[0].items[0].specs.step: does not match the regex" +
                " pattern ^(\\+|\\-)?(\\d+)(\\.\\d+)?\$"
        errors[4].getMessage() == "\$.properties[0].items[1].specs.step: is missing but it is required"
    }

    def "arrayStructUnhappyTest"() {
        given:
        JsonNode node = getUnhappyTestJsonNode("arrayStructUnhappy")

        expect:
        Set<ValidationMessage> errors = jsonSchemaService.getValidationMessages(node)
        errors.size() == 2

        errors[0].getMessage() == "\$.properties[0].items[0].items[0].sourceProperties[0].sourcePlatform: does not" +
                " have a value in the enumeration [0, 1]"
        errors[1].getMessage() == "\$.properties[0].items[0].items[0].specs.min: is missing but it is required"
    }

    def "arrayBasicUnhappyTest"() {
        given:
        JsonNode node = getUnhappyTestJsonNode("arrayBasicUnhappy")

        expect:
        Set<ValidationMessage> errors = jsonSchemaService.getValidationMessages(node)
        errors.size() == 2
        errors[0].getMessage() == "\$.profile.modelIdentifierDisplayName: is missing but it is required"
        errors[1].getMessage() == "\$.properties[0].items[0].specs.max: is missing but it is required"
    }

    def "bitUnhappyTest"() {
        given:
        JsonNode node = getUnhappyTestJsonNode("bitUnhappy")

        expect:
        Set<ValidationMessage> errors = jsonSchemaService.getValidationMessages(node)
        errors.size() == 7
        errors[0].getMessage() == "\$.profile.modelIdentifier: is missing but it is required"
        errors[1].getMessage() == "\$.profile.sourceProfiles: there must be a minimum of 1 items in the array"
        errors[2].getMessage() == "\$.properties[0].identifier: is missing but it is required"
        errors[3].getMessage() == "\$.properties[0].items[0].sourceItems[0].position: is missing but it is required"
        errors[4].getMessage() == "\$.properties[1].sourceProperties: there must be a minimum of 1 items in the array"
        errors[5].getMessage() == "\$.properties[1].dataType: does not have a value in the " +
                "enumeration [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]"
        errors[6].getMessage() == "\$.properties[1].dataType: string found, integer expected"
    }

}

通过上述测试用例可以看出,该 validator 对执行校验的结果 是很 pretty 的(经过JsonSchemaUtils。getGroupedMessage()方法的处理后)。通过error信息,可以明确知道错误的位置和错误的原因。这也是选择networknt/json-schema-validator来作为后端集成的 JSON schema 校验框架的一重要原因。

总结

JSON Schema 作为请求数据(json)的校验元数据,通过其Validator的实现,可以作为请求合法以及甚至部分完整的校验。在请求的数据结构和校验逻辑较为复杂的时候,不失为一种较为实用的解决方案的选择。