继上篇文章(What is the JSON schema)介绍了项目引入 JSON Schema 的背景和对其简略的说明。
当时调研的时候，在网上发现很少有后端对 JSON Schema 使用的博客的分享。本篇文章分享下后端项目( SpringBoot )对 JSON Schema 的集成，算是填补一下这块的空白吧。😁(哈哈)
校验引擎(Validator)实现的选择
首先，在编写好 JSON Schema 的文档后，需要选用解析/校验引擎的实现( Implementations )。
Java 语言的Validator实现库
Java 语言的 Validator 有如下的开源实现库可用：
-
- Supports: 2020-12, 2019-09 draft-07, -04
- License: Apache License, Version 2.0
- Notes: Validator for Eclipse Vert.x project JSON types. Includes custom keywords support, custom dialect support, asynchronous validation
- Information last updated: 2022-08-31
-
- Supports: 2020-12, 2019-09 draft-07, -06, -04, -03
- License: Apache License 2.0
- Information last updated: 2022-08-31
-
networknt/json-schema-validator
- Supports: 2020-12, 2019-09 draft-07, -06, -04
- Compliance: This implementation documents that you must set
handleNullableFieldtofalseto produce specification-compliant behavior. - License: Apache License 2.0
- Notes: Support OpenAPI 3.0 with Jackson parser
- Information last updated: 2022-08-31
-
- Supports: 2019-09 draft-07, -06
- License: GNU Affero General Public License v3.0
- Notes: Uses Maven for the project and Gson under the hood.
- Information last updated: 2022-08-31
-
- Supports: draft-07, -06, -04
- License: Apache License 2.0
- Information last updated: 2022-08-31
-
- Supports: draft-07, -06, -04
- License: Apache License 2.0
- Information last updated: 2022-08-31
Java 语言的 Validator 实现库的对比和选择
其中，主要考虑版社区活跃度(包括 stars 数)、对 specification 版本支持的丰富度、解析日志的友好性以及性能等几个方面来进行选择。
| 标题 | stars (更新于2023.3.9) | 版本支持 | 日志 |
|---|---|---|---|
| github.com/networknt/j… | 55 | 2020-12, 2019-09 draft-07, -06, -04 | 好 |
| github.com/jimblackler… | 30 | 2020-12, 2019-09 draft-07, -06, -04, -03 | -- |
| github.com/eclipse-ver… | 578 | 2020-12, 2019-09 draft-07, -04 | --- |
另外，根据 networknt/json-schema-validator 提供的性能测试数据，其性能远远优于其他开源实现。
综上，我最终选择了 networknt/json-schema-validator来作为后端集成的 JSON schema 校验框架。
项目的集成
集成步骤如下：
- 将写好的 schema 文件放到 resource 的文件夹下面，以
./resource/json/UTPSchema为例： - 将对应的依赖引入项目中，以Gradle为例：
implementation group: "com.networknt", name: "json-schema-validator", version: "1.0.52"
- 定义Config类：
@Configuration
public class JsonSchemaConfig {
@Value(value = "${utp.schema.path:json/UTPSchema.json}")
String schemaJsonPath;
@Bean
public JsonSchema getSchema() throws IOException {
// With automatic version detection
JsonNode schemaNode = JsonSchemaUtils.getJsonNodeFromClasspath(schemaJsonPath);
return JsonSchemaUtils.getJsonSchemaFromJsonNodeAutomaticVersion(schemaNode);
}
}
public final class JsonSchemaUtils {
private JsonSchemaUtils() {
}
private static final ObjectMapper MAPPER = new ObjectMapper();
public static JsonNode getJsonNodeFromClasspath(String name) throws IOException {
JsonNode jsonNode;
try (InputStream is1 = Thread.currentThread().getContextClassLoader().getResourceAsStream(name)) {
jsonNode = MAPPER.readTree(is1);
}
return jsonNode;
}
public static JsonNode getJsonNodeFromStringContent(String content) throws IOException {
return MAPPER.readTree(content);
}
public static JsonNode getJsonNodeFromUrl(String url) throws IOException {
return MAPPER.readTree(new URL(url));
}
// Automatically detect version for given JsonNode
public static JsonSchema getJsonSchemaFromJsonNodeAutomaticVersion(JsonNode jsonNode) {
JsonSchemaFactory factory = JsonSchemaFactory.getInstance(SpecVersionDetector.detect(jsonNode));
return factory.getSchema(jsonNode);
}
public static Map<String, List<String>> getGroupedMessage(Set<ValidationMessage> errors) {
Map<String, List<String>> collect = new TreeMap<>();
errors.stream()
.collect(Collectors.groupingBy(ValidationMessage::getPath))
.forEach((path, validationMessages) -> {
List<String> messageList =
validationMessages.stream().
map(ValidationMessage::getMessage)
.collect(Collectors.toList());
collect.put(path, messageList);
});
return collect;
}
}
- 在业务逻辑中使用
@Slf4j
@Component
@AllArgsConstructor
public class JsonSchemaService {
private JsonSchema jsonSchema;
public void validateByJsonSchema(JsonNode jsonNode) {
if (ObjectUtils.isEmpty(jsonNode)) {
throw new UtpValidationException(ResultStatus.Product.NO_REQUIRED_PARAM);
}
Set<ValidationMessage> errors = getValidationMessages(jsonNode);
if (!CollectionUtils.isEmpty(errors)) {
Map<String, List<String>> groupedMessage = JsonSchemaUtils.getGroupedMessage(errors);
throw new UtpValidationException(ResultCode.Product.INVALID_PARAMETER, JsonUtils.toJson(groupedMessage));
}
}
public Set<ValidationMessage> getValidationMessages(JsonNode jsonNode) {
return jsonSchema.validate(jsonNode);
}
}
- 测试用例
class JsonSchemaServiceTest extends Specification {
JsonSchemaService jsonSchemaService;
static final String UNHAPPY_PATH_PREFIX = "json/unhappy/"
static final String HAPPY_PATH_PREFIX = "json/happy/"
static final String SCHEMA_PATH = "json/UTPSchema.json"
public static JsonNode getUnhappyTestJsonNode(String fileName){
return JsonSchemaUtils.getJsonNodeFromClasspath(UNHAPPY_PATH_PREFIX + fileName + ".json")
}
def setup() {
//init schema With automatic version detection
JsonNode schemaNode = JsonSchemaUtils.getJsonNodeFromClasspath(SCHEMA_PATH)
jsonSchemaService = new JsonSchemaService(JsonSchemaUtils.getJsonSchemaFromJsonNodeAutomaticVersion(schemaNode))
}
@Unroll
def "happyPath"() {
given:
JsonNode node = JsonSchemaUtils.getJsonNodeFromClasspath(HAPPY_PATH_PREFIX+ fileName)
when:
jsonSchemaService.validateByJsonSchema(node)
then:
notThrown(UtpValidationException.class)
where:
fileName << ["text.json", "number.json", "boolean.json", "enum.json", "arrayWithBasic.json",
"arrayWithStruct.json", "struct.json", "bitwise.json"]
}
def "numberUnhappyTest"() {
given:
JsonNode node = getUnhappyTestJsonNode("numberUnhappy")
expect:
Set<ValidationMessage> errors = jsonSchemaService.getValidationMessages(node)
errors.size() == 5
errors[0].getMessage() == "\$.profile.sourceProfiles[0].sourceModelIdentifierVersion: is missing but it is required"
errors[1].getMessage() == "\$.properties[0].specs.min: is missing but it is required"
errors[2].getMessage() == "\$.properties[0].specs.step: does not match the regex pattern " +
"^(\\+|\\-)?(\\d+)(\\.\\d+)?\$"
errors[3].getMessage() == "\$.properties[1].specs.step: does not match the regex pattern" +
" ^(\\+|\\-)?(\\d+)(\\.\\d+)?\$"
errors[4].getMessage() == "\$.properties[2].specs.step: does not match the regex " +
"pattern ^(\\+|\\-)?(\\d+)(\\.\\d+)?\$"
}
def "textUnhappyTest"() {
given:
JsonNode node = getUnhappyTestJsonNode("textUnhappy")
expect:
Set<ValidationMessage> errors = jsonSchemaService.getValidationMessages(node)
errors.size() == 6
errors[0].getMessage() == "\$.profile.sourceProfiles[1].sourceModelIdentifier: is missing but it is required"
errors[1].getMessage() == "\$.properties[1].specs.length: does not match the regex pattern ^[1-9]\\d*\$"
errors[2].getMessage() == "\$.properties[1].specs.length: should be valid to any of the schemas integer"
errors[3].getMessage() == "\$.properties[2].sourceProperties[1].sourceDataType: does not have a value in the " +
"enumeration [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]"
errors[4].getMessage() == "\$.properties[2].specs.length: does not match the regex pattern ^[1-9]\\d*\$"
errors[5].getMessage() == "\$.properties[2].specs.length: should be valid to any of the schemas integer"
}
def "booleanUnhappyTest"() {
given:
JsonNode node = getUnhappyTestJsonNode("booleanUnhappy")
expect:
Set<ValidationMessage> errors = jsonSchemaService.getValidationMessages(node)
errors.size() == 1
errors[0].getMessage() == "\$.properties[0].items[0].sourceItems[0].position: is missing but it is required"
}
def "enumUnhappyTest"() {
given:
JsonNode node = getUnhappyTestJsonNode("enumUnhappy")
expect:
Set<ValidationMessage> errors = jsonSchemaService.getValidationMessages(node)
errors.size() == 7
errors[0].getMessage() == "\$.profile.modelIdentifierDisplayName: is missing but it is required"
errors[1].getMessage() == "\$.properties[0].sourceProperties[0].sourcePlatform: does not have a" +
" value in the enumeration [0, 1]"
errors[2].getMessage() == "\$.properties[0].readWriteMode: does not have a value in the enumeration [1, 2]"
errors[3].getMessage() == "\$.properties[0].items[1].value: is missing but it is required"
errors[4].getMessage() == "\$.properties[0].items[2].dataType: does not have a value in the enumeration [1, 2]"
errors[5].getMessage() == "\$.properties[0].items[2].sourceItems[0].sourceValue: should be valid to any of the schemas integer"
errors[6].getMessage() == "\$.properties[0].items[2].sourceItems[0].sourceValue: should be valid to any of the schemas string"
}
def "structUnhappyTest"() {
given:
JsonNode node = getUnhappyTestJsonNode("structUnhappy")
expect:
Set<ValidationMessage> errors = jsonSchemaService.getValidationMessages(node)
errors.size() == 5
errors[0].getMessage() == "\$.properties[0].sourceProperties[0].sourceDataType: is missing but it is required"
errors[1].getMessage() == "\$.properties[0].readWriteMode: does not have a value in the enumeration [1, 2]"
errors[2].getMessage() == "\$.properties[0].items[0].identifier: is missing but it is required"
errors[3].getMessage() == "\$.properties[0].items[0].specs.step: does not match the regex" +
" pattern ^(\\+|\\-)?(\\d+)(\\.\\d+)?\$"
errors[4].getMessage() == "\$.properties[0].items[1].specs.step: is missing but it is required"
}
def "arrayStructUnhappyTest"() {
given:
JsonNode node = getUnhappyTestJsonNode("arrayStructUnhappy")
expect:
Set<ValidationMessage> errors = jsonSchemaService.getValidationMessages(node)
errors.size() == 2
errors[0].getMessage() == "\$.properties[0].items[0].items[0].sourceProperties[0].sourcePlatform: does not" +
" have a value in the enumeration [0, 1]"
errors[1].getMessage() == "\$.properties[0].items[0].items[0].specs.min: is missing but it is required"
}
def "arrayBasicUnhappyTest"() {
given:
JsonNode node = getUnhappyTestJsonNode("arrayBasicUnhappy")
expect:
Set<ValidationMessage> errors = jsonSchemaService.getValidationMessages(node)
errors.size() == 2
errors[0].getMessage() == "\$.profile.modelIdentifierDisplayName: is missing but it is required"
errors[1].getMessage() == "\$.properties[0].items[0].specs.max: is missing but it is required"
}
def "bitUnhappyTest"() {
given:
JsonNode node = getUnhappyTestJsonNode("bitUnhappy")
expect:
Set<ValidationMessage> errors = jsonSchemaService.getValidationMessages(node)
errors.size() == 7
errors[0].getMessage() == "\$.profile.modelIdentifier: is missing but it is required"
errors[1].getMessage() == "\$.profile.sourceProfiles: there must be a minimum of 1 items in the array"
errors[2].getMessage() == "\$.properties[0].identifier: is missing but it is required"
errors[3].getMessage() == "\$.properties[0].items[0].sourceItems[0].position: is missing but it is required"
errors[4].getMessage() == "\$.properties[1].sourceProperties: there must be a minimum of 1 items in the array"
errors[5].getMessage() == "\$.properties[1].dataType: does not have a value in the " +
"enumeration [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]"
errors[6].getMessage() == "\$.properties[1].dataType: string found, integer expected"
}
}
通过上述测试用例可以看出，该 validator 对执行校验的结果 是很 pretty 的（经过JsonSchemaUtils。getGroupedMessage()方法的处理后）。通过error信息，可以明确知道错误的位置和错误的原因。这也是选择networknt/json-schema-validator来作为后端集成的 JSON schema 校验框架的一重要原因。
总结
JSON Schema 作为请求数据(json)的校验元数据，通过其Validator的实现，可以作为请求合法以及甚至部分完整的校验。在请求的数据结构和校验逻辑较为复杂的时候，不失为一种较为实用的解决方案的选择。
