背景
日常API自动化测试通过 testng中配置 TestListener将测试结果记录到klov reportserver，klov reportserver 用mongodb作为数据库，目前已经运行一年多，开始暴露一些性能问题。
- Report 页面出现用例结果展示缓慢
- Report 页面请求服务器数据超时
- Report 页面无法展示图片问题
Klov 分析
Klov collection 说明
-
Project 由extents manager 自动创建
-
Report 多个test 组成的 report
-
Test 对应一个测试用例
-
Log 步骤信息
-
Media 保存test中的图片数据
-
Exception 保存test中的堆栈数据
Report 数据关系说明
-
Project
-
Report
-
Test
-
Log
-
media
-
exception
-
-
-
Klov 数据量统计
数据主要分布在media 和log两个collection
db.stats
[ { "avgObjSize": 53655.480213145005, "collections": 6, "dataSize": 54001611716, "db": "klov", "fsTotalSize": 1056281419776, "fsUsedSize": 634706132992, "indexSize": 23547904, "indexes": 10, "objects": 1006451, "ok": 1, "scaleFactor": 1, "storageSize": 52946083840, "totalSize": 52969631744, "views": 0 }]
db.log.stats()
[
{
"avgObjSize": 29617,
"capped": false,
"count": 784891,
"freeStorageSize": 13484032,
"indexBuilds": [],
"indexSizes": {
"_id_": 14647296
},
"nindexes": 1,
"ns": "klov.log",
"ok": 1,
"scaleFactor": 1,
"size": 23246864593,
"storageSize": 22510948352,
"totalIndexSize": 14647296,
"totalSize": 22525595648,
---省略---
]
性能分析
Mongo 慢查询报告：
https://reportserver.***.cn/projects/OpenAPI-TestReport/launches/66dfb7ae4a99a87a4cfaab16/tests
Media collection
db.media.find({log:ObjectId("66dfb7d84a99a87a4cfaab1b")});
[
{
"_id": {"$oid": "66dfb7d84a99a87a4cfaab1c"},
"base64String": "data:image/png;base64,/9j/4AAQSkZJRgABAQEBLAEsAA....忽略部分数据....NG7PB9FbgSa7cC2bLwitVupuroMSyR4nd/f5pnWmNGZ0gm+",
"log": {"$oid": "66dfb7d84a99a87a4cfaab1b"},
"project": {"$oid": "656e83ec579c8205918aca5d"},
"report": {"$oid": "66dfb7ae4a99a87a4cfaab16"},
"test": {"$oid": "66dfb7ae4a99a87a4cfaab17"}
}
]
报告用logid作为查询条件，但默认情况下，klov不会为logid创建索引
db.media.getIndexes();
[ { "key": { "_id": 1 }, "name": "_id_", "v": 2 }]
db.media.createIndex({log:-1});
验证效果：
db.media.find({log:ObjectId("66dfb7d84a99a87a4cfaab1b")}).explain("executionStats");
添加索引前后的查询计划对比：
db.media.find({log:ObjectId("66dfb7d84a99a87a4cfaab1b")}).explain("executionStats");
无索引查询计划
[
{
"executionStats": {
"executionSuccess": true,
"nReturned": 1,
"executionTimeMillis": 57870,
"totalKeysExamined": 0,
"totalDocsExamined": 47703,
"executionStages": {
"stage": "COLLSCAN",
"filter": {
"log": {
"$eq": {"$oid": "66dfb7d84a99a87a4cfaab1b"}
}
},
"nReturned": 1,
"executionTimeMillisEstimate": 53855,
"works": 47705,
"advanced": 1,
"needTime": 47703,
"needYield": 0,
"saveState": 2822,
"restoreState": 2822,
"isEOF": 1,
"direction": "forward",
"docsExamined": 47703
}
},
"ok": 1,
"queryPlanner": {
"plannerVersion": 1,
"namespace": "klov.media",
"indexFilterSet": false,
"parsedQuery": {
"log": {
"$eq": {"$oid": "66dfb7d84a99a87a4cfaab1b"}
}
},
"winningPlan": {
"stage": "COLLSCAN",
"filter": {
"log": {
"$eq": {"$oid": "66dfb7d84a99a87a4cfaab1b"}
}
},
"direction": "forward"
},
"rejectedPlans": []
},
"serverInfo": {
"host": "25ebdda77712",
"port": 27017,
"version": "4.4.12",
"gitVersion": "51475a8c4d9856eb1461137e7539a0a763cc85dc"
}
}
]
添加索引查询计划
db.media.find({log:ObjectId("66dfb7d84a99a87a4cfaab1b")}).explain("executionStats");
[
{
"executionStats": {
"executionSuccess": true,
"nReturned": 1,
"executionTimeMillis": 5,
"totalKeysExamined": 1,
"totalDocsExamined": 1,
"executionStages": {
"stage": "FETCH",
"nReturned": 1,
"executionTimeMillisEstimate": 0,
"works": 2,
"advanced": 1,
"needTime": 0,
"needYield": 0,
"saveState": 0,
"restoreState": 0,
"isEOF": 1,
"docsExamined": 1,
"alreadyHasObj": 0,
"inputStage": {
"stage": "IXSCAN",
"nReturned": 1,
"executionTimeMillisEstimate": 0,
"works": 2,
"advanced": 1,
"needTime": 0,
"needYield": 0,
"saveState": 0,
"restoreState": 0,
"isEOF": 1,
"keyPattern": {
"log": -1
},
"indexName": "log_-1",
"isMultiKey": false,
"multiKeyPaths": {
"log": []
},
"isUnique": false,
"isSparse": false,
"isPartial": false,
"indexVersion": 2,
"direction": "forward",
"indexBounds": {
"log": ["[ObjectId('66dfb7d84a99a87a4cfaab1b'), ObjectId('66dfb7d84a99a87a4cfaab1b')]"]
},
"keysExamined": 1,
"seeks": 1,
"dupsTested": 0,
"dupsDropped": 0
}
}
},
"ok": 1,
"queryPlanner": {
"plannerVersion": 1,
"namespace": "klov.media",
"indexFilterSet": false,
"parsedQuery": {
"log": {
"$eq": {"$oid": "66dfb7d84a99a87a4cfaab1b"}
}
},
"winningPlan": {
"stage": "FETCH",
"inputStage": {
"stage": "IXSCAN",
"keyPattern": {
"log": -1
},
"indexName": "log_-1",
"isMultiKey": false,
"multiKeyPaths": {
"log": []
},
"isUnique": false,
"isSparse": false,
"isPartial": false,
"indexVersion": 2,
"direction": "forward",
"indexBounds": {
"log": ["[ObjectId('66dfb7d84a99a87a4cfaab1b'), ObjectId('66dfb7d84a99a87a4cfaab1b')]"]
}
}
},
"rejectedPlans": []
},
"serverInfo": {
"host": "25ebdda77712",
"port": 27017,
"version": "4.4.12",
"gitVersion": "51475a8c4d9856eb1461137e7539a0a763cc85dc"
}
}
]
Log collection
测试报告根据 testid 查询log 集合
db.log.find({test:ObjectId("67089f26b19da883475b241f")});
增加索引
db.log.createIndex({test:-1});
验证
db.log.find({test:ObjectId("67089f26b19da883475b241f")}).explain("executionStats");
Exception collection
一个test对应一个 exception，db.test.find({exception:{$ne:null}}).limit(1);
[
{
"_id": {"$oid": "656e83ec579c8205918aca5f"},
"bdd": false,
"categorized": false,
"childNodesLength": 0,
"description": "test: testP0_SUCCESS_OnePersonDiffPhotos",
"duration": 1,
"endTime": {"$date": "2023-12-05T01:59:08.099Z"},
"exception": {"$oid": "656e83ef579c8205918aca62"},
"exceptionName": "java.lang.AssertionError",
"leaf": true,
"level": 0,
"logCount": 5,
"mediaCount": 0,
"name": "testP0_SUCCESS_OnePersonDiffPhotos",
"project": {"$oid": "656e83ec579c8205918aca5d"},
"report": {"$oid": "656e83ec579c8205918aca5e"},
"reportName": "OpenAPI-reportjava.text.SimpleDateFormat@6b2ed43a",
"reportSeq": 1,
"startTime": {"$date": "2023-12-05T01:59:08.098Z"},
"status": "fail"
}
]
测试报告用exception id 作为关联查询条件, exception 有id字段默认索引，无需处理
db.exception.find({_id:ObjectId("656e83ef579c8205918aca62")});
[
{
"_id": {"$oid": "656e83ef579c8205918aca62"},
"name": "java.lang.AssertionError",
"project": {"$oid": "656e83ec579c8205918aca5d"},
"report": {"$oid": "656e83ec579c8205918aca5e"},
"stacktrace": "java.lang.AssertionError: Not equals expected [SUCCESS_PASS] but found [SUCCESS]\n\tat org.testng.Assert.fail(Assert.java:96)\n\tat org.testng.Assert.failNotEquals(Assert.java:776)\n\tat org.testng.Assert.assertEqualsImpl(Assert.java:137)\n\tat org.testng.Assert.assertEquals(Assert.java:118)\n\tat org.testng.Assert.assertEquals(Assert.java:453)\n\tat",
" ----省略---",
"testCount": 4
}
]
Test collection
Test集合查询有2个条件 project 和 report
增加了db.test.createIndex({ report: -1, project: 1 }); 但查询仍然使用了report index
db.test.find({
project: ObjectId("6569b25255bbe231a87d1831"),
report: ObjectId("6569b25255bbe231a87d1832")
}).explain("executionStats");
[ { "key": { "_id": 1 }, "name": "_id_", "v": 2 }, { "key": { "report": 1, "level": 1, "startTime": 1 }, "name": "report_1_level_1_startTime_1", "v": 2 }, { "key": { "name": 1, "project": 1, "startTime": -1 }, "name": "name_1_project_1_startTime_-1", "v": 2 }]
参考
Mongodb doc: www.mongodb.com/zh-cn/docs/…
db.currentOp()查看慢当前正在执行的查询
