在当今复杂多变的数据环境中,图数据库以其独特的数据结构和强大的查询能力,成为了处理复杂关系数据的首选工具。然而,随着数据量的不断增长和业务数据安全性的要求日益提高,图数据库的备份与还原成为了数据库管理中不可或缺的一环。
备份,作为数据保护的基本手段。它不仅能够防止因硬件故障、软件错误、人为操作失误或外部攻击等原因导致的数据丢失,还能在数据损坏或系统崩溃时迅速恢复业务运行,确保数据的连续性和完整性,减少因数据问题导致的业务中断和经济损失。对于图数据库而言,由于其数据模型的特殊性,备份过程需要特别关注图的结构、节点和边的完整性,以确保在还原时能够恢复原有的图结构和数据关系。
还原,则是在数据丢失、损坏或系统需要迁移、升级等情况下,将已备份的数据恢复到数据库中的过程。
GDMBASE提供了一种快速灵活的备份与还原策略,本文将详细介绍如何在GDMBASE中备份与还原数据。
核心概念
在使用GDMBASE的备份还原之前,我们先了解一些概念。
- 全量备份:将整个图库的数据进行完全的备份。
- 增量备份:基于指定的备份任务,将该时刻之后发生变更的数据进行备份。
- 还原:基于已备份的文件,将数据还原至该备份时刻。
- 备份还原任务管理:支持对备份和还原的任务进行查询、终止等。
- GDMBASE根据备份数据范围把备份又分为全库备份和指定图备份。
一般地,备份的数据默认存放在../gdmbase/backup;同时也支持自定义存储路径,在gstore.ini中可进行配置,配置参数名为BACKUP_DIR,该参数支持相对路径或绝对路径;我们可以进入存放路径查看备份的数据,全量备份的数据存放在id_full_time,增量备份的数据存放在id_incr_time。
操作步骤
GDMBASE是在数据库不停机的情况下进行的数据备份还原,我们可以在cypher-shell中执行备份与还原命令来实现相应的操作。
全库备份
查看数据库中的图信息。
SHOW DATABASES;
+---------------------------------------------------------------------------------+
| name | segmentCount | replicaCount | status | capacity | size |
+---------------------------------------------------------------------------------+
| "sys" | 1 | 1 | "online" | "0.0MB" | "35.5MB" |
| "default" | 1 | 1 | "online" | "0.0MB" | "0.0B" |
| "characterMap" | 3 | 1 | "online" | "0.0MB" | "6.3GB" |
| "socialNetwork" | 3 | 1 | "online" | "0.0MB" | "826.5MB" |
+---------------------------------------------------------------------------------+
执行命令进行全库备份。
BACKUP ALL DATABASES FULL TO allFull01 COMMENT 全库备份;
查看备份进度。
SHOW BACKUPS YIELD id,name,baseId,comment,status,progress WHERE name = 'allFull01';
+--------------------------------------------------------------+
| id | name | baseId | comment | status | progress |
+--------------------------------------------------------------+
| 1 | "allFull01" | 0 | "全库备份" | "Running"| 21 |
+--------------------------------------------------------------+
当状态为Running,表示任务还未完成;此时可以主动终止备份任务,操作如下。
STOP BACKUP allFull01;
终止备份任务后的任务状态如下。
SHOW BACKUPS YIELD id,name,baseId,comment,status,progress WHERE name = 'allFull01';
+--------------------------------------------------------------+
| id | name | baseId | comment | status | progress |
+--------------------------------------------------------------+
| 1 | "allFull01" | 0 | "全库备份" | "Stop" | 42 |
+--------------------------------------------------------------+
未完成的备份,其文件不可用于还原,建议删除备份(此时系统将同时删除该备份文件)以减少数据冗余。
DROP BACKUP allFull01;
删除备份任务后,重新执行全库备份。
BACKUP ALL DATABASES FULL TO allFull02 COMMENT 全库备份;
查看备份进度。
SHOW BACKUPS YIELD id,name,baseId,comment,status,progress WHERE name = 'allFull02';
+--------------------------------------------------------------+
| id | name | baseId | comment | status | progress |
+--------------------------------------------------------------+
| 2 | "allFull02" | 0 | "全库备份" | "Finish" | 100 |
+--------------------------------------------------------------+
看到状态变成Finish,即备份成功。
全库还原
为了方便对比验证,我们先删除库中的characterMap图。
DROP DATABASE characterMap;
图删除后,查看到图库中已经不存在characterMap图。
SHOW DATABASES;
+---------------------------------------------------------------------------------+
| name | segmentCount | replicaCount | status | capacity | size |
+---------------------------------------------------------------------------------+
| "sys" | 1 | 1 | "online" | "0.0MB" | "35.5MB" |
| "default" | 1 | 1 | "online" | "0.0MB" | "0.0B" |
| "socialNetwork" | 3 | 1 | "online" | "0.0MB" | "826.5MB" |
+---------------------------------------------------------------------------------+
执行还原操作,将数据恢复到allFull02的状态。
RESTORE ALL DATABASES restoreFull01 FROM allFull02 COMMENT 全库还原;
查看还原进度。
SHOW RESTORES YIELD id,restoreName,backupName,status,comment WHERE restoreName = 'restoreFull01';
+-------------------------------------------------------------+
| id | restoreName | backupName | status | comment |
+-------------------------------------------------------------+
| 1 | "restoreFull01" | "allFull02" | "Running" | "全库还原" |
+-------------------------------------------------------------+
状态为Running时,表示数据正在还原,与终止备份相同,此时可以通过执行指令 stop restore restoreFull01 主动终止还原任务。 继续检查还原的任务状态。
SHOW RESTORES YIELD id,restoreName,backupName,status,comment WHERE restoreName = 'restoreFull01';
+-------------------------------------------------------------+
| id | restoreName | backupName | status | comment |
+-------------------------------------------------------------+
| 1 | "restoreFull01" | "allFull02" | "Finish" | "全库还原" |
+-------------------------------------------------------------+
看到状态变成Finish即还原成功。 检查还原后的数据。
SHOW DATABASES;
+---------------------------------------------------------------------------------+
| name | segmentCount | replicaCount | status | capacity | size |
+---------------------------------------------------------------------------------+
| "sys" | 1 | 1 | "online" | "0.0MB" | "35.5MB" |
| "default" | 1 | 1 | "online" | "0.0MB" | "0.0B" |
| "characterMap" | 3 | 1 | "online" | "0.0MB" | "6.3GB" |
| "socialNetwork" | 3 | 1 | "online" | "0.0MB" | "826.5MB" |
+---------------------------------------------------------------------------------+
由上可见,还原后的数据恢复到了allFull02的状态,且所有图状态都变成了可用状态。
指定图备份
切换到characterMap图,并查看该图数据统计信息。
CALL db.meta.count();
+-------------------------------------+
| type | count |
+-------------------------------------+
| "labels" | 28 |
| "relationshipTypes" | 28 |
| "labelIndexes" | 28 |
| "relationshipTypeIndexes" | 28 |
| "vertices" | 4606215 |
| "edges" | 6127699 |
+-------------------------------------+
备份characterMap图数据。
BACKUP DATABASE FULL TO graphFull01 COMMENT 指定图备份;
查看graphFull01的备份进度。
SHOW BACKUPS YIELD id,name,baseId,comment,status,progress WHERE name = 'graphFull01';
+------------------------------------------------------------------+
| id | name | baseId | comment | status | progress |
+------------------------------------------------------------------+
| 3 | "graphFull01" | 0 | "指定图备份" | "Finish" | 100 |
+------------------------------------------------------------------+
指定图还原
为了方便对比验证,我们先删除部分数据。
MATCH (n)-[r:glmac]->() RETURN count(r);
+----------+
| count(r) |
+----------+
| 217 |
+----------+
MATCH (n)-[r:glmac]->() DELETE r;
部分数据删除后,再次查看characterMap图的数据统计信息。
CALL db.meta.count();
+-------------------------------------+
| type | count |
+-------------------------------------+
| "labels" | 28 |
| "relationshipTypes" | 28 |
| "labelIndexes" | 28 |
| "relationshipTypeIndexes" | 28 |
| "vertices" | 4606215 |
| "edges" | 6127482 |
+-------------------------------------+
执行还原操作,将数据恢复到graphFull01的状态。
RESTORE DATABASE graphRestore01 FROM graphFull01 COMMENT 还原指定图数据;
查看还原进度。
SHOW RESTORES YIELD id,restoreName,backupName,status,comment WHERE restoreName = 'graphRestore01';
+--------------------------------------------------------------------------+
| id | restoreName | backupName | status | comment |
+--------------------------------------------------------------------------+
| 2 | "graphRestore01" | "graphFull01" | "Finish" | "还原指定图数据" |
+--------------------------------------------------------------------------+
检查characterMap图还原后的数据。
call db.meta.count();
+-------------------------------------+
| type | count |
+-------------------------------------+
| "labels" | 28 |
| "relationshipTypes" | 28 |
| "labelIndexes" | 28 |
| "relationshipTypeIndexes" | 28 |
| "vertices" | 4606215 |
| "edges" | 6127699 |
+-------------------------------------+
可见还原完成后,数据恢复到了graphFull01的状态。
增量备份还原
本次增量备份还原操作是基于socialNetwork图进行的,为验证还原到任意备份时刻的数据正确性,过程较为冗长复杂,为方便大家理解,先简单的介绍一下大致的操作步骤:
添加图片注释,不超过 140 字(可选)
增量备份还原具体的操作步骤如下:
检查数据: 查看每个顶点标签下的数据。
CALL db.meta.groupLabels();
+-------------------------+
| label | count |
+-------------------------+
| "Post" | 135701 |
| "Person" | 1528 |
| "Organisation" | 7955 |
| "Comment" | 151043 |
| "Tag" | 16080 |
| "Forum" | 13750 |
| "Place" | 1460 |
| "TagClass" | 71 |
+-------------------------+
查看每个边标签下的数据。
CALL db.meta.groupRelationshipTypes();
+-----------------------------------------+
| relationshipType | count |
+-----------------------------------------+
| "PersonLikesPost" | 47215 |
| "ForumHasTagTag" | 47697 |
| "OrganisationIsLocatedInPlace" | 7955 |
| "TagClassIsSubclassOfTagClass" | 70 |
| "PersonLikesComment" | 62225 |
| "CommentIsLocatedInPlace" | 151043 |
| "CommentReplyOfComment" | 76787 |
| "ForumHasMemberPerson" | 123268 |
| "PersonIsLocatedInPlace" | 1528 |
| "PostHasTagTag" | 51118 |
| "PostHasCreatorPerson" | 135701 |
| "CommentHasTagTag" | 191303 |
| "ForumHasModeratorPerson" | 13750 |
| "PersonHasInterestTag" | 35475 |
| "PersonStudyAtOrganisation" | 1209 |
| "PlaceIsPartOfPlace" | 1454 |
| "CommentHasCreatorPerson" | 151043 |
| "CommentReplyOfPost" | 74256 |
| "ForumContainerOfPost" | 135701 |
| "PersonWorkAtOrganisation" | 3313 |
| "PostIsLocatedInPlace" | 135701 |
| "TagHasTypeTagClass" | 16080 |
| "PersonKnowsPerson" | 14073 |
+-----------------------------------------+
全量备份数据:
BACKUP DATABASE FULL TO graphFull02 COMMENT 图全量备份;
修改标签名称、添加数据: 等待备份任务graphFull02完成后,在socialNetwork图中执行以下操作。 修改节点标签Person的名称并添加一条新的数据。
CALL db.schema.renameLabel('Person','人');
CREATE (n:人{ _PRIMARY_KEY:"a10",id:8888, firstName:"张", lastName:"三", gender:"男"});
数据修改后,再次查看characterMap图的数据统计信息。
CALL db.meta.groupLabels();
+-------------------------+
| label | count |
+-------------------------+
| "Post" | 135701 |
| "Forum" | 13750 |
| "人" | 1529 |
| "TagClass" | 71 |
| "Organisation" | 7955 |
| "Tag" | 16080 |
| "Comment" | 151043 |
| "Place" | 1460 |
+-------------------------+
基于全量备份数据执行第一次增量备份:
BACKUP DATABASE INCREMENT TO graphInc01 BASE ON graphFull02 COMMENT 修改标签并添加数据后增量备份;
修改属性值、删除指定边标签及数据: 等待备份任务graphInc01完成后,继续在socialNetwork图中执行以下操作。 标签人的属性gender的值为male时,修改属性值为男。
MATCH (n:人) WHERE n.gender = 'male' SET n.gender = '男' RETURN count(n);
+----------+
| count(n) |
+----------+
| 750 |
+----------+
修改属性值后,抽样查看数据。
MATCH (n:人) RETURN n.firstName,n.lastName,n.gender LIMIT 10;
+-------------------------------------+
| n.firstName | n.lastName | n.gender |
+-------------------------------------+
| "张" | "三" | "男" |
| "Hồ Chí" | "Loan" | "男" |
| "Chengdong" | "Wu" | "男" |
| "Eugen" | "Bajt" | "female" |
| "Mikhail" | "Sheik" | "男" |
| "Yang" | "Zhang" | "男" |
| "Maria" | "Onopka" | "female" |
| "Rahul" | "Khan" | "female" |
| "R." | "Sharma" | "female" |
| "Walter" | "Schmidt" | "female" |
+-------------------------------------+
删除PersonLikesPost边标签,同时删除数据。
CALL db.dropRelationshipType('PersonLikesPost');
删除完成后,再次查看数据的统计信息。
CALL db.meta.groupRelationshipTypes();
+-----------------------------------------+
| relationshipType | count |
+-----------------------------------------+
| "CommentIsLocatedInPlace" | 151043 |
| "CommentReplyOfPost" | 74256 |
| "ForumContainerOfPost" | 135701 |
| "ForumHasModeratorPerson" | 13750 |
| "PersonIsLocatedInPlace" | 1528 |
| "TagHasTypeTagClass" | 16080 |
| "PersonLikesComment" | 62225 |
| "CommentReplyOfComment" | 76787 |
| "PersonStudyAtOrganisation" | 1209 |
| "PostHasTagTag" | 51118 |
| "OrganisationIsLocatedInPlace" | 7955 |
| "CommentHasTagTag" | 191303 |
| "ForumHasTagTag" | 47697 |
| "PersonHasInterestTag" | 35475 |
| "TagClassIsSubclassOfTagClass" | 70 |
| "PersonKnowsPerson" | 14073 |
| "PostHasCreatorPerson" | 135701 |
| "CommentHasCreatorPerson" | 151043 |
| "ForumHasMemberPerson" | 123268 |
| "PersonWorkAtOrganisation" | 3313 |
| "PostIsLocatedInPlace" | 135701 |
| "PlaceIsPartOfPlace" | 1454 |
+-----------------------------------------+
基于第一次增量备份数据执行第二次增量备份:
BACKUP DATABASE INCREMENT TO graphInc02 BASE ON graphInc01 COMMENT
修改属性值和删除边标签及数据后增量备份; 等待备份任务graphInc02状态变成Finish,即备份成功。
还原到数据全量备份的状态:
RESTORE DATABASE graphRestore02 FROM graphFull02;
检查数据: 等待还原任务graphRestore02状态变成Finish后,查看库中数据。
CALL db.meta.groupLabels();
+-------------------------+
| label | count |
+-------------------------+
| "Post" | 135701 |
| "Place" | 1460 |
| "TagClass" | 71 |
| "Organisation" | 7955 |
| "Forum" | 13750 |
| "Person" | 1528 |
| "Tag" | 16080 |
| "Comment" | 151043 |
+-------------------------+
MATCH (n:Person) RETURN n.firstName,n.lastName,n.gender LIMIT 10;
+----------------------------------------+
| n.firstName | n.lastName | n.gender |
+----------------------------------------+
| "A." | "Reddy" | "female" |
| "A." | "Khan" | "female" |
| "K." | "Sen" | "female" |
| "Hiroshi" | "Yamada" | "male" |
| "Abhishek" | "Khan" | "male" |
| "James" | "Williams" | "male" |
| "Bingjian" | "Zhang" | "female" |
| "Gordon" | "Calvert" | "male" |
| "Heinrich" | "Herrmann" | "female" |
| "Konstantinos" | "Angello" | "female" |
+----------------------------------------+
CALL db.meta.groupRelationshipTypes();
+-----------------------------------------+
| relationshipType | count |
+-----------------------------------------+
| "PersonLikesPost" | 47215 |
| "ForumHasTagTag" | 47697 |
| "OrganisationIsLocatedInPlace" | 7955 |
| "TagClassIsSubclassOfTagClass" | 70 |
| "PersonLikesComment" | 62225 |
| "CommentIsLocatedInPlace" | 151043 |
| "CommentReplyOfComment" | 76787 |
| "ForumHasMemberPerson" | 123268 |
| "PersonIsLocatedInPlace" | 1528 |
| "PostHasTagTag" | 51118 |
| "PostHasCreatorPerson" | 135701 |
| "CommentHasTagTag" | 191303 |
| "ForumHasModeratorPerson" | 13750 |
| "PersonHasInterestTag" | 35475 |
| "PersonStudyAtOrganisation" | 1209 |
| "PlaceIsPartOfPlace" | 1454 |
| "CommentHasCreatorPerson" | 151043 |
| "CommentReplyOfPost" | 74256 |
| "ForumContainerOfPost" | 135701 |
| "PersonWorkAtOrganisation" | 3313 |
| "PostIsLocatedInPlace" | 135701 |
| "TagHasTypeTagClass" | 16080 |
| "PersonKnowsPerson" | 14073 |
+-----------------------------------------+
由上可见,数据还原到了graphFull02的状态,所有数据均未发生变更。
还原数据到第二次增量备份的状态:
RESTORE DATABASE graphRestore03 FROM graphInc02;
检查数据: 等待还原任务graphRestore03状态变成Finish后,查看库中数据。
CALL db.meta.groupLabels();
+-------------------------+
| label | count |
+-------------------------+
| "Comment" | 151043 |
| "Forum" | 13750 |
| "Place" | 1460 |
| "Organisation" | 7955 |
| "Tag" | 16080 |
| "人" | 1529 |
| "Post" | 135701 |
| "TagClass" | 71 |
+-------------------------+
MATCH (n:人) RETURN n.firstName,n.lastName,n.gender LIMIT 10;
+-------------------------------------+
| n.firstName | n.lastName | n.gender |
+-------------------------------------+
| "Ali" | "Diori" | "男" |
| "Anson" | "Chen" | "female" |
| "Alejandro" | "Sumac" | "female" |
| "Mahinda" | "Perera" | "男" |
| "Faisal" | "Malik" | "男" |
| "Zheng" | "Liu" | "female" |
| "Nikki" | "Reyes" | "female" |
| "Shweta" | "Khan" | "男" |
| "Chen" | "Liu" | "female" |
| "Boris" | "Golovin" | "female" |
+-------------------------------------+
CALL db.meta.groupRelationshipTypes();
+-----------------------------------------+
| relationshipType | count |
+-----------------------------------------+
| "ForumHasTagTag" | 47697 |
| "OrganisationIsLocatedInPlace" | 7955 |
| "TagClassIsSubclassOfTagClass" | 70 |
| "PersonLikesComment" | 62225 |
| "CommentIsLocatedInPlace" | 151043 |
| "CommentReplyOfComment" | 76787 |
| "ForumHasMemberPerson" | 123268 |
| "PersonIsLocatedInPlace" | 1528 |
| "PostHasTagTag" | 51118 |
| "PostHasCreatorPerson" | 135701 |
| "CommentHasTagTag" | 191303 |
| "ForumHasModeratorPerson" | 13750 |
| "PersonHasInterestTag" | 35475 |
| "PersonStudyAtOrganisation" | 1209 |
| "PlaceIsPartOfPlace" | 1454 |
| "CommentHasCreatorPerson" | 151043 |
| "CommentReplyOfPost" | 74256 |
| "ForumContainerOfPost" | 135701 |
| "PersonWorkAtOrganisation" | 3313 |
| "PostIsLocatedInPlace" | 135701 |
| "TagHasTypeTagClass" | 16080 |
| "PersonKnowsPerson" | 14073 |
+-----------------------------------------+
由上可见,数据还原到了graphInc02的状态,所有变更数据也都还原成功了。
还原数据到第一次增量备份的状态:
RESTORE DATABASE graphRestore04 FROM graphInc01;
检查数据: 等待还原任务graphRestore04状态变成Finish后,查看库中数据。
CALL db.meta.groupLabels();
+-------------------------+
| label | count |
+-------------------------+
| "人" | 1529 |
| "TagClass" | 71 |
| "Organisation" | 7955 |
| "Tag" | 16080 |
| "Post" | 135701 |
| "Place" | 1460 |
| "Comment" | 151043 |
| "Forum" | 13750 |
+-------------------------+
MATCH (n:人) RETURN n.firstName,n.lastName,n.gender LIMIT 10;
+-------------------------------------+
| n.firstName | n.lastName | n.gender |
+-------------------------------------+
| "Ali" | "Diori" | "male" |
| "Anson" | "Chen" | "female" |
| "Alejandro" | "Sumac" | "female" |
| "Mahinda" | "Perera" | "male" |
| "Faisal" | "Malik" | "male" |
| "Zheng" | "Liu" | "female" |
| "Nikki" | "Reyes" | "female" |
| "Shweta" | "Khan" | "male" |
| "Chen" | "Liu" | "female" |
| "Boris" | "Golovin" | "female" |
+-------------------------------------+
CALL db.meta.groupRelationshipTypes();
+-----------------------------------------+
| relationshipType | count |
+-----------------------------------------+
| "PersonLikesPost" | 47215 |
| "ForumHasTagTag" | 47697 |
| "OrganisationIsLocatedInPlace" | 7955 |
| "TagClassIsSubclassOfTagClass" | 70 |
| "PersonLikesComment" | 62225 |
| "CommentIsLocatedInPlace" | 151043 |
| "CommentReplyOfComment" | 76787 |
| "ForumHasMemberPerson" | 123268 |
| "PersonIsLocatedInPlace" | 1528 |
| "PostHasTagTag" | 51118 |
| "PostHasCreatorPerson" | 135701 |
| "CommentHasTagTag" | 191303 |
| "ForumHasModeratorPerson" | 13750 |
| "PersonHasInterestTag" | 35475 |
| "PersonStudyAtOrganisation" | 1209 |
| "PlaceIsPartOfPlace" | 1454 |
| "CommentHasCreatorPerson" | 151043 |
| "CommentReplyOfPost" | 74256 |
| "ForumContainerOfPost" | 135701 |
| "PersonWorkAtOrganisation" | 3313 |
| "PostIsLocatedInPlace" | 135701 |
| "TagHasTypeTagClass" | 16080 |
| "PersonKnowsPerson" | 14073 |
+-----------------------------------------+
由上可见,数据还原到了graphInc01的状态,之前的数据均被还原,不存在该时刻之后发生变更的数据。 可见增量备份可以基于任意已备份的数据执行,还原能将数据恢复到任意已备份的状态。
常见问题
- 集群环境发生变更将会导致现有的备份包不可用,需要重新进行全量备份。这里的集群变更包括存储节点扩容/缩容、段迁移/拆分、副本扩容等操作。
- 指定图或者整个数据库被还原到任意时刻后,相当于构建了一个全新的图或者库,不能再基于原有的备份任务执行增量备份。
- 多副本时数据备份只会备份主副本数据,数据还原后该图变成单副本,想要恢复多副本存储,需手动扩容副本数。
- 如果备份任务正在进行中,不能直接删除备份,需要先手动终止备份任务,再执行删除操作。
- 人为或者故障原因导致还原中断,对应图状态会变成readonly,此时须重新还原数据至任务完成,图状态恢复为online即可。
总结
GDMBASE数据库的备份还原是保障数据安全性和业务连续性的重要手段。通过制定合理的备份策略和执行严格的备份还原流程,可以确保在系统发生故障时能快速速恢复数据并减少损失。希望这篇文章能够帮助读者更好地理解和应用GDMBASE的备份与还原策略。