1、Redis数据迁移工具小结：
- redis-dump: redis-dump 是一个将redis数据导入/导出为json格式数据的小工具。
- redis-port : 豌豆荚开发，最初是 codis 项目相关工具，支持实时同步redis 数据到 codis/redis等中。
- redis-migrate-tool： 是唯品会开源的redis数据迁移工具，可用于redis集群间的数据在线迁移。（不支持redis-4.0以上版本）
- redis-shake : 是阿里云的redis数据同步的工具。支持redis主从->redis-cluster，cluster<->cluster等多种redis架构的数据同步。
- 其他：Rump、redis-migration支持版本较老，没有找到官方部署和管理的文档。
2、Redis-shake 使用
Redis-shake下载地址：github.com/alibaba/Red…
wget https://github.com/alibaba/RedisShake/releases/download/release-v2.0.3-20200724/redis-shake-v2.0.3.tar.gz
tar -zxvf redis-shake-v2.0.2.tar.gz
1、配置文件
| 配置文件 | 参数 |
|---|---|
| source.type = | standalone：单节点、主从模式 sentinel：哨兵模式 cluster：集群模式 proxy：代理模式 |
| source.address = | 源IP地址：端口号 |
| source.password_raw = | 源redis密码 |
| source.auth_type = auth | 源redis认证类型，不要更改 |
| target.type = | standalone：单节点、主从模式 sentinel：哨兵模式 cluster：集群模式 proxy：代理模式 |
| target.address = | 目标IP地址：端口号 |
| target.password_raw = | 目标Redis密码 |
| target.auth_type = auth | 目标redis认证类型，不要更改 |
| key_exists = | 当源目的有重复key，是否进行覆写 rewrite表示源端覆盖目的端。 none表示一旦发生进程直接退出。 ignore表示保留目的端key，忽略源端的同步key。该值在rump模式下没有用。 |
| log.file = | 日志文件，不配置将打印到stdout |
| log.level = info | "none", "error", "warn", "info", "debug" 默认为info |
| pid_path = | pid文件路径 |
同步数据配置文件举例：
###单个节点到单个节点配置举例
source.type: standalone
source.address: 10.1.1.1:20441
source.password_raw: 12345
target.type: standalone
target.address: 10.1.1.1:20551
target.password_raw: 12345
###集群版cluster到集群版cluster配置举例
source.type: cluster
source.address: 10.1.1.1:20441;10.1.1.1:20443;10.1.1.1:20445
source.password_raw: 12345
target.type: cluster
target.address: 10.1.1.1:20551;10.1.1.1:20553;10.1.1.1:20555
target.password_raw: 12345
###自动发现机制
target.address: @10.1.1.1:20551 #将会自动探测到10.1.1.1:20551集群下的所有节点，并写入所有master
2、启动
启动二进制文件： ./redis-shake.linux -conf redis-shake.conf -type xxx
xxx为sync, restore, dump, decode, rump其中之一。
- decode: 把Redis RDB文件解析成人类可读的文件格式
- restore: 把Rdis RDB文件作为数据源，恢复到目标Redis实例中
- dump: 对源Redis实例中的数据dump到RDB文件中
- sync: 基于Redis sync/psync命令从源redis实例同步到目标Redis实例，该模式包含全量同步和增量同步两个阶段。（通过默认Redis Slave来实现）
- rump: 基于Redis scan命令的方式从源Redis同步到目标Redis实例。只支持全量同步，这种方式通常适用于源redis不支持sync/psync命令的场景
mac下请使用redis-shake.darwin，windows请用redis-shake.windows。
3、日志信息
同步分为三个阶段：
1、等待源端save rdb完毕
2020/10/14 16:45:05 [WARN] source.auth_type[auth] != auth
2020/10/14 16:45:05 [WARN] target.auth_type[auth] != auth
2020/10/14 16:45:05 [INFO] input password is empty, skip auth address[10.224.192.210:9000] with type[auth].
2020/10/14 16:45:05 [INFO] input password is empty, skip auth address[10.224.192.210:9000] with type[auth].
2020/10/14 16:45:05 [INFO] source rdb[10.224.192.210:9000] checksum[yes]
2020/10/14 16:45:05 [WARN]
______________________________
\ \ _ ______ |
\ \ / ___-=O'/|O'/__|
\ RedisShake, here we go !! _______\ / | / )
/ / '/-==__ _/__|/__=-| -GM
/ Alibaba Cloud / * \ | |
/ / (o)
------------------------------
if you have any problem, please visit https://github.com/alibaba/RedisShake/wiki/FAQ
2020/10/14 16:45:05 [INFO] redis-shake configuration: {"ConfVersion":1,"Id":"redis-shake","LogFile":"/home/redis-shake-v2.0.2/conf2/redis-shake.log","LogLevel":"info","SystemProfile":9310,"HttpProfile":9320,"Parallel":32,"SourceType":"standalone","SourceAddress":"10.224.192.210:9000","SourcePasswordRaw":"***","SourcePasswordEncoding":"***","SourceAuthType":"auth","SourceTLSEnable":false,"SourceRdbInput":["local"],"SourceRdbParallel":1,"SourceRdbSpecialCloud":"","TargetAddress":"10.224.192.209:9003","TargetPasswordRaw":"***","TargetPasswordEncoding":"***","TargetDBString":"-1","TargetAuthType":"auth","TargetType":"standalone","TargetTLSEnable":false,"TargetRdbOutput":"local_dump","TargetVersion":"5.0.9","FakeTime":"","KeyExists":"rewrite","FilterDBWhitelist":[],"FilterDBBlacklist":[],"FilterKeyWhitelist":[],"FilterKeyBlacklist":[],"FilterSlot":[],"FilterLua":false,"BigKeyThreshold":524288000,"Metric":true,"MetricPrintLog":false,"SenderSize":104857600,"SenderCount":4095,"SenderDelayChannelSize":65535,"KeepAlive":0,"PidPath":"/home/redis-shake-v2.0.2/conf2/redis-shake.pid","ScanKeyNumber":50,"ScanSpecialCloud":"","ScanKeyFile":"","Qps":200000,"ResumeFromBreakPoint":false,"Psync":true,"NCpu":0,"HeartbeatUrl":"","HeartbeatInterval":10,"HeartbeatExternal":"","HeartbeatNetworkInterface":"","ReplaceHashTag":false,"ExtraInfo":false,"SockFileName":"","SockFileSize":0,"FilterKey":null,"FilterDB":"","Rewrite":false,"SourceAddressList":["10.224.192.210:9000"],"TargetAddressList":["10.224.192.209:9003"],"SourceVersion":"5.0.9","HeartbeatIp":"127.0.0.1","ShiftTime":0,"TargetReplace":true,"TargetDB":-1,"Version":"bugfix-2.0.2,2d6ac6382b20989c22eb70130c78a1dd036cee66,go1.10.3,2020-05-06_11:07:40","Type":"sync"}
2020/10/14 16:45:05 [INFO] DbSyncer[0] starts syncing data from 10.224.192.210:9000 to [10.224.192.209:9003] with http[9321], enableResumeFromBreakPoint[false], slot boundary[-1, -1]
2020/10/14 16:45:05 [INFO] input password is empty, skip auth address[10.224.192.210:9000] with type[auth].
2020/10/14 16:45:05 [INFO] DbSyncer[0] psync connect '10.224.192.210:9000' with auth type[auth] OK!
2020/10/14 16:45:05 [INFO] DbSyncer[0] psync send listening port[9320] OK!
2020/10/14 16:45:05 [INFO] DbSyncer[0] try to send 'psync' command: run-id[?], offset[-1]
2020/10/14 16:45:05 [INFO] Event:FullSyncStart Id:redis-shake
2020/10/14 16:45:05 [INFO] DbSyncer[0] psync runid = 0c082937e64e59d3d8b10065bc33c4e4364c5451, offset = 168546, fullsync
2020/10/14 16:45:06 [INFO] DbSyncer[0] +
2020/10/14 16:45:07 [INFO] DbSyncer[0] -
2020/10/14 16:45:07 [INFO] DbSyncer[0] +
2、全量同步、显示百分比
2020/10/14 16:47:15 [INFO] Aux information key:redis-ver value:5.0.9
2020/10/14 16:47:15 [INFO] Aux information key:redis-bits value:64
2020/10/14 16:47:15 [INFO] Aux information key:ctime value:1602665105
2020/10/14 16:47:15 [INFO] Aux information key:used-mem value:9299464672
2020/10/14 16:47:15 [INFO] Aux information key:repl-stream-db value:0
2020/10/14 16:47:15 [INFO] Aux information key:repl-id value:0c082937e64e59d3d8b10065bc33c4e4364c5451
2020/10/14 16:47:15 [INFO] Aux information key:repl-offset value:168546
2020/10/14 16:47:15 [INFO] Aux information key:aof-preamble value:0
2020/10/14 16:47:15 [INFO] db_size:4616790 expire_size:0
2020/10/14 16:47:16 [INFO] DbSyncer[0] total = 4.94GB - 13.99MB [ 0%] entry=12289
2020/10/14 16:47:17 [INFO] DbSyncer[0] total = 4.94GB - 35.15MB [ 0%] entry=32012
2020/10/14 16:47:18 [INFO] DbSyncer[0] total = 4.94GB - 55.77MB [ 1%] entry=51438
。。。。。。。。。。。。。
2020/10/14 16:51:01 [INFO] DbSyncer[0] total = 4.94GB - 4.83GB [ 97%] entry=4517104
2020/10/14 16:51:02 [INFO] DbSyncer[0] total = 4.94GB - 4.86GB [ 98%] entry=4548538
2020/10/14 16:51:03 [INFO] DbSyncer[0] total = 4.94GB - 4.89GB [ 99%] entry=4575792
2020/10/14 16:51:04 [INFO] DbSyncer[0] total = 4.94GB - 4.92GB [ 99%] entry=4603839
2020/10/14 16:51:05 [INFO] DbSyncer[0] total = 4.94GB - 4.94GB [100%] entry=4616790
3、增量同步，显示到 sync rdb done后，当前dbSyncer进入增量同步
2020/10/14 16:51:05 [INFO] DbSyncer[0] sync rdb done
2020/10/14 16:51:05 [INFO] input password is empty, skip auth address[10.224.192.210:9000] with type[auth].
2020/10/14 16:51:05 [INFO] DbSyncer[0] FlushEvent:IncrSyncStart Id:redis-shake
2020/10/14 16:51:06 [INFO] DbSyncer[0] sync: +forwardCommands=35 +filterCommands=0 +writeBytes=140
2020/10/14 16:51:07 [INFO] DbSyncer[0] sync: +forwardCommands=1 +filterCommands=0 +writeBytes=4
2020/10/14 16:51:08 [INFO] DbSyncer[0] sync: +forwardCommands=0 +filterCommands=0 +writeBytes=0
2020/10/14 16:51:09 [INFO] DbSyncer[0] sync: +forwardCommands=0 +filterCommands=0 +writeBytes=0
只要不停止redis-shake进程，会一直进行数据同步
2020/10/14 17:08:48 [INFO] DbSyncer[0] sync: +forwardCommands=1329 +filterCommands=0 +writeBytes=177795
2020/10/14 17:08:49 [INFO] DbSyncer[0] sync: +forwardCommands=1134 +filterCommands=0 +writeBytes=151687
提醒： 被同步的服务器需要配置localhost:9003> CONFIG SET stop-writes-on-bgsave-error no 才可以被全量和增量同步完成
