使用Redis自动部署cluster集群

哭也要有君子的风度
164 阅读6分钟

持续创作，加速成长！这是我参与「掘金日新计划 · 10 月更文挑战」的第7天，点击查看活动详情

使用redis工具自动部署cluster集群

目录

环境准备 1.安装ruby环境 2.使用redis-trib自动部署cluster集群 2.1.所有节点安装redis 2.2.使用redis-trib部署cluster集群 2.3.手动调整三主三从交叉复制 2.4.查看集群完整性 2.5.验证hash分配是否均匀 2.6.查看集群分配的误差值

手动搭建集群便于理解集群创建的流程和细节，不过手动搭建集群需要很多步骤，当集群节点众多时，必然会加大 搭建集群的复杂度和运维成本，因此官方提供了redis-trib.rb的工具方便我们快速搭建集群

redis-trib.rb是采用ruby实现的redis集群管理工具，内部通过cluster相关命令帮我们简化集群创建、检查、 槽迁移和均衡等常见运维操作，使用前要安装ruby依赖环境

redis-trib.rb无法实现所有节点都交叉复制，总会有一个节点不交叉，因此在安装完cluster以后，需要手动 调整交叉

redis必须是6.2以上 ruby必须是2.3以上（centos7）

环境准备

IP主机名端口号节点
192.168.146.138redis-16380master
192.168.146.138redis-16381slave-->redis-2
192.168.146.139redis-26380master
192.168.146.139redis-26381slave-->redis-3
192.168.146.170redis-36380master
192.168.146.170redis-36381slave-->redis-1

1.安装ruby环境

只在使用redis-trib的机器上安装即可

1.安装ruby管理工具
[root@redis-1 ~]# yum -y install rubygems

2.移除官网源
[root@redis-1 ~]# gem sources --remove https://rubygems.org/
https://rubygems.org/ removed from sources

3.增加阿里云源
[root@redis-1 ~]# gem sources -a http://mirrors.aliyun.com/rubygems/
http://mirrors.aliyun.com/rubygems/ added to sources

4.更新缓存
[root@redis-1 ~]# gem update --system
ruby2.3.0以下版本执行会报错


[root@redis-1 ~]# gem update --system
Updating rubygems-update
Fetching: rubygems-update-3.3.20.gem (100%)
Successfully installed rubygems-update-3.3.20
Parsing documentation for rubygems-update-3.3.20
Installing ri documentation for rubygems-update-3.3.20

5.安装ruby支持redis的插件
[root@redis-1 /data]# gem install redis -v 3.3.5
Fetching: redis-3.3.5.gem (100%)
Successfully installed redis-3.3.5
Parsing documentation for redis-3.3.5
Installing ri documentation for redis-3.3.5
1 gem installed

图片.png

2.使用redis-trib自动部署cluster集群

2.1.所有节点安装redis

1.创建部署路径
mkdir -p /data/redis_cluster/redis_{6380,6381}/{conf,data,logs,pid}

==========首先必须安装了redis=============
wget https://repo.huaweicloud.com/redis/redis-5.0.5.tar.gz
tar xf redis-5.0.5.tar.gz -C /data/redis_cluster/
cd /data/redis_cluster/
ln -s redis-5.0.5/ redis
cd redis/src/
make && make install
========================================

2.准备配置文件
cat > /data/redis_cluster/redis_6380/conf/redis_6380.conf <<EOF
bind $(ifconfig |grep 192.168.146.255 |awk -F ' ' '{print $2}')
port 6380  
daemonize yes
logfile /data/redis_cluster/redis_6380/logs/redis_6380.log
pidfile /data/redis_cluster/redis_6380/pid/redis_6380.log
dbfilename "redis_6380.rdb"
dir /data/redis_cluster/redis_6380/data
cluster-enabled yes
cluster-config-file node_6380.conf
cluster-node-timeout 15000
save 60 10000
save 300 10
save 900 1
EOF

cat > /data/redis_cluster/redis_6381/conf/redis_6381.conf <<EOF
bind $(ifconfig |grep 192.168.146.255 |awk -F ' ' '{print $2}')
port 6381  
daemonize yes
logfile /data/redis_cluster/redis_6381/logs/redis_6381.log
pidfile /data/redis_cluster/redis_6381/pid/redis_6381.log
dbfilename "redis_6381.rdb"
dir /data/redis_cluster/redis_6381/data
cluster-enabled yes
cluster-config-file node_6381.conf
cluster-node-timeout 15000
save 60 10000
save 300 10
save 900 1
EOF

3.启动redis
[root@redis-1 ~]# ./redis_shell.sh start 6380
redis 6380实例启动成功!
tcp        0      0 192.168.146.138:16380   0.0.0.0:*               LISTEN      -
tcp        0      0 192.168.146.138:6380    0.0.0.0:*               LISTEN      68861/redis-server
[root@redis-1 ~]# ./redis_shell.sh start 6381
redis 6381实例启动成功!
tcp        0      0 192.168.146.138:16381   0.0.0.0:*               LISTEN      69065/redis-server
tcp        0      0 192.168.146.138:6381    0.0.0.0:*               LISTEN      69065/redis-server

图片.png

图片.png

2.2.使用redis-trib部署cluster集群

语法格式：./redis-trib.rb create --replicas 每个主节点的副本数量 主库ip:端口 从库ip端口 cluster节点地址

--replicas //指定主库的副本数量，也就是从库数量

使用redis-trib安装的cluster集群，总会有一个节点不是交叉复制的，需要手动调整，因为trib也是根据节点地址交叉对应，到了最后一个机器已经没有第二个可以与它交叉的机器，它只能和自己去复制

[root@redis-1 ~]# cd /data/redis_cluster/redis-5.0.5/src
[root@redis-1 /data/redis_cluster/redis-5.0.5/src]# redis-cli --cluster create 192.168.146.138:6380 192.168.146.139:6380 192.168.146.170:6380 192.168.146.138:6381 192.168.146.139:6381 192.168.146.170:6381 --cluster-replicas 1

安装完查看集群准备已经是可用的
[root@redis-1 /data/redis_cluster/redis-5.0.5/src]# redis-cli -h 192.168.146.138 -p6380
192.168.146.138:6380> CLUSTER INFO
cluster_state:ok                  #OK的状态
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:58
cluster_stats_messages_pong_sent:62
cluster_stats_messages_sent:120
cluster_stats_messages_ping_received:57
cluster_stats_messages_pong_received:58
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:120

图片.png

复制关系

图片.png

2.3.手动调整三主三从交叉复制

这里工具创建出来的集群就是高可用的状态，如果我们的状态不是高可用，我们可以手动去调整交叉复制关系

举个例子：redis-3的6381要成为redis-1的6380的主库，需要去redis-3的6381redis交互式操作
1.获取主节点的信息
[root@redis-1 ~]# redis-cli -h 192.168.146.138 -p 6381 cluster nodes | grep 6380 | awk '{print $1.$2}'
812ae8ccf55d8994f1f9d30a20f6cff42fb24b4a192.168.146.138:6380
ce75dacf45d3ad4f852b7fb4d359a295b8a2bcdd192.168.146.139:6380
b61b8d0421b94b9de7267dda6c6f401a42622047192.168.146.170:6380

2.配置三主三从交叉复制
[root@redis-1 ~]# redis-cli -h 192.168.146.138 -p 6381
192.168.146.138:6381> CLUSTER REPLICATE ce75dacf45d3ad4f852b7fb4d359a295b8a2bcdd
OK
[root@redis-2 ~]# redis-cli -h 192.168.146.139 -p 6381
192.168.146.139:6381> CLUSTER REPLICATE 812ae8ccf55d8994f1f9d30a20f6cff42fb24b4a
OK
[root@redis-3 ~]# redis-cli -h 192.168.146.170 -p 6381
192.168.146.170:6381> CLUSTER REPLICATE b61b8d0421b94b9de7267dda6c6f401a42622047
OK

3.查看集群信息已经交叉复制
[root@redis-1 ~]# redis-cli -h 192.168.146.138 -p 6381 cluster nodes 
812ae8ccf55d8994f1f9d30a20f6cff42fb24b4a 192.168.146.170:6380 master - 0 1612342768677 3 connected 10923-16383
bdd20b03b573b2def6a9ee5053a75867709fc908 192.168.146.138:6381 myself,slave ce75dacf45d3ad4f852b7fb4d359a295b8a2bcdd 0 0 4 connected
9b7641253ea66073d865accdd4460d2877f9ff5d 192.168.146.139:6381 slave 812ae8ccf55d8994f1f9d30a20f6cff42fb24b4a 0 1612342767669 5 connected
ce75dacf45d3ad4f852b7fb4d359a295b8a2bcdd 192.168.146.139:6380 master - 0 1612342766658 2 connected 5461-10922
904a0109976cae38e5a3059fd70ce2727a0ed8fb 192.168.146.170:6381 slave b61b8d0421b94b9de7267dda6c6f401a42622047 0 1612342769686 3 connected
b61b8d0421b94b9de7267dda6c6f401a42622047 192.168.146.138:6380 master - 0 1612342770189 1 connected 0-5460

2.4.查看集群完整性

[root@redis-1 ~]# cd /data/redis_cluster/redis-5.0.5/src/
[root@redis-1 /data/redis_cluster/redis-5.0.5/src]# redis-cli --cluster check 192.168.146.138:6380
192.168.146.138:6380 (41a548f6...) -> 0 keys | 5461 slots | 1 slaves.
192.168.146.139:6380 (42c4640e...) -> 0 keys | 5462 slots | 1 slaves.
192.168.146.170:6380 (1d32e8a7...) -> 0 keys | 5461 slots | 1 slaves.
[OK] 0 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 192.168.146.138:6380)
M: 41a548f683aeb5ab5c69ce9d07627559ec62545c 192.168.146.138:6380
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
M: 42c4640e45dd2177265678dfdf57850ca43d6afa 192.168.146.139:6380
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: d3dcbb5c1e1d7ce849c244f1cd7bb32e381fcaa8 192.168.146.139:6381
   slots: (0 slots) slave
   replicates 41a548f683aeb5ab5c69ce9d07627559ec62545c
S: da889dbad8fb7c822185838e2c6a246ef22a1a56 192.168.146.170:6381
   slots: (0 slots) slave
   replicates 42c4640e45dd2177265678dfdf57850ca43d6afa
S: 85ea3692c331d15ecf5e42a24a969d0b47c11c40 192.168.146.138:6381
   slots: (0 slots) slave
   replicates 1d32e8a767bbff81423dde229389b358b9744697
M: 1d32e8a767bbff81423dde229389b358b9744697 192.168.146.170:6380
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

#集群没问题就会输出ok

2.5.验证hash分配是否均匀

1.插入1000条数据
[root@redis-1 ~]# for i in {1..1000}
do
redis-cli -c -h 192.168.146.138 -p 6380 set wang_${i} zhen_${i}
done

2.查看每个节点的数据量
[root@redis-1 ~]# redis-cli -h 192.168.146.138 -p 6380 dbsize
(integer) 330
[root@redis-1 ~]# redis-cli -h 192.168.146.139 -p 6380 dbsize
(integer) 334
[root@redis-1 ~]# redis-cli -h 192.168.146.170 -p 6380 dbsize
(integer) 336

2.6.查看集群分配的误差值

[root@redis-1 /data/redis_cluster/redis-5.0.5/src]# redis-cli --cluster rebalance 192.168.146.138:6380
>>> Performing Cluster Check (using node 192.168.146.138:6380)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
*** No rebalancing needed! All nodes are within the 2.00% threshold.

图片.png

图片.png

avatar
文章
阅读
粉丝