Linux上安装Redis并搭建集群(二)

194 阅读15分钟

上一篇:Linux上安装Redis并搭建集群(一)

二、搭建Redis集群

1、安装ruby相关环境和包(ruby的安装踩了很多坑,所以放在首位先安装)

(1).安装ruby环境

yum install ruby
yum install rubygems

(2).安装ruby包

gem install redis-5.0.3.gem
2、创建集群节点文件目录

在/usr/local/redis-cluster目录下创建 6001 6002 6003 6004 6005 6006目录文件

mkdir 6001 6002 6003 6004 6005 6006
3、复制配置文件

从已经安装的redis中,复制redis.conf文件到上面的6个文件目录

 cp  redis.conf  /usr/local/redis-cluster/6001/6001.conf 

修改redis配置文件

# ip地址
bind 192.168.0.105
# 端口号
port 6001
# redis后台运行
daemonize yes
# pid
pidfile /usr/local/redis-cluster/6001/redis_6001.pid
# 开启集群
cluster-enabled yes
# 集群的配置 配置文件首次启动自动生成
cluster-config-file /user.local/redis-cluster/6001/nodes_6001.conf
# 集群超时时间
cluster-node-timeout 15000
# aof日志开启,每次写操作都记录一条日志
appendonly yes
appendfilename "appendonly-6001.aof"
loglevel notice
logfile ""

同理,同上修改其他文件夹的节点配置文件。

[root@localhost local]# cd redis-cluster/
[root@localhost redis-cluster]# ll
总用量 24
drwxr-xr-x. 2 root root 4096 2月   1 06:55 6001
drwxr-xr-x. 2 root root 4096 2月   1 06:56 6002
drwxr-xr-x. 2 root root 4096 2月   1 06:56 6003
drwxr-xr-x. 2 root root 4096 2月   1 06:56 6004
drwxr-xr-x. 2 root root 4096 2月   1 06:56 6005
drwxr-xr-x. 2 root root 4096 2月   1 06:57 6006
[root@localhost redis-cluster]# cd 6001
[root@localhost 6001]# ll
总用量 64
-rw-r--r--. 1 root root 62156 2月   1 06:55 6001.conf
[root@localhost 6001]# vim 6001.conf 
[root@localhost 6001]# vim 6001.conf 
[root@localhost 6001]# cp 6001.conf /usr/local/redis-cluster/6002/6002.conf
cp:是否覆盖"/usr/local/redis-cluster/6002/6002.conf"? y
[root@localhost 6001]# cp 6001.conf  /usr/local/redis-cluster/6003/6003.conf
cp:是否覆盖"/usr/local/redis-cluster/6003/6003.conf"? y
[root@localhost 6001]# cp 6001.conf  /usr/local/redis-cluster/6004/6004.conf
cp:是否覆盖"/usr/local/redis-cluster/6004/6004.conf"? y
[root@localhost 6001]# cp 6001.conf  /usr/local/redis-cluster/6005/6005.conf
cp:是否覆盖"/usr/local/redis-cluster/6005/6005.conf"? y
[root@localhost 6001]# cp 6001.conf  /usr/local/redis-cluster/6006/6006.conf
cp:是否覆盖"/usr/local/redis-cluster/6006/6006.conf"? y
[root@localhost 6001]# cd ..
[root@localhost redis-cluster]# cd 6002
[root@localhost 6002]# vim 6002.conf 
4、各个节点配置完成,开始启动服务
[root@localhost local]# cd redis-5.0.3
[root@localhost redis-5.0.3]# ls
00-RELEASENOTES  CONTRIBUTING  deps     Makefile   README.md   runtest          runtest-sentinel  src    utils
BUGS             COPYING       INSTALL  MANIFESTO  redis.conf  runtest-cluster  sentinel.conf     tests
[root@localhost redis-5.0.3]# cd src
[root@localhost src]# ./redis-server /usr/local/redis-cluster/6001/6001.conf
6719:C 01 Feb 2019 07:47:41.749 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
6719:C 01 Feb 2019 07:47:41.749 # Redis version=5.0.3, bits=32, commit=00000000, modified=0, pid=6719, just started
6719:C 01 Feb 2019 07:47:41.749 # Configuration loaded
[root@localhost src]# ./redis-server /usr/local/redis-cluster/6002/6002.conf
6724:C 01 Feb 2019 07:47:55.045 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
6724:C 01 Feb 2019 07:47:55.045 # Redis version=5.0.3, bits=32, commit=00000000, modified=0, pid=6724, just started
6724:C 01 Feb 2019 07:47:55.045 # Configuration loaded
[root@localhost src]# ./redis-server /usr/local/redis-cluster/6003/6003.conf
6729:C 01 Feb 2019 07:48:08.772 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
6729:C 01 Feb 2019 07:48:08.772 # Redis version=5.0.3, bits=32, commit=00000000, modified=0, pid=6729, just started
6729:C 01 Feb 2019 07:48:08.772 # Configuration loaded
[root@localhost src]# ./redis-server /usr/local/redis-cluster/6004/6004.conf
6734:C 01 Feb 2019 07:48:21.169 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
6734:C 01 Feb 2019 07:48:21.169 # Redis version=5.0.3, bits=32, commit=00000000, modified=0, pid=6734, just started
6734:C 01 Feb 2019 07:48:21.169 # Configuration loaded
[root@localhost src]# ./redis-server /usr/local/redis-cluster/6005/6005.conf
6740:C 01 Feb 2019 07:48:33.366 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
6740:C 01 Feb 2019 07:48:33.366 # Redis version=5.0.3, bits=32, commit=00000000, modified=0, pid=6740, just started
6740:C 01 Feb 2019 07:48:33.366 # Configuration loaded
[root@localhost src]# ./redis-server /usr/local/redis-cluster/6006/6006.conf
6745:C 01 Feb 2019 07:48:52.011 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
6745:C 01 Feb 2019 07:48:52.011 # Redis version=5.0.3, bits=32, commit=00000000, modified=0, pid=6745, just started
6745:C 01 Feb 2019 07:48:52.011 # Configuration loaded

启动服务用的是redis解压包下的src目录中的redis-server启动

[root@localhost src]# ps -ef | grep redis
root      6018     1  0 06:45 ?        00:00:07 ./bin/redis-server 127.0.0.1:6379
root      6720     1  0 07:47 ?        00:00:00 ./redis-server 192.168.0.105:6001 [cluster]           
root      6725     1  0 07:47 ?        00:00:00 ./redis-server 192.168.0.105:6002 [cluster]           
root      6730     1  0 07:48 ?        00:00:00 ./redis-server 192.168.0.105:6003 [cluster]           
root      6735     1  0 07:48 ?        00:00:00 ./redis-server 192.168.0.105:6004 [cluster]           
root      6741     1  0 07:48 ?        00:00:00 ./redis-server 192.168.0.105:6005 [cluster]           
root      6746     1  0 07:48 ?        00:00:00 ./redis-server 192.168.0.105:6006 [cluster]           
root      6756  6039  0 07:50 pts/0    00:00:00 grep redis
5、执行redis的创建集群命令创建集群(注意ip地址和端口号)
[root@localhost src]# ./redis-trib.rb create -- replicas 1 192.168.0.105:6001 192.168.0.105:6002 192.168.0.105:6003 192.168.0.105:6004 192.168.0.105:6005 192.168.0.105:6006

报出以下警告:

WARNING: redis-trib.rb is not longer available!
You should use redis-cli instead.

All commands and features belonging to redis-trib.rb have been moved
to redis-cli.
In order to use them you should call redis-cli with the --cluster
option followed by the subcommand name, arguments and options.

Use the following syntax:
redis-cli --cluster SUBCOMMAND [ARGUMENTS] [OPTIONS]

Example:
redis-cli --cluster create replicas 1 192.168.0.105:6001 192.168.0.105:6002 192.168.0.105:6003 192.168.0.105:6004 192.168.0.105:6005 192.168.0.105:6006

To get help about all subcommands, type:
redis-cli --cluster help

原因:这个创建集群的命令是redis5版本以前的,我的版本是redis5.0.3,5以后的版本改成了下面这条命令来创建了。

[root@localhost src]# ./redis-cli --cluster create 192.168.0.105:6001 192.168.0.105:6002 192.168.0.105:6003 192.168.0.105:6004 192.168.0.105:6005 192.168.0.105:6006 --cluster-replicas 1

出现下面的命令就说明你的集群已经搭建成功了!

>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 192.168.0.105:6004 to 192.168.0.105:6001
Adding replica 192.168.0.105:6005 to 192.168.0.105:6002
Adding replica 192.168.0.105:6006 to 192.168.0.105:6003
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: caf410b20574f7b714d252df872e5bf8b4a61a74 192.168.0.105:6001
   slots:[0-5460] (5461 slots) master
M: 138c98e4d1d1fe9c604fe777e8a22573b93d69cd 192.168.0.105:6002
   slots:[5461-10922] (5462 slots) master
M: a2cccb5d178a1135e0002929c4a8795567003164 192.168.0.105:6003
   slots:[10923-16383] (5461 slots) master
S: 50037c3a79a91c5255119d2fdf47c4c7f73b396d 192.168.0.105:6004
   replicates a2cccb5d178a1135e0002929c4a8795567003164
S: 5d681dcdf6d9bb4a8e5df6252bb50b07040a273a 192.168.0.105:6005
   replicates caf410b20574f7b714d252df872e5bf8b4a61a74
S: dc334ec3adec10a6c7cf85c84aaf5011e751e20f 192.168.0.105:6006
   replicates 138c98e4d1d1fe9c604fe777e8a22573b93d69cd
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.....
>>> Performing Cluster Check (using node 192.168.0.105:6001)
M: caf410b20574f7b714d252df872e5bf8b4a61a74 192.168.0.105:6001
   slots:[0-5460] (5461 slots) master
   13814765030593265665 additional replica(s)
M: 138c98e4d1d1fe9c604fe777e8a22573b93d69cd 192.168.0.105:6002
   slots:[5461-10922] (5462 slots) master
   685037151884148737 additional replica(s)
S: 5d681dcdf6d9bb4a8e5df6252bb50b07040a273a 192.168.0.105:6005
   slots: (0 slots) slave
   replicates caf410b20574f7b714d252df872e5bf8b4a61a74
M: a2cccb5d178a1135e0002929c4a8795567003164 192.168.0.105:6003
   slots:[10923-16383] (5461 slots) master
   685038113956823041 additional replica(s)
S: 50037c3a79a91c5255119d2fdf47c4c7f73b396d 192.168.0.105:6004
   slots: (0 slots) slave
   replicates a2cccb5d178a1135e0002929c4a8795567003164
S: dc334ec3adec10a6c7cf85c84aaf5011e751e20f 192.168.0.105:6006
   slots: (0 slots) slave
   replicates 138c98e4d1d1fe9c604fe777e8a22573b93d69cd
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

创建集群命令中 --replicas 1,1其实代表的是一个比例,就是主节点数/从节点数的比例。slot对于Redis集群而言,一个存放数据的地方,对应一个槽。对于每一个Master而言,会存在一个slot的范围,而Slave则没有。在Redis集群中,依然是Master可以读、写,而Slave只读。数据的写入,实际上是分布的存储在slot中。

6、数据测试

登录集群客户端,-c标识以集群方式登录

./redis-cli -h 192.168.0.105 -p 6001 -c
[root@localhost src]# ./redis-cli -h 192.168.0.105 -p 6001 -c
192.168.0.105:6001> set redis cluster-test
OK
192.168.0.105:6001> quit
[root@localhost src]# ./redis-cli -h 192.168.0.105 -p 6003 -c
192.168.0.105:6003> get redis
-> Redirected to slot [1151] located at 192.168.0.105:6001
"cluster-test"
192.168.0.105:6001> 

redis cluster在设计的时候,就考虑到了去中心化,去中间件,也就是说,集群中的每个节点都是平等的关系,都是对等的,每个节点都保存各自的数据和整个集群的状态。每个节点都和其他所有节点连接,而且这些连接保持活跃,这样就保证了我们只需要连接集群中的任意一个节点,就可以获取到其他节点的数据。

Redis集群并没有使用传统的一致性哈希来分配数据,而是采用另外一种叫做哈希槽(hash slot)的方式来分配的,一致性哈希对向集群中新增和删除实例的支持很好,但是哈希槽对向集群新增实例或者删除实例的话,需要额外的操作,需要手动的将slot重新平均的分配到新集群的实例中。

Redis 集群会把数据存在一个master节点,然后在这个master和其对应的salve之间进行数据同步。当读取数据时,也根据一致性哈希算法到对应的master节点获取数据。只有当一个master 挂掉之后,才会启动一个对应的salve节点,充当master。

需要注意的是:必须要3个或以上的主节点,否则在创建集群时会失败,并且当存活的主节点数小于总节点数的一半时,整个集群就无法提供服务了。

Redis Cluster中有一个16384长度的槽的概念,他们的编号为0、1、2、3……16382、16383。这个槽是一个虚拟的槽,并不是真正存在的。正常工作的时候,Redis Cluster中的每个Master节点都会负责一部分的槽,当有某个key被映射到某个Master负责的槽,那么这个Master负责为这个key提供服务,至于哪个Master节点负责哪个槽,这是可以由用户指定的,也可以在初始化的时候自动生成(redis-trib.rb脚本)。这里值得一提的是,在Redis Cluster中,只有Master才拥有槽的所有权,如果是某个Master的slave,这个slave只负责槽的使用,但是没有所有权。

7、集群添加节点
  • 添加主节点
# ./redis-cli --cluster add-node new_host:new_port existing_host:existing_port
# ./redis-cli --cluster add-node 192.168.0.105:6007 192.168.0.105:6001

192.168.0.105:6007 是新增的节点
192.168.0.105:6001 集群任一个旧节点

  • 添加丛节点
# ./redis-cli --cluster add-node new_host:new_port existing_host:existing_port \
                    --cluster-slave [--cluster-master-id <master_id>]
# ./redis-cli  add-node --slave --master-id e35403b9171c00c92935a7ba15cf90000f511b88  192.168.0.105:6008 192.168.0.105:6001

–slave,表示添加的是从节点
–master-id e35403b9171c00c92935a7ba15cf90000f511b88 ,主节点的node id,在这里是前面新添加的6008的node id
192.168.0.105:6008 新节点
192.168.0.105:6001 集群任一个旧节点

我这里演示天界主节点:
将6001文件 复制一份改为6007,然后修改6007.conf文件端口号也改为6007。redis实例添加到集群之前,一定要确保这个redis实例没有存储过数据,也不能持久化的数据文件,否则在添加的时候会报错的!

./redis-cli --cluster add-node 192.168.0.105:6007 192.168.0.105:6001
  • 重新分配slot

#./redis-cli reshard 192.168.0.105:6007 //下面是主要过程
How many slots do you want to move (from 1 to 16384)? 1000 //设置slot数1000
What is the receiving node ID? e35403b9171c00c92935a7ba15cf90000f511b88
//新节点node id
Please enter all the source node IDs.
Type ‘all’ to use all the nodes as source nodes for the hash slots.
Type ‘done’ once you entered all the source nodes IDs.
Source node #1:all //表示全部节点重新洗牌
Do you want to proceed with the proposed reshard plan (yes/no)? yes //确认重新分

  • 查看下集群情况
[root@localhost src]# ./redis-cli -h 192.168.0.105 -p 6001 -c
192.168.0.105:6001> cluster nodes
e35403b9171c00c92935a7ba15cf90000f511b88 192.168.0.105:6007@16007 master - 0 1549056838000 7 connected 0-332 5461-5794 10923-11255
5d681dcdf6d9bb4a8e5df6252bb50b07040a273a 192.168.0.105:6005@16005 slave caf410b20574f7b714d252df872e5bf8b4a61a74 0 1549056836851 5 connected
dc334ec3adec10a6c7cf85c84aaf5011e751e20f 192.168.0.105:6006@16006 slave 138c98e4d1d1fe9c604fe777e8a22573b93d69cd 0 1549056838868 6 connected
caf410b20574f7b714d252df872e5bf8b4a61a74 192.168.0.105:6001@16001 myself,master - 0 1549056835000 1 connected 333-5460
50037c3a79a91c5255119d2fdf47c4c7f73b396d 192.168.0.105:6004@16004 slave a2cccb5d178a1135e0002929c4a8795567003164 0 1549056838000 4 connected
138c98e4d1d1fe9c604fe777e8a22573b93d69cd 192.168.0.105:6002@16002 master - 0 1549056839877 2 connected 5795-10922
a2cccb5d178a1135e0002929c4a8795567003164 192.168.0.105:6003@16003 master - 0 1549056838000 3 connected 11256-16383

在这里插入图片描述

重新给192.168.0.105:6007 分配slot成功

7、集群删除节点
  • 删除从节点
./redis-cli --cluster del-node host:port <node-id>
./redis-cli --cluster del-node 192.168.0.105:6007 ‘e35403b9171c00c92935a7ba15cf90000f511b88 ’
  • 删除主节点

如果主节点有从节点,将从节点转移到其他主节点
如果主节点有slot,去掉分配的slot,然后在删除主节点

#./redis-cli reshard 192.168.0.105:6007 //取消分配的slot,下面是主要过程

How many slots do you want to move (from 1 to 16384)? 1000 //被删除master的所有slot数量
What is the receiving node ID? e35403b9171c00c92935a7ba15cf90000f511b88 //接收6007节点slot的master
Please enter all the source node IDs.
Type ‘all’ to use all the nodes as source nodes for the hash slots.
Type ‘done’ once you entered all the source nodes IDs.
Source node #1:e35403b9171c00c92935a7ba15cf90000f511b88 //被删除master的node-id
Source node #2:done

Do you want to proceed with the proposed reshard plan (yes/no)? yes //取消slot后,reshard
新增master节点后,也进行了这一步操作,当时是分配,现在去掉。反着的。

#redis-trib.rb del-node 192.168.0.105:6007 'e35403b9171c00c92935a7ba15cf90000f511b88 ’
新的master节点被删除了,这样就回到了,就是这篇文章开头,还没有添加节点的状态