如何在docker中搭建 redis-cluster

redis自学专栏

在mac系统中安装docker，并使用docker desktop 学习redis相关知识，在集群模块时，需要搭建集群，遇到相关问题在此记录。

参考文档 bitnami搭建集群：bitnami/redis-cluster

在docker中搭建redisCluster集群，可以使用bitnami的镜像，也可以使用 redis原生镜像，相比较来说使用bitnami镜像搭建，更加方便一些（不需要节点握手、分配节点槽、分配主从，这些都是自动化万能充的）。

以下以redis官方镜像搭建redis集群。

1.准备docker-compose.yml

新建一个文件，文件名为docker-compose.yml

services:
  redis-node-0:
    image: redis:latest
    container_name: redis-cluster-6370
    networks:
      redis-network:
        ipv4_address: 172.19.0.10
    ports:
      - "6370:6370"
      - "16370:16370"
    volumes:
      - ~/local/redis/cluster/redis0.conf:/usr/local/etc/redis/redis.conf
      - ~/local/redis/cluster/data0:/data
    environment:
      - 'REDIS_PASSWORD=bitnami'
      - 'REDISCLI_AUTH=bitnami'
      - 'REDIS_CLUSTER_CREATOR=yes'
      - 'REDIS_NODES=redis-cluster-6370 redis-cluster-6371 redis-cluster-6372 redis-cluster-6373 redis-cluster-6374 redis-cluster-6375'
    command:
      redis-server /usr/local/etc/redis/redis.conf

  redis-node-1:
    image: redis:latest
    container_name: redis-cluster-6371
    networks:
      redis-network:
        ipv4_address: 172.19.0.11
    ports:
      - "6371:6371"
      - "16371:16371"
    volumes:
      - ~/local/redis/cluster/redis1.conf:/usr/local/etc/redis/redis.conf
      - ~/local/redis/cluster/data1:/data
    environment:
      - 'REDIS_PASSWORD=bitnami'
      - 'REDISCLI_AUTH=bitnami'
      - 'REDIS_CLUSTER_CREATOR=yes'
      - 'REDIS_NODES=redis-cluster-6370 redis-cluster-6371 redis-cluster-6372 redis-cluster-6373 redis-cluster-6374 redis-cluster-6375'
    command:
      redis-server /usr/local/etc/redis/redis.conf

  redis-node-2:
    image: redis:latest
    container_name: redis-cluster-6372
    networks:
      redis-network:
        ipv4_address: 172.19.0.12
    ports:
      - "6372:6372"
      - "16372:16372"
    volumes:
      - ~/local/redis/cluster/redis2.conf:/usr/local/etc/redis/redis.conf
      - ~/local/redis/cluster/data2:/data
    environment:
      - 'REDIS_PASSWORD=bitnami'
      - 'REDISCLI_AUTH=bitnami'
      - 'REDIS_CLUSTER_CREATOR=yes'
      - 'REDIS_NODES=redis-cluster-6370 redis-cluster-6371 redis-cluster-6372 redis-cluster-6373 redis-cluster-6374 redis-cluster-6375'
    command:
      redis-server /usr/local/etc/redis/redis.conf

  redis-node-3:
    image: redis:latest
    container_name: redis-cluster-6373
    networks:
      redis-network:
        ipv4_address: 172.19.0.13
    ports:
      - "6373:6373"
      - "16373:16373"
    volumes:
      - ~/local/redis/cluster/redis3.conf:/usr/local/etc/redis/redis.conf
      - ~/local/redis/cluster/data3:/data
    environment:
      - 'REDIS_PASSWORD=bitnami'
      - 'REDISCLI_AUTH=bitnami'
      - 'REDIS_CLUSTER_CREATOR=yes'
      - 'REDIS_NODES=redis-cluster-6370 redis-cluster-6371 redis-cluster-6372 redis-cluster-6373 redis-cluster-6374 redis-cluster-6375'
    command:
      redis-server /usr/local/etc/redis/redis.conf

  redis-node-4:
    image: redis:latest
    container_name: redis-cluster-6374
    networks:
      redis-network:
        ipv4_address: 172.19.0.14
    ports:
      - "6374:6374"
      - "16374:16374"
    volumes:
      - ~/local/redis/cluster/redis4.conf:/usr/local/etc/redis/redis.conf
      - ~/local/redis/cluster/data4:/data
    environment:
      - 'REDIS_PASSWORD=bitnami'
      - 'REDISCLI_AUTH=bitnami'
      - 'REDIS_CLUSTER_CREATOR=yes'
      - 'REDIS_NODES=redis-cluster-6370 redis-cluster-6371 redis-cluster-6372 redis-cluster-6373 redis-cluster-6374 redis-cluster-6375'
    command:
      redis-server /usr/local/etc/redis/redis.conf

  redis-node-5:
    image: redis:latest
    container_name: redis-cluster-6375
    networks:
      redis-network:
        ipv4_address: 172.19.0.15
    ports:
      - "6375:6375"
      - "16375:16375"
    volumes:
      - ~/local/redis/cluster/redis5.conf:/usr/local/etc/redis/redis.conf
      - ~/local/redis/cluster/data5:/data
    depends_on:
      - redis-node-0
      - redis-node-1
      - redis-node-2
      - redis-node-3
      - redis-node-4
    environment:
      - 'REDIS_PASSWORD=bitnami'
      - 'REDISCLI_AUTH=bitnami'
      - 'REDIS_CLUSTER_REPLICAS=1'
      - 'REDIS_NODES=redis-cluster-6370 redis-cluster-6371 redis-cluster-6372 redis-cluster-6373 redis-cluster-6374 redis-cluster-6375'
      - 'REDIS_CLUSTER_CREATOR=yes'
    command:
      redis-server /usr/local/etc/redis/redis.conf

networks:
  redis-network:
    external: true

注意：有一些地方需要修改！！！

• [~]我在实践的时候使用的绝对路径，这里用相对路径来表达
• ip地址可以换自己的地址。我这里固定了一个redis-network的网络，而且不是新建的。
• networks，如果自己本地没有，则需要新建一个，新建的命令为下面的内容
• 自己映射的路径要有配置文件
• 完了之后，还要执行一系列的命令，让集群运行起来

新建网络内容举例

networks: 
 redis-network: 
  driver: bridge 
   ipam: 
    config: 
     - subnet: 172.19.0.0/16 
     - gateway: 172.19.0.1

我的文件夹结构

2.运行集群

集群运行起来之后，可以进入容器，然后使用命令进入某一个节点。比如： redis-cli -h 172.19.0.10 -p 6370 -a bitnami，（这里 ip和端口号要与你的一致，否则连接不了）。之后运行命令 CLUSTER info 查看集群状态，此时显示cluster_state:fail 未启动。我们需要对集群的机器进行分配槽，分配主从。

2.1.加入集群

使用命令cluster meet 172.19.0.11 6371 让6371加入到集群中去。同理，将6372 ~ 6375都加入到集群中。这里需要注意的是，只需要在一个机器中添加即可。

添加完成后，使用CLUSTER nodes 查看各个节点，就可以看到一共有6个节点，不过都是master节点，这个时候，需要给各个节点分配角色，是主节点还是从节点。

2.1.分配槽

使用命令给3个节点分配槽

redis-cli -h 10.8.13.106 -p 6370 -a bitnami cluster addslots $(seq 0 5461)
redis-cli -h 10.8.13.106 -p 6371 -a bitnami cluster addslots $(seq 5462 10922)
redis-cli -h 10.8.13.106 -p 6372 -a bitnami cluster addslots $(seq 10923 16383)

如果少分配槽，或者0~16383 有任何槽没有分配到节点，都是启动不成功的，因为槽没有分配完成。

少分配槽，就需要将对应的槽添加到节点中去，可以使用redis-cli -h 10.8.13.106 -p 6370 -a bitnami cluster ADDSLOTS 0 命令将槽添加进去。（我这里是将槽0添加进去了）

完成后示例：

2.3.分配主从

注意，分配主从需要进入到对应的从节点，然后执行命令。

进入从节点，让该从节点复制其主节点，命令如下：

redis-cli -h 10.8.13.106 -p 6373 -a bitnami
cluster replicate fb62e193fd0b733863764a8654fa0d88d209196c
redis-cli -h 10.8.13.106 -p 6374 -a bitnami
cluster replicate 17d1c5a8cd5b3d2ebbc281a6f5087b516ccd8113
redis-cli -h 10.8.13.106 -p 6375 -a bitnami
cluster replicate 701511073d1426a7826c4c8b0ae2a8466e294dcc

我这里，进入了3、4、5三个从节点，然后让他们去复制主节点。后面的是主节点的id，这个id，可以在任意节点，使用CLUSTER nodes查看。分配完的节点状态如下：

附：redis集群搭建我用的配置文件，这里的配置文件取自redis源码。redis0.conf

bind 0.0.0.0
protected-mode yes
# 要改
port 6370
tcp-backlog 511

timeout 0

tcp-keepalive 300

daemonize no

# pidfile /opt/bitnami/redis/tmp/redis.pid
pidfile /var/run/redis_6379.pid

loglevel notice

logfile ""

databases 16

always-show-logo no

set-proc-title yes

proc-title-template "{title} {listen-addr} {server-mode}"

locale-collate ""

stop-writes-on-bgsave-error yes

rdbcompression yes

rdbchecksum yes

dbfilename dump.rdb

rdb-del-sync-files no

dir ./

replica-serve-stale-data yes

replica-read-only yes

repl-diskless-sync yes

repl-diskless-sync-delay 5

repl-diskless-sync-max-replicas 0

repl-diskless-load disabled

repl-disable-tcp-nodelay no

replica-priority 100
acllog-max-len 128

# 这里要改，注意，如果有，这两个都要有。
requirepass bitnami
masterauth bitnami

lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
replica-lazy-flush no

lazyfree-lazy-user-del no

lazyfree-lazy-user-flush no

oom-score-adj no

oom-score-adj-values 0 200 800

disable-thp yes

appendonly no

appendfilename "appendonly.aof"

appenddirname "appendonlydir"

appendfsync everysec

no-appendfsync-on-rewrite no

auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

aof-load-truncated yes

aof-use-rdb-preamble yes

aof-timestamp-enabled no

cluster-enabled yes

cluster-config-file nodes.conf

# 这里要改 cluster-announce-ip 172.19.0.10
cluster-announce-ip 10.8.13.106
cluster-announce-port 6370
cluster-announce-bus-port 16370

slowlog-log-slower-than 10000

slowlog-max-len 128

latency-monitor-threshold 0

notify-keyspace-events ""

hash-max-listpack-entries 512
hash-max-listpack-value 64

list-max-listpack-size -2

list-compress-depth 0

set-max-intset-entries 512

set-max-listpack-entries 128
set-max-listpack-value 64

zset-max-listpack-entries 128
zset-max-listpack-value 64

hll-sparse-max-bytes 3000

stream-node-max-bytes 4096
stream-node-max-entries 100

activerehashing yes

client-output-buffer-limit normal 0 0 0
client-output-buffer-limit replica 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60

hz 10

dynamic-hz yes

aof-rewrite-incremental-fsync yes

rdb-save-incremental-fsync yes

jemalloc-bg-thread yes