docker compose rabbitmq cluster

概要

3个物理主机： 192.168.1.11 192.168.1.12 192.168.1.13

节点1

$ mkdir -p rabbitmq-cluster && cd rabbitmq-cluster

docker-compose.yml

$ cat docker-compose.yml

services:
  rabbit:
    container_name: rabbitmq
    image: rabbitmq:3-management
    restart: always
    hostname: rabbitmq1
    extra_hosts:
      #- "rabbitmq1:172.100.8.250"
      - "rabbitmq2:172.100.8.249"
      - "rabbitmq3:172.100.8.248"
    environment:
      - TZ=Asia/Shanghai
      - RABBITMQ_ERLANG_COOKIE=MY_COOKIE
      - RABBITMQ_DEFAULT_USER=admin
      - RABBITMQ_DEFAULT_PASS=admin123
      - RABBITMQ_DEFAULT_VHOST=my_vhost
    ports:
      - "4369:4369"
      - "5671:5671"
      - "5672:5672"
      - "15671:15671"
      - "15672:15672"
      - "25672:25672"
    volumes:
      - ./data:/var/lib/rabbitmq
      - ./logs:/var/log/rabbitmq

注意：务必将extra_hosts中的本机一行注释掉，否则容器会启动失败（一个hostname对应两个ip)

部署

$docker compose up -d$ docker compose ps

NAME       IMAGE                   COMMAND                   SERVICE   CREATED         STATUS         PORTS
rabbitmq   rabbitmq:3-management   "docker-entrypoint.s…"   rabbit    7 seconds ago   Up 6 seconds   0.0.0.0:4369->4369/tcp, :::4369->4369/tcp, 0.0.0.0:5671-5672->5671-5672/tcp, :::5671-5672->5671-5672/tcp, 0.0.0.0:15671-15672->15671-15672/tcp, :::15671-15672->15671-15672/tcp, 0.0.0.0:25672->25672/tcp, :::25672->25672/tcp, 15691-15692/tcp

$ docker exec -it rabbitmq /bin/bash

root@rabbitmq1:/# cat /etc/hosts
127.0.0.1       localhost
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.100.8.249   rabbitmq2
172.100.8.248   rabbitmq3
172.29.0.2      rabbitmq1
root@rabbitmq1:/# hostname -i
172.29.0.2

节点2

$ mkdir -p rabbitmq-cluster && cd rabbitmq-cluster

docker-compose.yml

$ cat docker-compose.yml

services:
  rabbit:
    container_name: rabbitmq
    image: rabbitmq:3-management
    restart: always
    hostname: rabbitmq2
    extra_hosts:
      - "rabbitmq1:172.100.8.250"
      #- "rabbitmq2:172.100.8.249"
      - "rabbitmq3:172.100.8.248"
    environment:
      - TZ=Asia/Shanghai
      - RABBITMQ_ERLANG_COOKIE=MY_COOKIE
      - RABBITMQ_DEFAULT_USER=admin
      - RABBITMQ_DEFAULT_PASS=admin123
      - RABBITMQ_DEFAULT_VHOST=my_vhost
    ports:
      - "4369:4369"
      - "5671:5671"
      - "5672:5672"
      - "15671:15671"
      - "15672:15672"
      - "25672:25672"
    volumes:
      - ./data:/var/lib/rabbitmq
      - ./logs:/var/log/rabbitmq
      - ./rabbitmq.sh:/home/rabbitmq.sh

修改内容：

• hostname: rabbitmq2
• extra_hosts: 注释本机hostname对应的行
• volumes增加rabbitmq.sh

添加加入集群的脚本

编写rabbitmq.sh启动脚本（磁盘节点）

cat > rabbitmq.sh <<EOF
rabbitmqctl stop_app
rabbitmqctl reset
rabbitmqctl join_cluster rabbit@rabbitmq1
rabbitmqctl start_app
EOF
chmod +x rabbitmq.sh

部署

$docker compose up -d$ docker compose ps

进入rabbitmq2节点的容器中，执行/home/rabbitmq.sh脚本

docker container exec -it rabbitmq /bin/bash
./home/rabbitmq.sh
rabbitmqctl cluster_status #查看集群状态

## 测试连通性
apt update && apt install curl -y 
curl http://rabbitmq1:15672/

节点3

docker-compose.yml

services:
  rabbit:
    container_name: rabbitmq
    image: rabbitmq:3-management
    restart: always
    hostname: rabbitmq3
    extra_hosts:
      - "rabbitmq1:172.100.8.250"
      - "rabbitmq2:172.100.8.249"
      #- "rabbitmq3:172.100.8.248"
    environment:
      - TZ=Asia/Shanghai
      - RABBITMQ_ERLANG_COOKIE=MY_COOKIE
      - RABBITMQ_DEFAULT_USER=admin
      - RABBITMQ_DEFAULT_PASS=admin123
      - RABBITMQ_DEFAULT_VHOST=my_vhost
    ports:
      - "4369:4369"
      - "5671:5671"
      - "5672:5672"
      - "15671:15671"
      - "15672:15672"
      - "25672:25672"
    volumes:
      - ./data:/var/lib/rabbitmq
      - ./logs:/var/log/rabbitmq
      - ./rabbitmq-ram.sh:/home/rabbitmq-ram.sh

添加加入集群的脚本

cat > rabbitmq-ram.sh <<EOF
rabbitmqctl stop_app
rabbitmqctl reset
rabbitmqctl join_cluster --ram rabbit@rabbitmq1
rabbitmqctl start_app
EOF
chmod +x rabbitmq-ram.sh

部署

$docker compose up -d$ docker compose ps
$　docker exec -it rabbitmq /bin/bash ./home/rabbitmq-ram.sh

检查状态

$ docker exec -it rabbitmq /bin/bash

root@rabbitmq3:/# rabbitmqctl cluster_status
Cluster status of node rabbit@rabbitmq3 ...
Basics

Cluster name: rabbit@rabbitmq3
Total CPU cores available cluster-wide: 60
Disk Nodes

rabbit@rabbitmq1
rabbit@rabbitmq2

RAM Nodes

rabbit@rabbitmq3

Running Nodes

rabbit@rabbitmq1
rabbit@rabbitmq2
rabbit@rabbitmq3

Versions

rabbit@rabbitmq3: RabbitMQ 3.13.3 on Erlang 26.2.5
rabbit@rabbitmq2: RabbitMQ 3.13.3 on Erlang 26.2.5
rabbit@rabbitmq1: RabbitMQ 3.13.3 on Erlang 26.2.5

CPU Cores

Node: rabbit@rabbitmq3, available CPU cores: 12
Node: rabbit@rabbitmq2, available CPU cores: 24
Node: rabbit@rabbitmq1, available CPU cores: 24

......

负载均衡

可以采用LVS 或 haproxy,nginx

参考

wiki.clay-wangzhi.com/docker/dock…