本次搭建的环境基本上是互联网中大型项目必会涉及到的中间件，主要是以下这些（部分需要根据需求选择版本）：
- MySQL 8.0（MySQL 5.7 也行，众所周知大部分公司用的还是 5.7）
- Redis 5.0.2 （这个主要是我最常用的）
- ES & Kibana
- RocketMQ
安装docker，官网：Get Docker
docker版本，Version: 24.0.6
sudo docker info
Client:
Version: 24.0.6
Context: desktop-linux
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.11.2-desktop.5
Path: /Users/liulvhua/.docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.23.0-desktop.1
Path: /Users/liulvhua/.docker/cli-plugins/docker-compose
dev: Docker Dev Environments (Docker Inc.)
Version: v0.1.0
Path: /Users/liulvhua/.docker/cli-plugins/docker-dev
extension: Manages Docker extensions (Docker Inc.)
Version: v0.2.20
Path: /Users/liulvhua/.docker/cli-plugins/docker-extension
init: Creates Docker-related starter files for your project (Docker Inc.)
Version: v0.1.0-beta.9
Path: /Users/liulvhua/.docker/cli-plugins/docker-init
sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
Version: 0.6.0
Path: /Users/liulvhua/.docker/cli-plugins/docker-sbom
scan: Docker Scan (Docker Inc.)
Version: v0.26.0
Path: /Users/liulvhua/.docker/cli-plugins/docker-scan
scout: Docker Scout (Docker Inc.)
Version: v1.0.9
Path: /Users/liulvhua/.docker/cli-plugins/docker-scout
Server:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 4
Server Version: 24.0.6
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 8165feabfdfe38c65b599c4993d227328c231fca
runc version: v1.1.8-0-g82f18fe
init version: de40ad0
Security Options:
seccomp
Profile: unconfined
cgroupns
Kernel Version: 6.4.16-linuxkit
Operating System: Docker Desktop
OSType: linux
Architecture: aarch64
CPUs: 10
Total Memory: 7.661GiB
Name: linuxkit-62317f06a9ba
ID: 8a94167e-d790-4075-945c-0eb09de5e75c
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
No Proxy: hubproxy.docker.internal
Experimental: false
Insecure Registries:
hubproxy.docker.internal:5555
127.0.0.0/8
Registry Mirrors:
https://14saylz9.mirror.aliyuncs.com/
Live Restore Enabled: false
WARNING: daemon is not using the default seccomp profile
环境安装与部署
以下所有环境都是单机部署的，现在大家用的电脑应该都是在8核16G以上，这个配置理论是够了，部署完 如果是跑项目可以暂时把其他无关的进程关闭。
使用 docker compose 进行部署，docker 源 使用 阿里云即可
准备：为每个环境新建一个文件夹，存放 docker-compose.yml 文件，所有的 compose 文件统一命名为：docker-compose.yml 并且执行命令统一为 docker-compose up -d
先拉取镜像，再启容器 拉取镜像可以通过 docker search 搜索需要的镜像，再通过 docker pull image:version 拉取到本地 也可以使用 docker 客户端：
mysql8.0 镜像
docker pull mysql:8.0
8.0: Pulling from library/mysql
edfaa6c9daad: Pull complete
bb9fa257184c: Pull complete
e2c9f07c7bd8: Pull complete
7865ad857d8c: Pull complete
1e0aaa5cd9dd: Pull complete
f6ed6e3f1776: Pull complete
c3aca5ebfaba: Pull complete
2deaa7499105: Pull complete
cfee2607bfc8: Pull complete
daac8111285e: Pull complete
9633671d9b4d: Pull complete
Digest: sha256:40ab5148cf849eb9858040893a28d1dc1ef3fe0d13637c75fd263aed316614c0
Status: Downloaded newer image for mysql:8.0
docker.io/library/mysql:8.0
mysql集群部署
创建docker-compose编排配置文档，docker-compose up -d 一键启动
# liulvhua @ MacBookPro in /usr/local/soft [16:55:12]
$ ls
es kibana mongo mq mysql nacos redis
# liulvhua @ MacBookPro in /usr/local/soft [16:55:14]
$ cd mysql
# liulvhua @ MacBookPro in /usr/local/soft/mysql [16:55:18]
$ ll
total 8
-rw-r--r-- 1 root wheel 1.9K 12 9 12:03 docker-compose.yml
# liulvhua @ MacBookPro in /usr/local/soft/mysql [16:55:21]
$ docker-compose up -d
[+] Running 4/4
✔ Network mysql_default Created 0.0s
✔ Container mysql-mysql-master-1 Started 0.0s
✔ Container mysql-mysql-slave-1 Started 0.0s
✔ Container mysql-setup-master-slave-1 Started
在 /usr/local/ 目录下新建一个mysql文件夹，docker-compose.yml内容如下
version: '3.1'
services:
mysql-master:
image: mysql:8.0
environment:
MYSQL_ROOT_PASSWORD: masterpassword
MYSQL_DATABASE: mydb
ports:
- "3306:3306"
#如果要挂载目录文件请释放开此配置 如果使用阿里云会报错请注释此内容
# volumes:
# - ./mysql:/var/lib/mysql
command: --server-id=1 --log-bin=mysql-bin --binlog-format=ROW --default-authentication-plugin=mysql_native_password
mysql-slave:
image: mysql:8.0
environment:
MYSQL_ROOT_PASSWORD: slavepassword
MYSQL_DATABASE: mydb
ports:
- "3307:3306"
#如果要挂载目录文件请释放开此配置如果使用阿里云会报错请注释此内容
# volumes:
# - ./mysqls:/var/lib/mysql
command: --server-id=2 --default-authentication-plugin=mysql_native_password
depends_on:
- mysql-master
setup-master-slave:
image: mysql:8.0
environment:
MYSQL_ROOT_PASSWORD: masterpassword
depends_on:
- mysql-master
- mysql-slave
command: >
bash -c "
echo 'Setting up master-slave replication...';
sleep 60;
MYSQL_PWD=masterpassword mysql -h mysql-master -uroot -e '
CREATE USER "replica"@"%" IDENTIFIED WITH mysql_native_password BY "replicapassword";
GRANT REPLICATION SLAVE ON *.* TO "replica"@"%";
FLUSH PRIVILEGES;
FLUSH TABLES WITH READ LOCK;
SHOW MASTER STATUS\G' > master_status;
MASTER_LOG_FILE=$$(cat master_status | grep File: | awk '{print $$2}');
MASTER_LOG_POS=$$(cat master_status | grep Position: | awk '{print $$2}');
MYSQL_PWD=slavepassword mysql -h mysql-slave -uroot -e '
CHANGE MASTER TO MASTER_HOST="mysql-master", MASTER_USER="replica", MASTER_PASSWORD="replicapassword", MASTER_LOG_FILE='$$MASTER_LOG_FILE', MASTER_LOG_POS='$$MASTER_LOG_POS';
START SLAVE;';
echo 'Replication setup complete.';
"
redis5.0.2镜像
docker pull redis:5.0.2
5.0.2: Pulling from library/redis
2216f9d9ff00: Pull complete
f30486ebbbf1: Pull complete
1c396b63e865: Pull complete
3d1675931bd4: Pull complete
5d54d906982e: Pull complete
ef8a1f2b8265: Pull complete
Digest: sha256:f57d1597d038a742dfba6acfaf48b10e6383466eea2aef95d1ee76f32633f959
Status: Downloaded newer image for redis:5.0.2
docker.io/library/redis:5.0.2
redis 集群部署
version: '2.2'
services:
redis-node1:
image: redis:5.0
## --cluster-announce-ip 你的公网IP 这部分参数 应对的是部分小伙伴用的 阿里云服务器出现网卡回环的问题 如果没有问题 可以不用写这个
command: redis-server --port 7000 --cluster-enabled yes --cluster-config-file /data/nodes.conf --appendonly yes --bind 0.0.0.0 --cluster-announce-ip 你的公网IP
ports:
- "7000:7000"
- "17000:17000"
volumes:
- ./data/node1:/data
networks:
- redis-cluster
redis-node2:
image: redis:5.0
command: redis-server --port 7001 --cluster-enabled yes --cluster-config-file /data/nodes.conf --appendonly yes --bind 0.0.0.0 --cluster-announce-ip 你的公网IP
ports:
- "7001:7001"
- "17001:17001"
volumes:
- ./data/node2:/data
networks:
- redis-cluster
redis-node3:
image: redis:5.0
command: redis-server --port 7002 --cluster-enabled yes --cluster-config-file /data/nodes.conf --appendonly yes --bind 0.0.0.0 --cluster-announce-ip 你的公网IP
ports:
- "7002:7002"
- "17002:17002"
volumes:
- ./data/node3:/data
networks:
- redis-cluster
redis-node4:
image: redis:5.0
command: redis-server --port 7003 --cluster-enabled yes --cluster-config-file /data/nodes.conf --appendonly yes --bind 0.0.0.0 --cluster-announce-ip 你的公网IP
ports:
- "7003:7003"
- "17003:17003"
volumes:
- ./data/node4:/data
networks:
- redis-cluster
redis-node5:
image: redis:5.0
command: redis-server --port 7004 --cluster-enabled yes --cluster-config-file /data/nodes.conf --appendonly yes --bind 0.0.0.0 --cluster-announce-ip 你的公网IP
ports:
- "7004:7004"
- "17004:17004"
volumes:
- ./data/node5:/data
networks:
- redis-cluster
redis-node6:
image: redis:5.0
command: redis-server --port 7005 --cluster-enabled yes --cluster-config-file /data/nodes.conf --appendonly yes --bind 0.0.0.0 --cluster-announce-ip 你的公网IP
ports:
- "7005:7005"
- "17005:17005"
volumes:
- ./data/node6:/data
networks:
- redis-cluster
redis-init:
image: redis:5.0
command: >
sh -c "redis-server --port 7000 --cluster-enabled yes --cluster-config-file /data/nodes.conf --appendonly yes --bind $$(hostname -i)
&& sleep 5
&& redis-cli --cluster create redis-node1:7000 redis-node2:7001 redis-node3:7002 redis-node4:7003 redis-node5:7004 redis-node6:7005 --cluster-replicas 1"
depends_on:
- redis-node1
networks:
- redis-cluster
networks:
redis-cluster:
这里遇到一个问题：
docker-compose up -d
[+] Running 7/7
✘ redis-node4 Error 2.7s
✘ redis-init Error 2.7s
✘ redis-node2 Error 2.7s
✘ redis-node6 Error 2.7s
✘ redis-node3 Error 2.7s
✘ redis-node1 Error 2.7s
✘ redis-node5 Error 2.7s
Error response from daemon: Head "https://registry-1.docker.io/v2/library/redis/manifests/5.0": Get "https://auth.docker.io/token?scope=repository%3Alibrary%2Fredis%3Apull&service=registry.docker.io": dialing auth.docker.io:443 with direct connection: resolving host auth.docker.io: lookup auth.docker.io: no such host
其实是redis的镜像没有匹配到，仔细看在docker镜像本地的是5.0.2
改过来就可以。
其他还可能遇到的问题：端口冲突，这种就很简单，docker-compose.yml 里的端口换一个就行了
修改后的 docker-compose.yml 如下
version: '2.2'
services:
redis-node1:
image: redis:5.0.2
## --cluster-announce-ip 你的公网IP 这部分参数 应对的是部分小伙伴用的 阿里云服务器出现网卡回环的问题 如果没有问题 可以不用写这个
command: redis-server --port 7100 --cluster-enabled yes --cluster-config-file /data/nodes.conf --appendonly yes --bind 0.0.0.0 --cluster-announce-ip 172.20.10.9
ports:
- "7100:7100"
- "17100:17100"
volumes:
- ./data/node1:/data
networks:
- redis-cluster
redis-node2:
image: redis:5.0.2
command: redis-server --port 7101 --cluster-enabled yes --cluster-config-file /data/nodes.conf --appendonly yes --bind 0.0.0.0 --cluster-announce-ip 172.20.10.9
ports:
- "7101:7101"
- "17101:17101"
volumes:
- ./data/node2:/data
networks:
- redis-cluster
redis-node3:
image: redis:5.0.2
command: redis-server --port 7102 --cluster-enabled yes --cluster-config-file /data/nodes.conf --appendonly yes --bind 0.0.0.0 --cluster-announce-ip 172.20.10.9
ports:
- "7102:7102"
- "17102:17102"
volumes:
- ./data/node3:/data
networks:
- redis-cluster
redis-node4:
image: redis:5.0.2
command: redis-server --port 7103 --cluster-enabled yes --cluster-config-file /data/nodes.conf --appendonly yes --bind 0.0.0.0 --cluster-announce-ip 172.20.10.9
ports:
- "7103:7103"
- "17103:17103"
volumes:
- ./data/node4:/data
networks:
- redis-cluster
redis-node5:
image: redis:5.0.2
command: redis-server --port 7104 --cluster-enabled yes --cluster-config-file /data/nodes.conf --appendonly yes --bind 0.0.0.0 --cluster-announce-ip 172.20.10.9
ports:
- "7104:7104"
- "17104:17104"
volumes:
- ./data/node5:/data
networks:
- redis-cluster
redis-node6:
image: redis:5.0.2
command: redis-server --port 7105 --cluster-enabled yes --cluster-config-file /data/nodes.conf --appendonly yes --bind 0.0.0.0 --cluster-announce-ip 172.20.10.9
ports:
- "7105:7105"
- "17105:17105"
"docker-compose.yml" 87L, 2679B
启动我们的redis集群服务 docker-compose up -d
# liulvhua @ MacBookPro in /usr/local/soft/redis [22:17:16]
$ docker-compose up -d
[+] Running 8/8
✔ Network redis_redis-cluster Created 0.0s
✔ Container redis-redis-node1-1 Started 0.0s
✔ Container redis-redis-node6-1 Started 0.1s
✔ Container redis-redis-node3-1 Started 0.1s
✔ Container redis-redis-node5-1 Started 0.1s
✔ Container redis-redis-node4-1 Started 0.1s
✔ Container redis-redis-node2-1 Started 0.1s
✔ Container redis-redis-init-1 Started
如果 你的集群起来不是自动的集群 请 进入容器执行这部分命令 IP写自己的IP
redis-cli --cluster create IP:7000 IP:7001 IP:7002 IP:7003 IP:7004 IP:7005 --cluster-replicas 1
--cluster-announce-ip 你的公网IP
elasticsearch镜像
docker pull docker.elastic.co/elasticsearch/elasticsearch:7.6.2
7.6.2: Pulling from elasticsearch/elasticsearch
c808caf183b6: Pull complete
d6caf8e15a64: Pull complete
b0ba5f324e82: Pull complete
d7e8c1e99b9a: Pull complete
85c4d6c81438: Pull complete
3119218fac98: Pull complete
914accf214bb: Pull complete
Digest: sha256:59342c577e2b7082b819654d119f42514ddf47f0699c8b54dc1f0150250ce7aa
Status: Downloaded newer image for docker.elastic.co/elasticsearch/elasticsearch:7.6.2
docker.elastic.co/elasticsearch/elasticsearch:7.6.2
es单节点部署
docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.6.2
docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.6.2
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
{"type": "server", "timestamp": "2023-12-09T08:33:42,411Z", "level": "WARN", "component": "o.e.b.JNANatives", "cluster.name": "docker-cluster", "node.name": "65427b6b18f4", "message": "unable to install syscall filter: ",
es集群部署
docker-compose.yml 内容如下：
version: '2.2'
services:
es01:
image: docker.elastic.co/elasticsearch/elasticsearch:7.6.2
container_name: es01
environment:
- node.name=es01
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es02,es03
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- data01:/usr/share/elasticsearch/data
ports:
- 9200:9200
networks:
- elastic
es02:
image: docker.elastic.co/elasticsearch/elasticsearch:7.6.2
container_name: es02
environment:
- node.name=es02
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es01,es03
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- data02:/usr/share/elasticsearch/data
networks:
- elastic
es03:
image: docker.elastic.co/elasticsearch/elasticsearch:7.6.2
container_name: es03
environment:
- node.name=es03
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es01,es02
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- data03:/usr/share/elasticsearch/data
networks:
- elastic
volumes:
data01:
driver: local
data02:
driver: local
data03:
driver: local
"docker-compose.yml" 72L, 1667B
前台启动：docker-compose up
后台启动：docker-compose up -d
docker-compose up
[+] Running 10/0
✔ Network es_elastic Created0.0s
✔ Volume "es_data01" Created0.0s
✔ Volume "es_data02" Created0.0s
✔ Volume "es_data03" Created0.0s
✔ Container es02 Created0.0s
✔ Container es01 Created0.0s
✔ Container es03 Created0.0s
! es01 The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested 0.0s
! es02 The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested 0.0s
! es03 The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested 0.0s
Attaching to es01, es02, es03
当然，es 也可以和 kibana 一起，docker-compose.yml 内容如下：
version: '2.3'
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:7.6.2
container_name: elasticsearch
environment:
- discovery.type=single-node
ports:
- "9200:9200"
- "9300:9300"
volumes:
- elasticsearch-data:/usr/share/elasticsearch/data
kibana:
image: elastic/kibana:8.11.2
container_name: kibana
environment:
- ELASTICSEARCH_URL=http://elasticsearch:9200
ports:
- "5601:5601"
depends_on:
- elasticsearch
volumes:
elasticsearch-data:
RocketMQ
version: '2.2'
services:
namesrv:
image: apache/rocketmq:4.9.5
container_name: rocketmq-namesrv
ports:
- "9876:9876"
command: "sh mqnamesrv"
broker:
image: apache/rocketmq:4.9.5
container_name: rocketmq-broker
environment:
- NAMESRV_ADDR=namesrv:9876
ports:
- "10911:10911"
- "10909:10909"
depends_on:
- namesrv
command: "sh mqbroker -n namesrv:9876"
apache/rocketmq
到现在的话，环境基本已经搭建起来了
参考：
docker 官网：docs.docker.com/get-docker/
elastic 官方文档：www.elastic.co/guide/en/el…
