首先搭建standalone集群
1、准备配置文件
1.1、复制默认的 flink-conf.yaml 配置文件,并修改以下配置,然后重命名为 jobmanager-conf.yaml 并放在 /home/debo/dockerData/flink 目录下
jobmanager.rpc.address: flink-jobmanager
jobmanager.bind-host: 0.0.0.0
#taskmanager.bind-host: localhost
#taskmanager.host: localhost
rest.address: 0.0.0.0
rest.bind-address: 0.0.0.0
1.2、复制默认的 flink-conf.yaml 配置文件,并修改以下配置,然后重命名为 taskmanager1-conf.yaml 并放在 /home/debo/dockerData/flink 目录下
jobmanager.rpc.address: flink-jobmanager
jobmanager.bind-host: 0.0.0.0
taskmanager.bind-host: 0.0.0.0
taskmanager.host: flink-taskmanager1
rest.address: 0.0.0.0
rest.bind-address: 0.0.0.0
1.3、复制默认的 flink-conf.yaml 配置文件,并修改以下配置,然后重命名为 taskmanager2-conf.yaml 并放在 /home/debo/dockerData/flink 目录下
jobmanager.rpc.address: flink-jobmanager
jobmanager.bind-host: 0.0.0.0
taskmanager.bind-host: 0.0.0.0
taskmanager.host: flink-taskmanager2
rest.address: 0.0.0.0
rest.bind-address: 0.0.0.0
1.4、复制默认的 flink-conf.yaml 配置文件,并修改以下配置,然后重命名为 taskmanager3-conf.yaml 并放在 /home/debo/dockerData/flink 目录下
jobmanager.rpc.address: flink-jobmanager
jobmanager.bind-host: 0.0.0.0
taskmanager.bind-host: 0.0.0.0
taskmanager.host: flink-taskmanager3
rest.address: 0.0.0.0
rest.bind-address: 0.0.0.0
2、执行 docker 命令
依次执行以下命令创建 flink 集群
docker run \
-d -p 8081:8081 \
--name flink-jobmanager \
--network flink-network \
--mount type=bind,src=/home/debo/dockerData/flink/jobmanager-conf.yaml,target=/opt/flink/conf/flink-conf.yaml flink:1.16.3-java8 jobmanager
docker run -d --name flink-taskmanager1 --network flink-network --mount type=bind,src=/home/debo/dockerData/flink/taskmanager1-conf.yaml,target=/opt/flink/conf/flink-conf.yaml flink:1.16.3-java8 taskmanager
docker run -d --name flink-taskmanager2 --network flink-network --mount type=bind,src=/home/debo/dockerData/flink/taskmanager2-conf.yaml,target=/opt/flink/conf/flink-conf.yaml flink:1.16.3-java8 taskmanager
docker run -d --name flink-taskmanager3 --network flink-network --mount type=bind,src=/home/debo/dockerData/flink/taskmanager3-conf.yaml,target=/opt/flink/conf/flink-conf.yaml flink:1.16.3-java8 taskmanager
部署单台flinkstandalone(建议)
编写docker-compose.yml
version: "2.1"
services:
jobmanager:
image: flink:latest
expose:
- "6123"
ports:
- "8081:8081"
command: jobmanager
environment:
- JOB_MANAGER_RPC_ADDRESS=jobmanager
taskmanager:
image: flink:latest
expose:
- "6121"
- "6122"
depends_on:
- jobmanager
command: taskmanager
links:
- "jobmanager:jobmanager"
environment:
- JOB_MANAGER_RPC_ADDRESS=jobmanager
在上面的配置中,我们定义了两个服务:jobmanager 和 taskmanager。jobmanager 服务负责管理作业,taskmanager 服务负责执行作业。我们将Flink的Web UI端口映射到宿主机的8081端口,以便查看作业状态和日志。
启动Flink集群
现在,我们可以使用Docker Compose来启动Flink集群。在终端中执行以下命令:
docker-compose up
http://localhost:8081/#/overview
单台方式2
拉取
docker pull flink:latest
创建
docker run -d --name flink_container -p 8081:8081 flink:latest
启动
docker start flink_container