官方文档:
大家可以先看官方的文档进行安装,如果有问题了,可以在看后续章节。
nightlies.apache.org/flink/flink…
nightlies.apache.org/flink/flink…
1-部署流程
1.1-首先我们需要安装Docker和K8s,我这里使用的是Docker Desktop
1.2-安装cert-manager
kubectl create -f https://github.com/jetstack/cert-manager/releases/download/v1.8.2/cert-manager.yaml
1.3-使用heml安装
安装方式一:
helm repo add flink-operator-repo https://downloads.apache.org/flink/flink-kubernetes-operator-1.0.1/
helm install flink-kubernetes-operator flink-operator-repo/flink-kubernetes-operator
如果失败,可以看看是否镜像拉取不下来,可以手动拉取一下:
docker pull ghcr.io/apache/flink-kubernetes-operator:1.0.1
安装成功后可以基于命令查看 kubectl get pod
安装方式二:
我们也可以自己下载源码去构建镜像:
docker build . -t <repo>/flink-kubernetes-operator:latest
基于自己构建的镜像去安装
helm install flink-kubernetes-operator helm/flink-kubernetes-operator --set image.repository=<repo>/flink-kubernetes-operator --set image.tag=latest
比如我打出来的是:
image: lizu/flink-kubernetes-operator
helm install flink-kubernetes-operator flink-operator-repo/flink-kubernetes-operator --set image.repository=lizu/flink-kubernetes-operator --set image.tag=latest
卸载:
helm uninstall flink-kubernetes-operator
如果需要提交到指定的namespace,并且指定了serviceAccount,则需要执行如下操作:
kubectl create namespace flink-operator
kubectl create serviceaccount flink --namespace=flink-operator
kubectl create clusterrolebinding flink-role --clusterrole=edit --serviceaccount=flink-operator:flink --namespace=flink-operator
1.4-提交任务,使用官方提供的demo
kubectl create -f https://raw.githubusercontent.com/apache/flink-kubernetes-operator/release-1.0/examples/basic.yaml
内容如下:
################################################################################
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
################################################################################
apiVersion: flink.apache.org/v1beta1
kind: FlinkDeployment
metadata:
name: basic-example
spec:
image: flink:1.15
flinkVersion: v1_15
flinkConfiguration:
taskmanager.numberOfTaskSlots: "2"
serviceAccount: flink
jobManager:
resource:
memory: "1024m"
cpu: 1
taskManager:
resource:
memory: "1024m"
cpu: 1
job:
jarURI: local:///opt/flink/examples/streaming/StateMachineExample.jar
parallelism: 2
upgradeMode: stateless
查看启动情况:
1.5-查看日志,开启ui,删除
kubectl logs -f deploy/basic-example
kubectl port-forward svc/basic-example-rest 8081
kubectl delete flinkdeployment/basic-example
2-部署方式
支持两种部署,可以直接去官方文档查看
The Flink Kubernetes Operator supports two main types of deployments: **Application** and **Session**
https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/custom-resource/overview/#session-cluster-deployments
详细的参数可以查看
https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/custom-resource/reference/
3-通过java代码提交
最后简单试一下基于java代码提交flink任务到k8s吧
<dependency>
<groupId>io.fabric8</groupId>
<artifactId>kubernetes-client</artifactId>
</dependency>
/**
lizu test
*/
Config config = new ConfigBuilder()
.withMasterUrl("https://kubernetes.docker.internal:6443")
.build();
KubernetesClient client = new DefaultKubernetesClient(config);
CustomResourceDefinitionContext context = new CustomResourceDefinitionContext.Builder()
.withGroup("flink.apache.org")
.withVersion("v1beta1")
.withScope("Namespaced")
.withName("flink-kubernetes-operator")
.withPlural("flinkdeployments")
.withKind("FlinkDeployment")
.build();
MixedOperation<GenericKubernetesResource, GenericKubernetesResourceList, Resource<GenericKubernetesResource>> resourceMixedOperation =
client.genericKubernetesResources(context);
resourceMixedOperation.inNamespace("default")
.load(K8sUtils.class.getResourceAsStream("/flink-pi.yaml"))
.createOrReplace();
//这里比较简单,提交之后还可以基于api进行监控。大家可以自行实现。
4-总结
基于官方operator方式提交任务，基本上安装官方文档来就可以运行起来。
后续会在运行简单demo的基础上去真正应用起来,记录更多实际中的问题。
