k8s dashboard 的http接口改造

2,881

背景: 玩过Kubernetes的人都知道,官方提供了一种集群web插件dashboard,使用官方示例可以快速的部署一套dashboard,可以方便相关人员进行集群概况预览.但是官方的实例默认使用了https并且需要通过证书或Token来进行统一认证,而dashboard这种内部基础工具增加了https和证书认证后不仅使得使用的成本高了起来,而且和内部的统一管理入口也不太好集成(通常内部系统都会统一使用nginx之类的代理工具进行统一代理).

本篇文章主要介绍下如何改造官方dashboard集群运行参数,将dashboard使用HTTP方式对外暴露.

官方Dashboard搭建

注意:当前官方版本是基于k8s 1.15的版本

官方已经给了在k8s集群运行的基础示例,直接运行即可:

$ wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-beta4/aio/deploy/recommended.yaml

$ kubectl apply -f recommended.yaml

$ kubectl  get pod,deploy,svc -n kubernetes-dashboard
NAME                                            READY   STATUS    RESTARTS   AGE
pod/dashboard-metrics-scraper-fb986f88d-dz4jd   1/1     Running   0          5m18s
pod/kubernetes-dashboard-6bb65fcc49-vsfnk       1/1     Running   0          5m18s

NAME                                              READY   UP-TO-DATE   AVAILABLE   AGE
deployment.extensions/dashboard-metrics-scraper   1/1     1            1           5m18s
deployment.extensions/kubernetes-dashboard        1/1     1            1           5m18s

NAME                                TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
service/dashboard-metrics-scraper   ClusterIP   10.253.61.79    <none>        8000/TCP   5m18s
service/kubernetes-dashboard        ClusterIP   10.253.233.70   <none>        443/TCP    5m18s

# 我们可以看到官方的dashboard帮我们启动了web-ui,并且帮我们启动了一个Metric服务
# 但是dashboard默认使用的https的443端口

# 测试下Dashboard是否正常
$ curl https://10.253.233.70:443 -k -I
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: no-store
Content-Length: 1262
Content-Type: text/html; charset=utf-8
Last-Modified: Thu, 29 Aug 2019 09:14:59 GMT
Date: Sun, 08 Sep 2019 04:27:08 GMT

如上,我们可以看到kubernetes-dashboard暴露了一个443端口,测试访问返回200,即证明dashboard暂时启动成功,至于如何去访问,你可以选择使用NodePort类型的Service将服务端口映射在node节点的随机端口,也可以选择使用Ingress之类的入口服务进行代理访问。

Dashboard的HTTP方式改造

k8s-dashboard项目是使用Golang语言开发的一个简单的k8s集群的web ui.我们查看到官方的Dockerfile核心配置如下:

dashboard-dockerfile

# 核心内容如下
$ cat Dockerfile
FROM scratch
ADD . /
EXPOSE 9090 8443
ENTRYPOINT ["/dashboard", "--insecure-bind-address=0.0.0.0", "--bind-address=0.0.0.0"]

可以看到,Dashboard项目其实暴露了一个非安全端口9090,因为该项目是前后端分离的,非安全端口主要用于前端项目去调用后端接口数据使用,但由于通常我们的dashboard都只是用作内部系统访问,安全风险会相对比较低,因此在考虑便捷性上面,我们可以将非安全端口对外使用.

修改recommended.yaml文件

  • 1.容器组暴露9090端口
  • 2.增加9090端口的http探针检测
  • 3.Service暴露9090端口
  • 4.Service采用NodePort类型
# service处的核心配置
kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  ports:
    - port: 443
      name: https
      targetPort: 8443
    - port: 9090
      name: http
      targetPort: 9090
  type: NodePort
  selector:
    k8s-app: kubernetes-dashboard

# deploy处的核心配置
      containers:
        - name: kubernetes-dashboard
          image: kubernetesui/dashboard:v2.0.0-beta4
          imagePullPolicy: Always
          ports:
            - containerPort: 8443
              protocol: TCP
            - containerPort: 9090
              protocol: TCP

          livenessProbe:
            httpGet:
              scheme: HTTPS
              path: /
              port: 8443
            httpGet:
              scheme: HTTP
              port: 9090
            initialDelaySeconds: 30
            timeoutSeconds: 30

# 再次更新配置
$ kubectl apply -f recommended.yaml

$ kubectl  get pod,deploy,svc -n kubernetes-dashboard
NAME                                            READY   STATUS             RESTARTS   AGE
pod/dashboard-metrics-scraper-fb986f88d-dz4jd   1/1     Running            0          53m
pod/kubernetes-dashboard-648df46b87-z727t       0/1     CrashLoopBackOff   11         28m

NAME                                              READY   UP-TO-DATE   AVAILABLE   AGE
deployment.extensions/dashboard-metrics-scraper   1/1     1            1           53m
deployment.extensions/kubernetes-dashboard        0/1     1            0           53m

NAME                                TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                        AGE
service/dashboard-metrics-scraper   ClusterIP   10.253.61.79    <none>        8000/TCP                       53m
service/kubernetes-dashboard        NodePort    10.253.233.70   <none>        443:48382/TCP,9090:43275/TCP   53m

# 访问非安全端口(也可以访问node节点的43275)
$ curl -I 10.253.233.70:9090
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: no-store
Content-Length: 1262
Content-Type: text/html; charset=utf-8
Last-Modified: Thu, 29 Aug 2019 09:14:59 GMT
Date: Sun, 08 Sep 2019 04:57:16 GMT

$ curl -I  172.16.21.24:43275
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: no-store
Content-Length: 1262
Content-Type: text/html; charset=utf-8
Last-Modified: Thu, 29 Aug 2019 09:14:59 GMT
Date: Sun, 08 Sep 2019 04:58:54 GMT

接下来就可以直接访问dashboard的页面了。

后续问题

此时,会发现一个问题,就是没有默认显示集群已有的资源,查看右上角的通知可以看到相关信息。

大概异常通知信息如下:

secrets is forbidden: User "system:serviceaccount:kubernetes-dashboard:kubernetes-dashboard" cannot list resource "secrets" in API group "" in the namespace "default"

其实很明显就是用户system:serviceaccount:kubernetes-dashboard:kubernetes-dashboard没有相关权限,在recommended.yaml增加想要dashboard去查看的资源即可.

# 增加一些想要在dashboard上操作的资源
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
rules:
  # Allow Metrics Scraper to get metrics from the Metrics server
  - apiGroups: ["metrics.k8s.io"]
    resources: ["pods", "nodes","namespaces","secrets","persistentvolumeclaims"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["","apps"]
    resources: ["pods", "nodes","namespaces","secrets","persistentvolumeclaims","replicasets","deployments","events"]
    verbs: ["get", "list", "watch"]

使用Http方式可以访问服务后,就可以直接将dashboard代理到Ingress或者其他内部的API网关中直接使用了。欢迎关注我的公众号Wechat.jpeg