08 traefik部署与灰度流量

998 阅读8分钟
  • 安装traefik,有两套方案。推荐方案1

一、方案一

1.1、使用helm安装traefik

[root@VM-16-14-centos ~]# kubectl get cs
Warning: v1 ComponentStatus is deprecated in v1.19+
NAME                 STATUS      MESSAGE                                                                                       ERROR
scheduler            Unhealthy   Get "http://127.0.0.1:10251/healthz": dial tcp 127.0.0.1:10251: connect: connection refused
etcd-0               Healthy     {"health":"true","reason":""}
controller-manager   Healthy     ok

[root@VM-16-14-centos ~]# kubectl get nodes
NAME              STATUS   ROLES                  AGE    VERSION
vm-16-14-centos   Ready    control-plane,master   4d6h   v1.22.2
vm-16-4-centos    Ready    <none>                 4d5h   v1.22.2
vm-16-6-centos    Ready    <none>                 4d5h   v1.22.2

[root@VM-16-14-centos ~]# helm repo add traefik https://helm.traefik.io/traefik
"traefik" has been added to your repositories

[root@VM-16-14-centos ~]# helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "aliyun" chart repository
...Successfully got an update from the "stable" chart repository
...Successfully got an update from the "elastic" chart repository
...Successfully got an update from the "harbor" chart repository
...Successfully got an update from the "apphub" chart repository
...Successfully got an update from the "traefik" chart repository
...Successfully got an update from the "bitnami" chart repository
...Successfully got an update from the "ingress-nginx" chart repository
...Successfully got an update from the "gitlab" chart repository
Update Complete. ⎈Happy Helming!⎈

[root@VM-16-14-centos ~]# helm install traefik traefik/traefik
NAME: traefik
LAST DEPLOYED: Thu Oct 28 23:47:01 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None

[root@VM-16-14-centos data]# kubectl get pods -A | grep traefik
default                traefik-676b85dd4d-4h7sw                     1/1     Running     0              2m55s

[root@VM-16-14-centos data]# kubectl get svc -A | grep traefik
default                traefik                              LoadBalancer   10.254.23.180    <pending>     80:32112/TCP,443:32700/TCP     3m

[root@VM-16-14-centos data]# kubectl get crd | grep traefik
ingressroutes.traefik.containo.us           2021-10-28T15:46:59Z
ingressroutetcps.traefik.containo.us        2021-10-28T15:46:59Z
ingressrouteudps.traefik.containo.us        2021-10-28T15:46:59Z
middlewares.traefik.containo.us             2021-10-28T15:46:59Z
middlewaretcps.traefik.containo.us          2021-10-28T15:47:00Z
serverstransports.traefik.containo.us       2021-10-28T15:47:00Z
tlsoptions.traefik.containo.us              2021-10-28T15:47:00Z
tlsstores.traefik.containo.us               2021-10-28T15:47:00Z
traefikservices.traefik.containo.us         2021-10-28T15:47:00Z

[root@VM-16-14-centos data]# kubectl get sa | grep traefik
traefik                   1         6m47s

1.2、配置

E1、LoadBalancer一直pending状态

  • GCE环境可用(自动分配公网IP、并绑定),非GCE勿用

image.png

[root@VM-16-14-centos ~]# kubectl edit svc traefik

  • 46行。若注释 默认即 ClusterIP模式,根据部署形态来定。这里修改为NodePort

image.png

  • 修改为DaemonSet模式
  • 原配置文件为Deployment模式,修改后对应的配置值都需要修改

image.png

1.2.1 部署形态

[root@VM-16-14-centos data]# helm show values traefik/traefik > traefik_values.yml

[root@VM-16-14-centos data]# vim traefik_values.yml
kind: Deployment
为
kind: DaemonSet

注释replicas
  #replicas: 1
  #progressDeadlineSeconds: 600
  #strategy:
    #rollingUpdate:
      #maxSurge: 1
      #maxUnavailable: 1
    #type: RollingUpdate
  #availableReplicas: 1
  #lastUpdateTime: "2021-10-28T15:47:02Z"
  #lastUpdateTime: "2021-10-28T15:47:42Z"  
  #readyReplicas: 1
  #replicas: 1
  #updatedReplicas: 1

1.2.2 配置

[root@VM-16-14-centos data]# vim traefik_values.yml

#修改为NodePort
  type: LoadBalancer
  
  type: NodePort

#开启访问日志
  access:
    # To enable access logs
    enabled: false

  access:
    # To enable access logs
    enabled: true

#开启dashborad
[root@VM-16-14-centos data]# vim traefik_values.yml
  traefik:
    port: 9000
    expose: false

  traefik:
    port: 9000
    expose: true  

# ture 即可通过9000 nodeport访问dashboard

1.3、helm基本使用

image.png

[root@VM-16-14-centos data]# helm show values traefik/traefik

1.4、traefik基本概念

当启动 Traefik 时,需要定义 entrypoints(入口点),然后,根据连接到这些 entrypoints 的路由来分析传入的请求,来查看他们是否与一组规则相匹配,如果匹配,则路由可能会将请求通过一系列中间件转换过后再转发到你的服务上去。在了解 Traefik 之前有几个核心概念我们必须要了解:

  • Providers 用来自动发现平台上的服务,可以是编排工具、容器引擎或者 key-value 存储等,比如 Docker、Kubernetes、File
  • Entrypoints 监听传入的流量(端口等…),是网络入口点,它们定义了接收请求的端口(HTTP 或者 TCP)。
  • Routers 分析请求(host, path, headers, SSL, …),负责将传入请求连接到可以处理这些请求的服务上去。
  • Services 将请求转发给你的应用(load balancing, …),负责配置如何获取最终将处理传入请求的实际服务。
  • Middlewares 中间件,用来修改请求或者根据请求来做出一些判断(authentication, rate limiting, headers, ...),中间件被附件到路由上,是一种在请求发送到你的服务之前(或者在服务的响应发送到客户端之前)调整请求的一种方法。

二、方案二

2.1、手动安装traefik

  • trefik,只有一个go可执行文件,理论可以运行在docker、k8s中,甚至单独部署
  • 官方文档,参考样例配置
  • doc.traefik.io/traefik/use…

三、IP与端口基本概念

3.1、IP

  1. POD IP image.png
  • POD IP,pod 的IP
  1. CLUSTER-IP image.png
  • CLUSTER IP,svc的IP,无论模式是 ClusterIP、NodePort
  • CLUSTER IP,分为 普通ServiceHeadless Service(主要供StatefulSet使用)

3.2、端口

  • NodePort、ClusterIP是基于svc。默认不配置type,则默认为ClusterIP模式。
  • ClusterIP只能用于集群内部访问,默认参数有:
  • port: 80 前端访问端口 。服务访问svc端口、集群内部访问端口
  • targetPort: 80 后端访问端口。pod控制器中定义端口、应用访问端口。targetPort是POD端口,从port和nodePort经过kube-proxy流入到后端pod的targetPort上,最后进入容器。
  • nodePort: NodePort,外部客户端访问svc端口。端口范围固定、但可改。如果不配置则生成随机端口,若指定则定义为固定端口。
  • hostPort: 容器端口与node端口路由、但没有意义、pod被重新调度时需要维护pod与所在node关系、且和node现有端口可能冲突。不建议使用这种端口映射方式。

3.3、NodePort与HostNetwork

  • nodePort模式主要占用的是svc的nodePort端口。而hostNetwork则需要占用物理机的80和443端口。
  • 在ingress,hostNetwork模式不再需要创建一个nodePort的svc,而是直接在每个节点都创建一个ingress-controller的容器,而且将该容器的网络模式设为hostNetwork。也就是说每个节点node物理机的80和443端口将会被ingress-controller中的nginx容器占用。当流量通过80/443端口进入时,将直接进入到nginx中。而后nginx根据ingress规则再将流量转发到对应的web应用容器中。

四、灰度流量

4.1、新建Deployment,使用不同镜像 v1/v2

  • 镜像v1/v2,镜像输出内容不同
  • Service,可写一个配置文件,用----分割;服务名必须全部为小写
#创建Deployment v1/v2,区别镜像指向不同,分别为v1/v2

[root@VM-16-14-centos a]# pwd
/data/myk8s/traefik/a
[root@VM-16-14-centos a]# cat go_hello_8080_dep_v1.yml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: go-hello8080-deploymentv1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: go-hello8080-podv1
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
  template:
    metadata:
      labels:
        app: go-hello8080-podv1
    spec:
      containers:
        - name: go-hello8080-containerv1
          #image: 127.0.0.1:81/mylibary/go_hello:v1  #镜像名称+版本
          image: 1.13.15.220:81/mylibary/go_hello_8080:v1  #镜像名称+版本
          imagePullPolicy: Always #表示镜像来源,IfNotPresent本地没有就从hub仓库拉取,Never表示只从本地
          #command: ["/bin/bash","-ce","tail -f /dev/null"]
          ports:
            - containerPort: 8080
      imagePullSecrets:
      #- name: harborsecret
      - name: harbor-registry-key

[root@VM-16-14-centos a]# cat go_hello_8080_dep_v2.yml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: go-hello8080-deploymentv2
spec:
  replicas: 1
  selector:
    matchLabels:
      app: go-hello8080-podv2
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
  template:
    metadata:
      labels:
        app: go-hello8080-podv2
    spec:
      containers:
        - name: go-hello8080-containerv2
          #image: 127.0.0.1:81/mylibary/go_hello:v1  #镜像名称+版本
          image: 1.13.15.220:81/mylibary/go_hello_8080:v2  #镜像名称+版本
          imagePullPolicy: Always #表示镜像来源,IfNotPresent本地没有就从hub仓库拉取,Never表示只从本地
          #command: ["/bin/bash","-ce","tail -f /dev/null"]
          ports:
            - containerPort: 8080
      imagePullSecrets:
      #- name: harborsecret
      - name: harbor-registry-key
      
#创建Service,可写一个配置文件,用----分割
[root@VM-16-14-centos a]# cat go_hello_8080_svc.yml
apiVersion: v1
kind: Service
metadata:
  name: go-hello8080-svc-v1
spec:
  selector:
    app: go-hello8080-podv1
  type: NodePort
  ports:
  - name: http
    port: 80
    targetPort: 8080

---

apiVersion: v1
kind: Service
metadata:
  name: go-hello8080-svc-v2
spec:
  selector:
    app: go-hello8080-podv2
  type: NodePort
  ports:
  - name: http
    port: 80
    targetPort: 8080

4.2、创建TraefikService

  • 这里灰度策略为区分权重。其中 v1 3,v2 1
# 创建TraefikService
[root@VM-16-14-centos a]# cat go_hello_8080_TraefikService.yml
apiVersion: traefik.containo.us/v1alpha1
kind: TraefikService
metadata:
  name: go-hello8080-svc-traefikservice-wrr
spec:
  weighted:
    services:
      - name: go-hello8080-svc-v1
        weight: 3
        port: 80
        kind: Service
      - name: go-hello8080-svc-v2
        weight: 1
        port: 80
        kind: Service

4.3、创建IngressRoute并引用TraefikService

# 创建IngressRoute并应用TraefikService
[root@VM-16-14-centos a]# cat go_hello_8080_TraefikService_ingressroute.yml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: go-hello8080-svc-traefikservice-ingressroute
  namespace: default
spec:
  entryPoints:
    - web
  routes:
  - match: Host(`kdb.sohu.com`)
    kind: Rule
    services:
    - name: go-hello8080-svc-traefikservice-wrr
      kind: TraefikService

4.4、验证灰度发布策略

  • v1 权重3,v2 权重1
[root@VM-16-14-centos a]#  kubectl get pods -A
default                go-hello8080-deploymentv1-854b877fc5-n9kdc   1/1     Running   0              33m
default                go-hello8080-deploymentv2-7577cbfd68-skkpv   1/1     Running   0              33m

[root@VM-16-14-centos a]#  kubectl get svc -A
NAMESPACE              NAME                                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                                     AGE
default                go-hello8080-svc-v1                  NodePort    10.254.128.101   <none>        80:32673/TCP                                68m
default                go-hello8080-svc-v2                  NodePort    10.254.221.122   <none>        80:32055/TCP                                90m
default                traefik                              NodePort    10.254.204.220   <none>        9000:32148/TCP,80:31400/TCP,443:30738/TCP   13d

 [root@VM-16-14-centos a]#  kubectl get ingressroute -A
NAMESPACE              NAME                                           AGE
default                go-hello8080-svc-traefikservice-ingressroute   86m

[root@VM-16-14-centos a]#  kubectl get traefikservices.traefik.containo.us -A
NAMESPACE   NAME                                  AGE
default     go-hello8080-svc-traefikservice-wrr   89m

  • 验证:

实验环境:traefik 80使用NodePort,因此写host后,访问kdb.sohu.com:31400/Hello
前3次进入v1,第4次请求进入v2。依次循环。灰度生效。

image.png

image.png

五、中间件

reference: