1.理解Ingress
使用 Ingress 时一般会有三个组件:
- 1.1 反向代理负载均衡器:向代理负载均衡器对进入集群的外部流量进行负载分发。现在反向代理负载均衡器通和Ingress Controller已经合并为一个组件（1.1和1.2你可以认为是一个，在kubernetes集群内以pod的方式存在）
- 1.2 Ingress Controller: Ingress Controller与Kubernetes的API交互，实时感知后端Service、pod的变化，再结合Ingress策略集生成配置，然后更新反向代理负载均衡器的配置，实现动态服务发现与更新。
- 1.3 Ingress（ingress策略集) : Ingress策略集定义了集群外的流量到达集群内的Service的规则。
2.介绍traefik
Traefik是一款开源的反向代理与负载均衡工具。它最大的优点是能够与常见的微服务系统直接整合，实现自动化动态配置。Traefik通过不断地跟 kubernetes API 打交道，实时的感知后端 service、pod 等变化，比如pod，service 增加与减少等；当得到这些变化信息后，Ingress自动更新配置并热重载 ，达到服务发现的作用。
3.环境介绍
3.1 集群节点
| hostname | ip address |
|---|---|
| k8s-master01 | 192.168.1.10 |
| k8s-node01 | 192.168.1.20 |
| k8s-node02 | 192.168.1.21 |
| k8s-node03 | 192.168.1.23 |
3.2 Ingress规则
| host | traefik-service | traefik-ingress-controller | service | pod |
|---|---|---|---|---|
| cheddar.minikube:30189 | traefik-ingress-service:80 | traefik-ingress-controller:80 | cheddar:80 | 10.244.2.180:80, 10.244.2.183:80,10.244.2.167:80 |
| stilton.minikube:30189 | traefik-ingress-service:80 | traefik-ingress-controller:80 | stilton:80 | 10.244.2.191:80, 10.244.2.172:80, 10.244.2.189:80 |
| wensleydale.minikube:30189 | traefik-ingress-service:80 | traefik-ingress-controller:80 | wensleydale:80 | 10.244.2.166:80, 10.244.2.177:80, 10.244.2.176:80 |
3.3 拓扑图
3.4 traefik dashboard
| host | traefik-service | traefik-ingress-controller |
|---|---|---|
| traefik-ui.minikube:32092 | traefik-ingress-service:8080 | traefik-ingress-controller:8080 |
4.设置RBAC(Role-Based Access Control)
首先要让traefik有相应的权限，将会进行一下设置：
- 1.为traefik创建一个ClusterRole，并给予足够的权限
- 2.为traefik创建一个ServiceAccount
- 3.通过ClusterRoleBinding关联1，2两步创建的ClusterRole和ServiceAccount，让使用ServiceAccount的用户（这里的用户指的是traefik,第8在创建traefik-ingress-controller的时候会关联ServiceAccount）拥有ClusterRole所设定的权限。
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: traefik-ingress-controller
rules:
- apiGroups:
- ""
resources: #资源1
- services
- endpoints
- secrets
verbs: #对资源1所拥有的的权限
- get
- list
- watch
- apiGroups:
- extensions
resources:
- ingresses #资源2
verbs: #对资源2所拥有的权限
- get
- list
- watch
- apiGroups:
- extensions
resources:
- ingresses/status
verbs:
- update
---
kind: ServiceAccount
apiVersion: v1
metadata:
name: traefik-ingress-controller
namespace: traefik
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: traefik-ingress-controller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: traefik-ingress-controller #绑定的ClusterRole的名字
subjects:
- kind: ServiceAccount
name: traefik-ingress-controller #绑定的ServiceAccount的名字
namespace: traefik
#查看创建的ClusterRole
[root@k8s-master01 ~]# kubectl describe clusterroles traefik-ingress-controller
Name: traefik-ingress-controller
Labels: <none>
Annotations: <none>
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
endpoints [] [] [get list watch]
secrets [] [] [get list watch]
services [] [] [get list watch]
ingresses.extensions [] [] [get list watch]
ingresses.extensions/status [] [] [update]
#查看创建的ServiceAccount
[root@k8s-master01 ~]# kubectl describe serviceaccount -n traefik traefik-ingress-controller
Name: traefik-ingress-controller
Namespace: traefik
Labels: <none>
Annotations: <none>
Image pull secrets: <none>
Mountable secrets: traefik-ingress-controller-token-jml65
Tokens: traefik-ingress-controller-token-jml65
Events: <none>
#查看创建的ClusterRoleBinding
[root@k8s-master01 ~]# kubectl describe clusterrolebinding traefik-ingress-controller -n traefik
Name: traefik-ingress-controller
Labels: <none>
Annotations: <none>
Role:
Kind: ClusterRole
Name: traefik-ingress-controller
Subjects:
Kind Name Namespace
---- ---- ---------
ServiceAccount traefik-ingress-controller traefik
5.创建Deployment
- 分别根据后端的服务，定义3个Deployment:cheddar,stilton,wensleydale。
- 通过定义Deployment来创建ReplicaSet,ReplicaSet会根据Deployment中定义tempalte来创建用户期望的副本数量的pod。
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
namespace: traefik
name: stilton
labels:
app: cheese
cheese: stilton
spec:
replicas: 3
selector:
matchLabels:
app: cheese
task: stilton
template:
metadata:
labels:
app: cheese
task: stilton
version: v0.0.1
spec:
containers:
- name: cheese
image: errm/cheese:stilton
ports:
- containerPort: 80
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
namespace: traefik
name: cheddar
labels:
app: cheese
cheese: cheddar
spec:
replicas: 3
selector:
matchLabels:
app: cheese
task: cheddar
template:
metadata:
labels:
app: cheese
task: cheddar
version: v0.0.1
spec:
containers:
- name: cheese
image: errm/cheese:cheddar
ports:
- containerPort: 80
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
namespace: traefik
name: wensleydale
labels:
app: cheese
cheese: wensleydale
spec:
replicas: 3
selector:
matchLabels:
app: cheese
task: wensleydale
template:
metadata:
labels:
app: cheese
task: wensleydale
version: v0.0.1
spec:
containers:
- name: cheese
image: errm/cheese:wensleydale
ports:
- containerPort: 80
#查看创建的Deployment
[root@k8s-master01 ~]# kubectl get deployments -n traefik -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
cheddar 3/3 3 3 18h cheese errm/cheese:cheddar app=cheese,task=cheddar
stilton 3/3 3 3 18h cheese errm/cheese:stilton app=cheese,task=stilton
traefik-ingress-controller 1/1 1 1 19h traefik-ingress-lb traefik:v1.7 k8s-app=traefik-ingress-lb
wensleydale 3/3 3 3 18h cheese errm/cheese:wensleydale app=cheese,task=wensleydale
#查看Deployment创建ReplicaSet
[root@k8s-master01 ~]# kubectl get replicasets -n traefik -o wide
NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
cheddar-845749dbd6 3 3 3 18h cheese errm/cheese:cheddar app=cheese,pod-template-hash=845749dbd6,task=cheddar
stilton-f89c97cdb 3 3 3 18h cheese errm/cheese:stilton app=cheese,pod-template-hash=f89c97cdb,task=stilton
traefik-ingress-controller-5c86bf9cd7 1 1 1 19h traefik-ingress-lb traefik:v1.7 k8s-app=traefik-ingress-lb,pod-template-hash=5c86bf9cd7
wensleydale-7c5ff658b 3 3 3 18h cheese errm/cheese:wensleydale app=cheese,pod-template-hash=7c5ff658b,task=wensleydale
#查看ReplicSet创建的pod
[root@k8s-master01 ~]# kubectl get pod -n traefik -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
cheddar-845749dbd6-hgpsg 1/1 Running 0 16h 10.244.2.180 k8s-node02 <none> <none>
cheddar-845749dbd6-kx8sx 1/1 Running 0 16h 10.244.2.183 k8s-node02 <none> <none>
cheddar-845749dbd6-v4nch 1/1 Running 0 17h 10.244.2.167 k8s-node02 <none> <none>
stilton-f89c97cdb-4wq79 1/1 Running 0 16h 10.244.2.191 k8s-node02 <none> <none>
stilton-f89c97cdb-772x5 1/1 Running 0 16h 10.244.2.172 k8s-node02 <none> <none>
stilton-f89c97cdb-gzwv9 1/1 Running 0 16h 10.244.2.189 k8s-node02 <none> <none>
traefik-ingress-controller-5c86bf9cd7-grgm5 1/1 Running 0 16h 10.244.2.173 k8s-node02 <none> <none>
wensleydale-7c5ff658b-7plfx 1/1 Running 0 17h 10.244.2.166 k8s-node02 <none> <none>
wensleydale-7c5ff658b-8gspc 1/1 Running 0 16h 10.244.2.177 k8s-node02 <none> <none>
wensleydale-7c5ff658b-ldzdp 1/1 Running 0 16h 10.244.2.176 k8s-node02 <none> <none>
6.创建Service
分别创建3个ClusterIP类型(默认类型)的service来负载均衡第5步创建的3组pod的流量。
---
apiVersion: v1
kind: Service
metadata:
namespace: traefik
name: stilton
spec:
ports:
- name: http
targetPort: 80
port: 80
selector:
app: cheese
task: stilton
---
apiVersion: v1
kind: Service
metadata:
namespace: traefik
name: cheddar
spec:
ports:
- name: http
targetPort: 80
port: 80
selector:
app: cheese
task: cheddar
---
apiVersion: v1
kind: Service
metadata:
namespace: traefik
name: wensleydale
spec:
ports:
- name: http
targetPort: 80
port: 80
selector:
app: cheese
task: wensleydale
查看创建的service
[root@k8s-master01 ~]# kubectl get service -o wide -n traefik
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
cheddar ClusterIP 10.101.171.243 <none> 80/TCP 25h app=cheese,task=cheddar
stilton ClusterIP 10.106.215.138 <none> 80/TCP 25h app=cheese,task=stilton
wensleydale ClusterIP 10.97.14.121 <none> 80/TCP 25h app=cheese,task=wensleydale
7.创建Ingress（Ingress策略集）
分别创建3个Ingress规则，定义访问的域名应该负载到哪个service。
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
namespace: traefik
name: cheese
spec:
rules:
- host: stilton.minikube #访问后端服务的域名
http:
paths:
- path: /
backend:
serviceName: stilton #负载后端服务的serivce
servicePort: http
- host: cheddar.minikube
http:
paths:
- path: /
backend:
serviceName: cheddar
servicePort: http
- host: wensleydale.minikube
http:
paths:
- path: /
backend:
serviceName: wensleydale
servicePort: http
查看创建的Ingress
[root@k8s-master01 ~]# kubectl describe ingresses -n traefik cheese
Name: cheese
Namespace: traefik
Address:
Default backend: default-http-backend:80 (<none>)
Rules:
Host Path Backends
---- ---- --------
stilton.minikube
/ stilton:http (10.244.2.172:80,10.244.2.189:80,10.244.2.191:80)
cheddar.minikube
/ cheddar:http (10.244.2.167:80,10.244.2.180:80,10.244.2.183:80)
wensleydale.minikube
/ wensleydale:http (10.244.2.166:80,10.244.2.176:80,10.244.2.177:80)
Annotations:
Events: <none>
8.配置traefik
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: traefik-ingress-controller
namespace: traefik
labels:
k8s-app: traefik-ingress-lb
spec:
replicas: 1
selector:
matchLabels:
k8s-app: traefik-ingress-lb
template:
metadata:
labels:
k8s-app: traefik-ingress-lb
name: traefik-ingress-lb
spec:
#使traefik拥有名为traefik-ingress-controller的ServiceAccount权限（在第2步中通过ClusterRoleBinding将ServiceAccount和ClusterRole进行绑定，让ServiceAccount拥有了相应的权限）
serviceAccountName: traefik-ingress-controller
terminationGracePeriodSeconds: 60
containers:
- image: traefik:v1.7
name: traefik-ingress-lb
ports:
- name: http
containerPort: 80
- name: admin
containerPort: 8080
args:
- --api
- --kubernetes
- --logLevel=INFO
---
kind: Service
apiVersion: v1
metadata:
name: traefik-ingress-service
namespace: traefik
spec:
selector:
k8s-app: traefik-ingress-lb
ports:
- protocol: TCP
port: 80 #traefik-ingress-controllert提供负载均衡器的内部端口
name: web
- protocol: TCP
port: 8080 #traefik-ingress-controllert提供Dashboard访问的内部端口
name: admin
type: NodePort #service的类型为NodePort，直接将traefik-ingress-controllert的服务对外暴露
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: traefik-web-ui
namespace: traefik
spec:
rules:
- host: traefik-ui.minikube #访问traefik dashboard的域名
http:
paths:
- path: /
backend:
serviceName: traefik-ingress-service #负载traefik dashboard服务的service
servicePort: web
#查看创建的traefik的Deployment
[root@k8s-master01 ~]# kubectl get deployments -n traefik -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
traefik-ingress-controller 1/1 1 1 19h traefik-ingress-lb traefik:v1.7 k8s-app=traefik-ingress-lb
[root@k8s-master01 ~]# kubectl get replicaset -n traefik -o wide
NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
traefik-ingress-controller-5c86bf9cd7 1 1 1 19h traefik-ingress-lb traefik:v1.7 k8s-app=traefik-ingress-lb,pod-template-hash=5c86bf9cd7
[root@k8s-master01 ~]# kubectl get pod -n traefik -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
traefik-ingress-controller-5c86bf9cd7-grgm5 1/1 Running 0 16h 10.244.2.173 k8s-node02 <none> <none>
#查看创建的traefik的service
[root@k8s-master01 ~]# kubectl get service -o wide -n traefik
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
traefik-ingress-service NodePort 10.97.120.98 <none> 80:30189/TCP,8080:32092/TCP 19h k8s-app=traefik-ingress-lb
#查看创建的traefik的ingress
[root@k8s-master01 ~]# kubectl describe ingresses -n traefik traefik-web-ui
Name: traefik-web-ui
Namespace: traefik
Address:
Default backend: default-http-backend:80 (<none>)
Rules:
Host Path Backends
---- ---- --------
traefik-ui.minikube
/ traefik-ingress-service:web (10.244.2.173:80)
Annotations:
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{},"name":"traefik-web-ui","namespace":"traefik"},"spec":{"rules":[{"host":"traefik-ui.minikube","http":{"paths":[{"backend":{"serviceName":"traefik-ingress-service","servicePort":"web"},"path":"/"}]}}]}}
Events: <none>
9.访问验证
9.1 客户端添加hosts记录
解析到集群中任意一台node都可以。
vim /etc/hosts
添加以下4行:
192.168.1.10 traefik-ui.minikube
192.168.1.10 cheddar.minikub
192.168.1.10 stilton.minikube
192.168.1.10 wensleydale.minikube
9.2 登录traefik dashboard
访问http://traefik-ui.minikube:32092
9.3 分别通过域名访问3个服务
访问http://cheddar.minikube:30189
