nginx service mesh结合nginx-ingress管理南北向流量

92 阅读5分钟

自动sideCar注入

sideCar可以代理pod的流量,便于实现微服务的可观察性、监控、日志记录、配置、断路器等功能,nginx service mesh可以自动向Pod中注入sideCar容器

# 将指定命名空间的sideCar添加上label,添加该label后,nginx  service mesh会自动向部署的pod注入sideCar
kubectl label namespaces <namespace name> injector.nsm.nginx.com/auto-inject=enabled

# 如果在自动注入之前有部署的资源,可以重启该资源让nginx service mesh 注入sideCar
# 使用如下命令可以重启pod
kubectl rollout restart <resource type>/<resource name>
# 示例
kubectl rollout restart deployment/frontend

查看自动注入sideCar前后对比结果

  • 注入sideCar之前可以看到Pod里面有一个容器

image.png

  • 注入sideCar之后Pod里面有两个容器,多出来的那个容器即为nginx service mesh自动注入的sideCar

image.png

部署nginx ingress controller

克隆仓库并切换到指定的目录

git clone https://github.com/nginxinc/kubernetes-ingress.git --branch v3.1.1
cd kubernetes-ingress/deployments

配置RBAC

给ingress controller创建命名空间和service account

kubectl apply -f common/ns-and-sa.yaml

为上面创建的service account绑定cluster role和cluster role binding

kubectl apply -f rbac/rbac.yaml

创建公共资源

创建一个configMap用来定制nginx配置

kubectl apply -f common/nginx-config.yaml

创建ingressClass资源

kubectl apply -f common/ingress-class.yaml

创建自定义资源

默认情况下需要创建下面的自定义资源,如果不创建的话Ingress controller会处于not ready的状态

kubectl apply -f common/crds/k8s.nginx.org_virtualservers.yaml
kubectl apply -f common/crds/k8s.nginx.org_virtualserverroutes.yaml
kubectl apply -f common/crds/k8s.nginx.org_transportservers.yaml
kubectl apply -f common/crds/k8s.nginx.org_policies.yaml

部署ingress controller

  • daemon-set目录:可以利用里面的文件创建deamonSet模式的ingress controller,使每个node都有一个pod
  • deployment目录:可以根据需要调整ingress controller数量
# 使用daemonset
kubectl apply -f daemon-set/nginx-ingress.yaml 

查看nginx ingress部署情况

image.png

访问ingress controller

  • 使用demonset方式创建ingress controller时,Ingress Controller容器的80和443端口映射到容器所在节点的相同端口,可以在运行了ingress controller的节点使用该节点的ip和端口访问
  • 使用deployment方式创建ingress controller时,需要创建一个nodetype类型或者loadbalancer的service来暴露,方法参考如何访问部署的ingress controller

部署bookinfo示例项目以测试集群业务是否可以正常跑通

bookinfo项目说明

总体架构

Bookinfo 应用分为四个单独的微服务:

  • productpage. 这个微服务会调用 details 和 reviews 两个微服务,用来生成页面。
  • details. 这个微服务中包含了书籍的信息。
  • reviews. 这个微服务中包含了书籍相关的评论。它还会调用 ratings 微服务。
  • ratings. 这个微服务中包含了由书籍评价组成的评级信息。

review微服务

reviews 微服务有 3 个版本

  • v1 版本不会调用 ratings 服务。
  • v2 版本会调用 ratings 服务,并使用 1 到 5 个黑色星形图标来显示评分信息。
  • v3 版本会调用 ratings 服务,并使用 1 到 5 个红色星形图标来显示评分信息。

项目整体架构

image.png

下载bookinfo 创建pod的yaml文件

wget https://docs.nginx.com/nginx-service-mesh/examples/bookinfo.yaml
# 执行
kubectl apply -f bookinfo.yaml

文件内容说明

# bookinfo.yaml
# Copyright 2017 Istio Authors
#
#   Licensed under the Apache License, Version 2.0 (the "License");
#   you may not use this file except in compliance with the License.
#   You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#   Unless required by applicable law or agreed to in writing, software
#   distributed under the License is distributed on an "AS IS" BASIS,
#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#   See the License for the specific language governing permissions and
#   limitations under the License.

##################################################################################################
# Details service
##################################################################################################
apiVersion: v1
kind: Service
metadata:
  name: details   #  创建details服务
  labels:
    app: details
    service: details
spec:
  ports:
  - port: 9080  # 该服务暴露的端口
    name: http  # 该端口的协议类型
  selector:
    app: details  # 将流量路由到被该标签选中的pod
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: bookinfo-details
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: details-v1  # 通过Deployment部署Pod
  labels:
    app: details
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: details 
      version: v1
  template:
    metadata:
      labels:
        app: details # 指定该pod的label
        version: v1
    spec:
      serviceAccountName: bookinfo-details
      containers:
      - name: details
        image: docker.io/istio/examples-bookinfo-details-v1:1.15.0
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9080
---
##################################################################################################
# Ratings service
##################################################################################################
apiVersion: v1
kind: Service
metadata:
  name: ratings
  labels:
    app: ratings
    service: ratings
spec:
  ports:
  - port: 9080
    name: http
  selector:
    app: ratings
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: bookinfo-ratings
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ratings-v1
  labels:
    app: ratings
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ratings
      version: v1
  template:
    metadata:
      labels:
        app: ratings
        version: v1
    spec:
      serviceAccountName: bookinfo-ratings
      containers:
      - name: ratings
        image: docker.io/istio/examples-bookinfo-ratings-v1:1.15.0
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9080
---
##################################################################################################
# Reviews service
##################################################################################################
apiVersion: v1
kind: Service
metadata:
  name: reviews
  labels:
    app: reviews
    service: reviews
spec:
  ports:
  - port: 9080
    name: http
  selector:
    app: reviews
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: bookinfo-reviews
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: reviews-v1
  labels:
    app: reviews
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: reviews
      version: v1
  template:
    metadata:
      labels:
        app: reviews
        version: v1
    spec:
      serviceAccountName: bookinfo-reviews
      containers:
      - name: reviews
        image: docker.io/istio/examples-bookinfo-reviews-v1:1.15.0
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9080
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: reviews-v2
  labels:
    app: reviews
    version: v2
spec:
  replicas: 1
  selector:
    matchLabels:
      app: reviews
      version: v2
  template:
    metadata:
      labels:
        app: reviews
        version: v2
    spec:
      serviceAccountName: bookinfo-reviews
      containers:
      - name: reviews
        image: docker.io/istio/examples-bookinfo-reviews-v2:1.15.0
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9080
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: reviews-v3
  labels:
    app: reviews
    version: v3
spec:
  replicas: 1
  selector:
    matchLabels:
      app: reviews
      version: v3
  template:
    metadata:
      labels:
        app: reviews
        version: v3
    spec:
      serviceAccountName: bookinfo-reviews
      containers:
      - name: reviews
        image: docker.io/istio/examples-bookinfo-reviews-v3:1.15.0
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9080
---
##################################################################################################
# Productpage services
##################################################################################################
apiVersion: v1
kind: Service
metadata:
  name: productpage
  labels:
    app: productpage
    service: productpage
spec:
  ports:
  - port: 9080
    name: http
  selector:
    app: productpage
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: bookinfo-productpage
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: productpage-v1
  labels:
    app: productpage
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: productpage
      version: v1
  template:
    metadata:
      labels:
        app: productpage
        version: v1
    spec:
      serviceAccountName: bookinfo-productpage
      containers:
      - name: productpage
        image: docker.io/istio/examples-bookinfo-productpage-v1:1.15.0
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9080
---

查看pod创建情况

image.png

下载bookinfo ingress文件

wget https://docs.nginx.com/nginx-service-mesh/examples/nginx-ingress-controller/bookinfo-ingress.yaml

修改为如下内容

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: bookinfo-ingress
spec:
  ingressClassName: nginx # use only with k8s version >= 1.18.0
  tls:
  rules:
  - host: www.52anime.cn  # 这里可以修改为自己想使用的域名
    http:
      paths:
      - path: /
        pathType: Prefix  # 前缀匹配
        backend:
          service:
            name: productpage    # 访问上面的/路径流量会转由productpage service处理
            port:
              number: 9080

使用修改文件创建ingress资源

kubectl apply -f bookinfo-ingress.yaml

查看ingress

image.png

访问网站,查看效果

image.png

使用prometheus和grafana就行监控