用istioctl看istio生成的envoy xds配置（二）

massivezh
249 阅读4分钟

第二部分部署了istio的规则，看下具体会生成怎样的envoy配置

配置环境

首先部署全量的DestinationRule，着重看reviews的部分

kubectl apply -f samples/bookinfo/networking/destination-rule-all.yaml
部分省略…………
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: reviews
spec:
  host: reviews
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2
  - name: v3
    labels:
      version: v3
---
部分省略…………

部署reviews v2 v3的VirtualService

kubectl apply -f samples/bookinfo/networking/virtual-service-reviews-50-v3.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
    - reviews
  http:
  - route:
    - destination:
        host: reviews
        subset: v2
      weight: 50
    - destination:
        host: reviews
        subset: v3
      weight: 50

查看下全局状态

# istioctl proxy-status
NAME                                                  CLUSTER        CDS        LDS        EDS        RDS          ECDS         ISTIOD                      VERSION
details-v1-7d88846999-6t7f5.default                   Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-7cd55d9dc4-pw4x4     1.15.0
gateway-57d696448d-vrml2.istio-ingress                Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-7cd55d9dc4-pw4x4     1.15.0
istio-egressgateway-775cf5d9b5-h24dz.istio-system     Kubernetes     SYNCED     SYNCED     SYNCED     NOT SENT     NOT SENT     istiod-7cd55d9dc4-pw4x4     1.15.0
istio-ingressgateway-ffbcc4c7f-9rw9j.istio-system     Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-7cd55d9dc4-pw4x4     1.15.0
productpage-v1-5f578dd9b7-kxbq2.default               Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-7cd55d9dc4-pw4x4     1.15.0
ratings-v1-754f9c4975-gbzrn.default                   Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-7cd55d9dc4-pw4x4     1.15.0
reviews-v1-69865ff55-vw54r.default                    Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-7cd55d9dc4-pw4x4     1.15.0
reviews-v2-789d584fbf-jd6rp.default                   Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-7cd55d9dc4-pw4x4     1.15.0
reviews-v3-64f8b69f99-9rhhm.default                   Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-7cd55d9dc4-pw4x4     1.15.0

从productpage的角度看到istio DestinationRule配置已经生效

root@VM-0-3-ubuntu:~/istio-1.15.0# istioctl proxy-config cluster productpage-v1-5f578dd9b7-kxbq2.default
SERVICE FQDN                                            PORT      SUBSET          DIRECTION     TYPE             DESTINATION RULE
ratings.default.svc.cluster.local                       9080      -               outbound      EDS              ratings.default
ratings.default.svc.cluster.local                       9080      v1              outbound      EDS              ratings.default
ratings.default.svc.cluster.local                       9080      v2              outbound      EDS              ratings.default
ratings.default.svc.cluster.local                       9080      v2-mysql        outbound      EDS              ratings.default
ratings.default.svc.cluster.local                       9080      v2-mysql-vm     outbound      EDS              ratings.default
reviews.default.svc.cluster.local                       9080      -               outbound      EDS              reviews.default
reviews.default.svc.cluster.local                       9080      v1              outbound      EDS              reviews.default
reviews.default.svc.cluster.local                       9080      v2              outbound      EDS              reviews.default
reviews.default.svc.cluster.local                       9080      v3              outbound      EDS              reviews.default
部分省略…………

查看详细的routes

# istioctl proxy-config routes productpage-v1-5f578dd9b7-kxbq2.default  --name 9080 -o json
部分省略，只看reviews部分…………
            {
                "name": "reviews.default.svc.cluster.local:9080",
                "domains": [
                    "reviews.default.svc.cluster.local",
                    "reviews.default.svc.cluster.local:9080",
                    "reviews",
                    "reviews:9080",
                    "reviews.default.svc",
                    "reviews.default.svc:9080",
                    "reviews.default",
                    "reviews.default:9080",
                    "172.16.253.81",
                    "172.16.253.81:9080"
                ],
                "routes": [
                    {
                        "match": {
                            "prefix": "/"
                        },
                        "route": {
                            "weightedClusters": {
                                "clusters": [
                                    {
                                        "name": "outbound|9080|v2|reviews.default.svc.cluster.local",
                                        "weight": 50
                                    },
                                    {
                                        "name": "outbound|9080|v3|reviews.default.svc.cluster.local",
                                        "weight": 50
                                    }
                                ],
                                "totalWeight": 100
                            },
部分省略…………
                        "metadata": {
                            "filterMetadata": {
                                "istio": {
                                    "config": "/apis/networking.istio.io/v1alpha3/namespaces/default/virtual-service/reviews"
                                }
                            }
                        },

这里可以发现一些有意思的地方

  1. domains包含所有匹配的url
  2. VirtualService规则都是一一对应生成route
  3. totalWeight是所有权重之和（万一不是100%呢？请看本文最后）
  4. 包含istio VirtualService原始配置的引用，方便debug

查看对应route的cluster

# istioctl proxy-config cluster productpage-v1-5f578dd9b7-kxbq2.default --fqdn reviews.default.svc.cluster.local \
#	--direction outbound --port 9080 -o json
部分省略…………
        "name": "outbound|9080||reviews.default.svc.cluster.local",
        "type": "EDS",
        "edsClusterConfig": {
            "edsConfig": {
                "ads": {},
                "initialFetchTimeout": "0s",
                "resourceApiVersion": "V3"
            },
            "serviceName": "outbound|9080||reviews.default.svc.cluster.local"
部分省略…………
        "metadata": {
            "filterMetadata": {
                "istio": {
                    "config": "/apis/networking.istio.io/v1alpha3/namespaces/default/destination-rule/reviews",
                    "default_original_port": 9080,
                    "services": [
                        {
                            "host": "reviews.default.svc.cluster.local",
                            "name": "reviews",
                            "namespace": "default"
 部分省略…………
        "name": "outbound|9080|v1|reviews.default.svc.cluster.local",
        "type": "EDS",
        "edsClusterConfig": {
            "edsConfig": {
                "ads": {},
                "initialFetchTimeout": "0s",
                "resourceApiVersion": "V3"
            },
            "serviceName": "outbound|9080|v1|reviews.default.svc.cluster.local"
部分省略…………
        "name": "outbound|9080|v2|reviews.default.svc.cluster.local",
        "type": "EDS",
        "edsClusterConfig": {
            "edsConfig": {
                "ads": {},
                "initialFetchTimeout": "0s",
                "resourceApiVersion": "V3"
            },
            "serviceName": "outbound|9080|v2|reviews.default.svc.cluster.local"
        },
 部分省略…………
        "name": "outbound|9080|v3|reviews.default.svc.cluster.local",
        "type": "EDS",
        "edsClusterConfig": {
            "edsConfig": {
                "ads": {},
                "initialFetchTimeout": "0s",
                "resourceApiVersion": "V3"
            },
            "serviceName": "outbound|9080|v3|reviews.default.svc.cluster.local"
        },

同样有一些关注点：

  1. 所有DestinationRule都是一一对应的cluster生成
  2. 包含istio destination-rule原始配置的引用，方便debug
  3. 除了显式定义的DestinationRule外，还有一条没有subset的cluster对应默认的kube Service（这一条在pod初始化注入sidecar时就有了）

部署VirtualService前后同一服务的区别

以review为例

# diff -u before after 
--- before      2022-09-09 19:15:35.343941273 +0800
+++ after       2022-09-09 19:16:19.123990192 +0800
@@ -191,12 +191,23 @@
                 ],
                 "routes": [
                     {
-                        "name": "default",
                         "match": {
                             "prefix": "/"
                         },
                         "route": {
-                            "cluster": "outbound|9080||reviews.default.svc.cluster.local",
+                            "weightedClusters": {
+                                "clusters": [
+                                    {
+                                        "name": "outbound|9080|v2|reviews.default.svc.cluster.local",
+                                        "weight": 50
+                                    },
+                                    {
+                                        "name": "outbound|9080|v3|reviews.default.svc.cluster.local",
+                                        "weight": 50
+                                    }
+                                ],
+                                "totalWeight": 100
+                            },
                             "timeout": "0s",
                             "retryPolicy": {
                                 "retryOn": "connect-failure,refused-stream,unavailable,cancelled,retriable-status-codes",
@@ -219,8 +230,15 @@
                                 "grpcTimeoutHeaderMax": "0s"
                             }
                         },
+                        "metadata": {
+                            "filterMetadata": {
+                                "istio": {
+                                    "config": "/apis/networking.istio.io/v1alpha3/namespaces/default/virtual-service/reviews"
+                                }
+                            }
+                        },
                         "decorator": {
-                            "operation": "reviews.default.svc.cluster.local:9080/*"
+                            "operation": "reviews:9080/*"
                         }
                     }
                 ],

可以看到几点变化：

  1. "name": "default"消失了
  2. 默认对应kube Service的一条cluster规则，变成了istio VirtualService对应的两条weightedClusters规则
  3. 新增istio的metadata，用于备注对应的istio原始配置
  4. operation的URL变成了缩略形式

由此可以看出这里的逻辑是，没有istio vs的情况下则根据kube svc生成规则，确保网络行为与没有istio时一致。

如果有istio vs则根据vs生成规则替代kube svc。

最后来看endpoints

# istioctl proxy-config endpoints productpage-v1-5f578dd9b7-kxbq2.default --cluster "outbound|9080||reviews.default.svc.cluster.local"
ENDPOINT             STATUS      OUTLIER CHECK     CLUSTER
172.16.0.12:9080     HEALTHY     OK                outbound|9080||reviews.default.svc.cluster.local
172.16.0.13:9080     HEALTHY     OK                outbound|9080||reviews.default.svc.cluster.local
172.16.0.14:9080     HEALTHY     OK                outbound|9080||reviews.default.svc.cluster.local

root@VM-0-3-ubuntu:~/istio-1.15.0# istioctl proxy-config endpoints productpage-v1-5f578dd9b7-kxbq2.default --cluster "outbound|9080|v1|reviews.default.svc.cluster.local"
ENDPOINT             STATUS      OUTLIER CHECK     CLUSTER
172.16.0.14:9080     HEALTHY     OK                outbound|9080|v1|reviews.default.svc.cluster.local

root@VM-0-3-ubuntu:~/istio-1.15.0# istioctl proxy-config endpoints productpage-v1-5f578dd9b7-kxbq2.default --cluster "outbound|9080|v2|reviews.default.svc.cluster.local"
ENDPOINT             STATUS      OUTLIER CHECK     CLUSTER
172.16.0.12:9080     HEALTHY     OK                outbound|9080|v2|reviews.default.svc.cluster.local

root@VM-0-3-ubuntu:~/istio-1.15.0# istioctl proxy-config endpoints productpage-v1-5f578dd9b7-kxbq2.default --cluster "outbound|9080|v3|reviews.default.svc.cluster.local"
ENDPOINT             STATUS      OUTLIER CHECK     CLUSTER
172.16.0.13:9080     HEALTHY     OK                outbound|9080|v3|reviews.default.svc.cluster.local

一个特例

最后的最后来看一下VirtualService所有权重之和不为100%会是什么情况

vim samples/bookinfo/networking/virtual-service-reviews-v2-v3.yaml 
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
    - reviews
  http:
  - route:
    - destination:
        host: reviews
        subset: v2
      weight: 50
    - destination:
        host: reviews
        subset: v3
      weight: 49

对应生成的routes

istioctl proxy-config routes productpage-v1-5f578dd9b7-kxbq2.default  --name 9080 -o json
                "routes": [
                    {
                        "match": {
                            "prefix": "/"
                        },
                        "route": {
                            "weightedClusters": {
                                "clusters": [
                                    {
                                        "name": "outbound|9080|v2|reviews.default.svc.cluster.local",
                                        "weight": 50
                                    },
                                    {
                                        "name": "outbound|9080|v3|reviews.default.svc.cluster.local",
                                        "weight": 49
                                    }
                                ],
                                "totalWeight": 99
                            },
avatar
文章
阅读
粉丝