（一）基础概念
①envoy的基本概念和术语
- Host：能够进行网络通信的实体（如移动设备，服务器上的应用程序）
- Downstream：发送请求并接受响应
- Upstream：上游主机连接来自Envoy的连接和请求，并返回响应
- Listener：监听器是命名网地址（例如：端口，unix domain socket等），可以被下游客户端连接。Envoy暴露一个或者多个监听器给下游主机连接。在Envoy中，listener可以绑定到端口上直接对外服务，也可以不绑定到端口上，而是接受其他listener转发的请求
- Cluster：集群是指Envoy连接到的逻辑上相同的一组上游主机。Envoy通过服务发现来发现集群的成员。可以选择通过主动健康检查来确定集群成员的健康状态。Envoy通过负载均衡策略决定将请求路由到那个集群成员
②Listeners：Envoy采用listener来接受和处理downstream发过来的请求，listener的处理逻辑是插件式的，可以通过配置不同的filter来插入不同的处理逻辑。
Virtual Listener：
Envoy创建了一个在15001端口监听的入口监听器。Iptable将请求截取后发向15001端口，该监听器接收后不进行业务处理，而是根据请求目的地分发给其他监听器处理。该监听器取名为"virtual"监听器也是这个原因
{
"version_info": "2019-07-26T08:48:04Z/8731",
"listener": {
"name": "virtual",
"address": {
"socket_address": {
"address": "0.0.0.0",
"port_value": 15001
}
},
"filter_chains": [
{
"filters": [
{
"name": "envoy.tcp_proxy",
"typed_config": {
"@type": "type.googleapis.com/envoy.config.filter.network.tcp_proxy.v2.TcpProxy",
"stat_prefix": "PassthroughCluster",
"cluster": "PassthroughCluster"
}
}
]
}
],
"use_original_dst": true
},
"last_updated": "2019-07-26T08:50:55.096Z"
}Inbound Listener：
当外部调用服务的请求到达Pod上15001的Virtual Listener上时，Virtual Listener根据请求目的地匹配到该listener，请求将被转发过来
{
"version_info": "2019-07-26T09:28:13Z/8805",
"listener": {
"name": "10.131.5.135_80",
"address": {
"socket_address": {
"address": "10.131.5.135",
"port_value": 80
}
},
"filter_chains": [
{
"filters": [
{
"name": "envoy.http_connection_manager",
"typed_config": {
"@type": "type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager",
"stat_prefix": "10.131.5.135_80",
"route_config": {
"name": "inbound|80|http-80|kunkka-caodi.mservice.svc.cluster.local",
"virtual_hosts": [
{
"name": "inbound|http|80",
"domains": [
"*"
],
"routes": [
{
"match": {
"prefix": "/"
},
"route": {
"cluster": "inbound|80|http-80|kunkka-caodi.mservice.svc.cluster.local",
"timeout": "0s",
"max_grpc_timeout": "0s"
},
"decorator": {
"operation": "kunkka-caodi.mservice.svc.cluster.local:80/*"
}
}
]
}
],
"validate_clusters": false
},
"http_filters": [
{
"name": "envoy.cors"
},
{
"name": "envoy.fault"
},
{
"name": "envoy.router"
}
],
"tracing": {
"client_sampling": {
"value": 100
},
"random_sampling": {
"value": 1
},
"overall_sampling": {
"value": 100
}
},
"server_name": "istio-envoy",
"use_remote_address": false,
"generate_request_id": true,
"forward_client_cert_details": "APPEND_FORWARD",
"set_current_client_cert_details": {
"subject": true,
"dns": true,
"uri": true
},
"upgrade_configs": [
{
"upgrade_type": "websocket"
}
],
"stream_idle_timeout": "0s"
}
}
]
}
],
"deprecated_v1": {
"bind_to_port": false
}
},
"last_updated": "2019-07-26T09:28:16.046Z"
}从上面的配置 "bind_to_port":false 可以得知该listener创建后并不会被绑定到tcp端口上直接接受网络上的数据，因此其他所有请求都转发自15001端口
Outbound Listener：
Envoy为网格中的外部服务按照端口创建多个listener，以用于处理出向请求。
首先需要区分入向请求和出向（发送给其他几个服务）请求
- 发给服务的的入向请求，virtual listener根据其目的IP和Port首先匹配到PodIP:80这个listener上，不会进入0.0.0.0_80 listener处理。
- 从Pod发出的出向请求，virtual listener无法找到和其目的IP完全匹配的listener，因此根据通配原则转交给0.0.0.0_80处理。
注意：出向请求的端口都是从service资源里面获取的。只有service资源里面才存储了对外暴露的端口和对内的端口信息
（二）envoy获取pilot的listener的代码流程分析
①服务端stream需要实现的方法
// AggregatedDiscoveryServiceServer is the server API for AggregatedDiscoveryService service.
type AggregatedDiscoveryServiceServer interface {
// This is a gRPC-only API.
StreamAggregatedResources(AggregatedDiscoveryService_StreamAggregatedResourcesServer) error
DeltaAggregatedResources(AggregatedDiscoveryService_DeltaAggregatedResourcesServer) error
}所以主要看pilot对于StreamAggregatedResources()方法的实现细节即可
②对于每个envoy的连接创建一个connectionNode资源，其需要根据envoy请求时的Request填充以下结构体
// Proxy contains information about an specific instance of a proxy (envoy sidecar, gateway,
// etc). The Proxy is initialized when a sidecar connects to Pilot, and populated from
// 'node' info in the protocol as well as data extracted from registries.
//
// In current Istio implementation nodes use a 4-parts '~' delimited ID.
// Type~IPAddress~ID~Domain
type Proxy struct {
// ClusterID specifies the cluster where the proxy resides.
ClusterID string
// Type specifies the node type. First part of the ID.
Type NodeType
// IPAddresses is the IP addresses of the proxy used to identify it and its
// co-located service instances. Example: "10.60.1.6". In some cases, the host
// where the poxy and service instances reside may have more than one IP address
IPAddresses []string
// ID is the unique platform-specific sidecar proxy ID. For k8s it is the pod ID and
// namespace.
ID string
// Locality is the location of where Envoy proxy runs.
Locality *core.Locality
// DNSDomain defines the DNS domain suffix for short hostnames (e.g.
// "default.svc.cluster.local")
DNSDomain string
// ConfigNamespace defines the namespace where this proxy resides
// for the purposes of network scoping.
// NOTE: DO NOT USE THIS FIELD TO CONSTRUCT DNS NAMES
ConfigNamespace string
// TrustDomain defines the trust domain of the certificate
TrustDomain string
// Metadata key-value pairs extending the Node identifier
Metadata map[string]string
// the sidecarScope associated with the proxy
SidecarScope *SidecarScope
// service instances associated with the proxy (只能是同一个集群下的配置下的serviceInstance)
ServiceInstances []*ServiceInstance
}③问题出现在此处
// GetProxyServiceInstances returns service instances co-located with a given proxy
func (c *Controller) GetProxyServiceInstances(proxy *model.Proxy) ([]*model.ServiceInstance, error) {
out := make([]*model.ServiceInstance, 0)
// There is only one IP for kube registry
proxyIP := proxy.IPAddresses[0]
pod := c.pods.getPodByIP(proxyIP)
if pod != nil {
// find proxy service by label selector, if not any
svcLister := listerv1.NewServiceLister(c.services.informer.GetIndexer())
if services, err := svcLister.GetPodServices(pod); err == nil && len(services) > 0 {
for _, svc := range services {
out = append(out, c.getProxyServiceInstancesByPod(pod, svc, proxy)...)
}
return out, nil
}
}
if len(out) == 0 {
if c.Env != nil {
c.Env.PushContext.Add(model.ProxyStatusNoService, proxy.ID, proxy, "")
status := c.Env.PushContext
if status == nil {
log.Infof("Empty list of services for pod %s %v", proxy.ID, c.Env)
}
} else {
log.Infof("Missing env, empty list of services for pod %s", proxy.ID)
}
}
return out, nil
}
pod := c.pods.getPodByIP(proxyIP)该方法是根据proxyIP获取pod相关的信息，内部的PodIP是从 ip := pod.Status.PodIP 获取到的。也就是从kubectl get pod -o wide获取到的。只要kubectl get pod -o wide获取不到对应的ip的话，pilot就拿不到这个pod对应的各种信息了，也就没有办法下发inbound listener了。导致探针检查的时候会出现404的错误
