grpc基于envoy治理 java实现 control panel

254 阅读4分钟

envoy的部署与架构:

见SRE空间:envoy

envoy基础(1.7版本的):

 PDF

业内厂商实现案例:

网易轻舟:

其中有几个关注点:

1.轻舟的这套服务网格技术的研发成员有C++的研发人员

2.轻舟的envoy部署模式是sidecar模式(要是按照这种模式部署,对雪球来说步子跨的是不是有些大?)

 PDF

从kong到envoy的演进:zhuanlan.zhihu.com/p/242260216…

其中有几个关注点:

1.从严选的架构可以看到在envoy之前,网易是有api网关规划的(当然雪球自研的openresty+warden有与之对应功能)

2.演进过程可以看出是兼容升级的(而我们目前旧的体系要实现哪些部分的梳理还欠缺)

 

Apisix Linkerd Envoy对比

envoy自身也有比较docs:www.servicemesher.com/envoy/intro…

nginx

haproxy

AWS ELB

SmartStack

Finagle

proxygen 和 wangle

gRPC

linkerd

nghttp2

 

首先Linkerd与Envoy的对比,作者在issue里有介绍:

github.com/envoyproxy/…

linkerd是一个独立的、开源的RPC路由代理,建立在Netty和Finagle上。

linkerd提供了许多Finagle的特性,包括感知延迟的负载平衡、连接池、断路、重试预算、截止日期、跟踪、细粒度检测,以及用于请求级路由的流量路由层。

linkerd提供了一个可插拔的服务发现接口(对Consul和ZooKeeper,以及Marathon和Kubernetes api提供标准支持)。

 

linkerd的内存和CPU要求明显高于envoy的。

然而,它的基础技术是广泛的生产测试和广泛部署。

与Envoy不同的是,linkerd提供了一种极简的配置语言,并且明确地不支持热加载,而是依赖于动态供应和服务抽象。

linkerd支持HTTP/1.1, Thrift, ThriftMux, HTTP/2(实验性)和gRPC(实验性)。

 

至于Apisix和Envoy

网上一大堆,找出来的基本上都是相互类似的

 APISIX 在响应延迟和 QPS 层面都略优于 Envoy, 由于 nginx 的多 worker 的协作方式在高并发场景下更有优势,得益于此, APISIX 在开启多个 worker 进程后性能提升较 Enovy 更为明显;

 

但是两者并不冲突, Envoy 的总线设计使它在处理东西向流量上有独特的优势, APISIX 在性能和延迟上的表现使它在处理南北向流量上具有海量的吞吐能力,根据自己的业务场景来选择合理的组件配合插件构建自己的服务才是正解。

grpc基于envoy治理实现

server端

这个使用envoy提供的ADS注册

路由规则和cluster、endpoint的注册则根据接口动态修改,这样整个设计模式就是按照service mesh概念里的控制面板抽离出来了

public static VirtualHost getVirtualHost(String name, String clusterName) {``    ``RouteAction routeAction = RouteAction.newBuilder()``            ``.setCluster(clusterName)``            ``.build();     ``HeaderMatcher headerMatcher = HeaderMatcher.newBuilder()``            ``.setName(``"UID"``)``            ``.setExactMatch(``"5784024476"``)``            ``.build();     ``RouteMatch routeMatch = RouteMatch.newBuilder()``            ``.setGrpc(GrpcRouteMatchOptions.newBuilder()) ``// 确定只匹配Grpc Match``            ``.setPrefix(``"/"``)``            ``.addHeaders(headerMatcher)``            ``.build();``    ``Route route = Route.newBuilder()``            ``.setMatch(routeMatch)``            ``.setRoute(routeAction)``            ``.build();``    ``VirtualHost virtualHost = VirtualHost.newBuilder()``            ``.setName(name)``            ``.addDomains(``"*"``)``            ``.addRoutes(route)``            ``.build();     ``return virtualHost;``}

 

服务发现动态配置,示例代码:

Endpoint endpoint = ClusterSnapshot.toEndpoint(host + ``":" + port);``ClusterLoadAssignment clusterLoadAssignment = simpleCache.getSnapshot(``0``).endpoints().resources().get(CLUSTER_NAME);``LocalityLbEndpoints localityLbEndpoints = LocalityLbEndpoints.newBuilder()``        ``.addLbEndpoints(LbEndpoint.newBuilder().setEndpoint(endpoint).build())``        ``.build(); List<ClusterLoadAssignment> assignments = ImmutableList.<ClusterLoadAssignment>builder()``        ``.add(clusterLoadAssignment.toBuilder().addEndpoints(localityLbEndpoints).build())``        ``.build(); Cluster.EdsClusterConfig edsClusterConfig = ClusterSnapshot.getEdsClusterConfig(CLUSTER_NAME, DISCOVERY_CLUSTER_NAME);``List<Cluster> clusters = ImmutableList.<Cluster>builder()``        ``.add(ClusterSnapshot.getCluster(CLUSTER_NAME, edsClusterConfig))``        ``.build();``Snapshot snapshot = Snapshot.create(``        ``clusters,``        ``assignments,``        ``ImmutableList.of(),``        ``ImmutableList.of(),``        ``ImmutableList.of(),``        ``version);``simpleCache.setSnapshot(``0``, snapshot);

 

client端

对于路由信息目前是在header(也就是GRPC的类Metadata)里面附加路由参数:

skyao.io/learning-gr…

提供对读取和写入元数据数值的访问,元数据数值在调用期间交换。

key 容许关联到多个值。

这个类不是线程安全,实现应该保证 header 的读取和写入不在多个线程中并发发生。

client代码实现:ClientHeaderInterceptor

public class ClientHeaderInterceptor ``implements ClientInterceptor {     ``private static final Logger logger = Logger.getLogger(ClientHeaderInterceptor.``class``.getName());``    ``Metadata.Key<String> cookie = Metadata.Key.of(``"cookie"``, Metadata.ASCII_STRING_MARSHALLER);     ``@Override``    ``public <ReqT, RespT> ClientCall<ReqT, RespT> interceptCall(MethodDescriptor<ReqT, RespT> method,``                                                               ``CallOptions callOptions, Channel next) {``        ``return new ForwardingClientCall.SimpleForwardingClientCall<ReqT, RespT>(next.newCall(method, callOptions)) {``            ``@Override``            ``public void start(Listener<RespT> responseListener, Metadata headers) {``                ``//此处为你登录后获得的cookie的值``                ``headers.put(cookie, ``"U=6371181803"``);``                ``super``.start(``new ForwardingClientCallListener.SimpleForwardingClientCallListener<RespT>(responseListener) {``                    ``@Override``                    ``public void onHeaders(Metadata headers) {``                        ``/**``                         ``* if you don't need receive header from server, you can``                         ``* use {@link io.grpc.stub.MetadataUtils#attachHeaders}``                         ``* directly to send header``                         ``*/``                        ``logger.info(``"header received from server:" + headers);``                        ``super``.onHeaders(headers);``                    ``}``                ``}, headers);``            ``}``        ``};``    ``}``}

调用,这部分先写死,后续更改为动态配置,示例代码:

static final Metadata.Key<String> COOKIE = Metadata.Key.of(``"cookie"``, Metadata.ASCII_STRING_MARSHALLER);``private final ManagedChannel originalChannel;``private final XUserDeviceServiceGrpc.XUserDeviceServiceBlockingStub blockingStub;``private final XUserDeviceServiceGrpc.XUserDeviceServiceBlockingStub headerBlockingStub; public XueqiuPushUserClientTest(String host, ``int port) {``    ``originalChannel = ManagedChannelBuilder.forAddress(host, port).usePlaintext(``true``).build();``    ``//将ClientHeaderInterceptor和原有的channel结合,生成包含拦截器的channel``    ``Channel channel = ClientInterceptors.intercept(originalChannel, ``new ClientHeaderInterceptor());``    ``blockingStub = XUserDeviceServiceGrpc.newBlockingStub(channel);     ``//如果只需要在客户端传送header,而不需要接受服务端的header可以简单调用MetadataUtils.attachHeaders注册meta数据而不用定义Interceptor``    ``XUserDeviceServiceGrpc.XUserDeviceServiceBlockingStub stub = XUserDeviceServiceGrpc.newBlockingStub(originalChannel);``    ``Metadata meta = ``new Metadata();``    ``meta.put(COOKIE, ``"U=6371181803"``);``    ``headerBlockingStub = MetadataUtils.attachHeaders(stub, meta);``}