阅读 402

Grpc, a framework different from REST !|Python 主题月

本文正在参加「Python主题月」,详情查看活动链接

GRPC

什么是GRPC?

随着微服务的流行,GRPC的发展势头也越来越盛。我们的后端项目,可能有很多微服务,而这些服务也许是有很多语言写的。比如身份验证的服务是Java,而消息通知的服务是Python,其他业务使用C等等。既然是微服务,我们这些服务之间获取还需要互相调用来协作完成业务,那么不同语言之间如何调用呢?GRPC就是这样的一种通信工具帮助你,让不同语言的程序可以互相交流。不需要你去处理语言之间的障碍,让你看起来就像调用自己写的函数一样简单。

GRPC顾名思义就是远程调用,为什么呢?来自官方文档的一句话:gRPC Remote Procedure Calls。

GRPC与REST的区别

  • REST是基于http/1.1,而GRPC是基于http/2。GRPC相对于REST要快很多。
  • 消息传输上,REST是JSON/XML,而GRPC是Protobuf,以二进制的形式传输,所以相对于JSON/XML要小很多。
  • GRPC API接口是非常严格的,必须明确的在proto文件中定义,REST则无需这样做。
  • GRPC代码的生成可以使用协议缓冲区编译器自动生成在GRPC项目内部,而REST需要借助三方工具(Swagger、OpenAPI)
  • GRPC可以通信流是双向的,而REST是单向的。
  • REST支持浏览器,而GRPC不支持,所以目前RPC 最常用的场景是 IOT 等硬件领域。

使用GRPC的前提

你必须知道以下几个概念

  • Protocol Buffers
  • 简单流(Unary RPC)
  • 客户端流(Server streaming RPC)
  • 服务端流(Client streaming RPC)
  • 双向流(Bidirectional streaming RPC)

GRPC in Python

下面以简单流为例,其他流可参考官方代码,各种语言的都能在这个仓库找到,routeguide这个示例包含了所有类型的grpc服务。使用注意:使用python开发前你必须安装grpcio-tools、grpcio。

python -m pip install grpcio
python -m pip install grpcio-tools
复制代码

1、编写proto文件

首先根据Protocol Buffers文件,来定义约束我们的服务。我们还是以我们最常见的helloworld为例。在项目根目录下创建文件夹protos,并在该文件夹下创建helloworld.proto文件,内容如下。

syntax = "proto3";
// To be compatible with Java configuration, it does not work in Python
// If false, only a single .java file will be generated for this .proto file.
// If true, separate .java files will be generated for each of the Java classes/enums/etc.
option java_multiple_files = true;
// The package you want to use for your generated Java/Kotlin classes.
option java_package = "com.wangscaler.examples.helloworld";
// The class name (and hence the file name) for the wrapper Java class you want to generate.
option java_outer_classname = "HelloWorldProto";
​
​
package helloworld;
// The greeting service definition.
service Greeter {
  // Sends a greeting
  rpc SayHello (HelloRequest) returns (HelloReply) {}
  // Sends another greeting
  rpc SayAuthor (AuthorRequest) returns (AuthorReply) {}
}
​
// The request message containing the user's name.
message HelloRequest {
  string name = 1;
}
​
// The request message containing the user's name.
message AuthorRequest {
  string name = 1;
  int32 age = 2;
}
​
// The response message containing the greetings
message HelloReply {
  string message = 1;
}
// The response message containing the greetings
message AuthorReply {
  string message = 1;
  int32 code = 2;
}
复制代码

我们定义了一个Greeter服务,并且这个服务提供了两个接口SayHello和SayAuthor。分别给这两个接口的请求参数和响应参数做了限制。

不同的是SayHello的入参是一个字符串类型的,响应参数也是一个字符串类型的;而AuthorReply入参多了一个int类型的age,而响应参数也多了个int类型的code。

2、自动生成代码

在项目根目录下执行

python -m grpc_tools.protoc -I./protos --python_out=. --grpc_python_out=. ./protos/helloworld.proto
复制代码

执行完之后,会在项目根路径下生成helloworld_pb2.py、helloworld_pb2_grpc.py两个python文件。

3、开发服务端

定义Greeter去继承helloworld_pb2_grpc.GreeterServicer,重写父类的SayHello、SayAuthor来实现我们的业务。

from concurrent import futures
import logging
import grpc
import helloworld_pb2
import helloworld_pb2_grpc
​
​
class Greeter(helloworld_pb2_grpc.GreeterServicer):
​
    def SayHello(self, request, context):
        print("Get a message from %s client" % request.name)
        return helloworld_pb2.HelloReply(message='Hello world, %s client !' % request.name)
​
    def SayAuthor(self, request, context):
        print("Hi author(%(name)s), your age is %(age)d" % {"name": request.name, "age": request.age})
        return helloworld_pb2.AuthorReply(
            message='Hello, %s ! ' % request.name, code=0)
​
​
def serve():
    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
    helloworld_pb2_grpc.add_GreeterServicer_to_server(Greeter(), server)
    server.add_insecure_port('[::]:50051')
    server.start()
    server.wait_for_termination()
​
​
if __name__ == '__main__':
    logging.basicConfig()
    serve()
​
​
复制代码

4、开发客户端(python)

import logging
import grpc
import helloworld_pb2
import helloworld_pb2_grpc
​
​
def run():
    # NOTE(gRPC Python Team): .close() is possible on a channel and should be
    # used in circumstances in which the with statement does not fit the needs
    # of the code.
    with grpc.insecure_channel('localhost:50051') as channel:
        stub = helloworld_pb2_grpc.GreeterStub(channel)
        hello_response = stub.SayHello(helloworld_pb2.HelloRequest(name='python'))
        author_response = stub.SayAuthor(helloworld_pb2.AuthorRequest(name='scaler', age=18))
    print("Greeter client received: " + hello_response.message)
    print("Greeter client received message: %(message)s and received code: %(code)d !" % {
        "message": author_response.message,
        "code": author_response.code})
​
​
if __name__ == '__main__':
    logging.basicConfig()
    run()
​
复制代码

执行之后控制台打印的消息如下:

Greeter client received: Hello world, python client !
Greeter client received message: Hello, scaler !  and received code: 0 !
复制代码

而我们服务器端的打印消息如下:

Get a message from python client
Hi author(scaler), your age is 18
复制代码

GRPC Java client

一开始我们就说了,GRPC可以兼容多语言的调用,所以我们的java客户端也是可以调用的上面的python的服务器端的SayHello和SayAuthor接口。

1、创建Maven项目

自行使用IDEA创建,不过多介绍。

2、复制上述的proto文件

在maven项目的src/main文件夹下创建proto文件夹,并将上述python中创建的proro文件,复制到这个文件夹下。

3、自动生成代码

使用mvn compile命令,将在target/generated-sources/protobuf/grpc-javatarget/generated-sources/protobuf/java生成我们需要的文件。

文件的包名就是我们proto文件中指定的java_package。

4、开发客户端(java)

在src/main/java下创建包和java_package一致,即和生成的代码的包名保持一致。

package com.wangscaler.examples;
​
import io.grpc.Channel;
import io.grpc.ManagedChannel;
import io.grpc.ManagedChannelBuilder;
import io.grpc.StatusRuntimeException;
​
import java.text.MessageFormat;
import java.util.concurrent.TimeUnit;
import java.util.logging.Level;
import java.util.logging.Logger;
​
/**
 * @author WangScaler
 * @date 2021/7/22 16:36
 */
​
​
public class Client {
    private static final Logger logger = Logger.getLogger(Client.class.getName());
    private final GreeterGrpc.GreeterBlockingStub blockingStub;
​
    public Client(Channel channel) {
        blockingStub = GreeterGrpc.newBlockingStub(channel);
    }
​
    public void greet(String name) {
        logger.info("Will try to greet " + name + " ...");
        HelloRequest helloRequest = HelloRequest.newBuilder().setName(name).build();
        AuthorRequest authorRequest = AuthorRequest.newBuilder().setName("wangscaler").setAge(18).build();
        HelloReply helloResponse;
        AuthorReply authorResponse;
        try {
            helloResponse = blockingStub.sayHello(helloRequest);
            authorResponse = blockingStub.sayAuthor(authorRequest);
        } catch (StatusRuntimeException e) {
            logger.log(Level.WARNING, "RPC failed: {0}", e.getStatus());
            return;
        }
        logger.info("Greeter client received: " + helloResponse.getMessage());
        logger.info(MessageFormat.format("Greeter client received message: {0} and received code: {1} ! ", authorResponse.getMessage(), authorResponse.getCode()));
    }
​
    public static void main(String[] args) throws Exception {
        String user = "java";
        String target = "localhost:50010";
        if (args.length > 0) {
            if ("--help".equals(args[0])) {
                System.err.println("Usage: [name [target]]");
                System.err.println("");
                System.err.println("  name    The name you wish to be greeted by. Defaults to " + user);
                System.err.println("  target  The server to connect to. Defaults to " + target);
                System.exit(1);
            }
            user = args[0];
        }
        if (args.length > 1) {
            target = args[1];
        }
​
        ManagedChannel channel = ManagedChannelBuilder.forTarget(target)
                .usePlaintext()
                .build();
        try {
            Client client = new Client(channel);
            client.greet(user);
        } finally {
            channel.shutdownNow().awaitTermination(5, TimeUnit.SECONDS);
        }
    }
}
​
复制代码

执行之后控制台打印的消息如下:

信息: Will try to greet java ...
七月 22, 2021 5:00:17 下午 com.wangscaler.examples.Client greet
信息: Greeter client received: Hello world, java client !
七月 22, 2021 5:00:17 下午 com.wangscaler.examples.Client greet
信息: Greeter client received message: Hello, wangscaler !  and received code: 0 ! 
复制代码

而我们python服务器端的打印消息如下:

Get a message from java client
Hi author(wangscaler), your age is 18
复制代码

Cool!我们的Java客户端像调用自己内部的函数一样,调用了远程的python服务器上的方法。这就是GRPC的强大之处。微服务之间的调用就像这一样轻松简单。

使用SSL身份验证

1、生成根证书和私钥

使用openssl生成ssl证书。

openssl req -newkey rsa:2048 -nodes -keyout server.key -x509 -days 365 -out server.crt
复制代码

注意:执行命令之后需要输入你的相关信息,如果你是在ssl本地测试,切记CN的值为localhost,此时你的客户端才可以通过localhost访问你的服务器。

2、修改服务器端

from concurrent import futures
import logging
import grpc
import helloworld_pb2
import helloworld_pb2_grpc
​
_ONE_DAY_IN_SECONDS = 60 * 60 * 24
​
​
class Greeter(helloworld_pb2_grpc.GreeterServicer):
​
    def SayHello(self, request, context):
        print("Get a message from %s client" % request.name)
        return helloworld_pb2.HelloReply(message='Hello world, %s client !' % request.name)
​
    def SayAuthor(self, request, context):
        print("Hi author(%(name)s), your age is %(age)d" % {"name": request.name, "age": request.age})
        return helloworld_pb2.AuthorReply(
            message='Hello, %s ! ' % request.name, code=0)
​
​
def serve():
    with open('server.key', 'rb') as f:
        private_key = f.read()
    with open('server.crt', 'rb') as f:
        certificate_chain = f.read()
    server_credentials = grpc.ssl_server_credentials(
        ((private_key, certificate_chain,),))
    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
    helloworld_pb2_grpc.add_GreeterServicer_to_server(Greeter(), server)
    server.add_secure_port('[::]:50051', server_credentials)
    server.start()
    server.wait_for_termination()
​
​
if __name__ == '__main__':
    logging.basicConfig()
    serve()
​
复制代码

至此我们的GRPC,就加入了SSL身份验证。

3、修改客户端(python)

我们使用之前的客户端去连接,发现连接失败,如下图所示。

image-20210727092307787.png 接下来,修改如下:

首先将openssl生成的server.key和server.crt复制到项目的根路径下。最后修改代码如下:

import logging
import grpc
import helloworld_pb2
import helloworld_pb2_grpc
​
​
def run():
    with open('server.crt', 'rb') as f:
        trusted_certs = f.read()
    credentials = grpc.ssl_channel_credentials(root_certificates=trusted_certs)
    with grpc.secure_channel('localhost:50051', credentials) as channel:
        stub = helloworld_pb2_grpc.GreeterStub(channel)
        hello_response = stub.SayHello(helloworld_pb2.HelloRequest(name='python'))
        author_response = stub.SayAuthor(helloworld_pb2.AuthorRequest(name='scaler', age=18))
    print("Greeter client received: " + hello_response.message)
    print("Greeter client received message: %(message)s and received code: %(code)d !" % {
        "message": author_response.message,
        "code": author_response.code})
​
​
if __name__ == '__main__':
    logging.basicConfig()
    run()
复制代码

再次运行客户端,控制台正常打印

Greeter client received: Hello world, python client !
Greeter client received message: Hello, scaler !  and received code: 0 !
复制代码

4、修改客户端(java)

修改之前测试连接。报错如下:

RPC failed: Status{code=UNAVAILABLE, description=Network closed for unknown reason, cause=null}
复制代码

修改如下,将openssl生成的server.crt复制到项目路径下,能找到就可以,不复制也行。

package com.wangscaler.examples;
​
import io.grpc.Channel;
import io.grpc.ManagedChannel;
import io.grpc.ManagedChannelBuilder;
import io.grpc.StatusRuntimeException;
import io.grpc.netty.shaded.io.grpc.netty.GrpcSslContexts;
import io.grpc.netty.shaded.io.grpc.netty.NegotiationType;
import io.grpc.netty.shaded.io.grpc.netty.NettyChannelBuilder;
import io.grpc.netty.shaded.io.netty.handler.ssl.SslContext;
import io.grpc.netty.shaded.io.netty.handler.ssl.SslContextBuilder;
​
import javax.net.ssl.SSLException;
import java.io.File;
import java.text.MessageFormat;
import java.util.concurrent.TimeUnit;
import java.util.logging.Level;
import java.util.logging.Logger;
​
/**
 * @author WangScaler
 * @date 2021/7/22 16:36
 */
​
​
public class Client {
    private static final Logger logger = Logger.getLogger(Client.class.getName());
    private final GreeterGrpc.GreeterBlockingStub blockingStub;
​
    public Client(Channel channel) {
        blockingStub = GreeterGrpc.newBlockingStub(channel);
    }
​
    public void greet(String name) {
        logger.info("Will try to greet " + name + " ...");
        HelloRequest helloRequest = HelloRequest.newBuilder().setName(name).build();
        AuthorRequest authorRequest = AuthorRequest.newBuilder().setName("wangscaler").setAge(18).build();
        HelloReply helloResponse;
        AuthorReply authorResponse;
        try {
            helloResponse = blockingStub.sayHello(helloRequest);
            authorResponse = blockingStub.sayAuthor(authorRequest);
        } catch (StatusRuntimeException e) {
            logger.log(Level.WARNING, "RPC failed: {0}", e.getStatus());
            return;
        }
        logger.info("Greeter client received: " + helloResponse.getMessage());
        logger.info(MessageFormat.format("Greeter client received message: {0} and received code: {1} ! ", authorResponse.getMessage(), authorResponse.getCode()));
    }
​
    private static SslContext buildSslContext(String trustCertCollectionFilePath) throws SSLException {
        SslContextBuilder builder = GrpcSslContexts.forClient();
        if (trustCertCollectionFilePath != null) {
            builder.trustManager(new File(trustCertCollectionFilePath));
        }
        return builder.build();
    }
​
    public static void main(String[] args) throws Exception {
        String user = "java";
        String target = "localhost:50010";
        if (args.length > 0) {
            if ("--help".equals(args[0])) {
                System.err.println("Usage: [name [target]]");
                System.err.println("");
                System.err.println("  name    The name you wish to be greeted by. Defaults to " + user);
                System.err.println("  target  The server to connect to. Defaults to " + target);
                System.exit(1);
            }
            user = args[0];
        }
        if (args.length > 1) {
            target = args[1];
        }
        String[] targets = new String[2];
        targets = target.split(":");
        String host = targets[0];
        int port = Integer.parseInt(targets[1]);
        SslContext sslContext = Client.buildSslContext("D://springboot/test-grpc/src/main/java/com/wangscaler/examples/server.crt");
        ManagedChannel channel = NettyChannelBuilder.forAddress(host, port)
                .negotiationType(NegotiationType.TLS)
                .sslContext(sslContext)
                .build();
        try {
            Client client = new Client(channel);
            client.greet(user);
        } finally {
            channel.shutdownNow().awaitTermination(5, TimeUnit.SECONDS);
        }
    }
}
​
复制代码

再次连接。控制台正常的调用到服务器端的函数。

信息: Will try to greet java ...
七月 27, 2021 9:38:51 上午 com.wangscaler.examples.Client greet
信息: Greeter client received: Hello world, java client !
七月 27, 2021 9:38:51 上午 com.wangscaler.examples.Client greet
信息: Greeter client received message: Hello, wangscaler !  and received code: 0 ! 
复制代码

参考信息

文章分类
后端