protobuf学习
protobuf学习protobuf是什么？protobuf的特点protobuf要解决的现实痛点protobuf的使用依赖protobuf的语法分析protobuf的编译、转换与目标生成protobuf的proto源文件和目标文件的映射关系proto元数据类型与具体主要语言之间的映射关系protobuf数据的序列化和反序列化
protobuf是什么？
protobuf即Protocol Buffers的简称，是由Google开发并于2018年开源的一种数据描述语言，用于描述一种轻便且高效的结构化数据存储格式。通俗的理解：protobuf是一个用于表示结构化的数据如何存储、如何传输的语言框架，和JSON、XML类似。其处理或描述的对象是【结构化的数据】，处理结果是一组有特定组织和表示含义的【纯二进制的数据流】，处理场景是【数据存储或者数据通信的传输】，典型应用是【数据库数据存储】和【基于RPC的网络通信】，所以我们完全可以把protobuf就看成是与JSON和XML一样的技术概念。
虽然从理解上，可以把protobuf与JSON、XML划等号，但是在技术层面上讲，它们之间还是存在很明显的区别的。
- JSON/XML更偏重于如何去组织生成结构化数据，即数据的表示和逻辑组织，数据的存储或传输则不太关注，结构化数据的存储和传输是以面向字节（或面向字符串、键值对）来实现的，即所见即所得，便于人工的检视和分析处理，数据内容本身附带有相关的辅助性描述内容如数据类型，因此在数据安全性、数据传输体量、网络吞吐率等方面比较欠缺，资源性开销比较大；
- protobuf更偏重于数据的存储和传输，其是纯二进制的数据内容，以面向比特来完成存储和传输，相对而言其在数据的表示和逻辑结构化组织上较JSON/XML性能差，人工检视和分析处理具有一定的难度。在数据安全性、数据传输体量、网络吞吐率等方面表现极佳，资源性开销较小。数据空间体量上比XML开销更小（约3
10倍）、传输速度比XML更快（约20100倍）。
protobuf的特点
- 语言无关性：它不依赖具体的开发语言，即无论是使用C语言开发的软件，还是使用C++、Python、JAVA等开发的软件，均可使用protobuf技术应用；
- 平台无关性：即不依赖具体的开发平台，Windows生态、Linux生态、Mac生态、x86生态、x64生态等均适用；
- 可扩展性：可扩展、更新数据结构，不影响和破坏原有的程序；
- 支持数据的序列化（即结构化数据的二进制化表示）和反序列化（即反二进制化提取出结构化数据）；
protobuf要解决的现实痛点
想象一下这样一个在实际开发中经常遇到的场景：
我们的客户端程序是使用JAVA开发的，可能会运行在不同的平台上如Linux、Windows或Android，而我们的服务器端程序通常是基于Linux平台并使用C++开发实现。那么在客户端和服务器端进行数据通信时可能会存在下列几种常见的设计模式：
- 发送端直接传输按字节对齐的结构体数据，只要结构体的声明为定长格式，如此在接收端只需按同样的数据结构体格式进行转换即可。即使结构体定义为变长模式，也无需付出太多额外的开销也可完成同样的数据发送、数据接收和数据解析的工作。这种设计模式是当前比较普遍的一种通信设计模式，需要双方事先已协商并定义好数据的结构体组织，并在实际通信前即要完成数据的encode、decode相关工作；一旦项目需求发生了某种必须的变更，对数据结构体的组织产生了必须的变更后，双方必须再次协商生成新的数据结构体组织，并再次针对新结构体重复进行数据的encode、decode相关工作。还必须要考虑对旧结构体组织的兼容性支持问题。
- 使用SOAP协议、JSON、XML等作为消息报文的格式载体，虽然因需求变更导致的数据结构同步变更产生的额外开销实际增加不了多少，也可以解决新旧结构体组织的兼容性问题，但是传输数据的实际体量却不够精简，网络负担较大，整体性能不高；
protobuf的使用依赖
- protobuf基础语言包：google.golang.org/protobuf，定义了protobuf相关的语法等；
- protoc：即protocol buffers compiler--protobuf编译器，用C++实现的编译工具，可以从托管站点获取最新的- [https://github.com/protocolbuffers/protobuf/releases](- github.com/protocolbuf…) 工具软件，其对应的源码仓库为 github.com/protocolbuf…。其设计原理大致是将用户编辑生成的.proto文件进行语法检查、词法语义检查，提取出结构化的数据，然后通过指定目标语言的插件工具如protoc-gen-go等转换成指定目标语言的文件列表。
- 目标语言插件：针对go开发，需要安装protoc-gen-go插件工具（github.com/golang/protobuf/protoc-gen-go）、protoc-gen-gogo插件工具（github.com/gogo/protobuf/protoc-gen-gogo@latest protoc-gen-gogo）、protoc-gen-swagger插件工具（github.com/grpc-ecosystem/grpc-gateway/protoc-gen-swagger）；
- gRPC工具：gRPC框架包（google.golang.org/grpc）;
- gRPC基于GO的工具插件：protoc-gen-go-grpc（google.golang.org/grpc/cmd/protoc-gen-go-grpc）;
protobuf的语法分析
protobuf是通过.proto文件来定义组织化数据结构的，主要涉计message（即数据元）和service（即RPC接口）两大项：
// protobuf syntax prompt, indicate which version of protobuf is used, general use version 3
// NOTE: protobuf using the decimal or {} as the statement segment seperate flag
syntax = "proto3";
// Explicit specific the import path, which will generate the target package name in GO
option go_package = "xxx/GoTargetPackageName"
// which package bellow strctured datas belonged to, may not the same with package name in GO
package main;
// also may import other packages defined using protobuf
import "google/protobuf/any.proto"
// Defined a enum type data type, the same as enum in C/C++
// Data type name must using all Upper Case Charactor
enum QUERY_TYPE {
// Type member using Upper-Camel-Case style rule;
// Recommended to using all Upper Case Charactor like the MACRO definations in C
QueryTypeOne = 0;
Protobuf using decimal as the seperate flag
QueryTypeTwo = 1;
}
// message data meta such as a struct data type in C/C++, a class data type in JAVA
message SampleMessageTypeName { // SampleMessageTypeName using Upper-Camel-Case style rule
// member of message meta using format: FiledRules ProtobufDataType FiledDataVar = FiledNumber
// NOTE: 1. ProtobufDataType may meta data type or user defined combine data type such as nested message;
// 2. FiledDataVar using Lower-Camel-Case style rule;
// 3. FiledNumber must start with 1;
// 4. Protobuf using decimal as the seperate flag;
// 5. If user not explicit specific the FiledRules, a named "implicit filed presence" Singular will used.
// For this situation, just simple understand it as optional is OK.
// 6. required is not supportd in proto3, instead user explicit set value to optional is the same as required purpose;
// required -- This data filed may must and only appear 1 times
required string stringVar1 = 1;
// optional -- This data filed may appear 0 or 1 times, that's mean it's optional
optional bytes bytesVar1 = 2;
// Combine and Nested data type is supported in protouf
message EmbeddedMessageTypeName {
optional string stringVar = 1;
required bytes bytesVar = 2;
}
// repeated -- This data filed may appear 0 or 1 or more than one times
repeated int32 repeatedInt32Var = 3;
// Using user self defined data type
optional QUERY_TYPE qtVar = 4;
// Using other packages definaed proto data type, must import the target package into this local package
optional OtherPacakge.TargetDataType otherDataTypeVar = 5;
// Using the nested message data type
optional EmbeddedMessageTypeName embedMessageDataTypeVar = 6;
}
// service interface meta such as an abstract interface class type in C++, which will supply serial RPC-API functions
service ServiceInterfaceTypeName { // ServiceInterfaceTypeName using Upper-Camel-Case style rule
// RPC API format: rpc ApiFunctionName(SearchRequest) returns(SearchResponse)
// NOTE: 1. the RPC function name using Upper-Camel-Case style rule;
// 2. The input parameter SearchRequest transmit into the function must be a message data meta instance;
// 3. The output parameter SearchResponse transmit out from the function must also be a message data meta instance;
// If body {} used, the decimal seperate flag is not needed.
rpc TodoAdded(TodoAddedRequest) returns(TodoAddedReply) {}
// If body {} not used, the decimal seperate flag is needed.
rpc TodoDeleted(TodoDeletedRequest) returns(TodoDeletedReply);
rpc TodoDone(TodoDoneRequest) returns(TodoDoneReply);
rpc TodoUnchecked(TodoUncheckedRequest) returns(TodoUncheckedReply);
}
protobuf的编译、转换与目标生成
本地实现了一个如下的.proto文件，内容如下：
// myexample.proto
syntax = "proto3";
// Explicit specific the import path
option go_package = "./myprotobuf";
// NOTE:
package myprotobufexample;
message MyMessageExample {
optional string stringMember1 = 1;
optional string stringMember2 = 2;
optional string stringRequest = 3;
optional string stringResponse = 4;
}
service MyServiceExample {
rpc QueryHello(MyMessageExample) returns(MyMessageExample);
}
protobuf的编译、转换和目标文件的生成是通过编译器protoc和目标语言的插件集如protoc-gen-go来实现的，（默认情况下，protobuf是不输出RPC服务代码的，但是可以安装gRPC的插件工具来显示生成RPC服务的相关代码"go install google.golang.org/grpc/cmd/protoc-gen-go-grpc"）语法格式为：
// protoc cmd format in protobuf website is bellow
// protoc --proto_path(InputFileDir) --go_out(ConvertTool&TargetOutFileDir) --go-grpc_out(PlugingRPC&TargetOutFileDir) TargetProtoFilesListPath
// NOTE: 1. --proto_path: the input .proto file's dir path, may general using -I instead of it and sometimes may omit via path statement with TargetProtoFilesListPath ;
// 2. --go_out: including two purpose, one is specific the message meta convert tool type to used, and the other is to specific the output files' dir path;
// 3. improt path: In protoc there is a named import path concept, which is the target files initial palcement path. protoc will use the --go_out as the final target palcement path.We may understand it as following:
// 3.1 protoc first generate the target files and place it at the import path;
// 3.2 protoc then cut the import path whole contents into the --go_out path;
// 4. --go_opt: go plugin tool options, there are following options serial:
// 4.1 paths=import: the default option mode, which will place the output file like .pb.go into the same path of the import path.
// e.g. import "example.com/project/protos/fizz", then example.com/project/protos/fizz/buzz.pb.go will generated and placed;
// 4.2 modlue=$PREFIX: the same result as paths=import;
// 4.3 paths=source_relative: place the output files at the same dir with input .proto files
// 5. --go-grpc_out: including two purpose, one is specific the RPC service convert tool type to used, and the other is to specific the output files's dir path;
// 6. For GO, .proto file's import path must explicit set, there are two method to implement:
// 6.1 using import statement in the .proto file: "option go_package="xxx/xxx/xxx"";
// 6.2 using cmd option M${PROTO_FILE}=${GO_IMPORT_PATH} of --go_opt;
// e.g. --go_opt=Mprotos/buzz.proto=example.com/project/protos/fizz
// --go_opt=Mprotos/bar.proto=example.com/project/protos/foo
// 7. The target output file name's prefix is the same as input .proto file;
// 8. The final target package name in go is specified by import path;
// e.g. If the import path is using "option go_package = "./targetpackage"",
// then the final go package statement will is "pacakge targetpackage"
protoc --proto_path=$SRC_DIR --go_out=$DST_DIR --go_opt=paths=source_relative --go-grpc_out=$DST_DIR foo.proto bar/baz.proto
// General format: protoc -I(InputFileDir) --xx_out(ConvertTool&TargetOutFileDir) TargetProtoFilesListPath
// $SRC_DIR: the path of the input target .proto file;
// --xx_out: the data convert tool to used, xx is target language related, for GO may --go_out, for C++ may --cpp_out;
// --xx-grpc_out: the service convert tool to used, xx is target language related, for GO may --go-grpc_out, for C++ may --cpp-grpc_out;
// $DATA_DST_DIR: the path of the target language files for message meta data will place, file format xxx.pb.go;
// $SERVICE_DST_DIR: the path of the target language files for service meta will place, file format xxx_grpc.pb.go;
// xxx.proto: the input target .proto files list;
protoc -I=$SRC_DIR --go_out=$DATA_DST_DIR --go-grpc_out=$SERVICE_DST_DIR $SRC_DIR/xxx.proto
对于本例，则使用如下的命令行内容：
// cd to the path of file myexample.proto protoc --go_out=./ --go-grpc_out=./ ./myexample.proto
运行结果如下所示：
protobuf的proto源文件和目标文件的映射关系
上述例程转换后将生成两个文件:
- myexample.pb.go：主要是对message数据的处理；
- myexample_grpc.pb.go：主要是对service的实现；
其中myexample.pb.go文件的内容如下：
// myexample.proto
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.26.0
// protoc v3.17.3
// source: myexample.proto
package myprotobuf
import (
protoreflect "google.golang.org/protobuf/reflect/protoreflect"
protoimpl "google.golang.org/protobuf/runtime/protoimpl"
reflect "reflect"
sync "sync"
)
const (
// Verify that this generated code is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)
// Verify that runtime/protoimpl is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)
)
type MyMessageExample struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
StringMember1 *string `protobuf:"bytes,1,opt,name=stringMember1,proto3,oneof" json:"stringMember1,omitempty"`
StringMember2 *string `protobuf:"bytes,2,opt,name=stringMember2,proto3,oneof" json:"stringMember2,omitempty"`
StringRequest *string `protobuf:"bytes,3,opt,name=stringRequest,proto3,oneof" json:"stringRequest,omitempty"`
StringResponse *string `protobuf:"bytes,4,opt,name=stringResponse,proto3,oneof" json:"stringResponse,omitempty"`
}
func (x *MyMessageExample) Reset() {
*x = MyMessageExample{}
if protoimpl.UnsafeEnabled {
mi := &file_myexample_proto_msgTypes[0]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *MyMessageExample) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*MyMessageExample) ProtoMessage() {}
func (x *MyMessageExample) ProtoReflect() protoreflect.Message {
mi := &file_myexample_proto_msgTypes[0]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use MyMessageExample.ProtoReflect.Descriptor instead.
func (*MyMessageExample) Descriptor() ([]byte, []int) {
return file_myexample_proto_rawDescGZIP(), []int{0}
}
func (x *MyMessageExample) GetStringMember1() string {
if x != nil && x.StringMember1 != nil {
return *x.StringMember1
}
return ""
}
func (x *MyMessageExample) GetStringMember2() string {
if x != nil && x.StringMember2 != nil {
return *x.StringMember2
}
return ""
}
func (x *MyMessageExample) GetStringRequest() string {
if x != nil && x.StringRequest != nil {
return *x.StringRequest
}
return ""
}
func (x *MyMessageExample) GetStringResponse() string {
if x != nil && x.StringResponse != nil {
return *x.StringResponse
}
return ""
}
var File_myexample_proto protoreflect.FileDescriptor
var file_myexample_proto_rawDesc = []byte{
0x0a, 0x0f, 0x6d, 0x79, 0x65, 0x78, 0x61, 0x6d, 0x70, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74,
0x6f, 0x12, 0x11, 0x6d, 0x79, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x65, 0x78, 0x61,
0x6d, 0x70, 0x6c, 0x65, 0x22, 0x89, 0x02, 0x0a, 0x10, 0x4d, 0x79, 0x4d, 0x65, 0x73, 0x73, 0x61,
0x67, 0x65, 0x45, 0x78, 0x61, 0x6d, 0x70, 0x6c, 0x65, 0x12, 0x29, 0x0a, 0x0d, 0x73, 0x74, 0x72,
0x69, 0x6e, 0x67, 0x4d, 0x65, 0x6d, 0x62, 0x65, 0x72, 0x31, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09,
0x48, 0x00, 0x52, 0x0d, 0x73, 0x74, 0x72, 0x69, 0x6e, 0x67, 0x4d, 0x65, 0x6d, 0x62, 0x65, 0x72,
0x31, 0x88, 0x01, 0x01, 0x12, 0x29, 0x0a, 0x0d, 0x73, 0x74, 0x72, 0x69, 0x6e, 0x67, 0x4d, 0x65,
0x6d, 0x62, 0x65, 0x72, 0x32, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x48, 0x01, 0x52, 0x0d, 0x73,
0x74, 0x72, 0x69, 0x6e, 0x67, 0x4d, 0x65, 0x6d, 0x62, 0x65, 0x72, 0x32, 0x88, 0x01, 0x01, 0x12,
0x29, 0x0a, 0x0d, 0x73, 0x74, 0x72, 0x69, 0x6e, 0x67, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74,
0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x48, 0x02, 0x52, 0x0d, 0x73, 0x74, 0x72, 0x69, 0x6e, 0x67,
0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x88, 0x01, 0x01, 0x12, 0x2b, 0x0a, 0x0e, 0x73, 0x74,
0x72, 0x69, 0x6e, 0x67, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x18, 0x04, 0x20, 0x01,
0x28, 0x09, 0x48, 0x03, 0x52, 0x0e, 0x73, 0x74, 0x72, 0x69, 0x6e, 0x67, 0x52, 0x65, 0x73, 0x70,
0x6f, 0x6e, 0x73, 0x65, 0x88, 0x01, 0x01, 0x42, 0x10, 0x0a, 0x0e, 0x5f, 0x73, 0x74, 0x72, 0x69,
0x6e, 0x67, 0x4d, 0x65, 0x6d, 0x62, 0x65, 0x72, 0x31, 0x42, 0x10, 0x0a, 0x0e, 0x5f, 0x73, 0x74,
0x72, 0x69, 0x6e, 0x67, 0x4d, 0x65, 0x6d, 0x62, 0x65, 0x72, 0x32, 0x42, 0x10, 0x0a, 0x0e, 0x5f,
0x73, 0x74, 0x72, 0x69, 0x6e, 0x67, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x42, 0x11, 0x0a,
0x0f, 0x5f, 0x73, 0x74, 0x72, 0x69, 0x6e, 0x67, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65,
0x32, 0x6a, 0x0a, 0x10, 0x4d, 0x79, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x45, 0x78, 0x61,
0x6d, 0x70, 0x6c, 0x65, 0x12, 0x56, 0x0a, 0x0a, 0x51, 0x75, 0x65, 0x72, 0x79, 0x48, 0x65, 0x6c,
0x6c, 0x6f, 0x12, 0x23, 0x2e, 0x6d, 0x79, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x65,
0x78, 0x61, 0x6d, 0x70, 0x6c, 0x65, 0x2e, 0x4d, 0x79, 0x4d, 0x65, 0x73, 0x73, 0x61, 0x67, 0x65,
0x45, 0x78, 0x61, 0x6d, 0x70, 0x6c, 0x65, 0x1a, 0x23, 0x2e, 0x6d, 0x79, 0x70, 0x72, 0x6f, 0x74,
0x6f, 0x62, 0x75, 0x66, 0x65, 0x78, 0x61, 0x6d, 0x70, 0x6c, 0x65, 0x2e, 0x4d, 0x79, 0x4d, 0x65,
0x73, 0x73, 0x61, 0x67, 0x65, 0x45, 0x78, 0x61, 0x6d, 0x70, 0x6c, 0x65, 0x42, 0x0e, 0x5a, 0x0c,
0x2e, 0x2f, 0x6d, 0x79, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x62, 0x06, 0x70, 0x72,
0x6f, 0x74, 0x6f, 0x33,
}
var (
file_myexample_proto_rawDescOnce sync.Once
file_myexample_proto_rawDescData = file_myexample_proto_rawDesc
)
func file_myexample_proto_rawDescGZIP() []byte {
file_myexample_proto_rawDescOnce.Do(func() {
file_myexample_proto_rawDescData = protoimpl.X.CompressGZIP(file_myexample_proto_rawDescData)
})
return file_myexample_proto_rawDescData
}
var file_myexample_proto_msgTypes = make([]protoimpl.MessageInfo, 1)
var file_myexample_proto_goTypes = []interface{}{
(*MyMessageExample)(nil), // 0: myprotobufexample.MyMessageExample
}
var file_myexample_proto_depIdxs = []int32{
0, // 0: myprotobufexample.MyServiceExample.QueryHello:input_type -> myprotobufexample.MyMessageExample
0, // 1: myprotobufexample.MyServiceExample.QueryHello:output_type -> myprotobufexample.MyMessageExample
1, // [1:2] is the sub-list for method output_type
0, // [0:1] is the sub-list for method input_type
0, // [0:0] is the sub-list for extension type_name
0, // [0:0] is the sub-list for extension extendee
0, // [0:0] is the sub-list for field type_name
}
func init() { file_myexample_proto_init() }
func file_myexample_proto_init() {
if File_myexample_proto != nil {
return
}
if !protoimpl.UnsafeEnabled {
file_myexample_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} {
switch v := v.(*MyMessageExample); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
}
file_myexample_proto_msgTypes[0].OneofWrappers = []interface{}{}
type x struct{}
out := protoimpl.TypeBuilder{
File: protoimpl.DescBuilder{
GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
RawDescriptor: file_myexample_proto_rawDesc,
NumEnums: 0,
NumMessages: 1,
NumExtensions: 0,
NumServices: 1,
},
GoTypes: file_myexample_proto_goTypes,
DependencyIndexes: file_myexample_proto_depIdxs,
MessageInfos: file_myexample_proto_msgTypes,
}.Build()
File_myexample_proto = out.File
file_myexample_proto_rawDesc = nil
file_myexample_proto_goTypes = nil
file_myexample_proto_depIdxs = nil
}
myexample_grpc.pb.go文件的内容如下所示：
// myexample.proto
// Code generated by protoc-gen-go-grpc. DO NOT EDIT.
// versions:
// - protoc-gen-go-grpc v1.3.0
// - protoc v3.17.3
// source: myexample.proto
package myprotobuf
import (
context "context"
grpc "google.golang.org/grpc"
codes "google.golang.org/grpc/codes"
status "google.golang.org/grpc/status"
)
// This is a compile-time assertion to ensure that this generated file
// is compatible with the grpc package it is being compiled against.
// Requires gRPC-Go v1.32.0 or later.
const _ = grpc.SupportPackageIsVersion7
const (
MyServiceExample_QueryHello_FullMethodName = "/myprotobufexample.MyServiceExample/QueryHello"
)
// MyServiceExampleClient is the client API for MyServiceExample service.
//
// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.
type MyServiceExampleClient interface {
QueryHello(ctx context.Context, in *MyMessageExample, opts ...grpc.CallOption) (*MyMessageExample, error)
}
type myServiceExampleClient struct {
cc grpc.ClientConnInterface
}
func NewMyServiceExampleClient(cc grpc.ClientConnInterface) MyServiceExampleClient {
return &myServiceExampleClient{cc}
}
func (c *myServiceExampleClient) QueryHello(ctx context.Context, in *MyMessageExample, opts ...grpc.CallOption) (*MyMessageExample, error) {
out := new(MyMessageExample)
err := c.cc.Invoke(ctx, MyServiceExample_QueryHello_FullMethodName, in, out, opts...)
if err != nil {
return nil, err
}
return out, nil
}
// MyServiceExampleServer is the server API for MyServiceExample service.
// All implementations must embed UnimplementedMyServiceExampleServer
// for forward compatibility
type MyServiceExampleServer interface {
QueryHello(context.Context, *MyMessageExample) (*MyMessageExample, error)
mustEmbedUnimplementedMyServiceExampleServer()
}
// UnimplementedMyServiceExampleServer must be embedded to have forward compatible implementations.
type UnimplementedMyServiceExampleServer struct {
}
func (UnimplementedMyServiceExampleServer) QueryHello(context.Context, *MyMessageExample) (*MyMessageExample, error) {
return nil, status.Errorf(codes.Unimplemented, "method QueryHello not implemented")
}
func (UnimplementedMyServiceExampleServer) mustEmbedUnimplementedMyServiceExampleServer() {}
// UnsafeMyServiceExampleServer may be embedded to opt out of forward compatibility for this service.
// Use of this interface is not recommended, as added methods to MyServiceExampleServer will
// result in compilation errors.
type UnsafeMyServiceExampleServer interface {
mustEmbedUnimplementedMyServiceExampleServer()
}
func RegisterMyServiceExampleServer(s grpc.ServiceRegistrar, srv MyServiceExampleServer) {
s.RegisterService(&MyServiceExample_ServiceDesc, srv)
}
func _MyServiceExample_QueryHello_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(MyMessageExample)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(MyServiceExampleServer).QueryHello(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: MyServiceExample_QueryHello_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(MyServiceExampleServer).QueryHello(ctx, req.(*MyMessageExample))
}
return interceptor(ctx, in, info, handler)
}
// MyServiceExample_ServiceDesc is the grpc.ServiceDesc for MyServiceExample service.
// It's only intended for direct use with grpc.RegisterService,
// and not to be introspected or modified (even as a copy)
var MyServiceExample_ServiceDesc = grpc.ServiceDesc{
ServiceName: "myprotobufexample.MyServiceExample",
HandlerType: (*MyServiceExampleServer)(nil),
Methods: []grpc.MethodDesc{
{
MethodName: "QueryHello",
Handler: _MyServiceExample_QueryHello_Handler,
},
},
Streams: []grpc.StreamDesc{},
Metadata: "myexample.proto",
}
.proto描述信息与.pb.go目标文件之间的元素对应关系如下表所示：
| .proto描述字段 | .pb.go描述字段 | 说明 |
|---|---|---|
| package myprotobufexample; | ---- | 无实际的对应关系 |
| postfix@go_package:myprotobuf | packae myprotobuf | go_package定义了目标文件的存放路径名以及生成的GO包名 |
| message MyMessageExample | sturct MyMessageExample | 结构化数据映射 |
| ---- | func (x *MyMessageExample) Reset() | 自动生成： 结构化数据相关的操作接口； |
| func (x *MyMessageExample) String() string | ||
| func (*MyMessageExample) ProtoMessage() | ||
| func (x *MyMessageExample) ProtoReflect() protoreflect.Message | ||
| func (*MyMessageExample) Descriptor() ([]byte, []int) | ||
| func (x *MyMessageExample) GetStringMember1() string | ||
| func (x *MyMessageExample) GetStringMember2() string | ||
| func (x *MyMessageExample) GetStringRequest() string | ||
| func (x *MyMessageExample) GetStringResponse() string | ||
| ---- | var File_myexample_proto protoreflect.FileDescriptor | 自动生成： 文件信息的一些属性和操作接口； |
| var file_myexample_proto_rawDesc = []byte | ||
| var file_myexample_proto_rawDescData = file_myexample_proto_rawDesc | ||
| func file_myexample_proto_rawDescGZIP() []byte | ||
| var file_myexample_proto_msgTypes = make | ||
| var file_myexample_proto_goTypes = []interface{} | ||
| var file_myexample_proto_depIdxs = []int32 | ||
| func init() | 自动生成： | |
| service MyServiceExample | type MyServiceExampleClient interface； type MyServiceExampleServer interface； type UnsafeMyServiceExampleServer interface； | 自动生成： 接口映射，包含服务端和客户端两个接口集和一个不安全的服务端接口； |
| rpc QueryHello(MyMessageExample) returns(MyMessageExample) | func (c *myServiceExampleClient) QueryHello(ctx context.Context, in MyMessageExample, opts ...grpc.CallOption) ( MyMessageExample, error) | 自动生成： RPC API，可供客户端和服务端接口调用； |
| ---- | func RegisterMyServiceExampleServer(s grpc.ServiceRegistrar, srv MyServiceExampleServer) | 自动生成： 用于向gRPC框架注册服务； |
| func _MyServiceExample_QueryHello_Handler | 自动生成： 用于当注册的服务触发被调用请求时gRPC可以及时回调进行具体处理； | |
| var MyServiceExample_ServiceDesc = grpc.ServiceDesc | 自动生成： RPC服务描述信息，描述了该服务接口提供了哪些相关服务以及对应的回调。由服务注册接口使用； |
proto元数据类型与具体主要语言之间的映射关系
| .proto Type | Notes | C++ Type | Python Type | Go Type |
|---|---|---|---|---|
| double | double | float | float64 | |
| float | float | float | float32 | |
| int32 | 使用变长编码，对于负值的效率很低，如果你的域有 可能有负值，请使用sint64替代 | int32 | int | int32 |
| uint32 | 使用变长编码 | uint32 | int/long | uint32 |
| uint64 | 使用变长编码 | uint64 | int/long | uint64 |
| sint32 | 使用变长编码，这些编码在负值时比int32高效的多 | int32 | int | int32 |
| sint64 | 使用变长编码，有符号的整型值。编码时比通常的 int64高效。 | int64 | int/long | int64 |
| fixed32 | 总是4个字节，如果数值总是比总是比228大的话，这 个类型会比uint32高效。 | uint32 | int | uint32 |
| fixed64 | 总是8个字节，如果数值总是比总是比256大的话，这 个类型会比uint64高效。 | uint64 | int/long | uint64 |
| sfixed32 | 总是4个字节 | int32 | int | int32 |
| sfixed32 | 总是4个字节 | int32 | int | int32 |
| sfixed64 | 总是8个字节 | int64 | int/long | int64 |
| bool | bool | bool | bool | |
| string | 一个字符串必须是UTF-8编码或者7-bit ASCII编码的文 本。 | string | str/unicode | string |
| bytes | 可能包含任意顺序的字节数据。 | string | str | []byte |
protobuf数据的序列化和反序列化
protobuf的数据存储或传输采用的是变长的编码规则，只保存有用的信息，尽可能的充分利用好每一个bit位，节省了大量空间和时间资源。
protobuf采用的是TLV编码格式（Tag-Length-Value，Length变长0~more）：
编码后的数据大致为如下的bit流：
具体的编码规则我们实际上无需太过于关心，protobuf自身已经可以很好的来完成编码和解码的相关工作了。我们作为用户只需要使用protobuf所提供的序列化和反序列化API接口即可以完成对数据的编解码。
关于protobuf数据的传输以及gRPC中的服务是如何调用以及进行参数传输的，以后会再补充或另行开文进行总结。
