LeaderElector是什么
在多副本的后端服务部署架构中,有时我们只能有一个副本来处理任务,其他副本只作为热备,这时就需要通过选举决定出主从,只有主才能处理任务,这时我们可以使用LeaderElector来帮助我么实现选主
选主模型
选主模型并不陌生,比较有代表性的就是K8S的一大基础组件etcd的raft算法,通过选主的模型来实现多副本数据强一致性
这里LeaderElector使用的比较简单,我们先从LeaderElectionRecord的字段来了解
// LeaderElectionRecord is the record that is stored in the leader election annotation.
// This information should be used for observational purposes only and could be replaced
// with a random string (e.g. UUID) with only slight modification of this code.
// TODO(mikedanese): this should potentially be versioned
type LeaderElectionRecord struct {
// HolderIdentity is the ID that owns the lease. If empty, no one owns this lease and
// all callers may acquire. Versions of this library prior to Kubernetes 1.14 will not
// attempt to acquire leases with empty identities and will wait for the full lease
// interval to expire before attempting to reacquire. This value is set to empty when
// a client voluntarily steps down.
HolderIdentity string `json:"holderIdentity"`
LeaseDurationSeconds int `json:"leaseDurationSeconds"`
AcquireTime metav1.Time `json:"acquireTime"`
RenewTime metav1.Time `json:"renewTime"`
LeaderTransitions int `json:"leaderTransitions"`
}
- HolderIdentity:主实例的ID
- LeaseDurationSeconds:任期
- AcquireTime:初次继任时间
- RenewTime:连任或继任时间
- LeaderTransitions:第几次选举 从上面的名称中,很容易的能联想出选举与继任的过程:
- 在无主时,开启选举,选出新主
- 在有主时,主不断连任来延续自己的任期
- 在主无法继任超出任期后,其他跟随者开启新的一轮选举,选主新主
ResourceLock
// Interface offers a common interface for locking on arbitrary
// resources used in leader election. The Interface is used
// to hide the details on specific implementations in order to allow
// them to change over time. This interface is strictly for use
// by the leaderelection code.
type Interface interface {
// Get returns the LeaderElectionRecord
Get(ctx context.Context) (*LeaderElectionRecord, []byte, error)
// Create attempts to create a LeaderElectionRecord
Create(ctx context.Context, ler LeaderElectionRecord) error
// Update will update and existing LeaderElectionRecord
Update(ctx context.Context, ler LeaderElectionRecord) error
// RecordEvent is used to record events
RecordEvent(string)
// Identity will return the locks Identity
Identity() string
// Describe is used to convert details on current resource lock
// into a string
Describe() string
}
上面提到的选主过程中,从开启选举到选出新主是通过ResourceLock实现的,也就是谁抢到了锁,谁就是主。
由于使用锁的客户端是不同进程的服务,所以就需要使用分布式锁,常见的分布式锁可以通过数据库、redis实现,这里因为是在k8s的环境中,所以使用k8s的api-server通过操作Resource资源来实现。
k8s的api-server是并发安全的,在并发创建、更新时,只会有其中一个并发是成功的,当并发创建时,其他并发会返回AlreadyExists错误;在并发更新时,会携带上次获取Resource资源的ResourceVersion,当率先被处理的并发线程被处理成功后,ResourceVersion会自动+1,其他并发线程再被处理时,由于ResourceVersion低于当前版本,会被拒绝更新
LeaderElector
了解了选举模型与ResourceLock后,LeaderElector的功能也就很清晰了
- 程序刚启动时,没有LeaderElectionRecord记录,多个跟随者并发创建,创建成功的会被选为主
- 跟随者在下一轮try时,会获取到已经创建的LeaderElectionRecord记录,检测记录中的继任时间与任期,由于并没有过期,所以不会做什么
- 领导者在下一轮try时,会更新自己的继任时间,避免过期
- 当领导者无法继任,跟随者检测到已经过期时,会更新LeaderElectionRecord,将领导者改为自己,成功更新的跟随者会获得这轮选举的胜利,成为新的领导者