kubelet初探: 不起眼的组件却最复杂易碎

77 阅读2分钟

概述

kubelet是Kubernetes初学者不会太注意的组件,因为作为一个Node Agent,从单体规模上和资源消耗上,它不太起眼。但是熟悉k8s常用能力的开发者知道,k8s还是提供了不少丰富的特性和假设规范的,这些特性的实现大多要放在kubelet,并且由于k8s的接口协议较多,kubelet都要处理好这些协议,比如CNI、CRI、CSI,因此kubelet成为了k8s名副其实的最复杂组件。

details

Kubelet

kubelet 主类非常大,囊括了整个kubelet的所有功能类,可以说kebelet的架构是集中式的,或者说架构设计比较粗犷。

type Kubelet struct {
    // ...
}

PodWorkers

NewMainKubelet 初始化了Kubelet主类的同时,开启podWorkers同步Pod的etcd状态。

	klet.podWorkers = newPodWorkers(
		klet.syncPod,
		klet.syncTerminatingPod,
		klet.syncTerminatedPod,

		kubeDeps.Recorder,
		klet.workQueue,
		klet.resyncInterval,
		backOffPeriod,
		klet.podCache,
	)
// If any step of this workflow errors, the error is returned, and is repeated
// on the next syncPod call.
func (kl *Kubelet) syncPod(ctx context.Context, updateType kubetypes.SyncPodType, pod, mirrorPod *v1.Pod, podStatus *kubecontainer.PodStatus) (isTerminal bool, err error) {
}

syncLoop

syncLoop 是 kubelet的主循环,其中采用了大量函数指针,由于Go对函数式编程支持的简陋,导致这些handler代码的实现异常难搞清楚,可见滥用函数在Go中是多么让人厌恶。

// Run starts the kubelet reacting to config updates
func (kl *Kubelet) Run(updates <-chan kubetypes.PodUpdate) {
    kl.pleg.Start()
	kl.syncLoop(updates, kl)
}

func (kl *Kubelet) syncLoop(updates <-chan kubetypes.PodUpdate, handler SyncHandler) {
    for {
        if !kl.syncLoopIteration(updates, handler, syncTicker.C, housekeepingTicker.C, plegCh) {
			break
		}
    }
}

// syncLoopIteration reads from various channels and dispatches pods to the
// given handler.
func (kl *Kubelet) syncLoopIteration(configCh <-chan kubetypes.PodUpdate, handler SyncHandler,
	syncCh <-chan time.Time, housekeepingCh <-chan time.Time, plegCh <-chan *pleg.PodLifecycleEvent) bool {
}

manager

kubelet 将众多功能拆分成了一个个manager负责,数量较多,具体实现可先略过。

kubepod.Manager

	mirrorPodClient := kubepod.NewBasicMirrorClient(klet.kubeClient, string(nodeName), nodeLister)
	klet.podManager = kubepod.NewBasicPodManager(mirrorPodClient, secretManager, configMapManager)

eviction.Manager

    evictionManager, evictionAdmitHandler := eviction.NewManager(klet.resourceAnalyzer, evictionConfig, killPodNow(klet.podWorkers, kubeDeps.Recorder), klet.podManager.GetMirrorPodByPod, klet.imageManager, klet.containerGC, kubeDeps.Recorder, nodeRef, klet.clock)

	klet.evictionManager = evictionManager

prober.Manager

	if kubeDeps.ProbeManager != nil {
		klet.probeManager = kubeDeps.ProbeManager
	} else {
		klet.probeManager = prober.NewManager(
			klet.statusManager,
			klet.livenessManager,
			klet.readinessManager,
			klet.startupManager,
			klet.runner,
			kubeDeps.Recorder)
	}

proberesults.Manager

	klet.livenessManager = proberesults.NewManager()
	klet.readinessManager = proberesults.NewManager()
	klet.startupManager = proberesults.NewManager()

secret.Manager/configmap.Manager

	klet.secretManager = secretManager
	klet.configMapManager = configMapManager

status.Manager

klet.statusManager = status.NewManager(klet.kubeClient, klet.podManager, klet)

volumemanager.VolumeManager

    klet.volumeManager = volumemanager.NewVolumeManager(
		kubeCfg.EnableControllerAttachDetach,
		nodeName,
		klet.podManager,
		klet.podWorkers,
		klet.kubeClient,
		klet.volumePluginMgr,
		klet.containerRuntime,
		kubeDeps.Mounter,
		kubeDeps.HostUtil,
		klet.getPodsDir(),
		kubeDeps.Recorder,
		keepTerminatedPodVolumes,
		volumepathhandler.NewBlockVolumePathHandler())

gc

kubelet 需要一些具有资源紧张探测和资源回收的能力,在kubelet中被称为GC,但是实际使用中GC的原本目的不一定能达到期望。

kubecontainer.GC

	klet.containerGC = containerGC
	klet.containerDeletor = newPodContainerDeletor(klet.containerRuntime, integer.IntMax(containerGCPolicy.MaxPerPodContainer, minDeadContainerInPod)

images.ImageGCManager

	imageManager, err := images.NewImageGCManager(klet.containerRuntime, klet.StatsProvider, kubeDeps.Recorder, nodeRef, imageGCPolicy, crOptions.PodSandboxImage)
	if err != nil {
		return nil, fmt.Errorf("failed to initialize image manager: %v", err)
	}
	klet.imageManager = imageManager

pleg

Pod Lifecycle Event Generator 算是kubelet少有的专有化名词,根据资料,是专门设计出用来轮询Pod状态,并包装成PodLifecycleEvent上报给主循环的,最最早期的kubelet应该是没有pleg的,因为可以在k8s的设计文档中找到pleg的设计提案。

值得注意的是,另一个Kubernetes生态项目 koordinator的agent koordlet也仿照kubelet实现了一个pleg的包,有时间可以对比这2个pleg的实现的优缺点。

	klet.pleg = pleg.NewGenericPLEG(klet.containerRuntime, plegChannelCapacity, plegRelistPeriod, klet.podCache, clock.RealClock{})

PodLifecycleEvent

// PodLifecycleEvent is an event that reflects the change of the pod state.
type PodLifecycleEvent struct {
	// The pod ID.
	ID types.UID
	// The type of the event.
	Type PodLifeCycleEventType
	// The accompanied data which varies based on the event type.
	//   - ContainerStarted/ContainerStopped: the container name (string).
	//   - All other event types: unused.
	Data interface{}
}