Kubernetes kube-controller-manager 控制中心机制源码深入剖析-Kubernetes商业环境实战

技术洞察TIC
1,440 阅读8分钟

专注于大数据及容器云核心技术解密，可提供全栈的大数据+云原生平台咨询方案，请持续关注本套博客。如有任何学术交流，可随时联系。更多内容请关注《数据云技术社区》公众号。

1 管理控制中心机制(Informer联动)

1.1 包结构

1.2 controller的启动去脉

  • kube-controller-manager 作为集群的管理控制中心，维护集群中的所有控制器，对维持集群的稳定和自我修复，实现高可用，副本控制等起关键作用。
  • Controller.run。主要解决自定义的资源变化事件回调处理逻辑
  • 在controller-manager的Run函数部分调用了InformerFactory.Start的方法，Start方法初始化各种类型的informer，并且每个类型起了个informer.Run的goroutine,Informer的作用就是监听和变化事件扔进队列中。
  • 三大核心函数：CreateControllerContext，StartControllers，ctx.InformerFactory.Start
k8s.io/kubernetes/cmd/kube-controller-manager/app/controllermanager.go
// Run runs the KubeControllerManagerOptions.  This should never exit.
func Run(c *config.CompletedConfig) error {
  
        //1:拿到对kube-APIserver中资源的操作句柄,创建控制器上下文 
        ctx, err := CreateControllerContext(c, rootClientBuilder, clientBuilder, stop)

        //2:初始化的所有控制器（包括apiserver的客户端，informer的回调函数等等）
        if err := StartControllers(ctx, saTokenControllerInitFunc, NewControllerInitializers(ctx.LoopMode)); err != nil {
            glog.Fatalf("error starting controllers: %v", err)
        }
        
        //3:启动Informer,并完成Controller最终的启动以及资源监听机制
        ctx.InformerFactory.Start(ctx.Stop)
        close(ctx.InformersStarted)
}

2 控制中心机制

2.1 控制中心入口之NewControllerManagerCommand

  • Kube-controller-manager的代码风格仍然是Cobra命令行框架。通过构造ControllerManagerCommand，然后执行command.Execute()函数。
k8s.io/kubernetes/cmd/kube-controller-manager/controller-manager.go
func main() {
    rand.Seed(time.Now().UTC().UnixNano())

    command := app.NewControllerManagerCommand()

    // TODO: once we switch everything over to Cobra commands, we can go back to calling
    // utilflag.InitFlags() (by removing its pflag.Parse() call). For now, we have to set the
    // normalize func and add the go flag set by hand.
    pflag.CommandLine.SetNormalizeFunc(utilflag.WordSepNormalizeFunc)
    pflag.CommandLine.AddGoFlagSet(goflag.CommandLine)
    // utilflag.InitFlags()
    logs.InitLogs()
    defer logs.FlushLogs()

    if err := command.Execute(); err != nil {
        fmt.Fprintf(os.Stderr, "%v\n", err)
        os.Exit(1)
    }
}

2.2 控制中心之NewKubeControllerManagerOptions

  • 构建ControllerManagerCommand基本的流程就是先构造NewControllerManagerCommand，添加Flags，执行Run函数。
  • NewKubeControllerManagerOptions是初始化controllerManager的参数，其中主要包括了各种controller的option
func NewControllerManagerCommand() *cobra.Command {

    s, err := options.NewKubeControllerManagerOptions()
    if err != nil {
        glog.Fatalf("unable to initialize command options: %v", err)
    }

    cmd := &cobra.Command{
        Use: "kube-controller-manager",
        Long: `The Kubernetes controller manager is a daemon that embeds
the core control loops shipped with Kubernetes. In applications of robotics and
automation, a control loop is a non-terminating loop that regulates the state of
the system. In Kubernetes, a controller is a control loop that watches the shared
state of the cluster through the apiserver and makes changes attempting to move the
current state towards the desired state. Examples of controllers that ship with
Kubernetes today are the replication controller, endpoints controller, namespace
controller, and serviceaccounts controller.`,
        Run: func(cmd *cobra.Command, args []string) {
            verflag.PrintAndExitIfRequested()
            utilflag.PrintFlags(cmd.Flags())

            c, err := s.Config(KnownControllers(), ControllersDisabledByDefault.List())
            if err != nil {
                fmt.Fprintf(os.Stderr, "%v\n", err)
                os.Exit(1)
            }

            if err := Run(c.Complete()); err != nil {
                fmt.Fprintf(os.Stderr, "%v\n", err)
                os.Exit(1)
            }
        },
    }
    s.AddFlags(cmd.Flags(), KnownControllers(), ControllersDisabledByDefault.List())

    return cmd
}

//初始化controllerManager的参数，其中主要包括了各种controller的option,例如DeploymentControllerOptions
//NewKubeControllerManagerOptions creates a new KubeControllerManagerOptions with a default config.
func NewKubeControllerManagerOptions() (*KubeControllerManagerOptions, error) {
    componentConfig, err := NewDefaultComponentConfig(ports.InsecureKubeControllerManagerPort)
    if err != nil {
        return nil, err
    }

    s := KubeControllerManagerOptions{
        Generic:         cmoptions.NewGenericControllerManagerConfigurationOptions(componentConfig.Generic),
        KubeCloudShared: cmoptions.NewKubeCloudSharedOptions(componentConfig.KubeCloudShared),
        AttachDetachController: &AttachDetachControllerOptions{
            ReconcilerSyncLoopPeriod: componentConfig.AttachDetachController.ReconcilerSyncLoopPeriod,
        },
        CSRSigningController: &CSRSigningControllerOptions{
            ClusterSigningCertFile: componentConfig.CSRSigningController.ClusterSigningCertFile,
            ClusterSigningKeyFile:  componentConfig.CSRSigningController.ClusterSigningKeyFile,
            ClusterSigningDuration: componentConfig.CSRSigningController.ClusterSigningDuration,
        },
        DaemonSetController: &DaemonSetControllerOptions{
            ConcurrentDaemonSetSyncs: componentConfig.DaemonSetController.ConcurrentDaemonSetSyncs,
        },
        DeploymentController: &DeploymentControllerOptions{
            ConcurrentDeploymentSyncs:      componentConfig.DeploymentController.ConcurrentDeploymentSyncs,
            DeploymentControllerSyncPeriod: componentConfig.DeploymentController.DeploymentControllerSyncPeriod,
        },
        DeprecatedFlags: &DeprecatedControllerOptions{
            RegisterRetryCount: componentConfig.DeprecatedController.RegisterRetryCount,
        },
        EndpointController: &EndpointControllerOptions{
            ConcurrentEndpointSyncs: componentConfig.EndpointController.ConcurrentEndpointSyncs,
        },
        GarbageCollectorController: &GarbageCollectorControllerOptions{
            ConcurrentGCSyncs:      componentConfig.GarbageCollectorController.ConcurrentGCSyncs,
            EnableGarbageCollector: componentConfig.GarbageCollectorController.EnableGarbageCollector,
        },
        HPAController: &HPAControllerOptions{
            HorizontalPodAutoscalerSyncPeriod:                   componentConfig.HPAController.HorizontalPodAutoscalerSyncPeriod,
            HorizontalPodAutoscalerUpscaleForbiddenWindow:       componentConfig.HPAController.HorizontalPodAutoscalerUpscaleForbiddenWindow,
            HorizontalPodAutoscalerDownscaleForbiddenWindow:     componentConfig.HPAController.HorizontalPodAutoscalerDownscaleForbiddenWindow,
            HorizontalPodAutoscalerDownscaleStabilizationWindow: componentConfig.HPAController.HorizontalPodAutoscalerDownscaleStabilizationWindow,
            HorizontalPodAutoscalerCPUInitializationPeriod:      componentConfig.HPAController.HorizontalPodAutoscalerCPUInitializationPeriod,
            HorizontalPodAutoscalerInitialReadinessDelay:        componentConfig.HPAController.HorizontalPodAutoscalerInitialReadinessDelay,
            HorizontalPodAutoscalerTolerance:                    componentConfig.HPAController.HorizontalPodAutoscalerTolerance,
            HorizontalPodAutoscalerUseRESTClients:               componentConfig.HPAController.HorizontalPodAutoscalerUseRESTClients,
        },
        JobController: &JobControllerOptions{
            ConcurrentJobSyncs: componentConfig.JobController.ConcurrentJobSyncs,
        },
        NamespaceController: &NamespaceControllerOptions{
            NamespaceSyncPeriod:      componentConfig.NamespaceController.NamespaceSyncPeriod,
            ConcurrentNamespaceSyncs: componentConfig.NamespaceController.ConcurrentNamespaceSyncs,
        },
        NodeIPAMController: &NodeIPAMControllerOptions{
            NodeCIDRMaskSize: componentConfig.NodeIPAMController.NodeCIDRMaskSize,
        },
        NodeLifecycleController: &NodeLifecycleControllerOptions{
            EnableTaintManager:     componentConfig.NodeLifecycleController.EnableTaintManager,
            NodeMonitorGracePeriod: componentConfig.NodeLifecycleController.NodeMonitorGracePeriod,
            NodeStartupGracePeriod: componentConfig.NodeLifecycleController.NodeStartupGracePeriod,
            PodEvictionTimeout:     componentConfig.NodeLifecycleController.PodEvictionTimeout,
        },
        PersistentVolumeBinderController: &PersistentVolumeBinderControllerOptions{
            PVClaimBinderSyncPeriod: componentConfig.PersistentVolumeBinderController.PVClaimBinderSyncPeriod,
            VolumeConfiguration:     componentConfig.PersistentVolumeBinderController.VolumeConfiguration,
        },
        PodGCController: &PodGCControllerOptions{
            TerminatedPodGCThreshold: componentConfig.PodGCController.TerminatedPodGCThreshold,
        },
        ReplicaSetController: &ReplicaSetControllerOptions{
            ConcurrentRSSyncs: componentConfig.ReplicaSetController.ConcurrentRSSyncs,
        },
        ReplicationController: &ReplicationControllerOptions{
            ConcurrentRCSyncs: componentConfig.ReplicationController.ConcurrentRCSyncs,
        },
        ResourceQuotaController: &ResourceQuotaControllerOptions{
            ResourceQuotaSyncPeriod:      componentConfig.ResourceQuotaController.ResourceQuotaSyncPeriod,
            ConcurrentResourceQuotaSyncs: componentConfig.ResourceQuotaController.ConcurrentResourceQuotaSyncs,
        },
        SAController: &SAControllerOptions{
            ConcurrentSATokenSyncs: componentConfig.SAController.ConcurrentSATokenSyncs,
        },
        ServiceController: &cmoptions.ServiceControllerOptions{
            ConcurrentServiceSyncs: componentConfig.ServiceController.ConcurrentServiceSyncs,
        },
        TTLAfterFinishedController: &TTLAfterFinishedControllerOptions{
            ConcurrentTTLSyncs: componentConfig.TTLAfterFinishedController.ConcurrentTTLSyncs,
        },
        SecureServing: apiserveroptions.NewSecureServingOptions().WithLoopback(),
        InsecureServing: (&apiserveroptions.DeprecatedInsecureServingOptions{
            BindAddress: net.ParseIP(componentConfig.Generic.Address),
            BindPort:    int(componentConfig.Generic.Port),
            BindNetwork: "tcp",
        }).WithLoopback(),
        Authentication: apiserveroptions.NewDelegatingAuthenticationOptions(),
        Authorization:  apiserveroptions.NewDelegatingAuthorizationOptions(),
    }

    s.Authentication.RemoteKubeConfigFileOptional = true
    s.Authorization.RemoteKubeConfigFileOptional = true
    s.Authorization.AlwaysAllowPaths = []string{"/healthz"}

    s.SecureServing.ServerCert.CertDirectory = "/var/run/kubernetes"
    s.SecureServing.ServerCert.PairName = "kube-controller-manager"
    s.SecureServing.BindPort = ports.KubeControllerManagerPort

    gcIgnoredResources := make([]kubectrlmgrconfig.GroupResource, 0, len(garbagecollector.DefaultIgnoredResources()))
    for r := range garbagecollector.DefaultIgnoredResources() {
        gcIgnoredResources = append(gcIgnoredResources, kubectrlmgrconfig.GroupResource{Group: r.Group, Resource: r.Resource})
    }

    s.GarbageCollectorController.GCIgnoredResources = gcIgnoredResources

    return &s, nil
}

2.3 控制中心之command.Execute()

  • 加载所有控制器，并将对应参数注入到控制器中
c, err := s.Config(KnownControllers(), ControllersDisabledByDefault.List())

// KnownControllers returns all known controllers's name
func KnownControllers() []string {
	ret := sets.StringKeySet(NewControllerInitializers(IncludeCloudLoops))

	// add "special" controllers that aren't initialized normally.  These controllers cannot be initialized
	// using a normal function.  The only known special case is the SA token controller which *must* be started
	// first to ensure that the SA tokens for future controllers will exist.  Think very carefully before adding
	// to this list.
	ret.Insert(
		saTokenControllerName,
	)

	return ret.List()
}

//k8s.io/kubernetes/cmd/kube-controller-manager/app/controllermanager.go
KnownControllers()中的NewControllerInitializers初始化所有的控制器

c, err := s.Config(KnownControllers(), ControllersDisabledByDefault.List())
func NewControllerInitializers(loopMode ControllerLoopMode) map[string]InitFunc {
    controllers := map[string]InitFunc{}
    controllers["endpoint"] = startEndpointController
    controllers["replicationcontroller"] = startReplicationController
    controllers["podgc"] = startPodGCController
    controllers["resourcequota"] = startResourceQuotaController
    controllers["namespace"] = startNamespaceController
    controllers["serviceaccount"] = startServiceAccountController
    controllers["garbagecollector"] = startGarbageCollectorController
    controllers["daemonset"] = startDaemonSetController
    controllers["job"] = startJobController
    controllers["deployment"] = startDeploymentController
    controllers["replicaset"] = startReplicaSetController
    controllers["horizontalpodautoscaling"] = startHPAController
    controllers["disruption"] = startDisruptionController
    controllers["statefulset"] = startStatefulSetController
    controllers["cronjob"] = startCronJobController
    controllers["csrsigning"] = startCSRSigningController
    controllers["csrapproving"] = startCSRApprovingController
    controllers["csrcleaner"] = startCSRCleanerController
    controllers["ttl"] = startTTLController
    controllers["bootstrapsigner"] = startBootstrapSignerController
    controllers["tokencleaner"] = startTokenCleanerController
    controllers["nodeipam"] = startNodeIpamController
    if loopMode == IncludeCloudLoops {
        controllers["service"] = startServiceController
        controllers["route"] = startRouteController
        // TODO: volume controller into the IncludeCloudLoops only set.
        // TODO: Separate cluster in cloud check from node lifecycle controller.
    }
    controllers["nodelifecycle"] = startNodeLifecycleController
    controllers["persistentvolume-binder"] = startPersistentVolumeBinderController
    controllers["attachdetach"] = startAttachDetachController
    controllers["persistentvolume-expander"] = startVolumeExpandController
    controllers["clusterrole-aggregation"] = startClusterRoleAggregrationController
    controllers["pvc-protection"] = startPVCProtectionController
    controllers["pv-protection"] = startPVProtectionController
    return controllers
}
  • 启动controller-manager的http服务和对应处理器，包括安全和非安全：BuildHandlerChain ,构造run的执行体。
k8s.io/kubernetes/cmd/kube-controller-manager/app/controllermanager.go
// Run runs the KubeControllerManagerOptions.  This should never exit.
func Run(c *config.CompletedConfig) error {
  
        //1:拿到对kube-APIserver中资源的操作句柄,创建控制器上下文 
        ctx, err := CreateControllerContext(c, rootClientBuilder, clientBuilder, stop)

        //2:初始化的所有控制器（包括apiserver的客户端，informer的回调函数等等）
        if err := StartControllers(ctx, saTokenControllerInitFunc, NewControllerInitializers(ctx.LoopMode)); err != nil {
            glog.Fatalf("error starting controllers: %v", err)
        }
        
        //3:启动Informer,并完成Controller最终的启动以及资源监听机制
        ctx.InformerFactory.Start(ctx.Stop)
        close(ctx.InformersStarted)
}

2.4 创建控制器上下文

  • 拿到对kube-APIserver中资源的操作句柄
  • 确认Kube-APIServer的健康（最多等待10s），然后拿获取连接
  • 创建控制器上下文
func CreateControllerContext(s *config.CompletedConfig, rootClientBuilder, clientBuilder controller.ControllerClientBuilder, stop <-chan struct{}) (ControllerContext, error) {
    //拿到对APIServer资源的操作句柄
    versionedClient := rootClientBuilder.ClientOrDie("shared-informers")
    sharedInformers := informers.NewSharedInformerFactory(versionedClient, ResyncPeriod(s)())

    // If apiserver is not running we should wait for some time and fail only then. This is particularly
    // important when we start apiserver and controller manager at the same time.
    //gaogao note : 10s内检查APIserver服务是否可用
    if err := genericcontrollermanager.WaitForAPIServer(versionedClient, 10*time.Second); err != nil {
        return ControllerContext{}, fmt.Errorf("failed to wait for apiserver being healthy: %v", err)
    }

    // Use a discovery client capable of being refreshed.
    discoveryClient := rootClientBuilder.ClientOrDie("controller-discovery")
    //note:  DiscoveryClient = discoveryClient.Discovery()
    cachedClient := cacheddiscovery.NewMemCacheClient(discoveryClient.Discovery())
    restMapper := restmapper.NewDeferredDiscoveryRESTMapper(cachedClient)
    go wait.Until(func() {
        restMapper.Reset()
    }, 30*time.Second, stop)

    availableResources, err := GetAvailableResources(rootClientBuilder)
    if err != nil {
        return ControllerContext{}, err
    }

    cloud, loopMode, err := createCloudProvider(s.ComponentConfig.CloudProvider.Name, s.ComponentConfig.ExternalCloudVolumePlugin,
        s.ComponentConfig.CloudProvider.CloudConfigFile, s.ComponentConfig.KubeCloudShared.AllowUntaggedCloud, sharedInformers)
    if err != nil {
        return ControllerContext{}, err
    }

    ctx := ControllerContext{
        ClientBuilder:      clientBuilder,
        InformerFactory:    sharedInformers,
        ComponentConfig:    s.ComponentConfig,
        RESTMapper:         restMapper,
        AvailableResources: availableResources,
        Cloud:              cloud,
        LoopMode:           loopMode,
        Stop:               stop,
        InformersStarted:   make(chan struct{}),
        ResyncPeriod:       ResyncPeriod(s),
    }
    return ctx, nil
}

2.5 初始化的所有控制器（完成注册Informer注册到Factory，并启动所有Controller）

  • 构造controller manager option，并转化为Config对象，执行Run函数。
  • 基于Config对象创建ControllerContext，其中包含InformerFactory。
  • 基于ControllerContext运行各种controller，各种controller的定义在NewControllerInitializers中。
  • 执行InformerFactory.Start(ControllerContext已经定义好controller)，实际执行了controllerContext.InformerFactory.Start(controllerContext.Stop)
  • 每种controller都会构造自身的结构体并执行对应的Run函数。
func StartControllers(ctx ControllerContext, startSATokenController InitFunc, controllers map[string]InitFunc) error {
    ···
    for controllerName, initFn := range controllers {
        if !ctx.IsControllerEnabled(controllerName) {
            glog.Warningf("%q is disabled", controllerName)
            continue
        }

        time.Sleep(wait.Jitter(ctx.ComponentConfig.GenericComponent.ControllerStartInterval.Duration, ControllerStartJitter))

        glog.V(1).Infof("Starting %q", controllerName)
        
        //note : initFn为初始化controller是创建的初始化函数
        started, err := initFn(ctx)
        ···
    }

    return nil
}
  • initFn 就是一个大而全的map[string]InitFunc 其中之一的函数，如startEndpointController。startEndpointController用于构建Controller和Informr并运行Controller.run
// NewControllerInitializers is a public map of named controller groups (you can start more than one in an init func)
// paired to their InitFunc.  This allows for structured downstream composition and subdivision.
func NewControllerInitializers(loopMode ControllerLoopMode) map[string]InitFunc {
	controllers := map[string]InitFunc{}
	controllers["endpoint"] = startEndpointController
	controllers["replicationcontroller"] = startReplicationController
	controllers["podgc"] = startPodGCController
	controllers["resourcequota"] = startResourceQuotaController
	controllers["namespace"] = startNamespaceController
	controllers["serviceaccount"] = startServiceAccountController
	controllers["garbagecollector"] = startGarbageCollectorController
	controllers["daemonset"] = startDaemonSetController
	controllers["job"] = startJobController
	controllers["deployment"] = startDeploymentController
	controllers["replicaset"] = startReplicaSetController
	controllers["horizontalpodautoscaling"] = startHPAController
	controllers["disruption"] = startDisruptionController
	controllers["statefulset"] = startStatefulSetController
	controllers["cronjob"] = startCronJobController
	controllers["csrsigning"] = startCSRSigningController
	controllers["csrapproving"] = startCSRApprovingController
	controllers["csrcleaner"] = startCSRCleanerController
	controllers["ttl"] = startTTLController
	controllers["bootstrapsigner"] = startBootstrapSignerController
	controllers["tokencleaner"] = startTokenCleanerController
	controllers["nodeipam"] = startNodeIpamController
	controllers["nodelifecycle"] = startNodeLifecycleController
	if loopMode == IncludeCloudLoops {
		controllers["service"] = startServiceController
		controllers["route"] = startRouteController
		controllers["cloud-node-lifecycle"] = startCloudNodeLifecycleController
		// TODO: volume controller into the IncludeCloudLoops only set.
	}
	controllers["persistentvolume-binder"] = startPersistentVolumeBinderController
	controllers["attachdetach"] = startAttachDetachController
	controllers["persistentvolume-expander"] = startVolumeExpandController
	controllers["clusterrole-aggregation"] = startClusterRoleAggregrationController
	controllers["pvc-protection"] = startPVCProtectionController
	controllers["pv-protection"] = startPVProtectionController
	controllers["ttl-after-finished"] = startTTLAfterFinishedController
	controllers["root-ca-cert-publisher"] = startRootCACertPublisher

	return controllers
}
  • Informer注册是在startDeploymentController方法中实现的，以Deployment为例，注册的目的其实就是Deployment Informer通过函数InformerFor注册进入Factory中，注册时实现了ListWatcher方法和Listener。而Controller.run实现了事件的回调处理。
  • 下图标识生成三个Informer
  • 下图ctx.InformerFactory.Apps().V1().Deployments()点进去，发现三个Informer生成的思路，并注册进了internalinterfaces.SharedINformerFactory中。
  • 三个Informer进一步通过Informer()注册进入Factory,并生成相关的EventHandler及ListWacher 
// NewDeploymentController creates a new DeploymentController.
func NewDeploymentController(dInformer appsinformers.DeploymentInformer, rsInformer appsinformers.ReplicaSetInformer, podInformer coreinformers.PodInformer, client clientset.Interface) (*DeploymentController, error) {
	eventBroadcaster := record.NewBroadcaster()
	eventBroadcaster.StartLogging(klog.Infof)
	eventBroadcaster.StartRecordingToSink(&v1core.EventSinkImpl{Interface: client.CoreV1().Events("")})

	if client != nil && client.CoreV1().RESTClient().GetRateLimiter() != nil {
		if err := metrics.RegisterMetricAndTrackRateLimiterUsage("deployment_controller", client.CoreV1().RESTClient().GetRateLimiter()); err != nil {
			return nil, err
		}
	}
	dc := &DeploymentController{
		client:        client,
		eventRecorder: eventBroadcaster.NewRecorder(scheme.Scheme, v1.EventSource{Component: "deployment-controller"}),
		queue:         workqueue.NewNamedRateLimitingQueue(workqueue.DefaultControllerRateLimiter(), "deployment"),
	}
	dc.rsControl = controller.RealRSControl{
		KubeClient: client,
		Recorder:   dc.eventRecorder,
	}

	dInformer.Informer().AddEventHandler(cache.ResourceEventHandlerFuncs{
		AddFunc:    dc.addDeployment,
		UpdateFunc: dc.updateDeployment,
		// This will enter the sync loop and no-op, because the deployment has been deleted from the store.
		DeleteFunc: dc.deleteDeployment,
	})
	rsInformer.Informer().AddEventHandler(cache.ResourceEventHandlerFuncs{
		AddFunc:    dc.addReplicaSet,
		UpdateFunc: dc.updateReplicaSet,
		DeleteFunc: dc.deleteReplicaSet,
	})
	podInformer.Informer().AddEventHandler(cache.ResourceEventHandlerFuncs{
		DeleteFunc: dc.deletePod,
	})

	dc.syncHandler = dc.syncDeployment
	dc.enqueueDeployment = dc.enqueue

	dc.dLister = dInformer.Lister()
	dc.rsLister = rsInformer.Lister()
	dc.podLister = podInformer.Lister()
	dc.dListerSynced = dInformer.Informer().HasSynced
	dc.rsListerSynced = rsInformer.Informer().HasSynced
	dc.podListerSynced = podInformer.Informer().HasSynced
	return dc, nil
  • 启动Informer和监听资源的事件,ListWacher
//SharedInformerFactory是一个informer工厂的接口定义。
// SharedInformerFactory a small interface to allow for adding an informer without an import cycle
type SharedInformerFactory interface {
    Start(stopCh <-chan struct{})
    InformerFor(obj runtime.Object, newFunc NewInformerFunc) cache.SharedIndexInformer
}

// Start initializes all requested informers.
func (f *sharedInformerFactory) Start(stopCh <-chan struct{}) {
    f.lock.Lock()
    defer f.lock.Unlock()

    for informerType, informer := range f.informers {
        if !f.startedInformers[informerType] {
            go informer.Run(stopCh)
            f.startedInformers[informerType] = true
        }
    }
}

2.6 InformerFactory.Start的调用者controllerContext

  • InformerFactory实际上是SharedInformerFactory，具体的实现逻辑在client-go中的informer的实现机制。
controllerContext.InformerFactory.Start(controllerContext.Stop)
close(controllerContext.InformersStarted)

2.7 总体架构图一览

  • 管理控制中心机制源码调用关系 
上图参考https://www.jianshu.com/p/ac9179007fe2，感谢原作者，本文主要融入新理解，全新解读。
  • Informer与controller关系一览

3 再总结

  • Kube-controller-manager对应的cmd部分的调用流程如下：Main-->NewControllerManagerCommand--> Run(c.Complete(), wait.NeverStop)-->StartControllers-->initFn(ctx)-->startDeploymentController/startStatefulSetController-->InformerFactory.Start->sts.NewStatefulSetController.Run/dc.NewDeploymentController.Run-->pkg/controller。
  • 其中CreateControllerContext函数用来创建各类型controller所需要使用的context，NewControllerInitializers初始化了各种类型的controller，其中就包括DeploymentController和StatefulSetController等。

4 最后

  • kube-controller-manager管理控制中心是和InformerFactory.Start遥相呼应的。
  • StartControllers完成了所有Controller的启动及回调处理逻辑，及Controller对应Informer的注册及ListWatcher函数。
  • Informer主要就是监听APIServer资源变化及放进队列中，Informer启动是通过controllerContext.InformerFactory.Start(controllerContext.Stop)完成的，但是首先通过InformerFor完成注册

专注于大数据及容器云核心技术解密，可提供全栈的大数据+云原生平台咨询方案，请持续关注本套博客。如有任何学术交流，可随时联系。更多内容请关注《数据云技术社区》公众号。

avatar
文章
阅读
粉丝