Koordinator-Scheduler 调度器源码解析

0 阅读8分钟

背景

本文将详细介绍 koordinator-Scheduler(后文简称,K-Scheduler) 是如何基于 K8s-Scheduling-framework 进行扩展,实现自己的增强逻辑。 我会以 K-Scheduler 的 RunPreFilterPlugins(对应K8s-Scheduling-framework的PreFilter扩展点)实现来展开,介绍其如何注册进原生framework,增强原生framework RunPreFilterPlugins功能。 最终理清 K-Scheduler 的完整调度流程。

建议:为了更好了解K-Scheduler的工作流程,阅读本文前可以先看我之前写的两篇内容 K8s-Scheduler调度器源码解析K8s Scheduling Framework 解析

Part I:K-Scheduler 核心数据结构

FrameWorkExtenderFactory

  • 角色:工厂 + 全局管理者

  • 位置framework_extender_factory.go:121

  • 主要职责

    • 存储全局共享资源(Koordinator clientset、informer factory 等)
    • 创建和管理 FrameworkExtender 实例(按 profile 缓存)
    • 拦截和代理 scheduler 的核心方法(SchedulePodNextPod
    • 管理全局监控和错误处理

FrameWorkExtender

// pkg/scheduler/frameworkext/interface.go
// 接口类型FrameworkExtender
type FrameworkExtender interface {
    framework.Framework // 嵌入原生 Framework 接口
    ExtendedHandle // 扩展的 Handle

    // Koord-scheduler 额外扩展的方法
    RunFindOneNodePlugin(ctx context.Context, cycleState *framework.CycleState, pod *corev1.Pod, result *framework.PreFilterResult) (string, *framework.Status)
    SetConfiguredPlugins(plugins *schedconfig.Plugins)

    RunReservationExtensionPreRestoreReservation(ctx context.Context, cycleState *framework.CycleState, pod *corev1.Pod) *framework.Status
    RunReservationExtensionRestoreReservation(ctx context.Context, cycleState *framework.CycleState, podToSchedule *corev1.Pod, matched []*ReservationInfo, unmatched []*ReservationInfo, nodeInfo *framework.NodeInfo) (PluginToReservationRestoreStates, *framework.Status)
    RunReservationExtensionFinalRestoreReservation(ctx context.Context, cycleState *framework.CycleState, pod *corev1.Pod, states PluginToNodeReservationRestoreStates) *framework.Status
    RunReservationExtensionPreRestoreReservationPreAllocation(ctx context.Context, cycleState *framework.CycleState, rInfo *ReservationInfo) *framework.Status
    RunReservationExtensionRestoreReservationPreAllocation(ctx context.Context, cycleState *framework.CycleState, rInfo *ReservationInfo, preAllocatable []*corev1.Pod, nodeInfo *framework.NodeInfo) (PluginToReservationRestoreStates, *framework.Status)
    RunReservationFilterPlugins(ctx context.Context, cycleState *framework.CycleState, pod *corev1.Pod, reservationInfo *ReservationInfo, nodeInfo *framework.NodeInfo) *framework.Status
    RunNominateReservationFilterPlugins(ctx context.Context, cycleState *framework.CycleState, pod *corev1.Pod, reservationInfo *ReservationInfo, nodeName string) *framework.Status
    RunReservationScorePlugins(ctx context.Context, cycleState *framework.CycleState, pod *corev1.Pod, reservationInfos []*ReservationInfo, nodeName string) (PluginToReservationScores, *framework.Status)
    RunReservationPreAllocationScorePlugins(ctx context.Context, cycleState *framework.CycleState, rInfo *ReservationInfo, pods []*corev1.Pod, nodeName string) (PluginToReservationScores, *framework.Status)
    RunNUMATopologyManagerAdmit(ctx context.Context, cycleState *framework.CycleState, pod *corev1.Pod, node *corev1.Node, numaNodes []int, policyType apiext.NUMATopologyPolicy, exclusivePolicy apiext.NumaTopologyExclusive, allNUMANodeStatus []apiext.NumaNodeStatus) *framework.Status
    RunResizePod(ctx context.Context, cycleState *framework.CycleState, pod *corev1.Pod, nodeName string) *framework.Status
}

// pkg/scheduler/frameworkext/framework_extender.go
// 实现类型frameworkExtenderImpl
// 实现了FrameWork Interface 所有的方法且内嵌 FrameWorkExtenderFactory 初始化的各类组件
// 例如:koordinatorClientSet,koordinatorSharedInformerFactory ...
type frameworkExtenderImpl struct {
    framework.Framework
    *errorHandlerDispatcher
    forgetPodHandlers []ForgetPodHandler

    schedulerFn       func() Scheduler
    configuredPlugins *schedconfig.Plugins
    monitor           *SchedulerMonitor

    koordinatorClientSet                koordinatorclientset.Interface
    koordinatorSharedInformerFactory    koordinatorinformers.SharedInformerFactory
    nodeResourceTopologyInformerFactory nrtinformers.SharedInformerFactory
    podLister                           listerscorev1.PodLister
    reservationLister                   listerschedulingv1alpha1.ReservationLister

    networkTopologyTreeManager networktopology.TreeManager

    // ***TransformersEnabled存放的都是重写原生调度器的插件集合,会在调度器创建前进行初始化
    // 这些Transformers就是Koord调度器插件扩展点的运行单元。
    preFilterTransformers         map[string]PreFilterTransformer
    filterTransformers            map[string]FilterTransformer
    scoreTransformers             map[string]ScoreTransformer
    postFilterTransformers        map[string]PostFilterTransformer
    preFilterTransformersEnabled  []PreFilterTransformer
    filterTransformersEnabled     []FilterTransformer
    scoreTransformersEnabled      []ScoreTransformer
    postFilterTransformersEnabled []PostFilterTransformer

    findOneNodePlugin FindOneNodePlugin
    preferNodesPlugin PreferNodesPlugin

    reservationCache                       ReservationCache
    reservationNominator                   ReservationNominator
    reservationFilterPlugins               []ReservationFilterPlugin
    reservationScorePlugins                []ReservationScorePlugin
    reservationPreBindPlugins              map[string]ReservationPreBindPlugin
    reservationRestorePlugins              []ReservationRestorePlugin
    reservationPreAllocationRestorePlugins []ReservationPreAllocationRestorePlugin

    resizePodPlugins         []ResizePodPlugin
    preBindExtensionsPlugins map[string]PreBindExtensions

    numaTopologyHintProviders []topologymanager.NUMATopologyHintProvider
    topologyManager           topologymanager.Interface

    metricsRecorder *metrics.MetricAsyncRecorder
}
  • 角色:单个 profile 的 framework 代理

  • 位置framework_extender.go (接口)

  • 实现frameworkExtenderImpl (结构体)

  • 主要职责

    • 嵌入原生 framework.Framework,继承所有原生功能

    • 重写所有 plugin 执行方法,插入 Transformer 钩子

    • 管理当前 profile 的 transformers 和自定义插件

    • 提供 K-Scheduler 特有的扩展能力

Run***Plugins:K8s Scheduling Framework 扩展点的运行承载方法的重写

type Framework interface {
    Handle
    PreEnqueuePlugins() []PreEnqueuePlugin
    EnqueueExtensions() []EnqueueExtensions
    QueueSortFunc() LessFunc
    RunPreFilterPlugins(ctx context.Context, state *CycleState, pod *v1.Pod) (*PreFilterResult, *Status)
    RunPostFilterPlugins(ctx context.Context, state *CycleState, pod *v1.Pod, filteredNodeStatusMap NodeToStatusMap) (*PostFilterResult, *Status)
    RunPreBindPlugins(ctx context.Context, state *CycleState, pod *v1.Pod, nodeName string) *Status
    RunPostBindPlugins(ctx context.Context, state *CycleState, pod *v1.Pod, nodeName string)
    RunReservePluginsReserve(ctx context.Context, state *CycleState, pod *v1.Pod, nodeName string) *Status
    RunReservePluginsUnreserve(ctx context.Context, state *CycleState, pod *v1.Pod, nodeName string)
    RunPermitPlugins(ctx context.Context, state *CycleState, pod *v1.Pod, nodeName string) *Status
    WaitOnPermit(ctx context.Context, pod *v1.Pod) *Status
    RunBindPlugins(ctx context.Context, state *CycleState, pod *v1.Pod, nodeName string) *Status
    HasFilterPlugins() bool
    HasPostFilterPlugins() bool
    // HasScorePlugins returns true if at least one Score plugin is defined.
    HasScorePlugins() bool
    // ListPlugins returns a map of extension point name to list of configured Plugins.
    ListPlugins() *config.Plugins
    // ProfileName returns the profile name associated to a profile.
    ProfileName() string
    // PercentageOfNodesToScore returns percentageOfNodesToScore associated to a profile.
    PercentageOfNodesToScore() *int32
    // SetPodNominator sets the PodNominator
    SetPodNominator(nominator PodNominator)
}

Koord-Scheduler在其pkg/scheduler/frameworkext/framework_extender.go文件中定义了这些Run***Plugins方法,例如下面我们按照RunPreFilterPlugins展开说明

func (ext *frameworkExtenderImpl) RunPreFilterPlugins(ctx context.Context, cycleState *framework.CycleState, pod *corev1.Pod) (*framework.PreFilterResult, *framework.Status) {
    trace := utiltrace.New("RunPreFilterPluginTransformers", utiltrace.Field{Key: "namespace", Value: pod.Namespace}, utiltrace.Field{Key: "name", Value: pod.Name})
    defer trace.LogIfLong(5 * time.Millisecond)
    // 前情提要:preFilterTransformersEnabled  []PreFilterTransformer
    // 每一个transformer是 PreFilterTransformer 类型
    // 包含两个方法:BeforePreFilter 和 AfterPreFilter
    
    // 1.执行 Koord PreFilterTransformer 的 BeforePreFilter 方法
    for _, transformer := range ext.preFilterTransformersEnabled {
       startTime := time.Now()
       trace.Step(fmt.Sprintf("BeforePrefilter %s begin", transformer.Name()))
       newPod, transformed, status := transformer.BeforePreFilter(ctx, cycleState, pod)
       trace.Step(fmt.Sprintf("BeforePrefilter %s done", transformer.Name()))
       ext.metricsRecorder.ObservePluginDurationAsync("BeforePreFilter", transformer.Name(), status.Code().String(), metrics.SinceInSeconds(startTime))
       if !status.IsSuccess() {
          status.SetFailedPlugin(transformer.Name())
          klog.ErrorS(status.AsError(), "Failed to run BeforePreFilter", "pod", klog.KObj(pod), "plugin", transformer.Name())
          return nil, status
       }
       if transformed {
          klog.V(5).InfoS("BeforePreFilter transformed", "transformer", transformer.Name(), "pod", klog.KObj(pod))
          pod = newPod
       }
    }

    // 2. 执行原生k8s-scheduling-framework的RunPreFilterPlugins接口定义的具体实现
    result, status := ext.Framework.RunPreFilterPlugins(ctx, cycleState, pod)
    if !status.IsSuccess() {
       return result, status
    }

    // 3. 执行 Koord PreFilterTransformer 的 AfterPreFilter 方法
    for _, transformer := range ext.preFilterTransformersEnabled {
       startTime := time.Now()
       trace.Step(fmt.Sprintf("AfterPrefilter %s begin", transformer.Name()))
       status = transformer.AfterPreFilter(ctx, cycleState, pod, result)
       trace.Step(fmt.Sprintf("AfterPrefilter %s done", transformer.Name()))
       ext.metricsRecorder.ObservePluginDurationAsync("AfterPreFilter", transformer.Name(), status.Code().String(), metrics.SinceInSeconds(startTime))
       if !status.IsSuccess() {
          status.SetFailedPlugin(transformer.Name())
          klog.ErrorS(status.AsError(), "Failed to run AfterPreFilter", "pod", klog.KObj(pod), "plugin", transformer.Name())
          return nil, status
       }
    }

    // ... something after

    return result, nil
}

在原生 Scheduler.ScheduleOne() 调用到 framework.RunPreFilterPlugins() 时,因为 Profile 已被替换成 frameworkExtenderImpl ,所以实际调用的是 Koordinator 重写的版本。 这里的 Profile 替换是在 Koord-Scheduler 启动前通过 SetUp方法 完成的,我们将在后面部分插件注册流程进行详细说明。

Part II:K-Scheduler 插件注册流程

复用官方kube-scheduler,然后以扩展的方式,注入 K-Scheduler 实现的高级调度功能

main

启动入口: 注册插件+构造调度器启动命令

var koordinatorPlugins = map[string]frameworkruntime.PluginFactory{
    loadaware.Name:               loadaware.New,
    nodenumaresource.Name:        nodenumaresource.New,
    reservation.Name:             reservation.New,
    coscheduling.Name:            coscheduling.New,
    deviceshare.Name:             deviceshare.New,
    elasticquota.Name:            elasticquota.New,
    defaultprebind.Name:          defaultprebind.New,
    noderesourcesfitplus.Name:    noderesourcesfitplus.New,
    scarceresourceavoidance.Name: scarceresourceavoidance.New,
    schedulinghint.Name:          schedulinghint.New,
}

func flatten(plugins map[string]frameworkruntime.PluginFactory) []app.Option {
    options := make([]app.Option, 0, len(plugins))
    for name, factoryFn := range plugins {
       options = append(options, app.WithPlugin(name, factoryFn))
    }
    return options
}

func main() {
    rand.Seed(time.Now().UnixNano())

    // Register custom plugins to the scheduler framework.
    // Later they can consist of scheduler profile(s) and hence
    // used by various kinds of workloads.
    command := app.NewSchedulerCommand(
       flatten(koordinatorPlugins)...,
    )

    logs.InitLogs()
    defer logs.FlushLogs()

    if err := command.Execute(); err != nil {
       os.Exit(1)
    }
}
  • koordinatorPlugins: 维护所有 koordinator-scheduler 插件Name插件构造函数 的映射;

  • flatten函数:展开 koordinatorPlugins 的每一个插件,对每一个插件调用 app.WithPlugin 函数来完成 out-of-tree plugins 的注册

app.NewSchedulerCommand

封装一个&cobra.Command,里面包含运行时依赖需要执行的命令和参数

// NewSchedulerCommand creates a *cobra.Command object with default parameters and registryOptions
func NewSchedulerCommand(registryOptions ...Option) *cobra.Command {
    opts := options.NewOptions()

    cmd := &cobra.Command{
       Use: "koord-scheduler",
       Long: `The Koordinator scheduler is a control plane process which assigns
Pods to Nodes. The scheduler implements based on kubernetes scheduling framework.
On the basis of compatibility with community scheduling capabilities, it provides 
richer advanced scheduling capabilities to address scheduling needs in co-located 
scenarios,ensuring the runtime quality of different workloads and users' demands 
for cost reduction and efficiency enhancement.
`,
       Run: func(cmd *cobra.Command, args []string) {
          if err := runCommand(cmd, opts, registryOptions...); err != nil {
             fmt.Fprintf(os.Stderr, "%v\n", err)
             os.Exit(1)
          }
       },
       Args: func(cmd *cobra.Command, args []string) error {
          for _, arg := range args {
             if len(arg) > 0 {
                return fmt.Errorf("%q does not take any arguments, got %q", cmd.CommandPath(), args)
             }
          }
          return nil
       },
    }

    nfs := opts.Flags
    verflag.AddFlags(nfs.FlagSet("global"))
    AddSyncBarrierFlags(nfs.FlagSet("global"))
    globalflag.AddGlobalFlags(nfs.FlagSet("global"), cmd.Name(), logs.SkipLoggingConfigurationFlags())
    frameworkext.AddFlags(nfs.FlagSet("extend"))
    fs := cmd.Flags()
    for _, f := range nfs.FlagSets {
       fs.AddFlagSet(f)
    }

    cols, _, _ := term.TerminalSize(cmd.OutOrStdout())
    cliflag.SetUsageAndHelpFunc(cmd, *nfs, cols)

    if err := cmd.MarkFlagFilename("config", "yaml", "yml", "json"); err != nil {
       klog.Background().Error(err, "Failed to mark flag filename")
    }

    return cmd
}

runCommand

构造&cobra.Command.Run 真正会执行的代码内容,关注其中的SetUp方法,这里是引入kube-scheduler然后将Koord-Scheduler基于K8s Frame Work增强的插件注册进去的地方。

// runCommand runs the scheduler.
func runCommand(cmd *cobra.Command, opts *options.Options, registryOptions ...Option) error {
    // ...

    cc, sched, extendedHandle, customWorkflow, err := Setup(ctx, opts, registryOptions...)
    if err != nil {
       return err
    }
    // add feature enablement metrics
    utilfeature.DefaultMutableFeatureGate.AddMetrics()
    return Run(ctx, cc, sched, extendedHandle, customWorkflow)
}

SetUp 初始化

func Setup(ctx context.Context, opts *options.Options, outOfTreeRegistryOptions ...Option) (*schedulerserverconfig.CompletedConfig, *scheduler.Scheduler, *frameworkext.FrameworkExtenderFactory, CustomWorkflow, error) {
    if cfg, err := latest.Default(); err != nil {
       return nil, nil, nil, nil, err
    } else {
       opts.ComponentConfig = cfg
    }

    if errs := opts.Validate(); len(errs) > 0 {
       return nil, nil, nil, nil, utilerrors.NewAggregate(errs)
    }

    c, err := opts.Config(ctx)
    if err != nil {
       return nil, nil, nil, nil, err
    }

    // Get the completed config
    // 1. 和kube-scheduler无异,都是构造一个私有不受外部改变的config
    cc := c.Complete()

    defaultprofile.AppendDefaultPlugins(cc.ComponentConfig.Profiles)

    informer.SetupCustomInformers(cc.InformerFactory)
    transformer.SetupTransformers(cc.InformerFactory, cc.KoordinatorSharedInformerFactory, cc.NodeResourceTopologyInformerFactory)

    metrics.Register()

    networkTopologyManager := networktopology.NewTreeManager(cc.KoordinatorSharedInformerFactory, cc.InformerFactory, cc.KoordinatorClient)

    // NOTE(joseph): K8s scheduling framework does not provide extension point for initialization.
    // Currently, only by copying the initialization code and implementing custom initialization.

    // ✅ 关键点 1
    // 2. frameworkExtenderFactory的初始化,构造koord plugins运行时依赖的组件。
    // koordinator这边复制原生kubernetes scheduler cmd包下的server.go和option.go 
    // 在自己的server.go option.go 里基于原生内容完成自己的扩展注册。
    frameworkExtenderFactory, err := frameworkext.NewFrameworkExtenderFactory(
       frameworkext.WithServicesEngine(cc.ServicesEngine),
       frameworkext.WithKoordinatorClientSet(cc.KoordinatorClient),
       frameworkext.WithKoordinatorSharedInformerFactory(cc.KoordinatorSharedInformerFactory),
       frameworkext.WithNodeResourceTopologySharedInformerFactory(cc.NodeResourceTopologyInformerFactory),
       frameworkext.WithNetworkTopologyManager(networkTopologyManager),
    )
    if err != nil {
       return nil, nil, nil, nil, err
    }

    // 3. 完成outOfTree注册
    // outOfTreeRegistryOptions的值来自于cmd/koord-scheduler/main.go flattern(KoordinatorPlugins)的结果
    outOfTreeRegistry := make(runtime.Registry)
    for _, option := range outOfTreeRegistryOptions {
       if err := option(frameworkExtenderFactory, outOfTreeRegistry); err != nil {
          return nil, nil, nil, nil, err
       }
    }

    recorderFactory := getRecorderFactory(&cc)
    completedProfiles := make([]kubeschedulerconfig.KubeSchedulerProfile, 0)
    
    // Create the scheduler.
    // 4. 创建原生 kube-scheduler 并传入koordinator自己定义的扩展插件 outOfTreeRegistry
    sched, err := scheduler.New(ctx,
       cc.Client,
       cc.InformerFactory,
       cc.DynInformerFactory,
       recorderFactory,
       scheduler.WithComponentConfigVersion(cc.ComponentConfig.TypeMeta.APIVersion),
       scheduler.WithKubeConfig(cc.KubeConfig),
       scheduler.WithProfiles(cc.ComponentConfig.Profiles...),
       scheduler.WithPercentageOfNodesToScore(cc.ComponentConfig.PercentageOfNodesToScore),
       // 传入koordiantor的扩展插件
       scheduler.WithFrameworkOutOfTreeRegistry(outOfTreeRegistry),
       scheduler.WithPodMaxBackoffSeconds(cc.ComponentConfig.PodMaxBackoffSeconds),
       scheduler.WithPodInitialBackoffSeconds(cc.ComponentConfig.PodInitialBackoffSeconds),
       scheduler.WithPodMaxInUnschedulablePodsDuration(cc.PodMaxInUnschedulablePodsDuration),
       scheduler.WithExtenders(cc.ComponentConfig.Extenders...),
       scheduler.WithParallelism(cc.ComponentConfig.Parallelism),
       scheduler.WithBuildFrameworkCapturer(func(profile kubeschedulerconfig.KubeSchedulerProfile) {
          // Profiles are processed during Framework instantiation to set default plugins and configurations. Capturing them for logging
          completedProfiles = append(completedProfiles, profile)
       }),
    )
    if err != nil {
       return nil, nil, nil, nil, err
    }
    if err := scheduleroptions.LogOrWriteConfig(klog.FromContext(ctx), opts.WriteConfigTo, &cc.ComponentConfig, completedProfiles); err != nil {
       return nil, nil, nil, nil, err
    }

    // extend framework to hook run plugin functions
    // ✅ 关键点 2
    // 将原生 Framework 替换为 Koordinator 的 FrameworkExtender
    // 所谓替换,就是遍历原生Profile,将自己重写过的plugin实现方法替换原生实现,从而完成扩展。
    // ***TransformersEnabled
    // 实现 Hook拦截 原生 Framework 逻辑,从而完成koord plugins 增强逻辑注入
    for k, fwk := range sched.Profiles {
       // 获取koord针对原生Profiles同名调度器扩展extender
       extender := frameworkExtenderFactory.GetExtender(k)
       if extender != nil {
          // 把原生定义插件填充给koord 的 extender(FrameworkExtender类型)
          extender.SetConfiguredPlugins(fwk.ListPlugins())
          // 每一个原生Profile其实是原生定义的Framework interface,
          // 这个Framework我们前面看到会在 koord 中的 FrameworkExtender 里作为一个内部字段出现
          // 这里,通过将Koord扩展过的FrameworkExtender覆盖原生的Profile(Framework)
          // 使得调度器执行SchedulerOne方法进行调度的时候,跑的PreFilter插槽点是Koord扩展过的RunPreFilters方法
          sched.Profiles[k] = extender
       }
    }

    frameworkExtenderFactory.InterceptSchedulerError(sched)
    frameworkExtenderFactory.InitScheduler(&frameworkext.SchedulerAdapter{Scheduler: sched})
    schedAdapter := frameworkExtenderFactory.Scheduler()

    eventhandlers.AddScheduleEventHandler(sched, schedAdapter, cc.InformerFactory, cc.KoordinatorSharedInformerFactory)
    reservationErrorHandler := eventhandlers.MakeReservationErrorHandler(
       sched,
       schedAdapter,
       frameworkExtenderFactory.KoordinatorClientSet(),
       frameworkExtenderFactory.KoordinatorSharedInformerFactory(),
    )
    frameworkExtenderFactory.RegisterErrorHandlerFilters(reservationErrorHandler, nil)

    for _, wf := range KnownWorkflowList {
       if wf.IsEnabled() {
          err = wf.Setup(ctx, &CustomWorkflowOptions{
             Sched:                      sched,
             SharedInformerFactory:      cc.InformerFactory,
             KubeClient:                 cc.Client,
             KoordSharedInformerFactory: cc.KoordinatorSharedInformerFactory,
             KoordClient:                cc.KoordinatorClient,
             RecorderFactory:            recorderFactory,
             KubeConfig:                 cc.KubeConfig,
          })
          if err != nil {
             return nil, nil, nil, nil, err
          }
          return &cc, sched, frameworkExtenderFactory, wf, nil
       }
    }

    return &cc, sched, frameworkExtenderFactory, nil, nil
}


// 我们来仔细看一下前面关键点2中如何实现Hook拦截(参考JAVA的代理类,frameworkExtenderImpl其实代理
// 了原生kube-scheduler的插件功能,并注入了自己的实现)
func (ext *frameworkExtenderImpl) SetConfiguredPlugins(plugins *schedconfig.Plugins) {
    // 用原生Profile里定义的Plguins来初始化configuredPlugins
    // configured预置的,正是koord进行扩展的base plugins,即原生Profile定义的Plugins
    ext.configuredPlugins = plugins
   
    // 对于PreFilter,Filter,Score,PostFilter这几个Plugin插槽
    // koord会用自己实现的同名transformer来进行替换
    for _, pl := range ext.configuredPlugins.PreFilter.Enabled {
       transformer := ext.preFilterTransformers[pl.Name]
       if transformer != nil {
          ext.preFilterTransformersEnabled = append(ext.preFilterTransformersEnabled, transformer)
       }
    }
    for _, pl := range ext.configuredPlugins.Filter.Enabled {
       transformer := ext.filterTransformers[pl.Name]
       if transformer != nil {
          ext.filterTransformersEnabled = append(ext.filterTransformersEnabled, transformer)
       }
    }
    for _, pl := range ext.configuredPlugins.Score.Enabled {
       transformer := ext.scoreTransformers[pl.Name]
       if transformer != nil {
          ext.scoreTransformersEnabled = append(ext.scoreTransformersEnabled, transformer)
       }
    }
    for _, pl := range ext.configuredPlugins.PostFilter.Enabled {
       transformer := ext.postFilterTransformers[pl.Name]
       if transformer != nil {
          ext.postFilterTransformersEnabled = append(ext.postFilterTransformersEnabled, transformer)
       }
    }
    klog.V(5).InfoS("Set configured transformer plugins",
       "PreFilterTransformer", len(ext.preFilterTransformersEnabled),
       "FilterTransformer", len(ext.filterTransformersEnabled),
       "ScoreTransformer", len(ext.scoreTransformersEnabled),
       "PostFilterTransformer", len(ext.postFilterTransformersEnabled))
}
  • 调用 Kubernetes 官方 scheduler.New() 创建基础调度器

  • 通过 scheduler.WithFrameworkOutOfTreeRegistry(outOfTreeRegistry) 传入 Koordinator 插件

  • 使用 PluginFactoryProxy 包装插件以支持扩展功能

Part III:K-Scheduler 插件运行流程

这里我们回顾Part I内 Run***Plugins 中 分析RunPreFilterPlugins这个扩展点的Case,来总结插件运行数据实体和运行时许流程,每个插件的其他扩展点运行也是类似的。

数据实体类图

image.png

插件运行时序图

image.png

K-Scheduler 调度流程图

Mermaid (4).jpg