kube-apiserver源码剖析与开发(六):创建一个 Pod 在 kukbe-apiserver 经历了什么

531 阅读8分钟

在前面的文章里我们说过,kube-apiserver 由 aggregatorServer、kubeAPIServer、apiExtensionsServer 三个 server 组成,他们各司其职,当自己处理不了请求时,就向下传递(委派)请求,处理的顺序如下:

pCZEG4A.png

虽然说 kube-apiserver 由三个 server 组成,但是只有 aggregatorServer 监听了端口,是请求的入口。底层用的是 golang 的 net/http 实现的,是一个标准的 tcp server

net/http/server.go

func (srv *Server) Serve(l net.Listener) error {
    ...
	for {
		rw, err := l.Accept()
		if err != nil {
			select {
			case <-srv.getDoneChan():
				return ErrServerClosed
			default:
			}
			if ne, ok := err.(net.Error); ok && ne.Temporary() {
				if tempDelay == 0 {
					tempDelay = 5 * time.Millisecond
				} else {
					tempDelay *= 2
				}
				if max := 1 * time.Second; tempDelay > max {
					tempDelay = max
				}
				srv.logf("http: Accept error: %v; retrying in %v", err, tempDelay)
				time.Sleep(tempDelay)
				continue
			}
			return err
		}
		connCtx := ctx
		if cc := srv.ConnContext; cc != nil {
			connCtx = cc(connCtx, rw)
			if connCtx == nil {
				panic("ConnContext returned nil")
			}
		}
		tempDelay = 0
		c := srv.newConn(rw)
		c.setState(c.rwc, StateNew, runHooks) // before Serve can return
		go c.serve(connCtx)
	}
}

主线程中 accept 请求,如果有请求过来另起一个协程进行处理,该协程会执行下面代码

net/http/server.go

func (sh serverHandler) ServeHTTP(rw ResponseWriter, req *Request) {
	handler := sh.srv.Handler
    
    ...
    
	// APIServerHandler
	// vendor/k8s.io/apiserver/pkg/server/handler.go
	handler.ServeHTTP(rw, req)
}

而这里的 handler 就是前面创建的 aggregatorServer 中 genericServer 的 Handler,它在如下代码中创建

// vendor/k8s.io/apiserver/pkg/server/handler.go

func NewAPIServerHandler(name string, s runtime.NegotiatedSerializer, handlerChainBuilder HandlerChainBuilderFn, notFoundHandler http.Handler) *APIServerHandler {
	nonGoRestfulMux := mux.NewPathRecorderMux(name)
	if notFoundHandler != nil {
		nonGoRestfulMux.NotFoundHandler(notFoundHandler)
	}

	gorestfulContainer := restful.NewContainer()
	gorestfulContainer.ServeMux = http.NewServeMux()
	gorestfulContainer.Router(restful.CurlyRouter{}) // e.g. for proxy/{kind}/{name}/{*}
	gorestfulContainer.RecoverHandler(func(panicReason interface{}, httpWriter http.ResponseWriter) {
		logStackOnRecover(s, panicReason, httpWriter)
	})
	gorestfulContainer.ServiceErrorHandler(func(serviceErr restful.ServiceError, request *restful.Request, response *restful.Response) {
		serviceErrorHandler(s, serviceErr, request, response)
	})
// vendor/k8s.io/apiserver/pkg/server/handler.go
// 在后面controller中往 gorestfulContainer 加ws
	director := director{
		name:               name,
		goRestfulContainer: gorestfulContainer,
		nonGoRestfulMux:    nonGoRestfulMux,
	}

	return &APIServerHandler{
		FullHandlerChain:   handlerChainBuilder(director),
		GoRestfulContainer: gorestfulContainer,
		NonGoRestfulMux:    nonGoRestfulMux,
		Director:           director,
	}
}

所以请求实际上是被 APIServerHandler 的 ServeHTTP 方法处理,如下:

// vendor/k8s.io/apiserver/pkg/server/handler.go

func (a *APIServerHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	a.FullHandlerChain.ServeHTTP(w, r)
}

请求就来到了 FullHandlerChain.ServeHTT 方法中。我们在前面的文章中说过,所有请求都会经过一条链,这条路链会在下面的函数中创建:

// vendor/k8s.io/apiserver/pkg/server/config.go

func DefaultBuildHandlerChain(apiHandler http.Handler, c *Config) http.Handler {
	handler := filterlatency.TrackCompleted(apiHandler)
	handler = genericapifilters.WithAuthorization(handler, c.Authorization.Authorizer, c.Serializer)
	handler = filterlatency.TrackStarted(handler, "authorization")

	if c.FlowControl != nil {
		workEstimatorCfg := flowcontrolrequest.DefaultWorkEstimatorConfig()
		requestWorkEstimator := flowcontrolrequest.NewWorkEstimator(
			c.StorageObjectCountTracker.Get, c.FlowControl.GetInterestedWatchCount, workEstimatorCfg)
		handler = filterlatency.TrackCompleted(handler)
		handler = genericfilters.WithPriorityAndFairness(handler, c.LongRunningFunc, c.FlowControl, requestWorkEstimator)
		handler = filterlatency.TrackStarted(handler, "priorityandfairness")
	} else {
		handler = genericfilters.WithMaxInFlightLimit(handler, c.MaxRequestsInFlight, c.MaxMutatingRequestsInFlight, c.LongRunningFunc)
	}

	handler = filterlatency.TrackCompleted(handler)
	handler = genericapifilters.WithImpersonation(handler, c.Authorization.Authorizer, c.Serializer)
	handler = filterlatency.TrackStarted(handler, "impersonation")

	handler = filterlatency.TrackCompleted(handler)
	handler = genericapifilters.WithAudit(handler, c.AuditBackend, c.AuditPolicyRuleEvaluator, c.LongRunningFunc)
	handler = filterlatency.TrackStarted(handler, "audit")

	failedHandler := genericapifilters.Unauthorized(c.Serializer)
	failedHandler = genericapifilters.WithFailedAuthenticationAudit(failedHandler, c.AuditBackend, c.AuditPolicyRuleEvaluator)

	failedHandler = filterlatency.TrackCompleted(failedHandler)
	handler = filterlatency.TrackCompleted(handler)
	handler = genericapifilters.WithAuthentication(handler, c.Authentication.Authenticator, failedHandler, c.Authentication.APIAudiences)
	handler = filterlatency.TrackStarted(handler, "authentication")

	handler = genericfilters.WithCORS(handler, c.CorsAllowedOriginList, nil, nil, nil, "true")

	// WithTimeoutForNonLongRunningRequests will call the rest of the request handling in a go-routine with the
	// context with deadline. The go-routine can keep running, while the timeout logic will return a timeout to the client.
	handler = genericfilters.WithTimeoutForNonLongRunningRequests(handler, c.LongRunningFunc)

	handler = genericapifilters.WithRequestDeadline(handler, c.AuditBackend, c.AuditPolicyRuleEvaluator,
		c.LongRunningFunc, c.Serializer, c.RequestTimeout)
	handler = genericfilters.WithWaitGroup(handler, c.LongRunningFunc, c.HandlerChainWaitGroup)
	if c.SecureServing != nil && !c.SecureServing.DisableHTTP2 && c.GoawayChance > 0 {
		handler = genericfilters.WithProbabilisticGoaway(handler, c.GoawayChance)
	}
	handler = genericapifilters.WithAuditAnnotations(handler, c.AuditBackend, c.AuditPolicyRuleEvaluator)
	handler = genericapifilters.WithWarningRecorder(handler)
	handler = genericapifilters.WithCacheControl(handler)
	handler = genericfilters.WithHSTS(handler, c.HSTSDirectives)
	if c.ShutdownSendRetryAfter {
		handler = genericfilters.WithRetryAfter(handler, c.lifecycleSignals.NotAcceptingNewRequest.Signaled())
	}
	handler = genericfilters.WithHTTPLogging(handler)
	if utilfeature.DefaultFeatureGate.Enabled(genericfeatures.APIServerTracing) {
		handler = genericapifilters.WithTracing(handler, c.TracerProvider)
	}
	handler = genericapifilters.WithLatencyTrackers(handler)
	handler = genericapifilters.WithRequestInfo(handler, c.RequestInfoResolver)
	handler = genericapifilters.WithRequestReceivedTimestamp(handler)
	handler = genericapifilters.WithMuxAndDiscoveryComplete(handler, c.lifecycleSignals.MuxAndDiscoveryComplete.Signaled())
	handler = genericfilters.WithPanicRecovery(handler, c.RequestInfoResolver)
	handler = genericapifilters.WithAuditID(handler)
	return handler
}

这个函数我们在前面的文章说过,它将所有需要执行的操作串联在一起,函数最后的的 handler 最先执行,开头的 handler 最后执行。根据上面代码,我们能看到创建一个 Pod 在这条链上经历了如下关键步骤:

  • Authentication:认证
  • Audit:审计
  • FlowControl:限速检测
  • Authorization:鉴权
  • apiHandler:如果上述步骤都顺利执行没有错误,则在链的末端开始执行请求的处理步骤

而 apiHandler 被传入的是下面的对象(director):

// vendor/k8s.io/apiserver/pkg/server/handler.go

director := director{
	name:               name,
	goRestfulContainer: gorestfulContainer,
	nonGoRestfulMux:    nonGoRestfulMux,
}

return &APIServerHandler{
	FullHandlerChain:   handlerChainBuilder(director),
	GoRestfulContainer: gorestfulContainer,
	NonGoRestfulMux:    nonGoRestfulMux,
	Director:           director,
}

所以接下来就是执行 director 的 ServeHTTP 方法:


func (d director) ServeHTTP(w http.ResponseWriter, req *http.Request) {
	path := req.URL.Path

	// check to see if our webservices want to claim this path
	for _, ws := range d.goRestfulContainer.RegisteredWebServices() {
		switch {
		case ws.RootPath() == "/apis":
			// if we are exactly /apis or /apis/, then we need special handling in loop.
			// normally these are passed to the nonGoRestfulMux, but if discovery is enabled, it will go directly.
			// We can't rely on a prefix match since /apis matches everything (see the big comment on Director above)
			if path == "/apis" || path == "/apis/" {
				klog.V(5).Infof("%v: %v %q satisfied by gorestful with webservice %v", d.name, req.Method, path, ws.RootPath())
				// don't use servemux here because gorestful servemuxes get messed up when removing webservices
				// TODO fix gorestful, remove TPRs, or stop using gorestful

				// func (s *GenericAPIServer) InstallAPIGroups(apiGroupInfos ...*APIGroupInfo) error
				d.goRestfulContainer.Dispatch(w, req)
				return
			}

		case strings.HasPrefix(path, ws.RootPath()):
			// ensure an exact match or a path boundary match
			if len(path) == len(ws.RootPath()) || path[len(ws.RootPath())] == '/' {
				klog.V(5).Infof("%v: %v %q satisfied by gorestful with webservice %v", d.name, req.Method, path, ws.RootPath())
				// don't use servemux here because gorestful servemuxes get messed up when removing webservices
				// TODO fix gorestful, remove TPRs, or stop using gorestful
				d.goRestfulContainer.Dispatch(w, req)
				return
			}
		}
	}
	// if we didn't find a match, then we just skip gorestful altogether
	klog.V(5).Infof("%v: %v %q satisfied by nonGoRestful", d.name, req.Method, path)
	d.nonGoRestfulMux.ServeHTTP(w, req)
}

这个方法内,首先会遍历注册在 aggregatorServer 内的路由,也就是 goRestFul 类型的路由,如果有匹配到 url,那么就在这里处理请求。我们本文讨论的是创建 Pod 的请求,它属于 legacyAPI(/api 为前缀,group 为空),路由是在 kubeAPIServer 中注册的,所以不会在 aggregatorServer 中处理,所以请求会走到最后的 nonGoRestfulMux.ServeHTTP 方法,nonGoRestfulMux 也是在 NewAPIServerHandler 中创建的

// vendor/k8s.io/apiserver/pkg/server/handler.go

func NewAPIServerHandler(name string, s runtime.NegotiatedSerializer, handlerChainBuilder HandlerChainBuilderFn, notFoundHandler http.Handler) *APIServerHandler {
	nonGoRestfulMux := mux.NewPathRecorderMux(name)
	if notFoundHandler != nil {
		nonGoRestfulMux.NotFoundHandler(notFoundHandler)
	}
	...
}
// vendor/k8s.io/apiserver/pkg/server/mux/pathrecorder.go

func NewPathRecorderMux(name string) *PathRecorderMux {
	ret := &PathRecorderMux{
		name:            name,
		pathToHandler:   map[string]http.Handler{},
		prefixToHandler: map[string]http.Handler{},
		mux:             atomic.Value{},
		exposedPaths:    []string{},
		pathStacks:      map[string]string{},
	}

	ret.mux.Store(&pathHandler{notFoundHandler: http.NotFoundHandler()})
	return ret
}

nonGoRestfulMux 的类型是 PathRecorderMux,它实现了 ServeHTTP 方法,所以接下来就是执行它的 ServeHTTP 方法。nonGoRestfulMux 是处理没有注册在 goRestFul 类型路由中的请求

// jetbrains://goland/navigate/reference?project=kubernetes&path=vendor/k8s.io/apiserver/pkg/server/mux/pathrecorder.go

func (m *PathRecorderMux) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	m.mux.Load().(*pathHandler).ServeHTTP(w, r)
}

// ServeHTTP makes it an http.Handler
func (h *pathHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	if exactHandler, ok := h.pathToHandler[r.URL.Path]; ok {
		klog.V(5).Infof("%v: %q satisfied by exact match", h.muxName, r.URL.Path)
		exactHandler.ServeHTTP(w, r)
		return
	}

	for _, prefixHandler := range h.prefixHandlers {
		if strings.HasPrefix(r.URL.Path, prefixHandler.prefix) {
			klog.V(5).Infof("%v: %q satisfied by prefix %v", h.muxName, r.URL.Path, prefixHandler.prefix)
			prefixHandler.handler.ServeHTTP(w, r)
			return
		}
	}
	klog.V(5).Infof("%v: %q satisfied by NotFoundHandler", h.muxName, r.URL.Path)
	h.notFoundHandler.ServeHTTP(w, r)
}

这里我们要看下,nonGoRestfulMux 的路由是在哪里注册的,只有知道了里面注册了什么,才能知道请求在这里怎么处理

nonGoRestfulMux 是在创建 aggregatorServer 时创建的,并注册路由

// vendor/k8s.io/kube-aggregator/pkg/apiserver/apiserver.go

func (c completedConfig) NewWithDelegate(...) {
    ...
    
    apiserviceRegistrationController := NewAPIServiceRegistrationController(informerFactory.Apiregistration().V1().APIServices(), s)
    	s.GenericAPIServer.AddPostStartHookOrDie("apiservice-registration-controller", func(context genericapiserver.PostStartHookContext) error {
    	go apiserviceRegistrationController.Run(context.StopCh, apiServiceRegistrationControllerInitiated)
    	select {
    	case <-context.StopCh:
    	case <-apiServiceRegistrationControllerInitiated:
    	}

	return nil
    })
}
// vendor/k8s.io/kube-aggregator/pkg/apiserver/apiservice_controller.go

func (c *APIServiceRegistrationController) Run(stopCh <-chan struct{}, handlerSyncedCh chan<- struct{}) {
	defer utilruntime.HandleCrash()
	defer c.queue.ShutDown()

	klog.Info("Starting APIServiceRegistrationController")
	defer klog.Info("Shutting down APIServiceRegistrationController")

	if !controllers.WaitForCacheSync("APIServiceRegistrationController", stopCh, c.apiServiceSynced) {
		return
	}

	/// initially sync all APIServices to make sure the proxy handler is complete
	if err := wait.PollImmediateUntil(time.Second, func() (bool, error) {
		services, err := c.apiServiceLister.List(labels.Everything())
		if err != nil {
			utilruntime.HandleError(fmt.Errorf("failed to initially list APIServices: %v", err))
			return false, nil
		}
		for _, s := range services {
			if err := c.apiHandlerManager.AddAPIService(s); err != nil {
				utilruntime.HandleError(fmt.Errorf("failed to initially sync APIService %s: %v", s.Name, err))
				return false, nil
			}
		}
		return true, nil
	}, stopCh); err == wait.ErrWaitTimeout {
		utilruntime.HandleError(fmt.Errorf("timed out waiting for proxy handler to initialize"))
		return
	} else if err != nil {
		panic(fmt.Errorf("unexpected error: %v", err))
	}
	close(handlerSyncedCh)

	// only start one worker thread since its a slow moving API and the aggregation server adding bits
	// aren't threadsafe
	go wait.Until(c.runWorker, time.Second, stopCh)

	<-stopCh
}

Run 方法里面也是一个标准的 list/watch 模式,首先等待同步所有 apiservice(见下图,k8s 内的一种 api 资源,类比 pod)到本地缓存,

pC2T45F.png转存失败,建议直接上传图片文件

然后调用 list 方法,从本地缓存获取所有 apiservice,然后遍历所有 apiservice ,逐个注册路由,注册过程如下

// vendor/k8s.io/kube-aggregator/pkg/apiserver/apiserver.go

func (s *APIAggregator) AddAPIService(apiService *v1.APIService) error {
	if proxyHandler, exists := s.proxyHandlers[apiService.Name]; exists {
		proxyHandler.updateAPIService(apiService)
		if s.openAPIAggregationController != nil {
			s.openAPIAggregationController.UpdateAPIService(proxyHandler, apiService)
		}
		if s.openAPIV3AggregationController != nil {
			s.openAPIV3AggregationController.UpdateAPIService(proxyHandler, apiService)
		}
		return nil
	}

	proxyPath := "/apis/" + apiService.Spec.Group + "/" + apiService.Spec.Version
	if apiService.Name == legacyAPIServiceName {
		proxyPath = "/api"
	}

	proxyHandler := &proxyHandler{
		localDelegate:              s.delegateHandler,
		proxyCurrentCertKeyContent: s.proxyCurrentCertKeyContent,
		proxyTransport:             s.proxyTransport,
		serviceResolver:            s.serviceResolver,
		egressSelector:             s.egressSelector,
	}
	proxyHandler.updateAPIService(apiService)
	if s.openAPIAggregationController != nil {
		s.openAPIAggregationController.AddAPIService(proxyHandler, apiService)
	}
	if s.openAPIV3AggregationController != nil {
		s.openAPIV3AggregationController.AddAPIService(proxyHandler, apiService)
	}
	s.proxyHandlers[apiService.Name] = proxyHandler
	s.GenericAPIServer.Handler.NonGoRestfulMux.Handle(proxyPath, proxyHandler)

	s.GenericAPIServer.Handler.NonGoRestfulMux.UnlistedHandlePrefix(proxyPath+"/", proxyHandler)

	if apiService.Name == legacyAPIServiceName {
		return nil
	}

	if s.handledGroups.Has(apiService.Spec.Group) {
		return nil
	}

	groupPath := "/apis/" + apiService.Spec.Group
	groupDiscoveryHandler := &apiGroupHandler{
		codecs:    aggregatorscheme.Codecs,
		groupName: apiService.Spec.Group,
		lister:    s.lister,
		delegate:  s.delegateHandler,
	}
	s.GenericAPIServer.Handler.NonGoRestfulMux.Handle(groupPath, groupDiscoveryHandler)
	s.GenericAPIServer.Handler.NonGoRestfulMux.UnlistedHandle(groupPath+"/", groupDiscoveryHandler)
	s.handledGroups.Insert(apiService.Spec.Group)
	return nil
}

首先判断该 apiservice 是否已经注册过?为什么需要做这个判断呢?因为这个方法不仅仅是在 aggregatorServer 启动的时候会调用,启动后 aggregatorServer 还会 watch apiservice 变化,如果有变化,需要更新路由。apiservice 为什么会变化呢?因为 apiservice 分两种,一种是 k8s 内置的,它不会发生变化,而由用户创建的 apiservice 可以被用户修改(如 apiservice 对应的后端 service 发生变化,就需要改变转发地址)。

然后判断 apiService 是否属于 legacyAPI(通过 legacyAPIServiceName = "v1." 判断),拼接这个 apiservice 转发的路径。在转发请求的时候就是根据这个路径来匹配的。

接着创建 proxyHandler, 用来处理请求,它的定义如下

// vendor/k8s.io/kube-aggregator/pkg/apiserver/handler_proxy.go

type proxyHandler struct {
	localDelegate http.Handler
	proxyCurrentCertKeyContent certKeyFunc
	proxyTransport             *http.Transport
	serviceResolver ServiceResolver
	handlingInfo atomic.Value
	egressSelector *egressselector.EgressSelector
}

type proxyHandlingInfo struct {
	local bool
	name string
	restConfig *restclient.Config
	transportBuildingError error
	proxyRoundTripper http.RoundTripper
	serviceName string
	serviceNamespace string
	serviceAvailable bool
	servicePort int32
}

proxyHandler 这个对象中有两个重要成员 localDelegate 和 handlingInfo,如果是本地(通过 proxyHandlingInfo.local 判断)的 apiservice(k8s 内置的),那么请求会转发到 localDelegate;如果是用户创建的 apiservice,那么请求转发到 proxyHandlingInfo.serviceName 对应的 service。

创建完 proxyHandler 然后将 proxyHandler 注册到 NonGoRestfulMux中,后续处理请求的时候,就可以通过 path 取出 handler 进行处理。

如果是 legacyAPI 类型的 apiservice,那么注册路由到这里就结束了。否则,还会注册 path 为 "/apis/" + apiService.Spec.Group 的路由。

好了,到这里我们本篇要讨论的 Pod 创建在 aggregatorServer 路由就创建完了。Pod api资源属于 legacyAPI 类型,所以请求会被 aggregatorServer.GenericAPIServer.Handler.NonGoRestfulMux ˙中注册的路由处理。我们再回到 PathRecorderMux 的 ServeHTTP 方法

//  vendor/k8s.io/apiserver/pkg/server/mux/pathrecorder.go

func (m *PathRecorderMux) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	m.mux.Load().(*pathHandler).ServeHTTP(w, r)
}

// ServeHTTP makes it an http.Handler
func (h *pathHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	if exactHandler, ok := h.pathToHandler[r.URL.Path]; ok {
		klog.V(5).Infof("%v: %q satisfied by exact match", h.muxName, r.URL.Path)
		exactHandler.ServeHTTP(w, r)
		return
	}

	for _, prefixHandler := range h.prefixHandlers {
		if strings.HasPrefix(r.URL.Path, prefixHandler.prefix) {
			klog.V(5).Infof("%v: %q satisfied by prefix %v", h.muxName, r.URL.Path, prefixHandler.prefix)
			prefixHandler.handler.ServeHTTP(w, r)
			return
		}
	}
	klog.V(5).Infof("%v: %q satisfied by NotFoundHandler", h.muxName, r.URL.Path)
	h.notFoundHandler.ServeHTTP(w, r)
}

创建一个 Pod 的 url 格式:/api/v1/namespaces/{namespace}/pods,可以匹配到 for 循环中的前缀匹配,handler 就是前面创建的 proxyHandler,他的 serveHTTP 方法如下

// vendor/k8s.io/kube-aggregator/pkg/apiserver/handler_proxy.go

func (r *proxyHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) {
	value := r.handlingInfo.Load()
	if value == nil {
		r.localDelegate.ServeHTTP(w, req)
		return
	}
	handlingInfo := value.(proxyHandlingInfo)
	if handlingInfo.local {
		if r.localDelegate == nil {
			http.Error(w, "", http.StatusNotFound)
			return
		}
		r.localDelegate.ServeHTTP(w, req)
		return
	}

	if !handlingInfo.serviceAvailable {
		proxyError(w, req, "service unavailable", http.StatusServiceUnavailable)
		return
	}

	if handlingInfo.transportBuildingError != nil {
		proxyError(w, req, handlingInfo.transportBuildingError.Error(), http.StatusInternalServerError)
		return
	}

	user, ok := genericapirequest.UserFrom(req.Context())
	if !ok {
		proxyError(w, req, "missing user", http.StatusInternalServerError)
		return
	}

	location := &url.URL{}
	location.Scheme = "https"
	rloc, err := r.serviceResolver.ResolveEndpoint(handlingInfo.serviceNamespace, handlingInfo.serviceName, handlingInfo.servicePort)
	if err != nil {
		klog.Errorf("error resolving %s/%s: %v", handlingInfo.serviceNamespace, handlingInfo.serviceName, err)
		proxyError(w, req, "service unavailable", http.StatusServiceUnavailable)
		return
	}
	location.Host = rloc.Host
	location.Path = req.URL.Path
	location.RawQuery = req.URL.Query().Encode()

	newReq, cancelFn := newRequestForProxy(location, req)
	defer cancelFn()

	if handlingInfo.proxyRoundTripper == nil {
		proxyError(w, req, "", http.StatusNotFound)
		return
	}

	proxyRoundTripper := handlingInfo.proxyRoundTripper
	upgrade := httpstream.IsUpgradeRequest(req)

	proxyRoundTripper = transport.NewAuthProxyRoundTripper(user.GetName(), user.GetGroups(), user.GetExtra(), proxyRoundTripper)

	if upgrade {
		transport.SetAuthProxyHeaders(newReq, user.GetName(), user.GetGroups(), user.GetExtra())
	}

	handler := proxy.NewUpgradeAwareHandler(location, proxyRoundTripper, true, upgrade, &responder{w: w})
	utilflowcontrol.RequestDelegated(req.Context())
	handler.ServeHTTP(w, newReq)
}

我们可以看到,创建一个 Pod 的请求会匹配到 handlingInfo.local 这个条件,所以请求就转到了下游 server,也就是 kubeAPIServer(准确来说是 kubeAPIServer 中的 genericServer,由 genericServer 中的 director 处理)。我们在上篇文章里面说了 kubeAPIServer 中路由注册的过程。

那么此时请求就来到了 kubeAPIServer 的 genericServer 中的 director.ServeHTTP 方法中,这里的处理流程就跟 aggregatorServer 的 genericServer 处理步骤一模一样了。

因为本篇文章讲的是创建 Pod 的请求,所以请求在 kubeAPIServer 的 goRestFul 类型路由中就被处理了。

创建 Pod 对应的 http 方法是 post,他的处理函数在 vendor/k8s.io/apiserver/pkg/endpoints/handlers/create.go 中,在上篇已经说明过,这里就不再附上代码,主要做了如下操作

  • 执行 mutatingAdmission 类型准入插件:该插件可以修改用户的请求体,比如插入 annotation 等
if mutatingAdmission, ok := admit.(admission.MutationInterface); ok && mutatingAdmission.Handles(admission.Create) {
	if err := mutatingAdmission.Admit(ctx, admissionAttributes, scope); err != nil {
		return nil, err
	}
}
  • 执行 validationAdmission 类型准入插件:该插件验证用户请求的合法性
  • 访问 etcd,写入数据

我们总结下创建一个 Pod 总体经历的事情:

  • Authentication:认证
  • Audit:审计
  • FlowControl:限速检测
  • Authorization:鉴权
  • 执行准入插件,包括 mutatingAdmission 和 validationAdmission 两种类型准入插件
  • 写 etcd

这里插一句,如果是获取一个 Pod 的话,和上面相比少的步骤就是准入插件的执行,不需要验证请求体的合法性和修改请求体。

以上就是创建一个 Pod 的经历了。