如何用Kubernetes优雅地关闭Golang pods

408 阅读3分钟

在这个例子中,我们将演示如何优雅地关闭一个 Golang 应用程序。这个例子现在已经被很多人熟知。然而,不为人知的是,这个解决方案在Kubernetes环境中是否像在本地环境中一样有效。这完全取决于Kubernetes pod/部署配置中的terminationGracePeriodSeconds 选项。简而言之,确保上下文超时值小于terminationGracePeriodSeconds 选项的值。默认情况下,如果你不配置它,它被设置为30 秒,所以你的上下文超时值应该是n < 30

我们的应用程序允许长时间运行的请求/连接在40秒内完成。然后我们将terminationGracePeriodSeconds 设置为60。这将防止我们的应用程序被Kubernetes过早地杀死。通常情况下,不配置这个值,将上下文超时设置为10秒这样的值也是可以的。你会看到下面的日志:

结构

├── Makefile
├── deploy
│   └── k8s
│       ├── deployment.yaml
│       └── service.yaml
├── docker
│   └── dev
│       └── Dockerfile
└── main.go

文件

Makefile

.PHONY: run
run:
	go run -race main.go

.PHONY: docker-push
docker-push:
	docker build -t you/api:latest -f ./docker/dev/Dockerfile .
	docker push you/api:latest
	docker rmi you/api:latest
	docker system prune --volumes --force

.PHONY: k8s-deploy
k8s-deploy:
	kubectl apply -f deploy/k8s/deployment.yaml
	kubectl apply -f deploy/k8s/service.yaml

Docker文件

FROM golang:1.15-alpine3.12 as build

WORKDIR /source
COPY . .

RUN CGO_ENABLED=0 go build -ldflags "-s -w" -o bin/main main.go

FROM alpine:3.12 as run

COPY --from=build /source/bin/main /main

ENTRYPOINT ["./main"]

main.go

package main

import (
	"context"
	"log"
	"net/http"
	"os"
	"os/signal"
	"syscall"
	"time"
)

func main() {
	// Create HTTP router.
	rtr := http.NewServeMux()
	rtr.HandleFunc("/", home)

	// Create HTTP sever.
	srv := &http.Server{
		Addr:    ":8080",
		Handler: rtr,
	}

	// Start HTTP server.
	go func() {
		if err := srv.ListenAndServe(); err != nil {
			log.Println("http server: listen and serve:", err)
		}
	}()
	log.Println("app: started")

	// Listen on application shutdown signals.
	listener := make(chan os.Signal, 1)
	signal.Notify(listener, os.Interrupt, syscall.SIGTERM)
	log.Println("app: received a shutdown signal:", <-listener)

	// Allow live connections a set period of time to complete.
	ctx, cancel := context.WithTimeout(context.Background(), time.Second*40)
	defer cancel()

	// Shutdown HTTP server.
	if err := srv.Shutdown(ctx); err != nil && err != http.ErrServerClosed {
		log.Println("app: dirty shutdown:", err)
		return
	}

	log.Println("app: clean shutdown")
}

// A dummy endpoint that simulates a long running request/connection/process.
func home(w http.ResponseWriter, _ *http.Request) {
	log.Println("sleeping!")
	time.Sleep(time.Second * 50)
	log.Println("woke up!")

	_, _ = w.Write([]byte("welcome home"))
}

service.yaml

apiVersion: v1
kind: Service

metadata:
  name: api-service

spec:
  type: NodePort
  selector:
    app: api
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080

部署.yaml

apiVersion: apps/v1
kind: Deployment

metadata:
  name: api-deployment
  labels:
    app: api

spec:
  replicas: 1
  selector:
    matchLabels:
      app: api
  template:
    metadata:
      labels:
        app: api
    spec:
      containers:
        - name: golang
          image: you/api:latest
          ports:
            - containerPort: 8080
      terminationGracePeriodSeconds: 60

测试

当没有实时连接/请求或有但提前完成时,重新启动部署后的pod日志将如下所示。应用程序已被很好地关闭,所有请求都已被送达:

$ kubectl logs -f pod/api-deployment-66cb684477-7tjrf
2021/02/14 20:33:59 app: started
2021/02/14 20:34:17 sleeping!
2021/02/14 20:34:34 app: received a shutdown signal: terminated
2021/02/14 20:34:34 http server: listen and serve: http: Server closed
2021/02/14 20:35:07 woke up!
2021/02/14 20:35:08 app: clean shutdown

当有一个长期运行的连接/请求时,重新启动部署后,pod日志将看起来像下面这样。应用程序已被中断,长期运行的请求没有被提供:

$ kubectl logs -f pod/api-deployment-6f6d994c98-vv8jt
2021/02/14 20:26:13 app: started
2021/02/14 20:28:01 sleeping!
2021/02/14 20:28:10 app: received a shutdown signal: terminated
2021/02/14 20:28:10 http server: listen and serve: http: Server closed
2021/02/14 20:28:50 app: dirty shutdown: context deadline exceeded

当有一个长期运行的连接/请求,并且terminationGracePeriodSeconds 值小于40秒,这是上下文超时,重新启动部署后,pod日志将看起来像下面这样。这个pod已经被杀死了,所以甚至没有 "关闭 "日志!

$ kubectl logs -f pod/api-deployment-b8d5dc94c-wqb6s
2021/02/14 20:14:55 app: started
2021/02/14 20:16:26 sleeping!
2021/02/14 20:16:32 app: received a shutdown signal: terminated
2021/02/14 20:16:32 http server: listen and serve: http: Server closed