k8s学习系列2pod的删除在删除pod的时候，通过--force选项可以提高删除pod的速度。如果不加的话，删除速度

pod的删除

kubectl delete pod podname --force

在删除pod的时候，通过--force选项可以提高删除pod的速度。如果不加的话，删除速度会慢，主要原因是kubernetes在删除pod的时候有个延期删除期（宽限期），默认是30s.可以通过参数terminationGracePeriodSeconds来指定。

pod hook

在整个pod的生命周期中，有两个hook是可以使用的

postStart 创建pod的时候，会随着pod里的主进程同时运行，没有先后顺序。

preStop 删除pod的时候，要先运行preStop里的程序，之后再关闭pod，这里需要注意的是，preStop必须要在pod宽限期内完成，如果没有完成的话，pod仍然是会被强制删除的。

静态pod

通常pod在master上统一管理，指定，分配。所谓静态pod，是指不是由master创建启动，在特定节点的kubelet上管理的，无法与Deployment或者DaemonSet进行关联。kubelete会为静态pod自动创建一个apiserver上的镜像pod，因此通过apiserver可以查询到，但是不能控制(删除等)

创建静态pod(在worker node上操作)

#1 查看kubelet运行的参数文件
[root@worker2 kubernetes]#  systemctl status kubelet
● kubelet.service - kubelet: The Kubernetes Node Agent
   Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
  Drop-In: /usr/lib/systemd/system/kubelet.service.d
           └─10-kubeadm.conf
   Active: active (running) since Sat 2023-07-22 03:20:24 EDT; 6 days ago
     Docs: https://kubernetes.io/docs/
kubelet的启动的参数文件是: /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf
#2编辑文件，添加Environment="KUBELET_STATIC_POD_PATH=--pod-manifest-path=/data/static_pod"
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS $KUBELET_STATIC_POD_PATH
/data/static_pod可以是任何指定的路径，必须保证是已经存在的路径。
$KUBELET_STATIC_POD_PATH一定要添加到ExecStart，否则是不生效的。
#3重启kubelet服务
[root@worker2 kubelet.service.d]# systemctl restart kubelet
Warning: The unit file, source configuration file or drop-ins of kubelet.service changed on disk. Run 'systemctl daemon-reload' to reload units.
[root@worker2 kubelet.service.d]# systemctl daemon-reload
[root@worker2 kubelet.service.d]# systemctl restart kubelet
[root@worker2 kubelet.service.d]# systemctl status kubelet
● kubelet.service - kubelet: The Kubernetes Node Agent
   Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
  Drop-In: /usr/lib/systemd/system/kubelet.service.d
           └─10-kubeadm.conf
   Active: active (running) since Sat 2023-07-29 00:26:19 EDT; 11s ago

通过systemctl cat kubelet查看通过systemd配置的与kubelet有关的所有配置文件路径与内容. /var/lib/kubelet/config.yaml也可以查看当前节点kubelet静态节点的路径为staticPodPath: /etc/kubernetes/manifests（默认路径）

[root@worker2 static_pod]# systemctl cat kubelet
# /usr/lib/systemd/system/kubelet.service
[Unit]
Description=kubelet: The Kubernetes Node Agent
Documentation=https://kubernetes.io/docs/
Wants=network-online.target
After=network-online.target

[Service]
ExecStart=/usr/bin/kubelet
Restart=always
StartLimitInterval=0
RestartSec=10

[Install]
WantedBy=multi-user.target

# /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf
# Note: This dropin only works with kubeadm and kubelet v1.11+
[Service]
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf"
Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
Environment="KUBELET_STATIC_POD_PATH=--pod-manifest-path=/data/static_pod"
# This is a file that "kubeadm init" and "kubeadm join" generates at runtime, populating the KUBELET_KUBEADM_ARGS variable dynamically
EnvironmentFile=-/var/lib/kubelet/kubeadm-flags.env
# This is a file that the user can use for overrides of the kubelet args as a last resort. Preferably, the user should use
# the .NodeRegistration.KubeletExtraArgs object in the configuration files instead. KUBELET_EXTRA_ARGS should be sourced from this file.
EnvironmentFile=-/etc/sysconfig/kubelet
ExecStart=
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS $KUBELET_STATIC_POD_PATH

删除静态pod

目前只能通过删除yaml文件的形式来删除静态pod

master node上静态pod的指定方式

在master node上通过systemctl status kubelet获取kubelet参数文件的位置

[root@master ~]# systemctl status kubelet
● kubelet.service - kubelet: The Kubernetes Node Agent
   Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
  Drop-In: /usr/lib/systemd/system/kubelet.service.d
           └─10-kubeadm.conf
   Active: active (running) since Sat 2023-07-22 03:03:31 EDT; 6 days ago

可以看到配置文件位置为/usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf

[root@master ~]# cat /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf
# Note: This dropin only works with kubeadm and kubelet v1.11+
[Service]
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf"
Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
# This is a file that "kubeadm init" and "kubeadm join" generates at runtime, populating the KUBELET_KUBEADM_ARGS variable dynamically
EnvironmentFile=-/var/lib/kubelet/kubeadm-flags.env
# This is a file that the user can use for overrides of the kubelet args as a last resort. Preferably, the user should use
# the .NodeRegistration.KubeletExtraArgs object in the configuration files instead. KUBELET_EXTRA_ARGS should be sourced from this file.
EnvironmentFile=-/etc/sysconfig/kubelet
ExecStart=
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS

在Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"中/var/lib/kubelet/config.yaml定义了静态pod所在路径。

[root@master ~]# grep static /var/lib/kubelet/config.yaml
staticPodPath: /etc/kubernetes/manifests

在master node上，apiserver, kube-proxy等静态配置文件都放在/etc/kubernetes/mainfests中

[root@wask8s1 ~]# ls -l /etc/kubernetes/manifests
total 16
-rw------- 1 root root 2372 Jul 22 02:54 etcd.yaml
-rw------- 1 root root 3343 Jul 22 02:55 kube-apiserver.yaml
-rw------- 1 root root 2740 Jul 22 02:55 kube-controller-manager.yaml
-rw------- 1 root root 1440 Jul 22 02:55 kube-scheduler.yaml

指定pod运行位置

给node添加标签

通过kubectl label node nodename key=value 给节点添加标签,多个标签可以用逗号隔开
kubectl label node --all kay=value，--all则表示给所有节点设置标签。
[root@master ~]# kubectl label node worker2 diskxx=ssdxx
node/worker2 labeled
[root@master ~]# kubectl get nodes worker2 --show-labels
NAME      STATUS   ROLES    AGE    VERSION   LABELS
worker2   Ready    <none>   239d   v1.26.4   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,diskxx=ssdxx,kubernetes.io/arch=amd64,kubernetes.io/hostname=worker2,kubernetes.io/os=linux
通过kubectl label node nodename key- 取消标签，key和-中间没有空格。
[root@master ~]# kubectl label node worker2 diskxx-
node/worker2 unlabeled
[root@master ~]# kubectl get nodes worker2 --show-labels
NAME      STATUS   ROLES    AGE    VERSION   LABELS
worker2   Ready    <none>   239d   v1.26.4   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=worker2,kubernetes.io/os=linux
通过kubectl label node worker1 node-role.kubernetes.io/worker=设置node role name
[root@master ~]# kubectl label node worker1 node-role.kubernetes.io/worker=
node/worker1 labeled
[root@master ~]# kubectl get nodes
NAME      STATUS   ROLES           AGE    VERSION
master   Ready    control-plane   239d   v1.26.4
worker1   Ready    worker          239d   v1.26.4
worker2   Ready    <none>          239d   v1.26.4
[root@master ~]# kubectl label node worker1 node-role.kubernetes.io/worker-
node/worker1 unlabeled
[root@master ~]# kubectl get nodes
NAME      STATUS   ROLES           AGE    VERSION
master   Ready    control-plane   239d   v1.26.4
worker1   Ready    <none>          239d   v1.26.4
worker2   Ready    <none>          239d   v1.26.4

在pod中通过nodeSelector参数可以让pod在含有特定标签的node上运行。需要注意nodeSelector的缩进与containers是同级的 如果不存在含有指定标签的node,则pod是创建不成功的。

节点的cordon与drain

如果想要某个node设置为不可用，可以对节点实施cordon或者drain操作，这样节点就会标记为schedulingDisabled，新创建的pod就不会分配到这些节点上。 cordon与drain的区别： drain比cordon多一个驱逐evicted的效果，即当我们对某个节点进行drain操作的时候，不仅把此节点标记为不可调度，且会把上面正在运行的pod删除。

取消drain的操作仍然是uncordon，没有undrain操作

节点taint以及pod的tolerations

如果给节点设置taint，那么只有设置了tolerations（容忍污点）的pod才能运行在此节点上。

kubectl taint nodes nodename keyname=valuename:NoSchedule

如果需要在含有taint的节点上运行pod，则定义pod的时候需要指定toleration属性。

tolerations:
- key: "key值"
  operator: "Equal"
  value:"value值"
  effect:"值"
operator的值有以下：
Equal: value需要和taint的value值一样(默认), value和effect的值要与节点的taint的值匹配才可以。
Exists: 可以不指定value的值

给节点设置及删除taint #operator的值等于Equal的情况

[root@master pod]# kubectl taint nodes worker2 keyxx=valuexx:NoSchedule
node/worker2 tainted
[root@master pod]# kubectl get nodes
NAME      STATUS   ROLES           AGE    VERSION
master   Ready    control-plane   240d   v1.26.4
worker1   Ready    <none>          240d   v1.26.4
worker2   Ready    <none>          240d   v1.26.4
[root@master pod]# kubectl describe nodes worker2 | grep -E '(Roles|Taints)'
Roles:              <none>
Taints:             keyxx=valuexx:NoSchedule

给node设置taint，是不会影响当前在node上运行的pod pod的toleration设置需要与node的所有污点匹配，才能成功创建pod，如果node上有多个污点设置的话，在创建pod的时候也要设置容忍所有的污点。

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: pod2
  name: pod2
spec:
  nodeSelector:
    diskxx: ssdxx
  tolerations:
  - key : "keyxx"
    operator: "Equal"
    value: "valuexx"
    effect: "NoSchedule"
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: pod2
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

#operator的值等于Exists的情况在设置taint的时候，如果value的值为非空，在pod的tolerations字段只能写Equal,不能写Exists

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: pod2
  name: pod2
spec:
  nodeSelector:
    diskxx: ssdxx
  tolerations:
  - key : "keyxx"
    operator: "Exists"
    value: "valuexx"
    effect: "NoSchedule"
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: pod2
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}
[root@wask8s1 pod]# kubectl apply -f podtaint.yaml
The Pod "pod2" is invalid: spec.tolerations[0].operator: Invalid value: core.Toleration{Key:"keyxx", Operator:"Exists", Value:"valuexx", Effect:"NoSchedule", TolerationSeconds:(*int64)(nil)}: value must be empty when `operator` is 'Exists'

可以通过设置value为空，来使用Exists

[root@master pod]# kubectl taint nodes worker2 keyxx=:NoSchedule --overwrite
node/worker2 modified
[root@master pod]# kubectl describe nodes worker2 | grep -E '(Roles|Taints)'
Roles:              <none>
Taints:             keyxx:NoSchedule
[root@master pod]# kubectl apply -f podtaint.yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: pod2
  name: pod2
spec:
  nodeSelector:
    diskxx: ssdxx
  tolerations:
  - key : "keyxx"
    operator: "Exists"
    effect: "NoSchedule"
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: pod2
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}
如果使用Equal的话，需要按照以下格式
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: pod2
  name: pod2
spec:
  nodeSelector:
    diskxx: ssdxx
  tolerations:
  - key : "keyxx"
    operator: "Equal"
    value: ""
    effect: "NoSchedule"
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: pod2
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

获取kubectl中消耗cpu资源最多的worker node

kubectl get nodes --selector='!node-role.kubernetes.io/control-plane' --no-headers=true | awk '{print $1}' | xargs -I {} kubectl top node {} --no-headers | awk 'NR>1{print $1,$3}' | sort -k2 -r | head -n 1 | awk '{print $1}'
其中可以通过kubectl top node查看每个node的资源消耗