本文已参与「新人创作礼」活动，一起开启掘金创作之路。
[root@webserver1 ~]# kubeadm init \
> --apiserver-advertise-address=192.168.1.142 \
> --image-repository registry.aliyuncs.com/google_containers \
> --kubernetes-version v1.24.1 \
> --service-cidr=10.68.0.0/16 \
> --pod-network-cidr=172.22.0.0/16
[init] Using Kubernetes version: v1.24.1
[preflight] Running pre-flight checks
[WARNING KubernetesVersion]: Kubernetes version is greater than kubeadm version. Please consider to upgrade kubeadm. Kubernetes version: 1.24.1. Kubeadm version: 1.23.x
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
W0529 18:28:30.214863 40622 images.go:80] could not find officially supported version of etcd for Kubernetes v1.24.1, falling back to the nearest etcd version (3.5.1-0)
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local webserver1] and IPs [10.68.0.1 192.168.1.142]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost webserver1] and IPs [192.168.1.142 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost webserver1] and IPs [192.168.1.142 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
W0529 18:28:36.461418 40622 images.go:80] could not find officially supported version of etcd for Kubernetes v1.24.1, falling back to the nearest etcd version (3.5.1-0)
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
journalctl -xeu kubelet
[root@webserver1 ~]# journalctl -xeu kubelet
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- The process /usr/bin/kubelet could not be executed and failed.
--
-- The error number returned by this process is 2.
5月 29 18:55:29 webserver1 systemd[1]: kubelet.service: main process exited, code=exited, status=203/EXEC
5月 29 18:55:29 webserver1 systemd[1]: Unit kubelet.service entered failed state.
5月 29 18:55:29 webserver1 systemd[1]: kubelet.service failed.
5月 29 18:55:39 webserver1 systemd[1]: kubelet.service holdoff time over, scheduling restart.
5月 29 18:55:39 webserver1 systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
-- Subject: Unit kubelet.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit kubelet.service has finished shutting down.
5月 29 18:55:39 webserver1 systemd[1]: Started kubelet: The Kubernetes Node Agent.
-- Subject: Unit kubelet.service has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit kubelet.service has finished starting up.
--
-- The start-up result is done.
5月 29 18:55:39 webserver1 systemd[35888]: Failed at step EXEC spawning /usr/bin/kubelet: No such file or directory
-- Subject: Process /usr/bin/kubelet could not be executed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- The process /usr/bin/kubelet could not be executed and failed.
--
-- The error number returned by this process is 2.
5月 29 18:55:39 webserver1 systemd[1]: kubelet.service: main process exited, code=exited, status=203/EXEC
5月 29 18:55:39 webserver1 systemd[1]: Unit kubelet.service entered failed state.
5月 29 18:55:39 webserver1 systemd[1]: kubelet.service failed.
经网上查阅，重新安装（或第一次安装）k8s，未经过kubeadm init 或者 kubeadm join后，kubelet会不断重启，这个是正常现象……，执行init或join后问题会自动解决，对此官网有如下描述，也就是此时不用理会kubelet.service。
Dockershim 自 1.24 版起已从 Kubernetes 项目中删除 如果要使用Docker的话 要降级 (降级到1.14.1解决了) Updated: Dockershim Removal FAQ | Kubernetes kubernetes.io/blog/2022/0…
yum earse kubelet
yum install -y docker kubelet kubeadm kubectl --disableexcludes=kubernetes
kubeadm reset -f
rm -rf .kube/
sudo rm -rf /etc/kubernetes/
sudo rm -rf /var/lib/kubelet/
sudo rm -rf /var/lib/etcd
解决方法
# 添加以下内容
vim /etc/docker/daemon.json
{
"exec-opts": ["native.cgroupdriver=systemd"]
}
# 重启docker
systemctl restart docker
# 重新初始化
kubeadm reset # 先重置
# 查看或修改docker的cgroup：
docker info | grep -i cgroup
vim /usr/lib/systemd/system/docker.service
ExecStart=/usr/bin/dockerd --exec-opt native.cgroupdriver=cgroupfs
# 查看或修改kubelet的cgroup drive ：
vim /etc/sysconfig/kubelet
KUBELET_EXTRA_ARGS="--cgroup-driver=cgroupfs"
vim /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf
# cgroup保持一致后，重启docker和kubelet
systemctl daemon-reload
systemctl restart docker
kubeadm init \
--apiserver-advertise-address=192.168.1.142 \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version v1.14.1 \
--service-cidr=10.68.0.0/16 \
--pod-network-cidr=172.22.0.0/16
