kubernetes集群证书续期

RikyLee
1,418 阅读10分钟

kubernetes证书

建立kubernetes集群会生成很多证书

[root@k8s-portal-master1 ~]# cd /etc/kubernetes/pki
[root@k8s-portal-master1 pki]# tree
.
├── apiserver.crt
├── apiserver-etcd-client.crt
├── apiserver-etcd-client.key
├── apiserver.key
├── apiserver-kubelet-client.crt
├── apiserver-kubelet-client.key
├── ca.crt
├── ca.key
├── etcd
│   ├── ca.crt
│   ├── ca.key
│   ├── healthcheck-client.crt
│   ├── healthcheck-client.key
│   ├── peer.crt
│   ├── peer.key
│   ├── server.crt
│   └── server.key
├── front-proxy-ca.crt
├── front-proxy-ca.key
├── front-proxy-client.crt
├── front-proxy-client.key
├── sa.key
└── sa.pub

1 directory, 22 files

kubernetes一共有如下证书

Etcd相关：

1、Etcd对外提供服务，要有一套etcd server证书
2、Etcd各节点之间进行通信，要有一套etcd peer证书
3、Kube-APIserver访问Etcd，要有一套etcd client证书

Kubernetes相关：

1、Kube-APIserver对外提供服务，要有一套kube-apiserver server证书
2、kube-scheduler、kube-controller-manager、kube-proxy、kubelet和其他可能用到的组件，需要访问kube-APIserver，要有一套kube-APIserver client证书
3、kube-controller-manager要生成服务的service account，要有一对用来签署service account的证书(CA证书)
4、kubelet对外提供服务，要有一套kubelet server证书
5、kube-APIserver需要访问kubelet，要有一套kubelet client证书

加起来共8套，同一个套内的证书必须是用同一个CA签署的，签署不同套证书的CA可以相同，也可以不同。例如，所有etcd server证书需要是同一个CA签署的，所有的etcd peer证书也需要是同一个CA签署的，而一个etcd server证书和一个etcd peer证书，完全可以是两个CA机构签署的，彼此没有任何关系，这算两套证书。

同一个套内的证书必须是同一个CA签署的原因在验证这些证书的一端，通常只能指定一个Root CA。这样一来，被验证的证书自然都需要是被这同一个Root CA对应的私钥签署，不然不能通过认证。

根证书与证书

通常我们配置https服务时需要到"权威机构"(CA)申请证书，过程是这样的：

  • 网站创建一个密钥对，提供公钥和组织以及个人信息给权威机构
  • 权威机构颁发证书
  • 浏览网页的朋友利用权威机构的根证书公钥解密签名，对比摘要，确定合法性
  • 客户端验证域名信息有效时间等（浏览器基本都内置各大权威机构的CA公钥）

这个证书包含如下内容：

  • 申请者公钥
  • 申请者组织和个人信息
  • 签发机构CA信息，有效时间，序列号等
  • 以上信息的签名

根证书又名自签名证书，也就是自己给自己颁发的证书。CA(Certificate Authority)被称为证书授权中心， kubernets中的ca证书就是根证书。

  • 密钥对：sa.key sa.pub
  • 根证书：ca.crt etcd/ca.crt
  • 私钥:ca.key 等 其它证书

Pod中的容器访问API Server(如dashboard访问API Server) 因为Pod的创建、销毁是动态的，所以要为它手动生成证书就不可行了。K8s使用了Service Account解决Pod 访问API Server的认证问题

默认情况下，每个 namespace 都会有一个 ServiceAccount，如果 Pod 在创建时没有指定 ServiceAccount就会使用 Pod 所属的 namespace 的 ServiceAccount,默认值/run/secrets/kubernates.io/serviceaccount/

查看kube-system命名空间下的pod

[root@k8s-portal-master1 pki]# kubectl exec  kube-proxy-24zcf -n kube-system -it -- /bin/sh #进入容器
# ls /run/secrets/kubernetes.io/serviceaccount
ca.crt  namespace  token

里面有ca.crt(根的证书) ,namespace,token3个文件

  • token是使用API Server私钥签名的JWT（json web token）。用于访问API Server时，Server端认证
  • ca.crt，根证书(是k8s中私有的)。用于Client端验证API Server发送的证书
  • namespace, 标识这个service-account-token的作用域名空间

service Account密钥对 sa.key sa.pub 提供给 kube-controller-manager使用，kube-controller-manager通过sa.key对token进行签名,master 节点通过公钥 sa.pub 进行签名的验证如kube-proxy是以pod 形式运行的,在pod中,直接使用service account与kube-apiserver进行认证, 此时就不需要再单独为kube-proxy创建证书了,会直接使用token校验。

查询证书有效期

由kubeadm部署的集群，所生成的证书证书有效期为一年，过期之后集群就不能再次使用了，我们可以对证书进行续期，这样集群就可以继续使用。可以通过如下命令查看具体过期时间：

[root@k8s-portal-master1 kubernetes]# kubeadm alpha certs check-expiration
CERTIFICATE                EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
admin.conf                 Dec 12, 2020 09:40 UTC   45d             no
apiserver                  Dec 12, 2020 09:43 UTC   45d             no
apiserver-etcd-client      Dec 12, 2020 09:43 UTC   45d             no
apiserver-kubelet-client   Dec 12, 2020 09:43 UTC   45d             no
controller-manager.conf    Dec 12, 2020 09:43 UTC   45d             no
etcd-healthcheck-client    Dec 12, 2020 09:43 UTC   45d             no
etcd-peer                  Dec 12, 2020 09:43 UTC   45d             no
etcd-server                Dec 12, 2020 09:43 UTC   45d             no
front-proxy-client         Dec 12, 2020 09:43 UTC   45d             no
scheduler.conf             Dec 12, 2020 09:43 UTC   45d             no

注意： 上面的列表中没有包含 kubelet.conf 因为 kubeadm 将 kubelet 配置为自动更新证书。

上述命令只能查询证书的过期时间，但是根证书的过期时间无法显示，我们可以通过另外一种方式查看证书的有限期

[root@k8s-portal-master1 pki]# openssl x509 -in /etc/kubernetes/pki/ca.crt -noout -text |grep ' Not '
            Not Before: Dec 13 09:40:51 2019 GMT
            Not After : Dec 10 09:40:51 2029 GMT
[root@k8s-portal-master1 pki]# openssl x509 -in /etc/kubernetes/pki/etcd/ca.crt -noout -text |grep ' Not '
            Not Before: Dec 13 09:40:50 2019 GMT
            Not After : Dec 10 09:40:50 2029 GMT
[root@k8s-portal-master1 pki]# openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -text |grep ' Not '
            Not Before: Dec 13 09:40:51 2019 GMT
            Not After : Oct 28 02:08:14 2021 GMT

根证书的默认有效期是十年，所以在根证书有限期内，我们只需要对普通证书进行续期即可。

修改系统时间模拟证书过期后，执行kubectl命令，会提示证书过期

[root@k8s-portal-master1 kubernetes]# kubectl get nodes
Unable to connect to the server: x509: certificate has expired or is not yet valid

证书续期

注意： 证书续期或者重新生成之前，请先备份原来的证书，以防万一，可以进行回滚操作。

用如下命令申请续期一年：

[root@k8s-portal-master1 kubernetes]# kubeadm alpha  certs renew all
certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself renewed
certificate for serving the Kubernetes API renewed
certificate the apiserver uses to access etcd renewed
certificate for the API server to connect to kubelet renewed
certificate embedded in the kubeconfig file for the controller manager to use renewed
certificate for liveness probes to healtcheck etcd renewed
certificate for etcd nodes to communicate with each other renewed
certificate for serving etcd renewed
certificate for the front proxy client renewed
certificate embedded in the kubeconfig file for the scheduler manager to use renewed

重新查看证书的有限期,可以看到证书的有限期已经延长了

[root@k8s-portal-master1 kubernetes]# kubeadm alpha certs check-expiration
CERTIFICATE                EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
admin.conf                 Oct 28, 2021 02:34 UTC   364d            no
apiserver                  Oct 28, 2021 02:34 UTC   364d            no
apiserver-etcd-client      Oct 28, 2021 02:34 UTC   364d            no
apiserver-kubelet-client   Oct 28, 2021 02:34 UTC   364d            no
controller-manager.conf    Oct 28, 2021 02:34 UTC   364d            no
etcd-healthcheck-client    Oct 28, 2021 02:34 UTC   364d            no
etcd-peer                  Oct 28, 2021 02:34 UTC   364d            no
etcd-server                Oct 28, 2021 02:34 UTC   364d            no
front-proxy-client         Oct 28, 2021 02:34 UTC   364d            no
scheduler.conf             Oct 28, 2021 02:34 UTC   364d            no

拷贝admin.conf到用户根目录.kube文件夹下覆盖原有的config文件

[root@k8s-portal-master1 kubernetes]# cp -i /etc/kubernetes/admin.conf ~/.kube/config

重启kubelet服务

[root@k8s-portal-master1 kubernetes]#  systemctl restart kubelet

如果是根证书过期了，那就只能重新签发新的根证书了,先对所有conf文件和证书进行备份

[root@k8s-portal-master1 kubernetes]# mkdir conf_bak
[root@k8s-portal-master1 kubernetes]# mkdir pki_bak
[root@k8s-portal-master1 kubernetes]# cp *.conf conf_bak/
[root@k8s-portal-master1 kubernetes]# cp -r pki/* pki_bak/
[root@k8s-portal-master1 kubernetes]# ll pki_bak/
total 56
-rw-r--r-- 1 root root 1277 Oct 28 10:23 apiserver.crt
-rw-r--r-- 1 root root 1090 Oct 28 10:23 apiserver-etcd-client.crt
-rw------- 1 root root 1675 Oct 28 10:23 apiserver-etcd-client.key
-rw------- 1 root root 1679 Oct 28 10:23 apiserver.key
-rw-r--r-- 1 root root 1099 Oct 28 10:23 apiserver-kubelet-client.crt
-rw------- 1 root root 1679 Oct 28 10:23 apiserver-kubelet-client.key
-rw-r--r-- 1 root root 1025 Oct 28 10:23 ca.crt
-rw------- 1 root root 1675 Oct 28 10:23 ca.key
drwxr-xr-x 2 root root  162 Oct 28 10:23 etcd
-rw-r--r-- 1 root root 1038 Oct 28 10:23 front-proxy-ca.crt
-rw------- 1 root root 1675 Oct 28 10:23 front-proxy-ca.key
-rw-r--r-- 1 root root 1058 Oct 28 10:23 front-proxy-client.crt
-rw------- 1 root root 1679 Oct 28 10:23 front-proxy-client.key
-rw------- 1 root root 1675 Oct 28 10:23 sa.key
-rw------- 1 root root  451 Oct 28 10:23 sa.pub
[root@k8s-portal-master1 kubernetes]# ll conf_bak
total 36
-rw------- 1 root root 5452 Oct 28 10:23 admin.conf
-rw------- 1 root root 5452 Oct 28 10:23 bootstrap-kubelet.conf
-rw------- 1 root root 5484 Oct 28 10:23 controller-manager.conf
-rw------- 1 root root 1853 Oct 28 10:23 kubelet.conf
-rw------- 1 root root 5436 Oct 28 10:23 scheduler.conf

从上述我们可以看出根证书一共有三套，分别是ca,front-proxy-ca,etcd-ca,还有一套密钥对，我们可以通过如下命名生成需要的密钥和根证书,但是采用证书会依赖根证书，所有如果根证书发生变化其他的证书都需要重新签发

  • 使用kubeadm init phase certs all --config kubeadm-config.yaml 来签发所有证书,kubeadm-config.yaml为初始化kubernetes集群时所用的配置文件

[root@k8s-portal-master1 test]# kubeadm init phase certs all --config kubeadm-config.yaml
W1028 14:40:37.461986   17087 configset.go:348] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local k8s-portal-master1] and IPs [10.96.0.1 10.3.175.168 10.3.175.165 10.3.175.166 10.3.175.167]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost k8s-portal-master1] and IPs [10.3.175.165 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost k8s-portal-master1] and IPs [10.3.175.165 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
  • 重新生成kubeconfig文件
[root@k8s-portal-master1 test]# kubeadm init phase kubeconfig all --config kubeadm-config.yaml
W1028 14:43:25.668596   17114 configset.go:348] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
  • 替换~/.kube/config文件

拷贝admin.conf到用户根目录.kube文件夹下覆盖原有的config文件

[root@k8s-portal-master1 kubernetes]# cp -i /etc/kubernetes/admin.conf ~/.kube/config
  • 依次重启master节点的docker和kubelet，确保master组件容器重启运行成功,至此证书更新完成
[root@k8s-portal-master1 kubernetes]# systemctl restart docker && systemctl restart kubelet

附录

  • 证书路径相关，证书应放置在建议的路径中（根路径为/etc/kubernetes/pki,以便 kubeadm使用）,无论使用什么位置，都应使用给定的参数指定路径
Default CNrecommended key pathrecommended cert pathcommandkey argumentcert argument
etcd-caetcd/ca.keyetcd/ca.crtkube-apiserver--etcd-cafile
kube-apiserver-etcd-clientapiserver-etcd-client.keyapiserver-etcd-client.crtkube-apiserver--etcd-keyfile--etcd-certfile
kubernetes-caca.keyca.crtkube-apiserver--client-ca-file
kubernetes-caca.keyca.crtkube-controller-manager--cluster-signing-key-file--client-ca-file, --root-ca-file, --cluster-signing-cert-file
kube-apiserverapiserver.keyapiserver.crtkube-apiserver--tls-private-key-file--tls-cert-file
kube-apiserver-kubelet-clientapiserver-kubelet-client.keyapiserver-kubelet-client.crtkube-apiserver--kubelet-client-key--kubelet-client-certificate
front-proxy-cafront-proxy-ca.keyfront-proxy-ca.crtkube-apiserver--requestheader-client-ca-file
front-proxy-cafront-proxy-ca.keyfront-proxy-ca.crtkube-controller-manager--requestheader-client-ca-file
front-proxy-clientfront-proxy-client.keyfront-proxy-client.crtkube-apiserver--proxy-client-key-file--proxy-client-cert-file
etcd-caetcd/ca.keyetcd/ca.crtetcd--trusted-ca-file, --peer-trusted-ca-file
kube-etcdetcd/server.keyetcd/server.crtetcd--key-file--cert-file
kube-etcd-peeretcd/peer.keyetcd/peer.crtetcd--peer-key-file--peer-cert-file
etcd-caetcd/ca.crtetcdctl--cacert
kube-etcd-healthcheck-clientetcd/healthcheck-client.keyetcd/healthcheck-client.crtetcdctl--key--cert
avatar
文章
阅读
粉丝