k8s-demo集群搭建详细步骤06：用cfssl 1.6.0生成集群各组件间HTTPS通讯需要的x509证书

Jason2020
592 阅读6分钟

本文已参与「新人创作礼」活动，一起开启掘金创作之路

这里一次准备好集群内部通讯需要的所有证书，你可以跳过本篇，在后续部署过程中需要相应证书的时候再创建

一、安装证书生产工具CFSSL

  • 本案例使用CloudFlare的PKI工具集cfssl创建所有证书
  • CSR：Certificate Signing Request 证书签名请求文件
[root@master1 ~]# mkdir -p /opt/k8s/bin
[root@master1 ~]# mkdir -p /opt/install/soft/cfssl
[root@master1 ~]# cd /opt/install/soft/cfssl
[root@master1 cfssl]# wget https://github.com/cloudflare/cfssl/releases/download/v1.6.0/cfssl_1.6.0_linux_amd64
[root@master1 cfssl]# mv cfssl_1.6.0_linux_amd64 /opt/k8s/bin/cfssl
[root@master1 cfssl]# wget https://github.com/cloudflare/cfssl/releases/download/v1.6.0/cfssljson_1.6.0_linux_amd64
[root@master1 cfssl]# mv cfssljson_1.6.0_linux_amd64 /opt/k8s/bin/cfssljson
[root@master1 cfssl]# wget https://github.com/cloudflare/cfssl/releases/download/v1.6.0/cfssl-certinfo_1.6.0_linux_amd64
[root@master1 cfssl]# mv cfssl-certinfo_1.6.0_linux_amd64 /opt/k8s/bin/cfssl-certinfo
[root@master1 cfssl]# chmod +x /opt/k8s/bin/cfssl*
  • 获取缺省配置，可以根据需要在这个基础上修改
[root@master1 ~]# mkdir -p /opt/install/cert
[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cfssl print-defaults config > ca-config.json
[root@master1 cert]# cfssl print-defaults csr > ca-csr.json

二、创建根证书

  • CA(Certificate Authority)是自签名的根证书，用来签名其它证书
  • 如果没有特殊说明，后续操作均在master1节点上执行
  • 证书临时存放目录 /opt/install/cert

1、创建根证书配置文件 ca-config.json

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cat > ca-config.json <<EOF
{
  "signing": {
    "default": {
      "expiry": "87600h"
    },
    "profiles": {
      "k8s-demo-server": {
        "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"
        ],
        "expiry": "87600h"
      },
      "k8s-demo-client": {
        "usages": [
            "signing",
            "key encipherment",
            "client auth"
        ],
        "expiry": "87600h"
      }
    }
  }
}
EOF
  • expiry: "87600h"：证书有效期设置为10年
  • signing：表示该证书可用于签名其它证书（生成的 ca.pem 证书中 CA=TRUE）
  • key encipherment：秘钥加密
  • server auth：表示client可以用该该证书对server提供的证书进行验证
  • client auth：表示server可以用该该证书对client提供的证书进行验证

2、创建证书签名请求文件 ca-csr.json

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cat > ca-csr.json <<EOF
{
  "CN": "k8s-demo-ca",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [{
    "C" : "China",
    "ST": "GuangDong",
    "L" : "ShenZhen",
    "O" : "k8s-demo",
    "OU": "jason@vip.qq.com"
  }],
  "ca": {
    "expiry": "87600h"
  }
}
EOF
  • CN ：Common Name，kube-apiserver从证书中提取该字段作为请求的用户名 (User Name)，浏览器使用该字段验证网站是否合法
  • C ：国家
  • ST ：州，省
  • L ： 地区，城市
  • O ：Organization，组织名，公司名称，kube-apiserver从证书中提取该字段作为请求用户所属的组 (Group)
  • OU ：组织内部单位名称，部门名称
  • 不同证书csr文件的CN、C、ST、L、O、OU组合必须不同，否则可能出现PEER'S CERTIFICATE HAS AN INVALID SIGNATURE错误

3、生成CA 根证书及私钥

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca
2022/04/08 23:58:28 [INFO] generating a new CA key and certificate from CSR
2022/04/08 23:58:28 [INFO] generate received request
2022/04/08 23:58:28 [INFO] received CSR
2022/04/08 23:58:28 [INFO] generating key: rsa-2048
2022/04/08 23:58:28 [INFO] encoded CSR
2022/04/08 23:58:28 [INFO] signed certificate with serial number 219577746226896537989503532764723745470224199241
[root@master1 cert]# ll
总用量 20
-rw-r--r-- 1 root root  472 4月   8 23:54 ca-config.json
-rw-r--r-- 1 root root 1078 4月   8 23:58 ca.csr
-rw-r--r-- 1 root root  251 4月   8 23:55 ca-csr.json
-rw------- 1 root root 1679 4月   8 23:58 ca-key.pem
-rw-r--r-- 1 root root 1371 4月   8 23:58 ca.pem
[root@master1 cert]#

4、分发证书

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# for node_ip in ${ALL_IPS[@]}
  do
    echo ">>> ${node_ip}"
    ssh root@${node_ip} "mkdir -p /opt/k8s/etc/cert"
    scp ca*.pem ca-config.json root@${node_ip}:/opt/k8s/etc/cert
  done
>>> 192.168.66.131
ca-key.pem                                 100% 1679   754.3KB/s   00:00
ca.pem                                     100% 1371     1.3MB/s   00:00
ca-config.json                             100%  472   728.7KB/s   00:00
>>> 192.168.66.132
ca-key.pem                                 100% 1679   564.6KB/s   00:00
ca.pem                                     100% 1371     2.0MB/s   00:00
ca-config.json                             100%  472   754.5KB/s   00:00
>>> 192.168.66.133
ca-key.pem                                 100% 1679   718.3KB/s   00:00
ca.pem                                     100% 1371     1.9MB/s   00:00
ca-config.json                             100%  472   703.0KB/s   00:00
>>> 192.168.66.134
ca-key.pem                                 100% 1679     1.9MB/s   00:00
ca.pem                                     100% 1371     1.1MB/s   00:00
ca-config.json                             100%  472   750.7KB/s   00:00
>>> 192.168.66.135
ca-key.pem                                 100% 1679     1.1MB/s   00:00
ca.pem                                     100% 1371     1.3MB/s   00:00
ca-config.json                             100%  472   640.3KB/s   00:00
>>> 192.168.66.136
ca-key.pem                                 100% 1679   812.2KB/s   00:00
ca.pem                                     100% 1371   717.5KB/s   00:00
ca-config.json                             100%  472   639.5KB/s   00:00
[root@master1 cert]#

三、生成集群管理员admin的证书（kubectl->kube-apiserver）

1、准备证书签名请求文件 kubectl-admin-csr.json

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cat > kubectl-admin-csr.json <<EOF
{
  "CN": "k8s-demo-admin",
  "hosts": [
    "192.168.66.131",
    "192.168.66.132",
    "192.168.66.133"
  ],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
 "names": [{
    "C" : "China",
    "ST": "GuangDong",
    "L" : "ShenZhen",
    "O" : "system:masters",
    "OU": "jason@vip.qq.com"
 }]
}
EOF
  • kube-apiserver 将提取的 User、Group 作为RBAC授权的用户标识
  • kubernetes使用RBAC进行角色权限控制，证书中的CN字段作为User，O字段作为Group
  • 这里管理员名字设定为k8s-demo-admin，你可以根据实际情况修改

2、生成集群管理员证书和私钥

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cfssl gencert -ca=/opt/k8s/etc/cert/ca.pem \
  -ca-key=/opt/k8s/etc/cert/ca-key.pem \
  -config=/opt/k8s/etc/cert/ca-config.json \
  -profile=k8s-demo-client kubectl-admin-csr.json | cfssljson -bare kubectl-admin
2022/04/09 00:07:38 [INFO] generate received request
2022/04/09 00:07:38 [INFO] received CSR
2022/04/09 00:07:38 [INFO] generating key: rsa-2048
2022/04/09 00:07:39 [INFO] encoded CSR
2022/04/09 00:07:39 [INFO] signed certificate with serial number 57990161219811929717399695305599374391535095535
[root@master1 cert]# ll kubectl-admin*
-rw-r--r-- 1 root root 1106 4月   9 00:07 kubectl-admin.csr
-rw-r--r-- 1 root root  303 4月   9 00:05 kubectl-admin-csr.json
-rw------- 1 root root 1675 4月   9 00:07 kubectl-admin-key.pem
-rw-r--r-- 1 root root 1493 4月   9 00:07 kubectl-admin.pem
[root@master1 cert]#
  • 如果授权部分操作权限给一个账户呢？证书如何生成？后面单独一篇文章来说RBAC
  • RBAC：Role Based Access Control
  • K8s还有其它五种的授权方式 AlwaysDeny、AlwaysAllow、ABAC、Webhook、Node

3、分发证书到3个Master节点+3Node节点（kube-nginx需要）

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# for node_ip in ${ALL_IPS[@]}
  do
    echo ">>> ${node_ip}"
    scp kubectl-admin*.pem root@${node_ip}:/opt/k8s/etc/cert
  done
>>> 192.168.66.131
kubectl-admin-key.pem                      100% 1675     1.1MB/s   00:00
kubectl-admin.pem                          100% 1493     2.3MB/s   00:00
>>> 192.168.66.132
kubectl-admin-key.pem                      100% 1675   955.3KB/s   00:00
kubectl-admin.pem                          100% 1493     1.7MB/s   00:00
>>> 192.168.66.133
kubectl-admin-key.pem                      100% 1675     1.6MB/s   00:00
kubectl-admin.pem                          100% 1493     2.1MB/s   00:00
>>> 192.168.66.134
kubectl-admin-key.pem                      100% 1675   721.5KB/s   00:00
kubectl-admin.pem                          100% 1493     1.8MB/s   00:00
>>> 192.168.66.135
kubectl-admin-key.pem                      100% 1675     1.3MB/s   00:00
kubectl-admin.pem                          100% 1493     1.7MB/s   00:00
>>> 192.168.66.136
kubectl-admin-key.pem                      100% 1675     1.5MB/s   00:00
kubectl-admin.pem                          100% 1493     1.7MB/s   00:00
[root@master1 cert]#

四、生成etcd节点之间通讯的证书

1、准备证书签名请求文件

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cat > etcd-csr.json <<EOF
{
  "CN": "k8s-demo-etcd",
  "hosts": [
    "192.168.66.131",
    "192.168.66.132",
    "192.168.66.133"
  ],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [{
    "C" : "China",
    "ST": "GuangDong",
    "L" : "ShenZhen",
    "O" : "k8s-demo",
    "OU": "jason@vip.qq.com"
  }]
}
EOF
  • UserName：k8s-demo-etcd
  • Group：k8s-demo

2、生成etcd证书

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cfssl gencert -ca=/opt/k8s/etc/cert/ca.pem \
    -ca-key=/opt/k8s/etc/cert/ca-key.pem \
    -config=/opt/k8s/etc/cert/ca-config.json \
    -profile=k8s-demo-server etcd-csr.json | cfssljson -bare etcd
2022/04/09 00:13:51 [INFO] generate received request
2022/04/09 00:13:51 [INFO] received CSR
2022/04/09 00:13:51 [INFO] generating key: rsa-2048
2022/04/09 00:13:51 [INFO] encoded CSR
2022/04/09 00:13:51 [INFO] signed certificate with serial number 643796563930902508188162284168305628433221534123
[root@master1 cert]# ll etcd*
-rw-r--r-- 1 root root 1094 4月   9 00:13 etcd.csr
-rw-r--r-- 1 root root  298 4月   9 00:12 etcd-csr.json
-rw------- 1 root root 1675 4月   9 00:13 etcd-key.pem
-rw-r--r-- 1 root root 1497 4月   9 00:13 etcd.pem
[root@master1 cert]#

3、分发到3个master节点

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# for node_ip in ${MASTER_IPS[@]}
  do
    echo ">>> ${node_ip}"
    ssh root@${node_ip} "mkdir -p /opt/k8s/etcd/cert"
    scp etcd*.pem root@${node_ip}:/opt/k8s/etcd/cert/
  done
>>> 192.168.66.131
etcd-key.pem                          100% 1675   364.0KB/s   00:00
etcd.pem                              100% 1497     1.7MB/s   00:00
>>> 192.168.66.132
etcd-key.pem                          100% 1675     1.3MB/s   00:00
etcd.pem                              100% 1497     1.8MB/s   00:00
>>> 192.168.66.133
etcd-key.pem                          100% 1675     1.6MB/s   00:00
etcd.pem                              100% 1497     2.2MB/s   00:00
[root@master1 cert]#

五、生成kube-apiserver证书

kube-apiserver访问etcd、kubelet等接口时使用

1、准备证书签名请求文件 apiserver-csr.json

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cat > apiserver-csr.json <<EOF
{
  "CN": "k8s-demo-apiserver",
  "hosts": [
    "192.168.66.131",
    "192.168.66.132",
    "192.168.66.133",
    "10.66.0.1",
    "127.0.0.1",
    "kubernetes",
    "kubernetes.default",
    "kubernetes.default.svc",
    "kubernetes.default.svc.cluster",
    "kubernetes.default.svc.${CLUSTER_DNS_DOMAIN}"
  ],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [{
    "C" : "China",
    "ST": "GuangDong",
    "L" : "ShenZhen",
    "O" : "k8s-demo",
    "OU": "jason@vip.qq.com"
  }]
}
EOF
  • 请打开apiserver-csr.json文件检查使用的环境变量是否正确

2、生成kube-apiserver证书

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cfssl gencert -ca=/opt/k8s/etc/cert/ca.pem \
  -ca-key=/opt/k8s/etc/cert/ca-key.pem \
  -config=/opt/k8s/etc/cert/ca-config.json \
  -profile=k8s-demo-server apiserver-csr.json | cfssljson -bare apiserver
2022/04/09 00:20:27 [INFO] generate received request
2022/04/09 00:20:27 [INFO] received CSR
2022/04/09 00:20:27 [INFO] generating key: rsa-2048
2022/04/09 00:20:27 [INFO] encoded CSR
2022/04/09 00:20:27 [INFO] signed certificate with serial number 608031939106242690821780017473811434790082829942
[root@master1 cert]# ll apiserver*
-rw-r--r-- 1 root root 1297 4月   9 00:20 apiserver.csr
-rw-r--r-- 1 root root  478 4月   9 00:17 apiserver-csr.json
-rw------- 1 root root 1679 4月   9 00:20 apiserver-key.pem
-rw-r--r-- 1 root root 1696 4月   9 00:20 apiserver.pem
[root@master1 cert]#

3、分发证书

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# for node_ip in ${MASTER_IPS[@]}
  do
    echo ">>> ${node_ip}"
    ssh root@${node_ip} "mkdir -p /opt/k8s/etc/cert"
    scp apiserver*.pem root@${node_ip}:/opt/k8s/etc/cert/
  done
>>> 192.168.66.131
apiserver-key.pem                       100% 1679     1.9MB/s   00:00
apiserver.pem                           100% 1696     2.2MB/s   00:00
>>> 192.168.66.132
apiserver-key.pem                       100% 1679     1.2MB/s   00:00
apiserver.pem                           100% 1696     2.2MB/s   00:00
>>> 192.168.66.133
apiserver-key.pem                       100% 1679     1.0MB/s   00:00
apiserver.pem                           100% 1696     1.9MB/s   00:00
[root@master1 cert]#

六、生成kube-controller-manager证书

1、准备证书签名请求文件 controller-manager-csr.json

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cat > controller-manager-csr.json <<EOF
{
    "CN": "k8s-demo-ctrl-mgr",
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "hosts": [
      "192.168.66.131",
      "192.168.66.132",
      "192.168.66.133"
    ],
    "names": [{
      "C" : "China",
      "ST": "GuangDong",
      "L" : "ShenZhen",
      "O" : "system:kube-controller-manager",
      "OU": "jason@vip.qq.com"
    }]
}
EOF
  • 用户名 k8s-demo-ctrl-mgr
  • 组名 system:kube-controller-manager
  • k8s内置的组有：
    • system:masters
    • system:authenticated
    • system:unauthenticated
    • system:monitoring
    • system:serviceaccounts
  • k8s 内置的用户名有：
    • system:kube-controller-manager
    • system:kube-scheduler
    • system:kube-proxy

证书里面没有用内置的组和用户名，需要通过 kubectl create clusterrolebinding 完成授权，在后面安装过程中有说明，这里先创建证书

2、生成kube-controller-manager证书

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cfssl gencert -ca=/opt/k8s/etc/cert/ca.pem \
  -ca-key=/opt/k8s/etc/cert/ca-key.pem \
  -config=/opt/k8s/etc/cert/ca-config.json \
  -profile=k8s-demo-server controller-manager-csr.json | cfssljson -bare controller-manager
2022/04/09 00:24:44 [INFO] generate received request
2022/04/09 00:24:44 [INFO] received CSR
2022/04/09 00:24:44 [INFO] generating key: rsa-2048
2022/04/09 00:24:44 [INFO] encoded CSR
2022/04/09 00:24:44 [INFO] signed certificate with serial number 225565725360415518875162934970060606729935751797
[root@master1 cert]# ll controller-manager*
-rw-r--r-- 1 root root 1131 4月   9 00:24 controller-manager.csr
-rw-r--r-- 1 root root  362 4月   9 00:23 controller-manager-csr.json
-rw------- 1 root root 1675 4月   9 00:24 controller-manager-key.pem
-rw-r--r-- 1 root root 1533 4月   9 00:24 controller-manager.pem
[root@master1 cert]#

3、分发证书到3个master节点

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# for node_ip in ${MASTER_IPS[@]}
  do
    echo ">>> ${node_ip}"
    scp controller-manager*.pem root@${node_ip}:/opt/k8s/etc/cert/
  done
>>> 192.168.66.131
controller-manager-key.pem           100% 1675     2.1MB/s   00:00
controller-manager.pem               100% 1533     2.3MB/s   00:00
>>> 192.168.66.132
controller-manager-key.pem           100% 1675   567.2KB/s   00:00
controller-manager.pem               100% 1533   489.6KB/s   00:00
>>> 192.168.66.133
controller-manager-key.pem           100% 1675   477.6KB/s   00:00
controller-manager.pem               100% 1533     1.6MB/s   00:00
[root@master1 cert]#

七、生成kube-scheduler证书

1、准备证书签名请求文件 scheduler-csr.json

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cat > scheduler-csr.json <<EOF
{
    "CN": "k8s-demo-scheduler",
    "hosts": [
      "192.168.66.131",
      "192.168.66.132",
      "192.168.66.133"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [{
      "C" : "China",
      "ST": "GuangDong",
      "L" : "ShenZhen",
      "O" : "system:kube-scheduler",
      "OU": "jason@vip.qq.com"
    }]
}
EOF
  • 用户名 k8s-demo-scheduler
  • 组 system:kube-scheduler

2、生成kube-scheduler证书

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cfssl gencert -ca=/opt/k8s/etc/cert/ca.pem \
  -ca-key=/opt/k8s/etc/cert/ca-key.pem \
  -config=/opt/k8s/etc/cert/ca-config.json \
  -profile=k8s-demo-server scheduler-csr.json | cfssljson -bare scheduler
2022/04/09 00:28:00 [INFO] generate received request
2022/04/09 00:28:00 [INFO] received CSR
2022/04/09 00:28:00 [INFO] generating key: rsa-2048
2022/04/09 00:28:00 [INFO] encoded CSR
2022/04/09 00:28:00 [INFO] signed certificate with serial number 492169199750482662715480367997375212380010366828
[root@master1 cert]# ll scheduler*
-rw-r--r-- 1 root root 1123 4月   9 00:28 scheduler.csr
-rw-r--r-- 1 root root  354 4月   9 00:27 scheduler-csr.json
-rw------- 1 root root 1679 4月   9 00:28 scheduler-key.pem
-rw-r--r-- 1 root root 1521 4月   9 00:28 scheduler.pem
[root@master1 cert]#

3、分发证书

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# for node_ip in ${MASTER_IPS[@]}
  do
    echo ">>> ${node_ip}"
    scp scheduler*.pem root@${node_ip}:/opt/k8s/etc/cert/
  done
>>> 192.168.66.131
scheduler-key.pem                100% 1679   754.6KB/s   00:00
scheduler.pem                    100% 1521     2.7MB/s   00:00
>>> 192.168.66.132
scheduler-key.pem                100% 1679     1.3MB/s   00:00
scheduler.pem                    100% 1521     1.4MB/s   00:00
>>> 192.168.66.133
scheduler-key.pem                100% 1679   456.4KB/s   00:00
scheduler.pem                    100% 1521   870.8KB/s   00:00
[root@master1 cert]#

八、生成kube-proxy证书

1、准备证书签名请求文件 kube-proxy-csr.json

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cat > kube-proxy-csr.json <<EOF
{
  "CN": "k8s-demo-kube-proxy",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [{
    "C" : "China",
    "ST": "GuangDong",
    "L" : "ShenZhen",
    "O" : "k8s-demo",
    "OU": "jason@vip.qq.com"
 }]
}
EOF

2、生成kube-proxy证书

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cfssl gencert -ca=/opt/k8s/etc/cert/ca.pem \
  -ca-key=/opt/k8s/etc/cert/ca-key.pem \
  -config=/opt/k8s/etc/cert/ca-config.json \
  -profile=k8s-demo-client kube-proxy-csr.json | cfssljson -bare kube-proxy
2022/04/09 00:30:28 [INFO] generate received request
2022/04/09 00:30:28 [INFO] received CSR
2022/04/09 00:30:28 [INFO] generating key: rsa-2048
2022/04/09 00:30:28 [INFO] encoded CSR
2022/04/09 00:30:28 [INFO] signed certificate with serial number 672155413999854433265972912221642992761461934015
2022/04/09 00:30:28 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
[root@master1 cert]# ll kube-proxy*
-rw-r--r-- 1 root root 1041 4月   9 00:30 kube-proxy.csr
-rw-r--r-- 1 root root  220 4月   9 00:29 kube-proxy-csr.json
-rw------- 1 root root 1675 4月   9 00:30 kube-proxy-key.pem
-rw-r--r-- 1 root root 1452 4月   9 00:30 kube-proxy.pem
[root@master1 cert]#

3、分发证书到所有节点

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# for node_ip in ${ALL_IPS[@]}
  do
    echo ">>> ${node_ip}"
    scp kube-proxy*.pem root@${node_ip}:/opt/k8s/etc/cert/
  done
>>> 192.168.66.131
kube-proxy-key.pem             100% 1675     1.7MB/s   00:00
kube-proxy.pem                 100% 1452     1.7MB/s   00:00
>>> 192.168.66.132
kube-proxy-key.pem             100% 1675   684.3KB/s   00:00
kube-proxy.pem                 100% 1452   676.7KB/s   00:00
>>> 192.168.66.133
kube-proxy-key.pem             100% 1675     1.0MB/s   00:00
kube-proxy.pem                 100% 1452     1.5MB/s   00:00
>>> 192.168.66.134
kube-proxy-key.pem             100% 1675     1.7MB/s   00:00
kube-proxy.pem                 100% 1452     1.7MB/s   00:00
>>> 192.168.66.135
kube-proxy-key.pem             100% 1675   684.3KB/s   00:00
kube-proxy.pem                 100% 1452   676.7KB/s   00:00
>>> 192.168.66.136
kube-proxy-key.pem             100% 1675     1.0MB/s   00:00
kube-proxy.pem                 100% 1452     1.5MB/s   00:00
[root@master1 cert]#

九、生成插件客户端证书（aggregator.client）

1、准备证书签名请求文件 aggregator-client-csr.json

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cat > aggregator-client-csr.json <<EOF
{
  "CN": "k8s-demo-aggregator",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
 "names": [{
    "C" : "China",
    "ST": "GuangDong",
    "L" : "ShenZhen",
    "O" : "k8s-demo",
    "OU": "jason@vip.qq.com"
 }]
}
EOF
  • CN名称需要用于kube-apiserver的--requestheader-allowed-names参数中，否则后续访问metrics时会提示权限不足

2、生成aggregator-client证书

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cfssl gencert -ca=/opt/k8s/etc/cert/ca.pem \
  -ca-key=/opt/k8s/etc/cert/ca-key.pem  \
  -config=/opt/k8s/etc/cert/ca-config.json  \
  -profile=k8s-demo-client aggregator-client-csr.json | cfssljson -bare aggregator-client
2022/04/09 00:34:53 [INFO] generate received request
2022/04/09 00:34:53 [INFO] received CSR
2022/04/09 00:34:53 [INFO] generating key: rsa-2048
2022/04/09 00:34:54 [INFO] encoded CSR
2022/04/09 00:34:54 [INFO] signed certificate with serial number 405145416807092219279574273423879427429396828806
2022/04/09 00:34:54 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
[root@master1 cert]# ll aggregator-client*
-rw-r--r-- 1 root root 1041 4月   9 00:34 aggregator-client.csr
-rw-r--r-- 1 root root  219 4月   9 00:33 aggregator-client-csr.json
-rw------- 1 root root 1679 4月   9 00:34 aggregator-client-key.pem
-rw-r--r-- 1 root root 1452 4月   9 00:34 aggregator-client.pem
[root@master1 cert]#

3、分发证书到3个master节点

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# for node_ip in ${MASTER_IPS[@]}
  do
    echo ">>> ${node_ip}"
    scp aggregator-client*.pem root@${node_ip}:/opt/k8s/etc/cert/
  done
>>> 192.168.66.131
aggregator-client-key.pem       100% 1679   869.1KB/s   00:00
aggregator-client.pem           100% 1452   657.8KB/s   00:00
>>> 192.168.66.132
aggregator-client-key.pem       100% 1679     1.3MB/s   00:00
aggregator-client.pem           100% 1452     1.8MB/s   00:00
>>> 192.168.66.133
aggregator-client-key.pem       100% 1679     1.1MB/s   00:00
aggregator-client.pem           100% 1452     1.7MB/s   00:00
[root@master1 cert]#

十、生成service account证书

1、准备证书签名请求文件 service-account-csr.json

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cat > service-account-csr.json <<EOF
{
  "CN": "k8s-demo-sa",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
 "names": [{
    "C" : "China",
    "ST": "GuangDong",
    "L" : "ShenZhen",
    "O" : "k8s-demo",
    "OU": "jason@vip.qq.com"
 }]
}
EOF
  • CN名称需要用于kube-apiserver的----service-account*参数中

2、生成service account证书

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# cfssl gencert -ca=/opt/k8s/etc/cert/ca.pem \
  -ca-key=/opt/k8s/etc/cert/ca-key.pem  \
  -config=/opt/k8s/etc/cert/ca-config.json  \
  -profile=k8s-demo-client service-account-csr.json | cfssljson -bare service-account
2022/04/09 12:04:29 [INFO] generate received request
2022/04/09 12:04:29 [INFO] received CSR
2022/04/09 12:04:29 [INFO] generating key: rsa-2048
2022/04/09 12:04:29 [INFO] encoded CSR
2022/04/09 12:04:29 [INFO] signed certificate with serial number 718789939225237580081317156388782928377752240000
2022/04/09 12:04:29 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
[root@master1 cert]# openssl x509 -in service-account.pem -pubkey -noout > service-account.pub
[root@master1 cert]# ll service-account*
-rw-r--r-- 1 root root 1029 4月   9 12:04 service-account.csr
-rw-r--r-- 1 root root  211 4月   9 12:01 service-account-csr.json
-rw------- 1 root root 1679 4月   9 12:04 service-account-key.pem
-rw-r--r-- 1 root root 1440 4月   9 12:04 service-account.pem
-rw-r--r-- 1 root root  451 4月   9 12:08 service-account.pub
[root@master1 cert]#

3、分发证书到3个master节点

[root@master1 ~]# cd /opt/install/cert
[root@master1 cert]# for node_ip in ${MASTER_IPS[@]}
  do
    echo ">>> ${node_ip}"
    scp service-account*.pem root@${node_ip}:/opt/k8s/etc/cert/
    scp service-account*.pub root@${node_ip}:/opt/k8s/etc/cert/
  done
>>> 192.168.66.131
service-account-key.pem           100% 1679     1.6MB/s   00:00
service-account.pem               100% 1440     1.1MB/s   00:00
service-account.pub               100%  451   214.4KB/s   00:00
>>> 192.168.66.132
service-account-key.pem           100% 1679     1.1MB/s   00:00
service-account.pem               100% 1440     1.9MB/s   00:00
service-account.pub               100%  451   304.1KB/s   00:00
>>> 192.168.66.133
service-account-key.pem           100% 1679     1.4MB/s   00:00
service-account.pem               100% 1440     1.9MB/s   00:00
service-account.pub               100%  451   287.2KB/s   00:00
[root@master1 cert]#

参考

  • 先用起来，通过操作实践认识kubernetes(k8s)，积累多了自然就理解了
  • 把理解的知识分享出来，自造福田，自得福缘
  • 追求简单，容易使人理解，知识的上下文也是知识的一部分，例如版本，时间等
  • 欢迎留言交流，也可以提出问题，一般在周末回复和完善文档
  • Jason@vip.qq.com 2022-4-8
avatar
文章
阅读
粉丝