二进制安装 ETCD 集群

胜似代码仔
344 阅读10分钟

下载二进制文件

下载地址：github.com/etcd-io/etc…

将对应架构的二进制文件下载到服务器上

ls
etcd-v3.5.17-linux-amd64.tar.gz

安装 etcd 二进制文件

解压缩文件

tar -zxvf etcd-v3.5.17-linux-amd64.tar.gz 
etcd-v3.5.17-linux-amd64/
etcd-v3.5.17-linux-amd64/etcd
etcd-v3.5.17-linux-amd64/etcdctl
etcd-v3.5.17-linux-amd64/etcdutl
etcd-v3.5.17-linux-amd64/README.md
etcd-v3.5.17-linux-amd64/README-etcdctl.md
etcd-v3.5.17-linux-amd64/READMEv2-etcdctl.md
etcd-v3.5.17-linux-amd64/README-etcdutl.md
etcd-v3.5.17-linux-amd64/Documentation/
etcd-v3.5.17-linux-amd64/Documentation/README.md
etcd-v3.5.17-linux-amd64/Documentation/dev-guide/
etcd-v3.5.17-linux-amd64/Documentation/dev-guide/apispec/
etcd-v3.5.17-linux-amd64/Documentation/dev-guide/apispec/swagger/
etcd-v3.5.17-linux-amd64/Documentation/dev-guide/apispec/swagger/rpc.swagger.json
etcd-v3.5.17-linux-amd64/Documentation/dev-guide/apispec/swagger/v3election.swagger.json
etcd-v3.5.17-linux-amd64/Documentation/dev-guide/apispec/swagger/v3lock.swagger.json

cd etcd-v3.5.17-linux-amd64 && ls
Documentation  etcd  etcdctl  etcdutl  README-etcdctl.md  README-etcdutl.md  README.md  READMEv2-etcdctl.md

将 etcd、etcdctl、etcdutl 三个二进制文件复制到 /usr/local/bin

sudo cp etcd* /usr/local/bin/

ls /usr/local/bin/etcd*
/usr/local/bin/etcd  /usr/local/bin/etcdctl  /usr/local/bin/etcdutl

查看 etcd 版本，二进制文件安装成功

etcd -version
etcd Version: 3.5.17
Git SHA: 507c0de
Go Version: go1.22.9
Go OS/Arch: linux/amd64

签发 kubernetes 中间证书

先用根证书签发一个中间CA证书，用于签发所有部署 k8s 所需要用到的终端证书

  • 先创建一个证书配置文件
cat config.json
{
    "signing": {
        "default": {
            "expiry": "8760h"
        },
        "profiles": {
            "kubernetes": {
                "usages": [
                    "signing",
                    "digital signature",
                    "key encipherment",
                    "cert sign",
                    "crl sign"
                ],
                "expiry": "43800h",
                "ca_constraint": {
                    "is_ca": true,
                    "max_path_len": 0
                }
            }
        }
    }
}
  • 创建 k8s 证书请求文件
cat kubernetes-ca-csr.json 
{
    "CN": "Kubernetes CA",
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "ST": "Guangdong",
            "L": "Guangzhou",
            "O": "Chillcy",
            "OU": "technology"
        }
    ]
}
  • 生成 kubernetes 中间证书
cfssl genkey -initca kubernetes-ca-csr.json | cfssljson -bare kubernetes-ca
2024/11/20 10:50:09 [INFO] generate received request
2024/11/20 10:50:09 [INFO] received CSR
2024/11/20 10:50:09 [INFO] generating key: rsa-2048
2024/11/20 10:50:10 [INFO] encoded CSR
2024/11/20 10:50:10 [INFO] signed certificate with serial number 229366379620715566146038720743393820010425636389

ls kubernetes*
kubernetes-ca.csr  kubernetes-ca-csr.json  kubernetes-ca-key.pem  kubernetes-ca.pem
  • 使用 Root CA 签发中间证书
cfssl sign -ca=root-ca.pem -ca-key=root-ca-key.pem -config=config.json -profile=kubernetes kubernetes-ca.csr | cfssljson -bare kubernetes-ca
2024/11/20 10:54:42 [INFO] signed certificate with serial number 631854689318449724773929915069298054864094763445
  • 查看证书信息
openssl x509 -in kubernetes-ca.pem -text -noout
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            6e:ad:58:1c:96:05:9b:4b:9e:ed:79:48:cc:4b:05:9d:76:c6:19:b5
        Signature Algorithm: sha512WithRSAEncryption
        Issuer: C = CN, ST = Guangdong, L = Guangzhou, O = Chillcy, OU = Technology, CN = Root CA
        Validity
            Not Before: Nov 20 02:50:00 2024 GMT
            Not After : Nov 19 02:50:00 2029 GMT
        Subject: C = CN, ST = Guangdong, L = Guangzhou, O = Chillcy, OU = technology, CN = Kubernetes CA
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:a2:f5:4d:7a:74:0f:a0:37:4d:cb:ee:e5:05:9a:
                    98:e0:b8:cf:bb:7a:69:2e:89:b9:c8:69:54:fb:2d:
                    43:02:99:0b:2d:da:93:ff:cd:fb:2b:28:0e:98:b8:
                    e2:dd:64:12:11:cc:15:0e:c5:ae:c4:3e:b2:b4:72:
                    7e:91:34:cc:01:91:a9:dd:d7:a6:7d:57:20:49:7b:
                    33:93:b8:cf:de:ef:f1:24:2f:05:67:da:ba:e1:7c:
                    2f:61:73:f0:22:54:a0:da:8d:21:18:fb:2b:37:6c:
                    c3:0e:87:b3:83:af:16:24:8e:e3:7c:b8:cb:1d:b9:
                    9f:f4:30:13:d8:c4:46:df:05:6e:ca:89:37:bf:13:
                    48:6f:8b:4e:0d:60:74:d5:76:1d:d1:a8:06:60:31:
                    ef:89:74:c3:77:9e:79:7f:a2:5b:ab:4f:2d:a5:5a:
                    0b:7c:c7:d3:e9:67:4e:47:8d:52:7e:05:c3:b7:81:
                    cf:cc:0c:81:13:0d:89:ee:8e:95:00:f8:5b:94:4d:
                    74:cd:9f:3b:43:3f:18:9e:81:99:fa:d7:35:17:ff:
                    7c:cb:7d:3a:c1:8d:83:ab:b1:62:4a:9e:af:e5:5d:
                    1b:fb:86:e9:fd:49:f5:90:4d:08:47:7f:4c:e8:95:
                    4e:f1:6f:71:86:a3:da:94:bd:2a:d9:dc:dc:7b:bf:
                    02:6b
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment, Certificate Sign, CRL Sign
            X509v3 Basic Constraints: critical
                CA:TRUE
            X509v3 Subject Key Identifier: 
                18:A0:A8:E0:1F:5A:A4:67:32:17:6E:49:90:C8:16:D3:C5:EF:55:2B
            X509v3 Authority Key Identifier: 
                60:6E:3D:27:E7:EE:41:1E:F7:EC:10:74:18:F5:11:97:D5:DD:10:74
    Signature Algorithm: sha512WithRSAEncryption
    Signature Value:
        3c:25:97:65:ee:57:c7:8e:d1:0f:d6:53:51:8f:a9:f4:97:23:
        60:a6:2d:e5:16:35:f2:6c:7c:9f:24:aa:99:44:4c:f7:ae:cb:
        f6:b8:98:6c:f1:a5:f5:21:71:08:00:4b:e5:a4:63:20:25:5d:
        d8:14:6a:94:a2:7b:86:40:41:94:de:e1:86:90:f3:8b:08:1e:
        f2:3f:ba:55:72:d8:4e:5e:6b:98:cf:b5:01:50:f9:c4:36:1b:
        73:55:ea:86:ab:4d:81:2c:f3:26:f4:a8:1b:b5:9c:e0:6a:24:
        63:2f:49:5c:37:d3:f7:4b:4f:8d:e8:b9:b2:1b:58:8e:cc:9c:
        18:29:5a:5a:93:b3:4b:af:5b:4b:d7:dd:cd:2e:c5:e6:56:9f:
        82:6d:71:58:20:2b:49:35:31:85:e4:6d:b6:19:6c:2d:8e:a4:
        c3:8e:b0:5c:e4:eb:a4:3c:ca:b8:93:e3:22:ab:90:90:2c:42:
        e5:75:45:49:a4:15:24:57:bf:6a:a9:32:33:f0:7b:42:c5:45:
        ac:6d:ca:dc:d9:d2:b4:dc:b9:2e:03:4a:66:f8:a8:df:57:54:
        cb:57:0c:14:d7:ec:76:64:2a:2d:f9:0b:34:7f:44:0a:6b:c7:
        85:e0:11:72:fe:fd:2e:b1:5f:44:5f:4b:d4:ed:a6:c1:ac:6e:
        3f:64:0b:97:ca:24:b9:6b:96:79:97:04:89:2e:b9:15:fe:34:
        82:5c:e7:fe:7c:1d:fc:25:4e:59:2b:57:2b:dd:76:d8:e7:af:
        6e:ae:3e:1a:4f:e0:08:5e:80:43:24:ba:90:5c:63:99:1c:09:
        30:56:2c:42:25:d2:7f:3e:6e:c2:c4:d6:d4:af:ad:71:01:6c:
        f2:fd:fc:9a:49:14:44:5a:b9:cb:72:f2:cb:3f:86:99:48:1f:
        6b:ef:af:37:cd:20:a0:c9:57:39:1e:4d:81:c1:4a:d9:7c:d1:
        d9:32:7d:0e:fd:4d:2d:9d:e1:89:d1:fc:96:8e:0c:b0:29:b4:
        a9:06:48:c3:70:ba:5e:1d:75:69:0c:a6:b2:67:36:cb:58:d6:
        66:b1:82:78:a0:58:74:73:c7:e9:17:34:a5:3f:bb:f4:e5:dd:
        de:7a:fc:a8:d0:4b:61:f6:ff:54:83:2c:d9:de:0a:ae:ee:4e:
        db:3b:f3:c0:11:5f:f5:ac:01:87:53:0c:17:2a:b8:f9:b7:89:
        97:15:14:ca:cc:cb:59:78:be:eb:73:e4:e5:71:d4:08:69:9c:
        2f:6a:29:39:7e:8d:4d:ae:58:7c:66:12:75:5c:45:71:40:37:
        a9:51:1f:ed:6d:10:89:ac:ae:63:5e:84:9b:d9:6a:59:0c:36:
        6f:2a:c2:25:92:84:39:db
  • 验证证书链
openssl verify -CAfile root-ca.pem kubernetes-ca.pem
kubernetes-ca.pem: OK

使用 kubernetes 中间证书签发 etcd 证书

  • 证书生成配置中添加 etcd 证书配置
cat config.json 
{
    "signing": {
        "default": {
            "expiry": "8760h"
        },
        "profiles": {
            "kubernetes": {
                "usages": [
                    "signing",
                    "digital signature",
                    "key encipherment",
                    "cert sign",
                    "crl sign"
                ],
                "expiry": "43800h",
                "ca_constraint": {
                    "is_ca": true,
                    "max_path_len": 0
                }
            },
            "etcd": {
                "usages": [
                    "digital signature", 
                    "key encipherment", 
                    "server auth",
                    "client auth"
                ],
                "expiry": "8760h"
            }
        }
    }
}
  • 创建 etcd 证书请求配置文件
cat etcd-csr.json 
{
    "CN": "etcd",
    "hosts": [
        "127.0.0.1",
        "localhost",
        "10.0.30.26",
        "10.0.30.31",
        "10.0.30.10"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "ST": "Guangdong",
            "L": "Guangzhou",
            "O": "Chillcy",
            "OU": "technology"
        }
    ]
}
  • 使用 kubernetes 证书签发 etcd 证书
cfssl gencert -ca=kubernetes-ca.pem -ca-key=kubernetes-ca-key.pem -config=config.json -profile=etcd etcd-csr.json | cfssljson -bare etcd
2024/11/22 10:05:41 [INFO] generate received request
2024/11/22 10:05:41 [INFO] received CSR
2024/11/22 10:05:41 [INFO] generating key: rsa-2048
2024/11/22 10:05:42 [INFO] encoded CSR
2024/11/22 10:05:42 [INFO] signed certificate with serial number 33753521303831965221471363974704161914649032879

ls etcd*
etcd.csr  etcd-csr.json  etcd-key.pem  etcd.pem

生成证书链并分发证书文件

cat kubernetes-ca.pem root-ca.pem > kubernetes-chain.pem

将 etcd 相关的证书文件和证书链都分发到要部署 etcd 集群的节点上

在各个节点上创建各自的 systemd 配置文件

cat /etc/systemd/system/etcd.service 
[Unit]
Description=etcd key-value store
Documentation=https://github.com/etcd-io/etcd
After=network.target

[Service]
Type=notify
Environment=ETCD_DATA_DIR=/var/lib/etcd
Environment=ETCD_NAME=etcd-1
ExecStart=/usr/local/bin/etcd \
  --listen-client-urls=https://10.0.30.26:2379 \
  --advertise-client-urls=https://10.0.30.26:2379 \
  --listen-peer-urls=https://10.0.30.26:2380 \
  --initial-advertise-peer-urls=https://10.0.30.26:2380 \
  --initial-cluster=etcd-1=https://10.0.30.26:2380,etcd-2=https://10.0.30.31:2380,etcd-3=https://10.0.30.10:2380 \
  --initial-cluster-token=etcd-cluster \
  --initial-cluster-state=new \
  --cert-file=/home/we8/k8s/etcd/etcd.pem \
  --key-file=/home/we8/k8s/etcd/etcd-key.pem \
  --trusted-ca-file=/home/we8/k8s/kubernetes-chain.pem \
  --client-cert-auth=true \
  --peer-cert-file=/home/we8/k8s/etcd/etcd.pem \
  --peer-key-file=/home/we8/k8s/etcd/etcd-key.pem  \
  --peer-trusted-ca-file=/home/we8/k8s/kubernetes-chain.pem \
  --peer-client-cert-auth=true
Restart=always
RestartSec=10s
LimitNOFILE=40000

[Install]
WantedBy=multi-user.target

ETCD_DATA_DIR etcd数据存放目录需要提前创建

每个节点启动 etcd 守护进程

sudo systemctl daemon-reload 
sudo systemctl enable etcd.service 
sudo systemctl start etcd.service
sudo systemctl status etcd.service

检查 etcd 集群部署成功

etcdctl endpoint health --endpoints=https://10.0.30.26:2379,https://10.0.30.31:2379,https://10.0.30.10:2379 --cacert=/home/we8/k8s/kubernetes-chain.pem --cert=/home/we8/k8s/etcd/etcd-server.pem --key=/home/we8/k8s/etcd/etcd-server-key.pem
https://10.0.30.26:2379 is healthy: successfully committed proposal: took = 9.451314ms
https://10.0.30.10:2379 is healthy: successfully committed proposal: took = 9.591302ms
https://10.0.30.31:2379 is healthy: successfully committed proposal: took = 12.274668ms

etcdctl member list --endpoints=https://10.0.30.26:2379,https://10.0.30.31:2379,https://10.0.30.10:2379 --cacert=/home/we8/k8s/kubernetes-chain.pem --cert=/home/we8/k8s/etcd/etcd-server.pem --key=/home/we8/k8s/etcd/etcd-server-key.pem
76ab2b1ab1144f01, started, etcd-3, https://10.0.30.10:2380, https://10.0.30.10:2379, false
9461193c4114ffda, started, etcd-1, https://10.0.30.26:2380, https://10.0.30.26:2379, false
bc3fd3a67c0778ff, started, etcd-2, https://10.0.30.31:2380, https://10.0.30.31:2379, false

etcdctl put greeting "Hello, etcd" --endpoints=https://10.0.30.26:2379,https://10.0.30.31:2379,https://10.0.30.10:2379 --cacert=/home/we8/k8s/kubernetes-chain.pem --cert=/home/we8/k8s/etcd/etcd-server.pem --key=/home/we8/k8s/etcd/etcd-server-key.pem
OK

etcdctl get greeting --endpoints=https://10.0.30.26:2379,https://10.0.30.31:2379,https://10.0.30.10:2379 --cacert=/home/we8/k8s/kubernetes-chain.pem --cert=/home/we8/k8s/etcd/etcd-server.pem --key=/home/we8/k8s/etcd/etcd-server-key.pem
greeting
Hello, etcd
avatar
文章
阅读
粉丝