为什么要首先配置etcd集群呢,因为etcd集群是Kubernetes的基础,所有数据都存储在这里
1.生成etcd证书
其实这里建议使用主机名,避免ip变化而导致的数据库不可用,但是我习惯用ip,为避免证书看起来不那么乱,所以我将不同的证书放置到不同的目录里面
cat > /etc/ssl/etcd/etcd-csr.json <<EOF
{
"CN": "etcd",
"hosts": [
"127.0.0.1",
"192.168.1.40",
"192.168.1.41",
"192.168.1.42"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "ChengDu",
"L": "ChengDu",
"O": "k8s",
"OU": "dessler"
}
]
}
EOF
cfssl gencert -ca=/etc/ssl/ca.pem \
-ca-key=/etc/ssl/ca-key.pem \
-config=/etc/ssl/ca-config.json \
-profile=kubernetes etcd-csr.json | cfssljson -bare etcd
ls
etcd.csr etcd-csr.json etcd-key.pem etcd.pem
2.分发证书及二进制文件
过程我就省略了,都是些基本的文件传输操作
3.准备证书环境
(自己注意数据保存目录,根据自己的情况决定)
chmod 644 /etc/ssl/etcd/etcd-key.pem
useradd -s /sbin/nologin -M etcd
mkdir -p /var/lib/etcd/
chown -R etcd:etcd /var/lib/etcd/
4.配置etcd的服务
这里需要特别注意就是etcd的名字,集群的3个节点是不能重复的,还有和这个名字和集群的配置(比如我这里的etcd01,etcd02,etcd03)
4.1服务器a配置etcd服务
cat > /usr/lib/systemd/system/etcd.service <<EOF
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
Documentation=https://github.com/coreos
[Service]
User=etcd
Type=notify
WorkingDirectory=/var/lib/etcd/
ExecStart=/usr/bin/etcd \\
--data-dir=/var/lib/etcd \\
--name=etcd01 \\
--cert-file=/etc/ssl/etcd/etcd.pem \\
--key-file=/etc/ssl/etcd/etcd-key.pem \\
--trusted-ca-file=/etc/ssl/ca.pem \\
--peer-cert-file=/etc/ssl/etcd/etcd.pem \\
--peer-key-file=/etc/ssl/etcd/etcd-key.pem \\
--peer-trusted-ca-file=/etc/ssl/ca.pem \\
--peer-client-cert-auth \\
--client-cert-auth \\
--listen-peer-urls=https://192.168.1.40:2380 \\
--initial-advertise-peer-urls=https://192.168.1.40:2380 \\
--listen-client-urls=https://192.168.1.40:2379,http://127.0.0.1:2379 \\
--advertise-client-urls=https://192.168.1.40:2379 \\
--initial-cluster-token=etcd-cluster-0 \\
--initial-cluster=etcd01=https://192.168.1.40:2380,etcd02=https://192.168.1.41:2380,etcd03=https://192.168.1.42:2380 \\
--initial-cluster-state=new
Restart=on-failure
RestartSec=5
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
EOF
4.2服务器a配置etcd服务
cat > /usr/lib/systemd/system/etcd.service <<EOF
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
Documentation=https://github.com/coreos
[Service]
User=etcd
Type=notify
WorkingDirectory=/var/lib/etcd/
ExecStart=/usr/bin/etcd \\
--data-dir=/var/lib/etcd \\
--name=etcd02 \\
--cert-file=/etc/ssl/etcd/etcd.pem \\
--key-file=/etc/ssl/etcd/etcd-key.pem \\
--trusted-ca-file=/etc/ssl/ca.pem \\
--peer-cert-file=/etc/ssl/etcd/etcd.pem \\
--peer-key-file=/etc/ssl/etcd/etcd-key.pem \\
--peer-trusted-ca-file=/etc/ssl/ca.pem \\
--peer-client-cert-auth \\
--client-cert-auth \\
--listen-peer-urls=https://192.168.1.41:2380 \\
--initial-advertise-peer-urls=https://192.168.1.41:2380 \\
--listen-client-urls=https://192.168.1.41:2379,http://127.0.0.1:2379 \\
--advertise-client-urls=https://192.168.1.41:2379 \\
--initial-cluster-token=etcd-cluster-0 \\
--initial-cluster=etcd01=https://192.168.1.40:2380,etcd02=https://192.168.1.41:2380,etcd03=https://192.168.1.42:2380 \\
--initial-cluster-state=new
Restart=on-failure
RestartSec=5
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
EOF
4.3服务器a配置etcd服务
cat > /usr/lib/systemd/system/etcd.service <<EOF
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
Documentation=https://github.com/coreos
[Service]
User=etcd
Type=notify
WorkingDirectory=/var/lib/etcd/
ExecStart=/usr/bin/etcd \\
--data-dir=/var/lib/etcd \\
--name=etcd03 \\
--cert-file=/etc/ssl/etcd/etcd.pem \\
--key-file=/etc/ssl/etcd/etcd-key.pem \\
--trusted-ca-file=/etc/ssl/ca.pem \\
--peer-cert-file=/etc/ssl/etcd/etcd.pem \\
--peer-key-file=/etc/ssl/etcd/etcd-key.pem \\
--peer-trusted-ca-file=/etc/ssl/ca.pem \\
--peer-client-cert-auth \\
--client-cert-auth \\
--listen-peer-urls=https://192.168.1.42:2380 \\
--initial-advertise-peer-urls=https://192.168.1.42:2380 \\
--listen-client-urls=https://192.168.1.42:2379,http://127.0.0.1:2379 \\
--advertise-client-urls=https://192.168.1.42:2379 \\
--initial-cluster-token=etcd-cluster-0 \\
--initial-cluster=etcd01=https://192.168.1.40:2380,etcd02=https://192.168.1.41:2380,etcd03=https://192.168.1.42:2380 \\
--initial-cluster-state=new
Restart=on-failure
RestartSec=5
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
EOF
5.检查集群状态
etcdctl --endpoints=https://192.168.1.40:2379,https://192.168.1.41:2379,https://192.168.1.42:2379 \
> --cert-file=/etc/ssl/etcd/etcd.pem \
> --ca-file=/etc/ssl/ca.pem \
> --key-file=/etc/ssl/etcd/etcd-key.pem \
> cluster-health
member e42b05792291d17 is healthy: got healthy result from https://192.168.1.41:2379
member 68326dea8aa5233d is healthy: got healthy result from https://192.168.1.40:2379
member e10fc861b3b13bc0 is healthy: got healthy result from https://192.168.1.42:2379
注意的几个地方:name 需要一一对应,ip地址也容易出错
集群状态正常,etd集群创建成果
- 说明:
- User:指定以 k8s 账户运行
- WorkingDirectory、--data-dir:指定工作目录和数据目录为/var/lib/etcd,需在启动服务前创建这个目录
- --name:指定节点名称,当--initial-cluster-state 值为 new 时,--name 的参数值必须位于 --initial-cluster 列表中
- --cert-file、--key-file:etcd server 与 client 通信时使用的证书和私钥
- --trusted-ca-file:签名 client 证书的 CA 证书,用于验证 client 证书
- --peer-cert-file、--peer-key-file:etcd 与 peer 通信使用的证书和私钥
- --peer-trusted-ca-file:签名 peer 证书的 CA 证书,用于验证 peer 证书