Kubernetes最佳实践之核心服务之etcd入门(一)

云原生生态圈
1,061 阅读4分钟

etcd是kubernetes容器编排服务强依赖的分布式键值存储服务，主要存储kubernetes中主要的元数据信息以及kube-apiserver和各个work节点之间的信息同步的数据，因此学好etcd是学习Kubernetes的基础。

介绍如何快速的部署一个三节点的ETCD集群，这里我们在GITHUB上快速的下载一个etcd资源，然后我们进行解压后，全部在解压后的目录下进行操作。注意解压出来的文件里面有etcd和etcdctl文件

部署etcd节点

在每一个Node节点上设置好环境变量，如下:

TOKEN=token-01
CLUSTER_STATE=new
NAME_1=node1
NAME_2=node2
NAME_3=node3
HOST_1=172.16.132.144
HOST_2=172.16.132.145
HOST_3=172.16.132.146
CLUSTER=${NAME_1}=http://${HOST_1}:2380,${NAME_2}=http://${HOST_2}:2380,${NAME_3}=http://${HOST_3}:2380

在每一个节点上分别运行etcd服务

# 在节点1上进行执行
THIS_NAME=${NAME_1}
THIS_IP=${HOST_1}
./etcd --data-dir=data.etcd --name ${THIS_NAME} \
	--initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://${THIS_IP}:2380 \
	--advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://${THIS_IP}:2379 \
	--initial-cluster ${CLUSTER} \
	--initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}

# 在节点2上进行执行
THIS_NAME=${NAME_2}
THIS_IP=${HOST_2}
./etcd --data-dir=data.etcd --name ${THIS_NAME} \
	--initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://${THIS_IP}:2380 \
	--advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://${THIS_IP}:2379 \
	--initial-cluster ${CLUSTER} \
	--initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}

# 在节点3上进行执行
THIS_NAME=${NAME_3}
THIS_IP=${HOST_3}
./etcd --data-dir=data.etcd --name ${THIS_NAME} \
	--initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://${THIS_IP}:2380 \
	--advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://${THIS_IP}:2379 \
	--initial-cluster ${CLUSTER} \
	--initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}

或者使用etcd的服务发现辅助集群的安装

curl https://discovery.etcd.io/new?size=3
https://discovery.etcd.io/a748540bbcd110d29cbb12c4217bc8a8

# grab this token
TOKEN=token-01
CLUSTER_STATE=new
NAME_1=machine-1
NAME_2=machine-2
NAME_3=machine-3
HOST_1=172.16.132.144
HOST_2=172.16.132.145
HOST_3=172.16.132.146
DISCOVERY=https://discovery.etcd.io/a748540bbcd110d29cbb12c4217bc8a8

THIS_NAME=${NAME_1}
THIS_IP=${HOST_1}
./etcd --data-dir=data.etcd --name ${THIS_NAME} \
	--initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://${THIS_IP}:2380 \
	--advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://${THIS_IP}:2379 \
	--discovery ${DISCOVERY} \
	--initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}

THIS_NAME=${NAME_2}
THIS_IP=${HOST_2}
./etcd --data-dir=data.etcd --name ${THIS_NAME} \
	--initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://${THIS_IP}:2380 \
	--advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://${THIS_IP}:2379 \
	--discovery ${DISCOVERY} \
	--initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}

THIS_NAME=${NAME_3}
THIS_IP=${HOST_3}
./etcd --data-dir=data.etcd --name ${THIS_NAME} \
	--initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://${THIS_IP}:2380 \
	--advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://${THIS_IP}:2379 \
	--discovery ${DISCOVERY} \
	--initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}

现在等到每个节点上的ETCD服务全部运行起来了，我们既可以通过etcdctl命令进行查看集群的状态以及成员列表

# 在任意一个节点上打开一个新的连接终端执行以下命令
export ETCDCTL_API=3
HOST_1=172.16.132.144
HOST_2=172.16.132.145
HOST_3=172.16.132.146
ENDPOINTS=$HOST_1:2379,$HOST_2:2379,$HOST_3:2379

suoper@node1:~/etcd-v3.2.27-linux-amd64$ ./etcdctl --endpoints=$ENDPOINTS member list
3e7e6811e232c854, started, machine-1, http://172.16.132.144:2380, http://172.16.132.144:2379
53e4363fdc66f410, started, machine-2, http://172.16.132.145:2380, http://172.16.132.145:2379
d42aaa003bf00aed, started, machine-3, http://172.16.132.146:2380, http://172.16.132.146:2379

# 友好的输出格式，
suoper@node1:~/etcd-v3.2.27-linux-amd64$ ./etcdctl --write-out="table" --endpoints=$ENDPOINTS member list
+------------------+---------+-----------+----------------------------+----------------------------+
|        ID        | STATUS  |   NAME    |         PEER ADDRS         |        CLIENT ADDRS        |
+------------------+---------+-----------+----------------------------+----------------------------+
| 3e7e6811e232c854 | started | machine-1 | http://172.16.132.144:2380 | http://172.16.132.144:2379 |
| 53e4363fdc66f410 | started | machine-2 | http://172.16.132.145:2380 | http://172.16.132.145:2379 |
| d42aaa003bf00aed | started | machine-3 | http://172.16.132.146:2380 | http://172.16.132.146:2379 |
+------------------+---------+-----------+----------------------------+----------------------------+

访问etcd集群

02_etcdctl_access_etcd_2016051001

put command to write:

etcdctl --endpoints=$ENDPOINTS put foo "Hello World!"

get to read from etcd:

etcdctl --endpoints=$ENDPOINTS get foo
etcdctl --endpoints=$ENDPOINTS --write-out="json" get foo

通过前缀进行查询

03_etcdctl_get_by_prefix_2016050501 
etcdctl --endpoints=$ENDPOINTS put web1 value1
etcdctl --endpoints=$ENDPOINTS put web2 value2
etcdctl --endpoints=$ENDPOINTS put web3 value3

etcdctl --endpoints=$ENDPOINTS get web --prefix

对键值进行删除

04_etcdctl_delete_2016050601 
etcdctl --endpoints=$ENDPOINTS put key myvalue
etcdctl --endpoints=$ENDPOINTS del key

etcdctl --endpoints=$ENDPOINTS put k1 value1
etcdctl --endpoints=$ENDPOINTS put k2 value2
etcdctl --endpoints=$ENDPOINTS del k --prefix

事务性写

txn to wrap multiple requests into one transaction:

05_etcdctl_transaction_2016050501 
etcdctl --endpoints=$ENDPOINTS put user1 bad
etcdctl --endpoints=$ENDPOINTS txn --interactive

compares:
value("user1") = "bad"      

success requests (get, put, delete):
del user1  

failure requests (get, put, delete):
put user1 good

观察(监控)

watch 查看指定KEY的动态的变化:

06_etcdctl_watch_2016050501 
etcdctl --endpoints=$ENDPOINTS watch stock1
etcdctl --endpoints=$ENDPOINTS put stock1 1000

etcdctl --endpoints=$ENDPOINTS watch stock --prefix
etcdctl --endpoints=$ENDPOINTS put stock1 10
etcdctl --endpoints=$ENDPOINTS put stock2 20

租期

lease 给键设置一个生命周期:

07_etcdctl_lease_2016050501 
etcdctl --endpoints=$ENDPOINTS lease grant 300
# lease 2be7547fbc6a5afa granted with TTL(300s)

etcdctl --endpoints=$ENDPOINTS put sample value --lease=2be7547fbc6a5afa
etcdctl --endpoints=$ENDPOINTS get sample

etcdctl --endpoints=$ENDPOINTS lease keep-alive 2be7547fbc6a5afa
etcdctl --endpoints=$ENDPOINTS lease revoke 2be7547fbc6a5afa
# or after 300 seconds
etcdctl --endpoints=$ENDPOINTS get sample

分布式锁

lock for distributed lock:

08_etcdctl_lock_2016050501 
etcdctl --endpoints=$ENDPOINTS lock mutex1

# another client with the same name blocks
etcdctl --endpoints=$ENDPOINTS lock mutex1

elect

elect 用于领导选举:

09_etcdctl_elect_2016050501 
etcdctl --endpoints=$ENDPOINTS elect one p1

# another client with the same name blocks
etcdctl --endpoints=$ENDPOINTS elect one p2

集群的状态

说明初始化集群中每个节点上的信息

10_etcdctl_endpoint_2016050501 
suoper@node1:~/etcd-v3.2.27-linux-amd64$ ./etcdctl --write-out=table --endpoints=$ENDPOINTS endpoint status
+---------------------+------------------+---------+---------+-----------+-----------+------------+
|      ENDPOINT       |        ID        | VERSION | DB SIZE | IS LEADER | RAFT TERM | RAFT INDEX |
+---------------------+------------------+---------+---------+-----------+-----------+------------+
| 172.16.132.144:2379 | 3e7e6811e232c854 |  3.2.27 |   25 kB |     false |         2 |          8 |
| 172.16.132.145:2379 | 53e4363fdc66f410 |  3.2.27 |   25 kB |      true |         2 |          8 |
| 172.16.132.146:2379 | d42aaa003bf00aed |  3.2.27 |   25 kB |     false |         2 |          8 |
+---------------------+------------------+---------+---------+-----------+-----------+------------+

suoper@node1:~/etcd-v3.2.27-linux-amd64$ ./etcdctl --write-out="table" --endpoints=$ENDPOINTS endpoint health
+---------------------+--------+------------+-------+
|      ENDPOINT       | HEALTH |    TOOK    | ERROR |
+---------------------+--------+------------+-------+
| 172.16.132.144:2379 |   true | 3.193782ms |       |
| 172.16.132.145:2379 |   true | 2.154854ms |       |
| 172.16.132.146:2379 |   true | 4.344392ms |       |
+---------------------+--------+------------+-------+

snapshot

snapshot 使用快照进行数据的备份，迁移，以及灾备恢复

11_etcdctl_snapshot_2016051001 
etcdctl --endpoints=$ENDPOINTS snapshot save my.db

Snapshot saved at my.db

etcdctl --write-out=table --endpoints=$ENDPOINTS snapshot status my.db

+---------+----------+------------+------------+
|  HASH   | REVISION | TOTAL KEYS | TOTAL SIZE |
+---------+----------+------------+------------+
| c55e8b8 |        9 |         13 | 25 kB      |
+---------+----------+------------+------------+

migrate

migrate to transform etcd v2 to v3 data:

12_etcdctl_migrate_2016061602 
# write key in etcd version 2 store
export ETCDCTL_API=2
etcdctl --endpoints=http://$ENDPOINT set foo bar

# read key in etcd v2
etcdctl --endpoints=$ENDPOINTS --output="json" get foo

# stop etcd node to migrate, one by one

# migrate v2 data
export ETCDCTL_API=3
etcdctl --endpoints=$ENDPOINT migrate --data-dir="default.etcd" --wal-dir="default.etcd/member/wal"

# restart etcd node after migrate, one by one

# confirm that the key got migrated
etcdctl --endpoints=$ENDPOINTS get /foo

member

member to add,remove,update membership:

13_etcdctl_member_2016062301 
# For each machine
TOKEN=my-etcd-token-1
CLUSTER_STATE=new
NAME_1=etcd-node-1
NAME_2=etcd-node-2
NAME_3=etcd-node-3
HOST_1=10.240.0.13
HOST_2=10.240.0.14
HOST_3=10.240.0.15
CLUSTER=${NAME_1}=http://${HOST_1}:2380,${NAME_2}=http://${HOST_2}:2380,${NAME_3}=http://${HOST_3}:2380

# For node 1
THIS_NAME=${NAME_1}
THIS_IP=${HOST_1}
etcd --data-dir=data.etcd --name ${THIS_NAME} \
	--initial-advertise-peer-urls http://${THIS_IP}:2380 \
	--listen-peer-urls http://${THIS_IP}:2380 \
	--advertise-client-urls http://${THIS_IP}:2379 \
	--listen-client-urls http://${THIS_IP}:2379 \
	--initial-cluster ${CLUSTER} \
	--initial-cluster-state ${CLUSTER_STATE} \
	--initial-cluster-token ${TOKEN}

# For node 2
THIS_NAME=${NAME_2}
THIS_IP=${HOST_2}
etcd --data-dir=data.etcd --name ${THIS_NAME} \
	--initial-advertise-peer-urls http://${THIS_IP}:2380 \
	--listen-peer-urls http://${THIS_IP}:2380 \
	--advertise-client-urls http://${THIS_IP}:2379 \
	--listen-client-urls http://${THIS_IP}:2379 \
	--initial-cluster ${CLUSTER} \
	--initial-cluster-state ${CLUSTER_STATE} \
	--initial-cluster-token ${TOKEN}

# For node 3
THIS_NAME=${NAME_3}
THIS_IP=${HOST_3}
etcd --data-dir=data.etcd --name ${THIS_NAME} \
	--initial-advertise-peer-urls http://${THIS_IP}:2380 \
	--listen-peer-urls http://${THIS_IP}:2380 \
	--advertise-client-urls http://${THIS_IP}:2379 \
	--listen-client-urls http://${THIS_IP}:2379 \
	--initial-cluster ${CLUSTER} \
	--initial-cluster-state ${CLUSTER_STATE} \
	--initial-cluster-token ${TOKEN}

Then replace a member with member remove and member add commands:

# get member ID
export ETCDCTL_API=3
HOST_1=10.240.0.13
HOST_2=10.240.0.14
HOST_3=10.240.0.15
etcdctl --endpoints=${HOST_1}:2379,${HOST_2}:2379,${HOST_3}:2379 member list

# remove the member
MEMBER_ID=278c654c9a6dfd3b
etcdctl --endpoints=${HOST_1}:2379,${HOST_2}:2379,${HOST_3}:2379 \
	member remove ${MEMBER_ID}

# add a new member (node 4)
export ETCDCTL_API=3
NAME_1=etcd-node-1
NAME_2=etcd-node-2
NAME_4=etcd-node-4
HOST_1=10.240.0.13
HOST_2=10.240.0.14
HOST_4=10.240.0.16 # new member
etcdctl --endpoints=${HOST_1}:2379,${HOST_2}:2379 \
	member add ${NAME_4} \
	--peer-urls=http://${HOST_4}:2380

在一个现有的集群中加入一个新的节点，需要使用 --initial-cluster-state existing 选项:


# 提醒: 如果要从同一个快磁盘上添加一个新成员的时候，一定要确保已经删除老成员的数据目录
#
# restart with 'existing' flag
TOKEN=my-etcd-token-1
CLUSTER_STATE=existing
NAME_1=etcd-node-1
NAME_2=etcd-node-2
NAME_4=etcd-node-4
HOST_1=10.240.0.13
HOST_2=10.240.0.14
HOST_4=10.240.0.16 # new member
CLUSTER=${NAME_1}=http://${HOST_1}:2380,${NAME_2}=http://${HOST_2}:2380,${NAME_4}=http://${HOST_4}:2380

THIS_NAME=${NAME_4}
THIS_IP=${HOST_4}
etcd --data-dir=data.etcd --name ${THIS_NAME} \
	--initial-advertise-peer-urls http://${THIS_IP}:2380 \
	--listen-peer-urls http://${THIS_IP}:2380 \
	--advertise-client-urls http://${THIS_IP}:2379 \
	--listen-client-urls http://${THIS_IP}:2379 \
	--initial-cluster ${CLUSTER} \
	--initial-cluster-state ${CLUSTER_STATE} \
	--initial-cluster-token ${TOKEN}

Auth

auth,user,role for authentication:

14_etcdctl_auth_2016062301 
export ETCDCTL_API=3
ENDPOINTS=$HOST_1:2379,$HOST_2:2379,$HOST_3:2379

etcdctl --endpoints=${ENDPOINTS} role add root
etcdctl --endpoints=${ENDPOINTS} role grant-permission root readwrite foo # 给key进行授权
etcdctl --endpoints=${ENDPOINTS} role get root 

etcdctl --endpoints=${ENDPOINTS} user add root
etcdctl --endpoints=${ENDPOINTS} user grant-role root root
etcdctl --endpoints=${ENDPOINTS} user get root

etcdctl --endpoints=${ENDPOINTS} auth enable
# 现在所有的客户端连接都需要进行验证

etcdctl --endpoints=${ENDPOINTS} --user=root:123 put foo bar
etcdctl --endpoints=${ENDPOINTS} get foo
etcdctl --endpoints=${ENDPOINTS} --user=root:123 get foo
etcdctl --endpoints=${ENDPOINTS} --user=root:123 get foo1

或扫描关注二维码，关注更多动态

avatar
文章
阅读
粉丝