准备阶段

使用 VMware 创建出三个虚机环境，其中一个作为 K8S 集群中的 Master 节点（高可用部署建议三个 Master 节点），另外两个作为 Work 节点（Work 节点可随时动态加入集群）

具体配置如下：

类型	IP地址	系统版本	配置
Master主节点	192.168.19.136	CentOS Linux release 7.9.2009 (Core)	4核4G
Work工作节点1	192.168.19.137	CentOS Linux release 7.9.2009 (Core)	4核4G
Work工作节点2	192.168.19.138	CentOS Linux release 7.9.2009 (Core)	4核4G

使用 Xshell 连接，后续都使用 shell 命令进行操作

初始化

注：需要在所有节点上执行初始化步骤

初始化之设置主机名

hostnamectl set-hostname master # 在 master 节点执行
hostnamectl set-hostname work1 # 在 work1 节点执行
hostnamectl set-hostname work2 # 在 work2 节点执行

添加 DNS 解析

cat >> /etc/hosts <<EOF
192.168.19.136 master
192.168.19.137 work1
192.168.19.138 work2
EOF

重新登陆后生效

初始化之关闭防火墙

关闭防火墙，清理防火墙规则，设置默认转发策略

systemctl stop firewalld
systemctl disable firewalld
iptables -F && iptables -X && iptables -F -t nat && iptables -X -t nat
iptables -P FORWARD ACCEPT

初始化之关闭 swap 分区

关闭 swap 分区，否则 kubelet 会启动失败

swapoff -a
sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

初始化之关闭 SELinux

关闭 SELinux，否则 kubelet 挂载目录时可能报错 Permission denied

setenforce 0
sed -i 's/^SELINUX=.*/SELINUX=disabled/' /etc/selinux/config

初始化之安装 Docker

使用 yum 安装

sudo yum install -y yum-utils device-mapper-persistent-data lvm2
sudo yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
sudo yum install -y docker-ce-18.09.9 docker-ce-cli-18.09.9 containerd.io

普通用户（ root 用户，请忽略）需要设置成非 root 用户也能执行 docker ，需要将用户加入 docker 组 （例如你的登录用户名是 togettoyou ）

sudo usermod -aG docker togettoyou # 需要重启生效

配置阿里镜像，进入 cr.console.aliyun.com/cn-hangzhou… 申请专属镜像加速器，并设置 systemd 为 Docker 的驱动程序

sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{
  "registry-mirrors": ["改为你申请的加速器地址"],
  "exec-opts": ["native.cgroupdriver=systemd"]
}
EOF

重载 daemon、重启 docker 、启用 docker 自启服务

sudo systemctl daemon-reload
sudo service docker restart
sudo systemctl enable docker.service

下载 Docker Compose

curl -L https://get.daocloud.io/docker/compose/releases/download/1.25.4/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose

配置执行权限

sudo chmod +x /usr/local/bin/docker-compose

检查是否安装成功

docker -v
docker-compose -v

初始化之安装 kubelet kubeadm kubectl

cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

sudo yum install -y kubelet-1.16.7 kubeadm-1.16.7 kubectl-1.16.7

检查是否安装成功：

kubelet --version
kubeadm version
kubectl version

初始化之修改内核的运行参数

cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables  = 1
net.ipv4.ip_forward                 = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF

# 应用配置
sysctl --system

初始化之配置 kubelet

sudo mkdir -p /etc/systemd/system/kubelet.service.d
sudo tee /etc/systemd/system/kubelet.service.d/10-proxy-ipvs.conf <<-'EOF'
# 启用 ipvs 相关内核模块
[Service]
ExecStartPre=-/sbin/modprobe ip_vs
ExecStartPre=-/sbin/modprobe ip_vs_rr
ExecStartPre=-/sbin/modprobe ip_vs_wrr
ExecStartPre=-/sbin/modprobe ip_vs_sh
EOF
sudo systemctl daemon-reload
systemctl enable kubelet.service

Master 节点部署

初始化 master 节点：

sudo kubeadm init --image-repository registry.cn-hangzhou.aliyuncs.com/google_containers \
      --kubernetes-version v1.16.7 \
      --pod-network-cidr 10.244.0.0/16 \
      --v 5 \
      --ignore-preflight-errors=all

如遇失败，可以使用 sudo kubeadm reset 恢复重设后，根据错误提示解决后重新部署。

根据提示，要使非 root 用户可以运行 kubectl，请运行以下命令 ：

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

或者，如果你是 root 用户，则可以运行 ：

export KUBECONFIG=/etc/kubernetes/admin.conf

也可以永久加入到环境变量中：

sudo tee /etc/profile <<-'EOF'
export KUBECONFIG=/etc/kubernetes/admin.conf
EOF

source /etc/profile

添加 flannel 网络插件：

# 添加 flannel 网络插件，需要在 kubeadm init 时设置 --pod-network-cidr=10.244.0.0/16
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/v0.14.0-rc1/Documentation/kube-flannel.yml

网络原因可能无法加载 flannel yml url，可使用以下命令从标准输入创建：

cat <<EOF | kubectl apply -f -
---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: psp.flannel.unprivileged
  annotations:
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: docker/default
    seccomp.security.alpha.kubernetes.io/defaultProfileName: docker/default
    apparmor.security.beta.kubernetes.io/allowedProfileNames: runtime/default
    apparmor.security.beta.kubernetes.io/defaultProfileName: runtime/default
spec:
  privileged: false
  volumes:
  - configMap
  - secret
  - emptyDir
  - hostPath
  allowedHostPaths:
  - pathPrefix: "/etc/cni/net.d"
  - pathPrefix: "/etc/kube-flannel"
  - pathPrefix: "/run/flannel"
  readOnlyRootFilesystem: false
  # Users and groups
  runAsUser:
    rule: RunAsAny
  supplementalGroups:
    rule: RunAsAny
  fsGroup:
    rule: RunAsAny
  # Privilege Escalation
  allowPrivilegeEscalation: false
  defaultAllowPrivilegeEscalation: false
  # Capabilities
  allowedCapabilities: ['NET_ADMIN', 'NET_RAW']
  defaultAddCapabilities: []
  requiredDropCapabilities: []
  # Host namespaces
  hostPID: false
  hostIPC: false
  hostNetwork: true
  hostPorts:
  - min: 0
    max: 65535
  # SELinux
  seLinux:
    # SELinux is unused in CaaSP
    rule: 'RunAsAny'
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: flannel
rules:
- apiGroups: ['extensions']
  resources: ['podsecuritypolicies']
  verbs: ['use']
  resourceNames: ['psp.flannel.unprivileged']
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - nodes
  verbs:
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - nodes/status
  verbs:
  - patch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: flannel
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: flannel
subjects:
- kind: ServiceAccount
  name: flannel
  namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: flannel
  namespace: kube-system
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: kube-flannel-cfg
  namespace: kube-system
  labels:
    tier: node
    app: flannel
data:
  cni-conf.json: |
    {
      "name": "cbr0",
      "cniVersion": "0.3.1",
      "plugins": [
        {
          "type": "flannel",
          "delegate": {
            "hairpinMode": true,
            "isDefaultGateway": true
          }
        },
        {
          "type": "portmap",
          "capabilities": {
            "portMappings": true
          }
        }
      ]
    }
  net-conf.json: |
    {
      "Network": "10.244.0.0/16",
      "Backend": {
        "Type": "vxlan"
      }
    }
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: kube-flannel-ds
  namespace: kube-system
  labels:
    tier: node
    app: flannel
spec:
  selector:
    matchLabels:
      app: flannel
  template:
    metadata:
      labels:
        tier: node
        app: flannel
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: kubernetes.io/os
                operator: In
                values:
                - linux
      hostNetwork: true
      priorityClassName: system-node-critical
      tolerations:
      - operator: Exists
        effect: NoSchedule
      serviceAccountName: flannel
      initContainers:
      - name: install-cni
        image: quay.io/coreos/flannel:v0.14.0-rc1
        command:
        - cp
        args:
        - -f
        - /etc/kube-flannel/cni-conf.json
        - /etc/cni/net.d/10-flannel.conflist
        volumeMounts:
        - name: cni
          mountPath: /etc/cni/net.d
        - name: flannel-cfg
          mountPath: /etc/kube-flannel/
      containers:
      - name: kube-flannel
        image: quay.io/coreos/flannel:v0.14.0-rc1
        command:
        - /opt/bin/flanneld
        args:
        - --ip-masq
        - --kube-subnet-mgr
        resources:
          requests:
            cpu: "100m"
            memory: "50Mi"
          limits:
            cpu: "100m"
            memory: "50Mi"
        securityContext:
          privileged: false
          capabilities:
            add: ["NET_ADMIN", "NET_RAW"]
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        volumeMounts:
        - name: run
          mountPath: /run/flannel
        - name: flannel-cfg
          mountPath: /etc/kube-flannel/
      volumes:
      - name: run
        hostPath:
          path: /run/flannel
      - name: cni
        hostPath:
          path: /etc/cni/net.d
      - name: flannel-cfg
        configMap:
          name: kube-flannel-cfg
EOF

Work 节点加入集群

在 work1 和 work2 节点通过执行 master 中 kubeadm init 步骤打印出的 kubeadm join 命令来加入集群：

kubeadm join 192.168.19.136:6443 --token gtv9xk.e5n390w7ry70029t \
    --discovery-token-ca-cert-hash sha256:e4a9199f2b6138251efc0b6d55eb7d1ef81287f522dae2ab1a6a90bba4973001

查看集群

在 master 节点执行

kubectl get nodes -o wide

如果在 work 节点也想使用 kubectl 工具，需要将 master 节点中的 /etc/kubernetes/admin.conf 拷贝到 work 节点下

scp /etc/kubernetes/admin.conf root@work1:/etc/kubernetes/admin.conf
scp /etc/kubernetes/admin.conf root@work2:/etc/kubernetes/admin.conf

到这里就成功搭建了拥有 一个Master 节点，两个 Work 节点的 Kubernetes 集群。

测试集群

在集群中测试部署 nginx 服务，在 master 节点运行以下命令创建

kubectl create namespace test-zjh # 创建命名空间

cat <<EOF | kubectl apply -f -
apiVersion: apps/v1 # API版本
kind: Deployment # 资源对象类型为 Deployment
metadata:
  name: zjh-nginx # Deployment的名称
  namespace: test-zjh # 命名空间
spec:
  selector: # 定位需要管理的 Pod，通过Pod的labels标签定位
    matchLabels:
      app: test-nginx
  replicas: 2 # 要部署的个数，k8s自动扩容分配
  template: # 要部署的 Pod
    metadata:
      labels:
        app: test-nginx
    spec:
      containers:
        - name: web
          image: nginx:latest
          ports:
            - containerPort: 80

---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service # Service的名称
  namespace: test-zjh # 命名空间
spec:
  ports:
    - port: 80 # Service虚端口
      targetPort: 80 # 容器端口
      nodePort: 30088 # 暴露端口
  selector: # 指定如何选择 Pod
    app: test-nginx
  type: NodePort # 指定为Node的IP地址
EOF

可以看见，分别在 work1 和 work2 创建了 pod，浏览器访问

最后

感谢您的阅读，更多分享请关注公众号【寻寻觅觅的Gopher】