手把手教你搭建生产级别kubernetes集群前言半年前，写过如何从零搭建K8S，那么为什么现在又写呢？em...这次采

前言

半年前，写过如何从零搭建K8S，那么为什么现在又写呢？

em...这次采用 Kubespray 搭建k8s集群(官方推荐生产环境采用这种方式)

几种方案对比：

部署方案	优点	缺点
Kubeadm	官方出品	部署较麻烦、升级麻烦、GitHub Star较少
Kubespray	官方出品、部署较简单、适用于生产	镜像需科学上网
Kops	适用于生产、GitHub Star最多	无法跨平台
Rancher	适用于生产、很完善、UI交互很棒	付费

折腾了一周之后，坑我差不多踩完了，本文将带领大家一起避坑

前期准备

最好准备4台机器，当然我只准备了3台，不要问我什么(穷，机器跑不起)

主机	系统版本	配置	IP
Ansible	CentOS 8.3	2核2G	192.168.1.103
Master	CentOS 8.3	2核2G	192.168.1.104
Node1	CentOS 8.3	2核2G	192.168.1.105
Node2	CentOS 8.3	2核2G	192.168.1.106

Node2，我并没有添加(内存爆表了)，你们可以自己添加

本次部署，采用KubeSpray版本为 2.15.0

Master、Node节点的操作

对Ansible、Master、Node节点执行以下几个操作：关闭防火墙、禁用SELinux、禁用交换分区

关闭防火墙

# 停止防火墙
[root@k8s-master k8s]$ systemctl stop firewalld.service

# 禁止firewall开机启动
[root@k8s-master k8s]$ systemctl disable firewalld.service

# 查看防火墙状态
[root@k8s-master k8s]$ firewall-cmd --state
not running

禁用SELinux

# 修改/etc/selinux/config, 设置SELINUX=disabled. 如下图
[root@k8s-master k8s]$ vim /etc/selinux/config
SELINUX=disabled

# 查看SELinux状态
[root@k8s-master k8s]$ sestatus
SELinux status:                 disabled

禁用交换分区

方法一：修改配置

# 编辑/etc/fstab, 将swap注释掉. 重启机器.
[root@k8s-master k8s]$ vim /etc/fstab
#/dev/mapper/cl-swap     swap                    swap    defaults        0 0

命令关掉，重启后会失效

# 关掉
[root@k8s-master k8s]$ swapoff -a
# 查看交换分区状态
[root@k8s-master k8s]$ free -m
              total        used        free      shared  buff/cache   available
Mem:           1816        1488          88          17         239         158
Swap:             0           0           0

若没有关掉，则会报，如下图

[ERROR Swap]: running with swap on is not supported. Please disable swap

没有禁用交换分区，报错的情况

配置Ansibe连接其他机器

Ansibe相当于脚本，需要操作Master、Node节点，那需要通过SSH的方式去连接机器

# 生成ssh公钥和私钥，不需要设置密码的话，就直接按下3个回车
[root@k8s-master k8s]$ ssh-keygen

[root@k8s-master k8s]$ ssh-copy-id root@192.168.1.104
[root@k8s-master k8s]$ ssh-copy-id root@192.168.1.105

安装Kubespray

安装前准备

Kubespray 我们只需安装到Ansible机器上即可，所以以下操作都是对Ansible机器操作

EPEL (Extra Packages for Enterprise Linux)：自动配置yum的软件仓库 ansible：自动化运维工具 Jinja2：基于python的模板引擎

[root@k8s-master k8s]$ sudo yum -y install epel-release

# 安装Python环境，Kubespray是Python写的
[root@k8s-master k8s]$ sudo yum install python36 -y 

# 安装pip，看到网上好多小伙伴都是这种方式安装，但我的不行，若你也不行的话那就用脚本的方式安装
[root@k8s-master k8s]$ yum install python-pip

# 安装pip后，再次升级pip
[root@k8s-master k8s]$ pip install --upgrade pip

# 安装ansible
[root@k8s-master k8s]$ sudo yum install -y ansible

# 升级 jinja2，否则可能会导致kubespray安装失败
[root@k8s-master k8s]$ pip install jinja2 --upgrade

脚本方式安装pip

yum install python-pip 方式安装成功的小伙伴，就不需要执行这步了

# 下载pip
[root@k8s-master k8s]$ wget https://bootstrap.pypa.io/get-pip.py

# 安装pip
[root@k8s-master k8s]$ python3 get-pip.py

开始安装KubeSpray

OK，到此，Ansible前期准备就已经完成，接下开始安装KubeSpray

下载KubeSpray源码

KubeSpray官方源码

# 我用的是2.15.0分支，或者直接下载zip，我就是这么干的
[root@k8s-master k8s]$ git clone https://github.com/kubernetes-sigs/kubespray.git

# 文件夹重命名，方便后面使用
[root@k8s-master k8s]$ mv kubespray-2.15.0 kuberspray

直接下载zip

源码拉取完后，开始安装

[root@k8s-master k8s]$ cd kuberspray

# 安装依赖(pip使用国内镜像超快，否则会出现超时报错的问题)
sudo pip3 install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple

# 拷贝配置文件
cp -rfp inventory/sample inventory/mycluster

# 使用脚本配置 ansible inventory 文件
declare -a IPS=(192.168.1.104 192.168.1.105)
CONFIG_FILE=inventory/mycluster/hosts.yaml python3 contrib/inventory_builder/inventory.py ${IPS[@]}

# 查看或修改默认参数，你可以修改为你需要的版本(选性操作，看你需不需要)
[root@k8s-master k8s]$ cat inventory/mycluster/group_vars/k8s-cluster/k8s-cluster.yml
kube_version: v1.19.7
kube_network_plugin: calico
kube_service_addresses: 10.233.0.0/18
ube_pods_subnet: 10.233.64.0/18
kube_proxy_mode: ipvs

恭喜kuberspray安装成功了，接下来部署集群

部署k8s集群

cd kubespray
vim inventory/mycluster/hosts.yaml

all:
  hosts:
    node1:
      ansible_host: 192.168.1.104
      ip: 192.168.1.104
      access_ip: 192.168.1.104
    node2:
      ansible_host: 192.168.1.105
      ip: 192.168.1.105
      access_ip: 192.168.1.105
  children:
    kube-master:
      hosts:
        node1:
        node2:
    kube-node:
      hosts:
        node1:
        node2:
    etcd:
      hosts:
        node1:
    k8s-cluster:
      children:
        kube-master:
        kube-node:
    calico-rr:
      hosts: {}

kube-master 是定义主节点 kube-node 是定义从节点

cd kubespray
# 执行部署
ansible-playbook -i inventory/mycluster/hosts.yaml --become -vvv --become-user=root cluster.yml

-vvv 是打印详细日志，可选

替换拉取镜像

当看到拉取镜像失败的时候，那要恭喜你，说明你前面安装都顺利了

Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

先吐槽一下k8s.gcr.io这个镜像到底有恶心，劳资作为个科学上网用户，都无法拉取这镜像 quay.io 科学上网可以拉取

遇到无法拉取的镜像，可性构建镜像，参考前文如何借用阿里云构建国外镜像

尝试以下方式拉取镜像(失败)

原因registry.aliyuncs.com/google_containers 镜像不齐全

正在我折腾要写脚本替换镜像的时候，发觉还有更优的方法(网上搜到很多都是采用脚本的方式去替换镜像，但那些文章都比较旧，大多数是2018年)

其实阿里云也有提供镜像，就在registry.aliyuncs.com/google_containers，所以我们需要替换成这个镜像源就行

[root@k8s-master k8s]$ vim inventory/mycluster/group_vars/k8s-cluster/k8s-cluster.yml
kube_image_repo: "registry.aliyuncs.com/google_containers"

老老实实地使用笨方法

等docker安装成功后，手动去拉取镜像，修改tag(PS: 后面再折腾其他办法优化)

# 编辑以下脚本
[root@node1 k8s]$ vim pull_images.sh
docker pull nginx:1.19

docker pull registry.cn-hangzhou.aliyuncs.com/owater/coredns:1.7.0
docker tag registry.cn-hangzhou.aliyuncs.com/owater/coredns:1.7.0 k8s.gcr.io/coredns:1.7.0
docker rmi registry.cn-hangzhou.aliyuncs.com/owater/coredns:1.7.0

docker pull registry.cn-hangzhou.aliyuncs.com/owater/k8s-dns-node-cache:1.16.0 
docker tag registry.cn-hangzhou.aliyuncs.com/owater/k8s-dns-node-cache:1.16.0 k8s.gcr.io/dns/k8s-dns-node-cache:1.16.0
docker rmi registry.cn-hangzhou.aliyuncs.com/owater/k8s-dns-node-cache:1.16.0

docker pull registry.cn-hangzhou.aliyuncs.com/owater/cluster-proportional-autoscaler-amd64:1.8.3
docker tag registry.cn-hangzhou.aliyuncs.com/owater/cluster-proportional-autoscaler-amd64:1.8.3 k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3
docker rmi registry.cn-hangzhou.aliyuncs.com/owater/cluster-proportional-autoscaler-amd64:1.8.3

docker pull registry.cn-hangzhou.aliyuncs.com/owater/kube-apiserver:v1.19.7
docker tag registry.cn-hangzhou.aliyuncs.com/owater/kube-apiserver:v1.19.7 k8s.gcr.io/kube-apiserver:v1.19.7
docker rmi registry.cn-hangzhou.aliyuncs.com/owater/kube-apiserver:v1.19.7

docker pull registry.cn-hangzhou.aliyuncs.com/owater/kube-controller-manager:v1.19.7
docker tag registry.cn-hangzhou.aliyuncs.com/owater/kube-controller-manager:v1.19.7 k8s.gcr.io/kube-controller-manager:v1.19.7
docker rmi registry.cn-hangzhou.aliyuncs.com/owater/kube-controller-manager:v1.19.7

docker pull registry.cn-hangzhou.aliyuncs.com/owater/kube-scheduler:v1.19.7
docker tag registry.cn-hangzhou.aliyuncs.com/owater/kube-scheduler:v1.19.7 k8s.gcr.io/kube-scheduler:v1.19.7
docker rmi registry.cn-hangzhou.aliyuncs.com/owater/kube-scheduler:v1.19.7

docker pull registry.cn-hangzhou.aliyuncs.com/owater/kube-proxy:v1.19.7
docker tag registry.cn-hangzhou.aliyuncs.com/owater/kube-proxy:v1.19.7 k8s.gcr.io/kube-proxy:v1.19.7
docker rmi registry.cn-hangzhou.aliyuncs.com/owater/kube-proxy:v1.19.7

docker pull registry.cn-hangzhou.aliyuncs.com/owater/etcd:v3.4.13
docker tag registry.cn-hangzhou.aliyuncs.com/owater/etcd:v3.4.13 quay.io/coreos/etcd:v3.4.13
docker rmi registry.cn-hangzhou.aliyuncs.com/owater/etcd:v3.4.13

docker pull registry.cn-hangzhou.aliyuncs.com/owater/cni:v3.16.5
docker tag registry.cn-hangzhou.aliyuncs.com/owater/cni:v3.16.5 quay.io/calico/cni:v3.16.5
docker rmi registry.cn-hangzhou.aliyuncs.com/owater/cni:v3.16.5

docker pull registry.cn-hangzhou.aliyuncs.com/owater/kube-controllers:v3.16.5
docker tag registry.cn-hangzhou.aliyuncs.com/owater/kube-controllers:v3.16.5 quay.io/calico/kube-controllers:v3.16.5
docker rmi registry.cn-hangzhou.aliyuncs.com/owater/kube-controllers:v3.16.5

docker pull registry.cn-hangzhou.aliyuncs.com/owater/node:v3.16.5
docker tag registry.cn-hangzhou.aliyuncs.com/owater/node:v3.16.5 quay.io/calico/node:v3.16.5
docker rmi registry.cn-hangzhou.aliyuncs.com/owater/node:v3.16.5

执行脚本

# 赋予脚本权限
[root@node1 k8s]$ chmod +x pull_images.sh
# 执行脚本
[root@node1 k8s]$ ./pull_images.sh

OK，这个版本Kubespray需要的镜像拉取完成了

github资源比较慢

某些资源需要从是github，所以没有科学上网或者github访问比较慢的同学就需要耐心等待了

测试下载速度，此时需要你的控制画面是什么静止状态，需耐心等待，甚至会出现超时的报错

校验是否成功

随意一个节点查看

[root@node1 k8s]$ kubectl get nodes
NAME    STATUS   ROLES    AGE   VERSION
node1   Ready    master   23m   v1.19.7
node2   Ready    master   22m   v1.19.7

踩坑

安装kuberspray依赖时

从错误中也可以看出是下载文件出现了超时，所以只需要设置超时时间即可解决，或者修改pip源

pip3 --default-timeout=50 install -r requirements.txt
# 或者
pip3 install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple

[root@k8s-master k8s]$ pip3 install -r requirements.txt
Requirement already satisfied: ansible==2.9.16 in /usr/lib/python3.6/site-packages (from -r requirements.txt (line 1)) (2.9.16)
Requirement already satisfied: PyYAML in /usr/lib64/python3.6/site-packages (from ansible==2.9.16->-r requirements.txt (line 1)) (3.12)
Requirement already satisfied: cryptography in /usr/lib64/python3.6/site-packages (from ansible==2.9.16->-r requirements.txt (line 1)) (2.3)
Collecting jinja2==2.11.1
  Using cached Jinja2-2.11.1-py2.py3-none-any.whl (126 kB)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/lib64/python3.6/site-packages (from jinja2==2.11.1->-r requirements.txt (line 2)) (0.23)
Collecting jmespath==0.9.5
  Using cached jmespath-0.9.5-py2.py3-none-any.whl (24 kB)
Collecting netaddr==0.7.19
  Downloading netaddr-0.7.19-py2.py3-none-any.whl (1.6 MB)
     |██████████▌                     | 532 kB 7.7 kB/s eta 0:02:23ERROR: Exception:
     
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 438, in _error_catcher
    yield
  File "/usr/local/lib/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 519, in read
    data = self._fp.read(amt) if not fp_closed else b""
  File "/usr/local/lib/python3.6/site-packages/pip/_vendor/cachecontrol/filewrapper.py", line 62, in read
    data = self.__fp.read(amt)
  File "/usr/lib64/python3.6/http/client.py", line 459, in read
    n = self.readinto(b)
  File "/usr/lib64/python3.6/http/client.py", line 503, in readinto
    n = self.fp.readinto(b)
  File "/usr/lib64/python3.6/socket.py", line 586, in readinto
    return self._sock.recv_into(b)
  File "/usr/lib64/python3.6/ssl.py", line 971, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/lib64/python3.6/ssl.py", line 833, in read
    return self._sslobj.read(len, buffer)
  File "/usr/lib64/python3.6/ssl.py", line 590, in read
    v = self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 224, in _main
    status = self.run(options, args)
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/cli/req_command.py", line 180, in wrapper
    return func(self, options, args)
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 321, in run
    reqs, check_supported_wheels=not options.target_dir
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 122, in resolve
    requirements, max_rounds=try_to_avoid_resolution_too_deep,
  File "/usr/local/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 445, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/usr/local/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 339, in resolve
    failure_causes = self._attempt_to_pin_criterion(name, criterion)
  File "/usr/local/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 207, in _attempt_to_pin_criterion
    criteria = self._get_criteria_to_update(candidate)
  File "/usr/local/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 198, in _get_criteria_to_update
    for r in self._p.get_dependencies(candidate):
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/provider.py", line 172, in get_dependencies
    for r in candidate.iter_dependencies(with_requires)
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/provider.py", line 171, in <listcomp>
    r
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 257, in iter_dependencies
    requires = self.dist.requires() if with_requires else ()
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 239, in dist
    self._prepare()
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 226, in _prepare
    dist = self._prepare_distribution()
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 319, in _prepare_distribution
    self._ireq, parallel_builds=True,
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 480, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 505, in _prepare_linked_requirement
    self.download_dir, hashes,
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 257, in unpack_url
    hashes=hashes,
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 130, in get_http_url
    from_path, content_type = download(link, temp_dir.path)
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/network/download.py", line 163, in __call__
    for chunk in chunks:
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/cli/progress_bars.py", line 168, in iter
    for x in it:
  File "/usr/local/lib/python3.6/site-packages/pip/_internal/network/utils.py", line 88, in response_chunks
    decode_content=False,
  File "/usr/local/lib/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 576, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/usr/local/lib/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 541, in read
    raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
  File "/usr/lib64/python3.6/contextlib.py", line 99, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/local/lib/python3.6/site-packages/pip/_vendor/urllib3/response.py", line 443, in _error_catcher
    raise ReadTimeoutError(self._pool, None, "Read timed out.")
pip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out.

镜像无法拉取问题

fatal: [node1 -> 192.168.1.104]: FAILED! => {
    "attempts": 4,
    "changed": true,
    "cmd": [
        "/usr/bin/docker",
        "pull",
        "k8s.gcr.io/coredns:1.7.0"
    ],
    "delta": "0:00:15.116121",
    "end": "2021-01-16 11:35:22.398178",
    "invocation": {
        "module_args": {
            "_raw_params": "/usr/bin/docker pull k8s.gcr.io/coredns:1.7.0",
            "_uses_shell": false,
            "argv": null,
            "chdir": null,
            "creates": null,
            "executable": null,
            "removes": null,
            "stdin": null,
            "stdin_add_newline": true,
            "strip_empty_ends": true,
            "warn": true
        }
    },
    "msg": "non-zero return code",
    "rc": 1,
    "start": "2021-01-16 11:35:07.282057",
    "stderr": "Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)",
    "stderr_lines": [
        "Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
    ],
    "stdout": "",
    "stdout_lines": []
}

执行上文中pull_images.sh脚本即可

部署中出现问题

SSH超时问题

镜像拉取完开始部署的时候，出现SSH超时问题

Timeout (12s) waiting for privilege escalation prompt:

解决方法：

修改 /etc/ansible/ansible.cfg 中超时时间

# SSH timeout
timeout = 30

不同版本containerd.io，podman冲突问题

由于我的CentOS 之前装过 containerd.io-1.2.6-3.3.el7.x86_64.rpm，所以建议CentOS最好是干净的，也不用事先安装Docker

"msg": "Depsolve Error occured: \n Problem: problem with installed package podman-2.0.5-5.module_el8.3.0+512+b3b58dca.x86_64\n  - package podman-2.0.5-5.module_el8.3.0+512+b3b58dca.x86_64 requires runc >= 1.0.0-57, but none of the providers can be installed\n  - package containerd.io-1.2.6-3.3.el7.x86_64 conflicts with containerd provided by containerd.io-1.3.9-3.1.el8.x86_64\n  - package containerd.io-1.3.9-3.1.el8.x86_64 conflicts with containerd provided by containerd.io-1.2.6-3.3.el7.x86_64\n  - package containerd.io-1.3.9-3.1.el8.x86_64 conflicts with runc provided by containerd.io-1.2.6-3.3.el7.x86_64\n  - cannot install both containerd.io-1.3.9-3.1.el8.x86_64 and containerd.io-1.2.6-3.3.el7.x86_64\n  - package containerd.io-1.3.9-3.1.el8.x86_64 conflicts with runc provided by runc-1.0.0-68.rc92.module_el8.3.0+475+c50ce30b.x86_64\n  - package containerd.io-1.3.9-3.1.el8.x86_64 obsoletes runc provided by runc-1.0.0-68.rc92.module_el8.3.0+475+c50ce30b.x86_64\n  - conflicting requests\n  - package runc-1.0.0-64.rc10.module_el8.3.0+479+69e2ae26.x86_64 is filtered out by modular filtering",

解决方案：

# 查看是否安装 Podman
[root@k8s-Node1 k8s]$ rpm -q podman

# 删除Podman
[root@k8s-Node1 k8s]$ dnf remove podman

buildah冲突

原因同上

"msg": "Depsolve Error occured: \n Problem: problem with installed package buildah-1.15.1-2.module_el8.3.0+475+c50ce30b.x86_64\n  - package buildah-1.15.1-2.module_el8.3.0+475+c50ce30b.x86_64 requires runc >= 1.0.0-26, but none of the providers can be installed\n  - package containerd.io-1.2.6-3.3.el7.x86_64 conflicts with containerd provided by containerd.io-1.3.9-3.1.el8.x86_64\n  - package containerd.io-1.3.9-3.1.el8.x86_64 conflicts with containerd provided by containerd.io-1.2.6-3.3.el7.x86_64\n  - package containerd.io-1.3.9-3.1.el8.x86_64 conflicts with runc provided by containerd.io-1.2.6-3.3.el7.x86_64\n  - cannot install both containerd.io-1.3.9-3.1.el8.x86_64 and containerd.io-1.2.6-3.3.el7.x86_64\n  - package containerd.io-1.3.9-3.1.el8.x86_64 conflicts with runc provided by runc-1.0.0-68.rc92.module_el8.3.0+475+c50ce30b.x86_64\n  - package containerd.io-1.3.9-3.1.el8.x86_64 obsoletes runc provided by runc-1.0.0-68.rc92.module_el8.3.0+475+c50ce30b.x86_64\n  - conflicting requests\n  - package runc-1.0.0-56.rc5.dev.git2abd837.module_el8.3.0+569+1bada2e4.x86_64 is filtered out by modular filtering\n  - package runc-1.0.0-64.rc10.module_el8.3.0+479+69e2ae26.x86_64 is filtered out by modular filtering",

解决方案：

# 查看是否安装 buildah
rpm -q buildah

# 删除buildah
dnf remove buildah

Jinja2 需要升级

<192.168.1.105> (0, b'\n{"cmd": "rpm -qa | grep epel-release || rpm -ivh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm", "stdout": "epel-release-7-13.noarch", "stderr": "", "rc": 0, "start": "2021-01-16 01:21:40.213172", "end": "2021-01-16 01:21:42.432345", "delta": "0:00:02.219173", "changed": true, "invocation": {"module_args": {"_raw_params": "rpm -qa | grep epel-release || rpm -ivh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm", "_uses_shell": true, "warn": true, "stdin_add_newline": true, "strip_empty_ends": true, "argv": null, "chdir": null, "executable": null, "creates": null, "removes": null, "stdin": null}}, "warnings": ["Consider using the yum, dnf or zypper module rather than running \'rpm\'.  If you need to use command because yum, dnf or zypper is insufficient you can add \'warn: false\' to this command task or set \'command_warnings=False\' in ansible.cfg to get rid of this message."]}\n', b'')
fatal: [node2]: FAILED! => {
    "msg": "The conditional check 'epel_task_result|succeeded' failed. The error was: template error while templating string: no filter named 'succeeded'. String: {% if epel_task_result|succeeded %} True {% else %} False {% endif %}"
}

FAILED - RETRYING: download_file | Download item (3 retries left).Result was: {
    "attempts": 2,
    "changed": false,
    "elapsed": 10,
    "invocation": {
        "module_args": {
            "attributes": null,
            "backup": null,
            "checksum": "sha256:c63ef1842533cd7888c7452cab9f320dcf45fc1c173e9d40abb712d45992db24",
            "client_cert": null,
            "client_key": null,
            "content": null,
            "delimiter": null,
            "dest": "/tmp/releases/kubeadm-v1.19.7-amd64",
            "directory_mode": null,
            "follow": false,
            "force": false,
            "force_basic_auth": false,
            "group": null,
            "headers": null,
            "http_agent": "ansible-httpget",
            "mode": "0755",
            "owner": "root",
            "regexp": null,
            "remote_src": null,
            "selevel": null,
            "serole": null,
            "setype": null,
            "seuser": null,
            "sha256sum": "",
            "src": null,
            "timeout": 10,
            "tmp_dest": null,
            "unsafe_writes": null,
            "url": "https://storage.googleapis.com/kubernetes-release/release/v1.19.7/bin/linux/amd64/kubeadm",
            "url_password": null,
            "url_username": null,
            "use_proxy": true,
            "validate_certs": true
        }
    },
    "msg": "failed to create temporary content file: The read operation timed out",
    "retries": 5
}

总结

Kubespray 搭建整体来还是比较折腾的，主要是镜像无法拉取问题，不然也就只是几行命令的事情

接下来继续折腾了，还是继续分享更多k8s相关技术点