基于kubespray部署k8s部分修改

381 阅读2分钟

如果用的镜像是某些基于CentOS修改的,但是改了release名称的镜像,可以sed直接替换Centos|RedHat为你的系统名


roles/etcd/tasks/upd_ca_trust.yml

- name: Gen_certs | target ca-certificate store file
  set_fact:
    ca_cert_path: |-
      {% if ansible_os_family == "Debian" -%}
      /usr/local/share/ca-certificates/etcd-ca.crt
      {%- elif ansible_os_family == "RedHat" -%} # 修改为你的镜像名
      /etc/pki/ca-trust/source/anchors/etcd-ca.crt

我这边使用了代理缓存模式,理论上依赖只需要下载一次,但是每次重新跑都会校验下载的二进制包,很多时候都会失败,这里加一点逻辑:如果已经下载好了,那么就跳过重复的下载校验。



#  vi group_vars/all/all.yml
---
download_run_once: true
download_localhost: false
download_force_cache: true
download_validate_certs: false





/tmp/kubespray_cache
├── cni-plugins-linux-amd64-v1.2.0.tgz
├── containerd-1.6.15-linux-amd64.tar.gz
├── crictl-v1.25.0-linux-amd64.tar.gz
├── etcd-v3.5.6-linux-amd64.tar.gz
├── images
│   ├── docker.io_kubeovn_kube-ovn_v1.10.7.tar
│   └── ghcr.io_k8snetworkplumbingwg_multus-cni_v3.8-amd64.tar
├── kubeadm-v1.25.6-amd64
├── kubelet-v1.25.6-amd64
├── nerdctl-1.0.0-linux-amd64.tar.gz
├── runc
└── runc.amd64


# 这个目录即为缓存目录,这个目录部署完成后,可以备份一下,后面reset集群,可以避免重复下载


# 修改部分逻辑

# vi /root/kubespray/roles/download/tasks/download_file.yml


     82   - name: File exist ?
     83     stat:
     84       path: "{{ file_path_cached }}"
     85     register: dest_file
     86
     87   - debug:
     88       msg: "{{ file_path_cached }} exists"
     89     when: dest_file.stat.exists
     90
     91   - debug:
     92       msg: "{{ file_path_cached }} not exists"
     93     when: not dest_file.stat.exists
     94
     95   # This must always be called, to check if the checksum matches. On no-match the file is re-downloaded.
     96   # This task will avoid logging it's parameters to not leak environment passwords in the log
     97   - name: download_file | Download item
     98     get_url:
     99       url: "{{ valid_mirror_urls | random }}"
    100       dest: "{{ file_path_cached if download_force_cache else download.dest }}"
    101       owner: "{{ omit if download_localhost else (download.owner | 
    ...
    117     when: not dest_file.stat.exists  # 跳过

至于镜像拉取慢的问题,这可能需要自己建一个代理



 /usr/local/bin/nerdctl -n k8s.io pull --quiet  registry.k8s.io/coredns/coredns:v1.9.3
INFO[0030] trying next host                              error="failed to do request: Head \"https://asia-east1-docker.pkg.dev/v2/k8s-artifacts-prod/images/coredns/coredns/manifests/v1.9.3\": dial tcp [2404:6800:4008:c04::52]:443: i/o timeout" host=registry.k8s.io
FATA[0030] failed to resolve reference "registry.k8s.io/coredns/coredns:v1.9.3": failed to do request: Head "https://asia-east1-docker.pkg.dev/v2/k8s-artifacts-prod/images/coredns/coredns/manifests/v1.9.3": dial tcp [2404:6800:4008:c04::52]:443: i/o timeout



  • 最好先安装好最新python3,目前3.11.2,最好使用python3虚拟环境,否则无法安装kubespray依赖(requirements.txt)
  • 存在一些镜像是基于Centos定制的,但是发行系统名改掉了,这里可以直接将CentOS或者federa更名为你的镜像名
  • 基于github下载的二进制存在重复下载,可以考虑在all.yaml使用缓存模式
  • pause等容器镜像拉取往往都有问题,可以考虑使用国内免费代理
  • multus的安装有问题,可以考虑手动安装,将multus的安装代码删掉
  • 安全组开放 最简单的是开放ipv4 v6所有协议所有端口
# k8s依赖国内代理

registry.k8s.io  --> registry.lank8s.cn
k8s.gcr.io --> lank8s.cn
gcr.io --> gcr.lank8s.cn


## k8s ali代理
k8s.gcr.io --> registry.aliyuncs.com/google_containers
k8s.gcr.io/coredns:1.6.5 == registry.aliyuncs.com/google_containers/coredns:1.6.5

docker pull registry.aliyuncs.com/google_containers/coredns:1.6.5
docker tag registry.aliyuncs.com/google_containers/coredns:1.6.5 k8s.gcr.io/coredns:1.6.5
docker rmi registry.aliyuncs.com/google_containers/coredns:1.6.5