k8s环境搭建

TangLyan
412 阅读1分钟

前置条件

系统配置

  1. 设置各服务器hostname
hostnamectl --static set-hostname k8s-master01

  1. 将各服务器hostname加入到 /etc/hosts中；

  2. 关闭selinux

setenforce 0 && sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

内网开启selinux会带来一些排错困难，以及现在很多软件本身也不支持selinux

  1. 关闭swap
 swapoff -a
 sed -i "s/^\/dev\/mapper\/centos-swap/#&/" /etc/fstab
  1. 关闭防火墙
systemctl stop firewalld

如果不关闭防火墙，那么需要在防火墙上开启一系列端口，内网环境下为了简单所以将其进行关闭

  1. 配置路由
    cat <<EOF >  /etc/sysctl.d/k8s.conf
    net.bridge.bridge-nf-call-ip6tables = 1
    net.bridge.bridge-nf-call-iptables = 1
    EOF

bridge-nf 使得 netfilter 可以对 Linux 网桥上的 IPv4/ARP/IPv6 包过滤

  1. 重启配置
sysctl --system

同步服务器

集群都需要执行的命令：

  • 设置时区
ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
  • 安装chrony
yum -y install chrony

主节点执行：

  • 设置配置
cat chrony.conf
server k8s-master-01 iburst
allow 100.66.0.0/16
allow 127.0.0.0/8
local stratum 10
  • 拷贝文件
cp chrony.conf /etc/chrony.conf

从节点执行：

  • 设置配置
cat chrony.conf
server k8s-master-01 iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
logdir /var/log/chrony
  • 拷贝文件
cp chrony.conf /etc/chrony.conf

统一执行：

  • 重启
systemctl restart chronyd
  • 重新同步
chronyc -a makestep

docker配置

  1. 删除之前已经安装得到docker;
yum install -y docker docker-common docker-selinux docker-engine
  1. 安装docker前置依赖包；
yum install -y yum-utils device-mapper-persistent-data lvm2
  1. 获取docker最新更新源;
  2. 安装docker；
yum -y isntall docker-ce
  1. 创建docekr配置目录；
mkdir -p /data/docker
mkdir /etc/docker/certs.d/harbor.com/ # harbor证书
  1. 拷贝默认daemon.json目录；
{
  "registry-mirrors": ["xxxx"],
  "data-root": "/data/docker",
  "exec-opts": ["native.cgroupdriver=cgroupfs"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2",
  "storage-opts": [
    "overlay2.override_kernel_check=true"
  ]
}
  1. 拷贝Harbor证书

etcd安装

ssl证书生成

  • 安装cfssl, cfssl-certinfo, cfssl-json

  • 获取默认配置文件

cfssl print-defaults config > ca-config.json
  • 修改配置文件
{
  "signing": {
    "default": {
      "expiry": "8760h"    # 有效时间
    },
    "profiles": {
      "kubernetes": {
        "usages": [
            "signing",	
            "key encipherment",
            "server auth",
            "client auth"
        ],
        "expiry": "8760h"   # 有效时间
      }
    }
  }
}
  • 获取ca证书签名请求csr
#打印csr模板文件从而进行修改
#cfssl print-defaults csr > ca-csr.json

#vim ca-csr.json
{
  "CN": "kubernetes",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "ShangHai",
      "L": "ShangHai",
      "O": "k8s",
      "OU": "System"
    }
  ]
}
  • CA证书和私钥
./cfssl gencert -initca ca-csr.json |   ./cfssljson -bare ca
  • 生成服务器证书
./cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kubernetes-csr.json | ./cfssljson -bare kubernetes

etcd安装

  • 下载安装包
  • 解压并安装
mv /usr/local/etcd-v3.4.14-linux-amd64 /usr/local/etcd
mkdir -p /usr/local/etcd/ssl
mkdir -p /usr/local/etcd/conf
cp ca.pem /usr/local/etcd/ssl/ca.pem 
cp kubernetes.pem /usr/local/etcd/ssl/kubernetes.pem 
cp kubernetes-key.pem /usr/local/etcd/ssl/kubernetes-key.pem

  • 拷贝etcd配置

  • 拷贝etcd.service

[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
EnvironmentFile=/usr/local/etcd/conf/etcd.conf
ExecStart=/usr/local/etcd/etcd
--name=${ETCD_NAME}
--data-dir=${ETCD_DATA_DIR}
--logger=${ETCD_LOGGER}
--log-outputs=${ETCD_LOG_OUTPUT}
--debug=${ETCD_DEBUG}
--listen-peer-urls=${ETCD_LISTEN_PEER_URLS}
--listen-client-urls=${ETCD_LISTEN_CLIENT_URLS}
--advertise-client-urls=${ETCD_ADVERTISE_CLIENT_URLS}
--initial-advertise-peer-urls=${ETCD_INITIAL_ADVERTISE_PEER_URLS}
--initial-cluster=${ETCD_INITIAL_CLUSTER}
--initial-cluster-token=${ETCD_INITIAL_CLUSTER_TOKEN}
--initial-cluster-state=${ETCD_INITIAL_CLUSTER_STATE}
--cert-file=${ETCD_CERT_FILE}
--key-file=${ETCD_KEY_FILE}
--peer-cert-file=${ETCD_PEER_CERT_FILE}
--peer-key-file=${ETCD_PEER_KEY_FILE}
--trusted-ca-file=${ETCD_TRUSTED_CA_FILE}
--client-cert-auth=${ETCD_CLIENT_CERT_AUTH}
--peer-client-cert-auth=${ETCD_PEER_CLIENT_CERT_AUTH}
--peer-trusted-ca-file=${ETCD_PEER_TRUSTED_CA_FILE}
Restart=always
RestartSec=1
StartLimitIntervalSec=0

[Install]
WantedBy=multi-user.target
  • 设置自启动
systemctl enable etcd
systemctl start etcd

K8S安装

机器类型分为：主机，从机；

所有机器操作；

  • 设置安装源
  • 下载安装
 yum -y install kubectl-1.21.1 kubelet-1.21.1 kubeadm-1.21.1
  • 设置kubelet自启动
systemctl enable kubelet
systemctl restart kubelet
  • 下载安装kubernetes所需的镜像（除去coredns）（使用阿里云镜像代理）；
kubeadm config images list --kubernetes-version v1.21.1 | grep -v coredns
  • 将下载的镜像都改名，例如；
docker tag "k8s.gcr.io/kube-apiserver:v1.21.1" "xxxxxx/kube-apiserver:v1.21.1"

  • 下载coredns镜像

  • 修改coredns镜像+v;

主机操作

  • 创建etcd证书保存目录；
mkdir -p /etc/kubernetes/pki/etcd

  • 拷贝etcd证书到上面目录地址；

  • 拷贝所需的配置文件，包括calico.yaml，nginx-ingress，kubeadm-init.yaml；

kubeadm-init.yaml

apiVersion: kubeadm.k8s.io/v1beta2
bootstrapTokens:
  - groups:
      - system:bootstrappers:kubeadm:default-node-token
    token: abcdef.0123456789abcdef
    ttl: 24h0m0s
    usages:
      - signing
      - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: {{ ansible_facts.default_ipv4.address }}         # 设置成当前节点ip
  bindPort: 6443
nodeRegistration:
  criSocket: /var/run/dockershim.sock
  name: k8s-master-01                            # 设置成当前节点主机名
  taints:
    - effect: NoSchedule
      key: node-role.kubernetes.io/master
---
apiServer:
  certSANs:
    - sase.k8s.master.com
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns:
  type: CoreDNS
etcd:
  external:
    endpoints:
      - https://{{ groups["etcd"][0] }}:2379
      - https://{{ groups["etcd"][1] }}:2379
      - https://{{ groups["etcd"][2] }}:2379
    caFile: /etc/kubernetes/pki/etcd/ca.pem                 # 搭建etcd集群时生成的ca证书
    certFile: /etc/kubernetes/pki/etcd/kubernetes.pem # 搭建etcd集群时生成的客户端证书
    keyFile: /etc/kubernetes/pki/etcd/kubernetes-key.pem  # 搭建etcd集群时生成的客户端密钥
kind: ClusterConfiguration
kubernetesVersion: 1.21.1
networking:
  dnsDomain: cluster.local
  podSubnet: 192.168.0.0/16
  serviceSubnet: 10.96.0.0/12
scheduler: {}
  • 安装k8s
kubeadm init --v=5 --config=/root/kubeadm-init.1.21.1.yaml
  • 拷贝kubectl认证文件
cp /etc/kubernetes/admin.conf $HOME/.kube/config
  • 修改配置
sed -i "s/- --port/#&/"   /etc/kubernetes/manifests/kube-controller-manager.yaml
sed -i "s/- --port/#&/"  /etc/kubernetes/manifests/kube-scheduler.yaml

  • 重启kubelet

  • 安装calico，ingress，metrics

  • 获取join token

kubeadm token create --print-join-command

从机操作

  • 执行token
avatar
文章
阅读
粉丝