安装minikube，需要先安装docker与kubectl，
安装 Docker：www.cnblogs.com/jhxxb/p/114…
安装 kubectl：kubernetes.io/docs/tasks/…
一、安装kubectl
1、Install kubectl binary with curl on Linux （使用下载安装包安装）
1.Download the latest release with the command:
[root@VM-12-10-centos ~]# curl -LO https://dl.k8s.io/release/v1.28.3/bin/linux/amd64/kubectl
2.Validate the binary (optional)
[root@VM-12-10-centos ~]# curl -LO "https://dl.k8s.io/v1.28.3/bin/linux/amd64/kubectl.sha256"
[root@VM-12-10-centos ~]# echo "$(cat kubectl.sha256) kubectl" | sha256sum --check
kubectl: OK
3.Install kubectl
[root@VM-12-10-centos ~]# install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
4.Test to ensure the version you installed is up-to-date:
[root@VM-12-10-centos ~]# kubectl version --client
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.0", GitCommit:"a866cbe2e5bbaa01cfd5e969aa3e033f3282a8a2", GitTreeState:"clean", BuildDate:"2022-08-23T17:44:59Z", GoVersion:"go1.19", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.7
[root@VM-12-10-centos ~]# kubectl version --client --output=yaml
clientVersion:
buildDate: "2022-08-23T17:44:59Z"
compiler: gc
gitCommit: a866cbe2e5bbaa01cfd5e969aa3e033f3282a8a2
gitTreeState: clean
gitVersion: v1.28.3
goVersion: go1.19
major: "1"
minor: "25"
platform: linux/amd64
kustomizeVersion: v4.5.7
2、Install using native package management（在线安装）
Red Hat-based distributions(Centos / RHEL / Fedora)，采用国内源，国外源不可用
cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
[root@VM-12-10-centos ~]# setenforce 0
[root@VM-12-10-centos ~]# yum install -y kubectl
3、Verify kubectl configuration
为了让 kubectl 找到和访问 Kubernetes 集群，它需要一个 kubeconfig 文件，该文件在使用 kube-up.sh 创建集群或成功部署 Minikube 集群时自动创建。 默认情况下，kubectl 配置位于 ~/.kube/config。
通过获取集群状态检查 kubectl 是否正确配置：
[root@VM-12-10-centos ~]# kubectl cluster-info
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
The connection to the server localhost:8080 was refused - did you specify the right host or port?
[root@VM-12-10-centos ~]# kubectl cluster-info dump
The connection to the server localhost:8080 was refused - did you specify the right host or port?
上述错误，如果您打算在笔记本电脑上（本地）运行 Kubernetes 集群，则需要先安装 Minikube 之类的工具，然后重新运行上述命令。
[root@VM-12-10-centos ~]# kubectl version --client
二、安装Minikube
1、直接安装（国内不可用）
[root@VM-12-10-centos ~]# install minikube-linux-amd64 /usr/local/bin/minikube
[root@VM-12-10-centos ~]# minikube start
* minikube v1.26.1 on Centos 8.2.2004 (amd64)
* Unable to pick a default driver. Here is what was considered, in preference order:
* Alternatively you could install one of these drivers:
- docker: Not installed: exec: "docker": executable file not found in $PATH
- kvm2: Not installed: exec: "virsh": executable file not found in $PATH
- podman: Not installed: exec: "podman": executable file not found in $PATH
- vmware: Not installed: exec: "docker-machine-driver-vmware": executable file not found in $PATH
- virtualbox: Not installed: unable to find VBoxManage in $PATH
- qemu2: Not installed: exec: "qemu-system-x86_64": executable file not found in $PATH
X Exiting due to DRV_NOT_DETECTED: No possible driver was detected. Try specifying --driver, or see https://minikube.sigs.k8s.io/docs/sta
提示要安装Docker，下面安装Docker：
[root@VM-12-10-centos ~]# yum install -y yum-utils \
device-mapper-persistent-data \
lvm2
[root@VM-12-10-centos ~]# yum-config-manager \
--add-repo \
http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
[root@VM-12-10-centos ~]# yum install docker-ce docker-ce-cli containerd.io
[root@VM-12-10-centos ~]# docker -v
Docker version 20.10.17, build 100c701
[root@VM-12-10-centos ~]# systemctl enable docker
再次启动minikube：
[root@VM-12-10-centos ~]# minikube start --driver=docker
* minikube v1.26.1 on Centos 8.2.2004 (amd64)
* Using the docker driver based on user configuration
X Exiting due to PROVIDER_DOCKER_NOT_RUNNING: "docker version --format -" exit status 1: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
* Suggestion: Start the Docker service
* Documentation: https://minikube.sigs.k8s.io/docs/drivers/docker/
[root@VM-12-10-centos ~]# system restart docker //因为docker没有启动民致上述错误，启动docker
[root@VM-12-10-centos ~]# minikube start --driver=docker
[root@VM-12-10-centos ~]# minikube start --driver=docker
* minikube v1.26.1 on Centos 8.2.2004 (amd64)
* Using the docker driver based on user configuration
* The "docker" driver should not be used with root privileges. If you wish to continue as root, use --force.
* If you are running minikube within a VM, consider using --driver=none:
* https://minikube.sigs.k8s.io/docs/reference/drivers/none/
X Exiting due to DRV_AS_ROOT: The "docker" driver should not be used with root privileges.
[root@VM-12-10-centos ~]# minikube start --force --driver=docker // 直接加 --force
[root@VM-12-10-centos ~]# minikube start --force --driver=docker
* minikube v1.26.1 on Centos 8.2.2004 (amd64)
! minikube skips various validations when --force is supplied; this may lead to unexpected behavior
! Specified Kubernetes version 1.26.1 is newer than the newest supported version: v1.24.3. Use `minikube config defaults kubernetes-version` for details.
* Using the docker driver based on existing profile
* The "docker" driver should not be used with root privileges. If you wish to continue as root, use --force.
* If you are running minikube within a VM, consider using --driver=none:
* https://minikube.sigs.k8s.io/docs/reference/drivers/none/
* Tip: To remove this root owned cluster, run: sudo minikube delete
* Starting control plane node minikube in cluster minikube
* Pulling base image ...
* Updating the running docker "minikube" container ...
! This container is having trouble accessing https://k8s.gcr.io
* To pull new external images, you may need to configure a proxy: https://minikube.sigs.k8s.io/docs/reference/networking/proxy/
! The image 'k8s.gcr.io/coredns/coredns:v1.8.6' was not found; unable to add it to cache.
! The image 'k8s.gcr.io/kube-scheduler:v1.26.1' was not found; unable to add it to cache.
! The image 'k8s.gcr.io/kube-apiserver:v1.26.1' was not found; unable to add it to cache.
! The image 'k8s.gcr.io/kube-controller-manager:v1.26.1' was not found; unable to add it to cache.
! The image 'k8s.gcr.io/etcd:3.5.0-0' was not found; unable to add it to cache.
! The image 'k8s.gcr.io/pause:3.6' was not found; unable to add it to cache.
! The image 'k8s.gcr.io/kube-proxy:v1.26.1' was not found; unable to add it to cache.
! The image 'gcr.io/k8s-minikube/storage-provisioner:v5' was not found; unable to add it to cache.
* Preparing Kubernetes v1.26.1 on Docker 20.10.17 ...
X Unable to load cached images: loading cached images: stat /root/.minikube/cache/images/amd64/k8s.gcr.io/kube-scheduler_v1.26.1: no such file or directory
X Exiting due to K8S_INSTALL_FAILED: updating control plane: downloading binaries: downloading kubelet: download failed: https://storage.googleapis.com/kubernetes-release/release/v1.26.1/bin/linux/amd64/kubelet?checksum=file:https://storage.googleapis.com/kubernetes-release/release/v1.26.1/bin/linux/amd64/kubelet.sha256: getter: &{Ctx:context.Background Src:https://storage.googleapis.com/kubernetes-release/release/v1.26.1/bin/linux/amd64/kubelet?checksum=file:https://storage.googleapis.com/kubernetes-release/release/v1.26.1/bin/linux/amd64/kubelet.sha256 Dst:/root/.minikube/cache/linux/amd64/v1.26.1/kubelet.download Pwd: Mode:2 Umask:---------- Detectors:[0x37f9288 0x37f9288 0x37f9288 0x37f9288 0x37f9288 0x37f9288 0x37f9288] Decompressors:map[bz2:0x37f9288 gz:0x37f9288 tar:0x37f9288 tar.bz2:0x37f9288 tar.gz:0x37f9288 tar.xz:0x37f9288 tar.zst:0x37f9288 tbz2:0x37f9288 tgz:0x37f9288 txz:0x37f9288 tzst:0x37f9288 xz:0x37f9288 zip:0x37f9288 zst:0x37f9288] Getters:map[file:0xc000b4f7c0 http:0xc000e74cd0 https:0xc000e74d20] Dir:false ProgressListener:0x37b5280 Insecure:false DisableSymlinks:false Options:[0x1a5d600]}: invalid checksum: Error downloading checksum file: bad response code: 404
*
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
主要是连接不上google的k8s.gcr.io，需要改用国内站点。
2、删除minikube，重新安装（采用国内源安装）
- 停止运行
[root@VM-12-10-centos ~]# minikube stop - 执行卸载命令
[root@VM-12-10-centos ~]# minikube delete
或：[root@VM-12-10-centos ~]#minikube delete --all # 清空
// 删除应用：
[root@VM-12-10-centos ~]# docker ps -aq # 查询有的应用
[root@VM-12-10-centos ~]# docker stop xxx #删除上面的应用
[root@VM-12-10-centos ~]# rm -rf ~/.kube ~/.minikube
[root@VM-12-10-centos ~]# sudo rm /usr/local/bin/localkube /usr/local/bin/minikube
[root@VM-12-10-centos ~]# systemctl stop '*kubelet*.mount'
[root@VM-12-10-centos ~]# sudo rm -rf /etc/kubernetes/
[root@VM-12-10-centos ~]# docker system prune -af --volumes
3. 重新下载安装：
阿里国内源到目前可下载1.23.1版本，最新官方版本是1.26.1 （2022-08-28）
[root@VM-12-10-centos ~]# curl -Lo minikube https://kubernetes.oss-cn-hangzhou.aliyuncs.com/minikube/releases/v1.23.1/minikube-linux-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/
用国内源启动：
[root@VM-12-10-centos ~]# minikube start --force --driver=docker --image-mirror-country='cn'
[root@VM-12-10-centos bin]# minikube start --registry-mirror=https://registry.docker-cn.com
* minikube v1.23.1 on Centos 8.2.2004 (amd64)
* Automatically selected the docker driver. Other choices: none, ssh
* The "docker" driver should not be used with root privileges.
* If you are running minikube within a VM, consider using --driver=none:
* https://minikube.sigs.k8s.io/docs/reference/drivers/none/
X Exiting due to DRV_AS_ROOT: The "docker" driver should not be used with root privileges.
[root@VM-12-10-centos bin]# cd ~
[root@VM-12-10-centos ~]# minikube start --force --driver=docker --image-mirror-country='cn'
* minikube v1.23.1 on Centos 8.2.2004 (amd64)
! minikube skips various validations when --force is supplied; this may lead to unexpected behavior
* Using the docker driver based on user configuration
* The "docker" driver should not be used with root privileges.
* If you are running minikube within a VM, consider using --driver=none:
* https://minikube.sigs.k8s.io/docs/reference/drivers/none/
* minikube 1.26.0 is available! Download it: https://github.com/kubernetes/minikube/releases/tag/v1.26.0
* To disable this notice, run: 'minikube config set WantUpdateNotification false'
* Using image repository registry.cn-hangzhou.aliyuncs.com/google_containers
* Starting control plane node minikube in cluster minikube
* Pulling base image ...
! The image 'registry.cn-hangzhou.aliyuncs.com/google_containers/coredns/coredns:v1.8.4' was not found; unable to add it to cache.
> registry.cn-hangzhou.aliyun...: 355.40 MiB / 355.40 MiB 100.00% 5.74 MiB
* Creating docker container (CPUs=2, Memory=3800MB) ...
* Preparing Kubernetes v1.22.1 on Docker 20.10.8 ...
X Unable to load cached images: loading cached images: stat /root/.minikube/cache/images/registry.cn-hangzhou.aliyuncs.com/google_containers/coredns/coredns_v1.8.4: no such file or directory
> kubelet.sha256: 64 B / 64 B [--------------------------] 100.00% ? p/s 0s
> kubectl.sha256: 64 B / 64 B [--------------------------] 100.00% ? p/s 0s
> kubeadm.sha256: 64 B / 64 B [--------------------------] 100.00% ? p/s 0s
> kubeadm: 43.71 MiB / 43.71 MiB [-------------] 100.00% 14.38 MiB p/s 3.2s
> kubectl: 44.73 MiB / 44.73 MiB [-------------] 100.00% 13.06 MiB p/s 3.6s
> kubelet: 146.25 MiB / 146.25 MiB [-------------] 100.00% 8.93 MiB p/s 17s
- Generating certificates and keys ...
- Booting up control plane ...
- Configuring RBAC rules ...
* Verifying Kubernetes components...
- Using image registry.cn-hangzhou.aliyuncs.com/google_containers/storage-provisioner:v5
* Enabled addons: storage-provisioner, default-storageclass
! /usr/local/bin/kubectl is version 1.25.0, which may have incompatibilites with Kubernetes 1.22.1.
- Want kubectl v1.22.1? Try 'minikube kubectl -- get pods -A'
* Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
也可以下载安装包：github.com/AliyunConta…，但没有安装成功。
3、验证：
[root@VM-12-10-centos ~]# kubectl get pods -A
打开Kubernetes可视化：
[root@VM-12-10-centos ~]# minikube dashboard // 或： minikube dashboard --url
dashboard启动后，只能本机容器访问，使用kubectl proxy --port=端口号（一个没有被占用的就可以） --address='机器内网地址' --accept-hosts='^.*' & 以上命令代理后，访问 http://机器公网IP:上面配置的端口号
使用proxy代理到虚拟机的指定端口:
[root@VM-12-10-centos ~]# kubectl proxy --port=8001 --address='0.0.0.0' --accept-hosts='^.'
或：kubectl proxy --port=8001 --address='0.0.0.0' --accept-hosts='^$' &
启动后宿主机访问链接：
http://43.138.239.76:8001/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/#/overview?namespace=_all
由于采用云服务器，要配置防火墙端口：
如要停止或退出 kubectl proxy，两种方式: 一种直接Ctrl+C
第二种，如果没有在控制台，可先查询端口
[root@VM-12-10-centos ~]# sudo ss -tunlp //查看端口使用
或[root@VM-12-10-centos ~]# netstat -tnlp | grep :80 //查询指定端口通过grep过滤
直接杀死进程：kill -9 pid //pid 进程号
访问主页如下：
[root@VM-12-10-centos ~]# minikube status
minikube
type: Control Plane
host: Running
kubelet: Running
apiserver: Running
kubeconfig: Configured
4、配置多节点：
[root@VM-12-10-centos ~]# minikube node list
添加节点：
[root@VM-12-10-centos ~]# minikube node add #相当于增加一个docker 实例
[root@VM-12-10-centos ~]# minikube node list
以上为minikube的安装过程。下面为在minikube上安装应用。 添加minikube的组件：
#查看minikube中的所有组件
minikube addons list
#启动ingress组件
minikube addons enable ingress
#查看ingress是否启动
kubectl get pods -n kube-system
三、应用创建
创建应用可直接用命令方式，也可采用yaml文档方式，生产中多数采用yaml方式。
[root@VM-12-10-centos ~]# kubectl create deployment nginx --image=nginx # 或指定版本：kubectl create deployment nginx --image=nginx:latest
[root@VM-12-10-centos ~]# kubectl expose deployment nginx --port=80 --type=NodePort # 暴露服务
[root@VM-12-10-centos ~]# minikube service --url nginx # 获取访问地址
[root@VM-12-10-centos ~]# minikube service nginx #进入服务
[root@VM-12-10-centos ~]# kubectl delete deployment nginx # 删除部署
[root@VM-12-10-centos ~]# kubectl delete service nginx # 删除服务
查看pod、services、node情况：
[root@VM-12-10-centos ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-6799fc88d8-4bm79 1/1 Running 0 45m 172.17.0.2 minikube-m02 <none> <none>
root@VM-12-10-centos ~]# kubectl get services -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 2d6h <none>
nginx NodePort 10.99.98.248 <none> 80:31569/TCP 44m app=nginx
[root@VM-12-10-centos ~]# kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
minikube Ready control-plane,master 2d6h v1.22.1 192.168.49.2 <none> Ubuntu 20.04.2 LTS 4.18.0-348.7.1.el8_5.x86_64 docker://20.10.8
minikube-m02 Ready <none> 71m v1.22.1 192.168.49.3 <none> Ubuntu 20.04.2 LTS 4.18.0-348.7.1.el8_5.x86_64 docker://20.10.8
minikube-m03 Ready <none> 68m v1.22.1 192.168.49.4 <none> Ubuntu 20.04.2 LTS 4.18.0-348.7.1.el8_5.x86_64 docker://20.10.8
四、服务暴露的向种方法
Kubernetes暴露网络服务的几种方式，一种直接暴露pod为访问地址：分别为：port_forward、hostNetwork、hostPort，一种采用serivce方式，分别为：NodePort、loadBalancer、Ingress；
(minikube.sigs.k8s.io/docs/start/)
Minikube 的节点物理仍是内网 IP，当向外提供 Service Nodeport 服务（包括LoadBalancer）时需要在本地上访问或通过代理转发
1、NodePort
- 这里没有通过service对外暴露，是直接把pod转为服务，直接访问pod。
[root@VM-12-10-centos ~]# kubectl create deployment nginx --image=nginx #或指定版本：kubectl create deployment nginx --image=nginx:latest
[root@VM-12-10-centos ~]# kubectl get pods -o wide
注意：默认情况下，Pod 只能通过 Kubernetes 集群中的内部 IP 地址访问。 要使得容器可以从 Kubernetes 虚拟网络的外部访问，你必须将 Pod 暴露为 Kubernetes Service，方法有多种。（参考：www.jianshu.com/p/4ff31ca13…）
这里采用端口转发port-forward来实现，kubectl port-forward 通过端口转发映射本地端口到指定的应用端口，从而访问集群中的应用程序(Pod).
[root@VM-12-10-centos ~]# kubectl port-forward nginx-6799fc88d8-j8bdw 30000:80 --address='0.0.0.0' & // &表示后台运行
或
[root@VM-12-5-centos ~]# kubectl port-forward service/nginx 30000:80 --address='0.0.0.0' &
[root@VM-12-10-centos ~]# ss -tunlp
[root@VM-12-10-centos ~]# curl localhost:30000
使用外部浏览器：
- 使用hostNetwork: true，直接定义Pod网络的方式发布服务
[root@VM-12-10-centos ~]# kubectl delete deployment nginx
如果在POD中使用"hostNetwork: true"配置网络，pod中运行的应用程序可以直接看到宿主主机的网络接口，宿主机所在的局域网上所有网络接口都可以访问到该应用程序及端口。对于以 hostNetwork 方式运行的 Pod，应显式设置其 DNS 策略 为"ClusterFirstWithHostNet"
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
# 使用主机网络
hostNetwork: true
# 该设置是使POD使用k8s的dns，dns配置在/etc/resolv.conf文件中
# 如果不加，pod默认使用所在宿主主机使用的DNS，这样会导致容器
# 内不能通过service name访问k8s集群中其他POD
dnsPolicy: ClusterFirstWithHostNet
containers:
- name: nginx
image: nginx:latest
ports:
- containerPort: 80
[root@VM-12-10-centos ~]# kubectl apply -f nginx.yaml # 或 kubectl create -f nginx.yaml
使用 hostNetwork，pod 实际上用的是 pod 宿主机的网络地址空间：即 pod IP 是宿主机 IP，而非 cni 分配的 pod IP，端口是宿主机网络监听接口。
[root@VM-12-10-centos ~]# kubectl get pods -o wide
[root@VM-12-10-centos ~]# curl 192.168.49.3
[root@VM-12-10-centos ~]# kubectl exec -it nginx-56958756fb-wp6sl bash
root@minikube-m02:/# curl 192.168.49.3
- HostPort方式
这是一种直接定义Pod网络的方式。hostPort是直接将容器的端口与所调度的节点上的端口路由，这样用户就可以通过宿主机的IP加上端口来访问Pod，这里是没有创建server，创建是deployment。
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
dnsPolicy: ClusterFirstWithHostNet
containers:
- name: nginx
image: nginx:1.7.9
ports:
- containerPort: 80
hostPort: 8000
name: http
protocol: TCP
- containerPort: 443
hostPort: 44300
name: https
protocol: TCP
[root@VM-12-10-centos ~]# kubectl create -f nginx.yaml
[root@VM-12-10-centos ~]# kubectl get pods -o wide
这里172.17.0.2是docker的 IP地址，访问测试 如下，通过PODIP+containerPort 和 node节点IP+hostPort 都可以正常访问到
[root@VM-12-10-centos ~]# docker exec -it 3fcb489804c4 ip addr
[root@VM-12-10-centos ~]# kubectl exec -it nginx-6657d4c4cf-bkf4c bash
[root@VM-12-10-centos ~]# kubectl get node -o wide
[root@VM-12-10-centos ~]# curl 192.168.49.3:8000
hostPort 与 hostNetwork 本质上都是暴露 pod 所在节点 IP 给终端用户，因为 pod 生命周期并不固定，随时都有可能异常重建，故 IP 的不确定最终导致用户使用上的不方便；此外宿主机端口占用也导致不能在同一台机子上有多个程序使用同一端口。因此一般情况下，不建议使用 hostPort 方式.
使用 hostPort，pod IP 并非宿主机 IP，而是 cni 分配的 pod IP，跟其他普通的 pod 使用一样的 ip 分配方式，端口并非宿主机网络监听端口，只是使用了 DNAT 机制将 hostPort 指定的端口映射到了容器的端口之上（可以通过 iptables 命令进行查看）。外部访问此 pod 时，仍然使用宿主机和 hostPort 方式。pod ip 跟宿主机 ip
因为Pod重新调度的时候该Pod被调度到的宿主机可能会变动，用户必须自己维护一个Pod与所在宿主机的对应关系。
这种网络方式可以用来做 nginx [Ingress controller]。外部流量都需要通过kubenretes node节点的80和443端口
- 使用ClusterIP
为相同的服务创建一个 VIP，只能用于集群内的访问，外网无法访问 ClusterIP。
[root@VM-12-10-centos ~]# kubectl delete deployment nginx-deployment
[root@VM-12-10-centos ~]# kubectl delete service nginx-service
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels: # 这里是定义Deployment的标签
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx # 选择关联Deployment标签
template:
metadata:
labels: # 给Pod定义一个标签，方便其他服务关联这个Pod
app: nginx
spec:
dnsPolicy: ClusterFirstWithHostNet
containers:
- name: nginx
image: nginx:latest
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx-service
spec:
selector: # Service 的selector 指定标签 app:nginx 来进行对Pod进行关联 ；(这里的app:nginx就是上面Deployment配置里labels定义的标签 )
app: nginx
ports:
- protocol: TCP
port: 80
targetPort: 80
type: ClusterIP
[root@VM-12-10-centos ~]# kubectl apply -f nginx.yaml
访问方法，通过proxy+api，开启proxy：
kubectl proxy --address='0.0.0.0' --accept-hosts='^*$' &
地址: http://ip:8001/api/v1/namespaces/default/services/nginx-service:http/proxy/
如果上述访问不到，可试：http://ip:8001/api/v1/namespaces/default/services/nginx-service
- 使用NodePort
Kubernetes中的service默认情况下都是使用的ClusterIP这种类型，这样的service会产生一个ClusterIP，这个IP只能在集群内部访问，要想让外部能够直接访问service，需要将service type修改为 nodePort，访问任意一个NodeIP:nodePort都将路由到ClusterIP。在某些场景下我们可以使用 Kubernetes 的 Proxy 模式来访问服务。
可采用命令方式或者yaml方式来创建服务：
[root@VM-12-10-centos ~]# kubectl delete deployment nginx
[root@VM-12-10-centos ~]# kubectl delete service nginx
[root@VM-12-10-centos ~]# kubectl create deployment nginx --image=nginx #先创建pod
1）yaml文件方式
apiVersion: v1
kind: Service
metadata:
name: nginx
spec:
selector:
app: nginx
type: NodePort
ports:
- name: http
port: 80
targetPort: 80
protocol: TCP
nodePort: 30000
[root@VM-12-10-centos ~]# kubectl create -f nginx.yaml
[root@VM-12-10-centos ~]# minikube service --url nginx # 获取访问地址
[root@VM-12-10-centos ~]# minikube service nginx #进入服务
[root@VM-12-10-centos ~]# kubectl get svc nginx
[root@VM-12-10-centos ~]# netstat -anp | grep 30000
没有端口发布，上面的是curl历史访问记录，一直没有办法访问：
[root@VM-12-10-centos ~]# kubectl describe svc/nginx
这里的EndPoints是none，说明service并没有绑定到任何一个pod，所以没法转发，那服务访问不通也是理所当然的事情了。
回过头去看了一下yaml文件果然发现了问题，在service中，缺少selector，所以没能绑定到pod，添加selector及对应标签之后，删除之前部署的应用。重新编辑nginx.yaml文件，因为刚才没有定义Deployment，修改如下，NodePort端口可以指定也可以不指定，不指定时由系统自行指定。
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels: # 这里是定义Deployment的标签
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx # 选择关联Deployment标签
template:
metadata:
labels: # 给Pod定义一个标签，方便其他服务关联这个Pod
app: nginx
spec:
containers:
- name: nginx
image: nginx:latest
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx-service
spec:
selector: # Service 的selector 指定标签 app:nginx 来进行对Pod进行关联 ；(这里的app:nginx就是上面Deployment配置里labels定义的标签 )
app: nginx
ports:
- protocol: TCP
port: 80
targetPort: 80
nodePort: 30000
type: NodePort
[root@VM-12-10-centos ~]# kubectl create -f nginx.yaml
[root@VM-12-10-centos ~]# kubectl describe svc/nginx-service
配置ipvs模式：
[root@VM-12-10-centos ~]# kubectl get pod -n kube-system
[root@VM-12-10-centos ~]# kubectl logs kube-proxy-vzrr9 -n kube-system
[root@VM-12-10-centos ~]# kubectl edit cm kube-proxy -n kube-system
[root@VM-12-10-centos ~]# kubectl edit cm kube-proxy -n kube-system
ipvs:
excludeCIDRs: null
minSyncPeriod: 0s
scheduler: ""
strictARP: false
syncPeriod: 30s
kind: KubeProxyConfiguration
metricsBindAddress: 127.0.0.1:10249
mode: "ipvs"
在master 和worker1上，所有不正确的worker上，执行操作：
cat > /etc/sysconfig/modules/ipvs.modules <<EOF
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack_ipv4
EOF
chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules && lsmod | grep -e ip_vs -e nf_conntrack_ipv4
删除原来的POD，会自动重启kube-proxy 的pod
[root@VM-12-10-centos ~]# kubectl get pod -n kube-system | grep kube-proxy |awk '{system("kubectl delete pod "$1" -n kube-system")}'
[root@VM-12-10-centos ~]# kubectl get pod -n kube-system -o wide
[root@VM-12-10-centos ~]# kubectl logs kube-proxy-v76kd -n kube-system
[root@VM-12-10-centos ~]# curl 192.168.49.2:30000 #只在node IP访问成功，外部访问不存功
[root@VM-12-10-centos ~]# minikube service nginx-service --url
http://192.168.49.2:30000
[root@VM-12-10-centos ~]# kubectl get service nginx-service --output='jsonpath="{.spec.ports[0].nodePort}"' # 获取notepord端口
"30000"
上述还是不能外面访问，只能在主机上访问，采用命令方式重试
[root@VM-12-10-centos ~]# kubectl create deployment nginx --image=nginx #或指定版本：kubectl create deployment nginx --image=nginx:latest\
[root@VM-12-10-centos ~]# kubectl expose deployment nginx --port=80 --type=NodePort #暴露服务
[root@VM-12-10-centos ~]# curl 192.168.49.2:30463
上述方式只能本机上访问，Host外部访问只能采用以下方式：
[root@VM-12-10-centos ~]# kubectl port-forward svc/nginx-service 30000:80 --address='0.0.0.0' & #端口转发
[root@VM-12-10-centos ~]# route -n # 查看网关，或 netstat -r
2、采用LoadBalancer
- 命令方式
[root@VM-12-5-centos ~]# kubectl create deployment nginx --image=nginx:latest
[root@VM-12-5-centos ~]# kubectl expose deployment nginx --type=LoadBalancer --port=8080
//In another window, start the tunnel to create a routable IP for the ‘nginx’ deployment:
[root@VM-12-5-centos ~]# minikube tunnel
//To find the routable IP, run this command and examine the `EXTERNAL-IP` column:
[root@VM-12-5-centos ~]# kubectl get services nginx
Your deployment is now available at <EXTERNAL-IP>:8080
- 使用yarm
[root@VM-12-10-centos ~]# kubectl delete deployment nginx-deployment
[root@VM-12-10-centos ~]# kubectl delete service nginx-service
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels: # 这里是定义Deployment的标签
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx # 选择关联Deployment标签
template:
metadata:
labels: # 给Pod定义一个标签，方便其他服务关联这个Pod
app: nginx
spec:
dnsPolicy: ClusterFirstWithHostNet
containers:
- name: nginx
image: nginx:latest
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx-service
spec:
selector: # Service 的selector 指定标签 app:nginx 来进行对Pod进行关联 ；(这里的app:nginx就是上面Deployment配置里labels定义的标签 )
app: nginx
ports:
- protocol: TCP
port: 80
targetPort: 80
nodePort: 30008
type: LoadBalancer
[root@VM-12-10-centos ~]# kubectl create -f nginx.yaml
或：
[root@VM-12-10-centos ~]# kubectl create deployment nginx --image=nginx:latest
[root@VM-12-10-centos ~]# kubectl expose deployment nginx --type=LoadBalancer --port=80 --target-port=80
[root@VM-12-10-centos ~]# kubectl get svc
对于支持负载均衡器的云服务平台而言，平台将提供一个外部 IP 来访问该服务。 在 Minikube 上，LoadBalancer 使得服务可以通过命令 minikube service 访问。
运行下面的命令：
[root@VM-12-10-centos ~]# minikube service nginx
[root@VM-12-10-centos ~]# minikube service nginx --url
http://192.168.49.2:30874
[root@VM-12-10-centos ~]# curl 192.168.49.2:30874
minikube 是一款基于 Kubernetes 的定位于快速验证功能的小型容器编排环境。
由于它的定位特性，我们在使用中会发现 minikube 虚拟出了一个 IP 作为自身的节点 IP，该 IP 和宿主机不同。对于 NodePort 类型的 Service 也没有办法通过 127.0.0.1 访问。我们必须通过 minikube ip 找到 minikube 的 IP 并通过它来访问 NodePort 类型的 service。
另外可以留意到 LoadBalancer 类型的 service 在默认情况下 external IP 为 ，为了要能够访问到 service，通过官网（minikube.sigs.k8s.io/docs/handbo…）可以看到，在启动服务之前，需要先执行 minikube tunnel，建立隧道和服务通信。在执行 minikube tunnel 后，可以发现 external IP 被设置成和 cluster IP 一样的值，这时候就可以通过 : 来访问 service 了。
[root@VM-12-10-centos ~]# minikube tunnel
minikube的Cluster-IP会路由到应用的Cluster-IP上面。
[root@VM-12-10-centos ~]# kubectl get svc
external-ip 没有一直处于 状态，external IP 被设置成和 cluster IP 一样的值。
[root@VM-12-10-centos ~]# minikube ip
[root@VM-12-10-centos ~]# curl 10.107.146.217
上述方式只能本机上访问Node，Host外部访问只能采用以下方式：
[root@VM-12-10-centos ~]# kubectl port-forward svc/nginx-service 8001:80 --address='0.0.0.0' & #端口转发
6、采用Ingress
Enable ingress addon:
minikube addons enable ingress
The following example creates simple echo-server services and an Ingress object to route to these services.
kind: Pod
apiVersion: v1
metadata:
name: foo-app
labels:
app: foo
spec:
containers:
- name: foo-app
image: 'kicbase/echo-server:1.0'
---
kind: Service
apiVersion: v1
metadata:
name: foo-service
spec:
selector:
app: foo
ports:
- port: 8080
---
kind: Pod
apiVersion: v1
metadata:
name: bar-app
labels:
app: bar
spec:
containers:
- name: bar-app
image: 'kicbase/echo-server:1.0'
---
kind: Service
apiVersion: v1
metadata:
name: bar-service
spec:
selector:
app: bar
ports:
- port: 8080
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: example-ingress
spec:
rules:
- http:
paths:
- pathType: Prefix
path: /foo
backend:
service:
name: foo-service
port:
number: 8080
- pathType: Prefix
path: /bar
backend:
service:
name: bar-service
port:
number: 8080
---
Apply the contents
kubectl apply -f https://storage.googleapis.com/minikube-site-examples/ingress-example.yaml
执行中可能会出错：
解决方法：
[root@VM-12-5-centos ~]# kubectl get ValidatingWebhookConfiguration
NAME WEBHOOKS AGE
ingress-nginx-admission 1 6h5m
[root@VM-12-5-centos ~]# kubectl delete -A ValidatingWebhookConfiguration ingress-nginx-admission
validatingwebhookconfiguration.admissionregistration.k8s.io "ingress-nginx-admission" deleted
重新执行：
kubectl apply -f https://storage.googleapis.com/minikube-site-examples/ingress-example.yaml
查看安装情况：
kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
example-ingress nginx * <your_ip_here> 80 5m45s
Note for Docker Desktop Users:
To get ingress to work you’ll need to open a new terminal window and run minikube tunnel and in the following step use 127.0.0.1 in place of <ip_from_above>.
Now verify that the ingress works
$ curl <ip_from_above>/foo
Request served by foo-app
...
$ curl <ip_from_above>/bar
Request served by bar-app
...
参考：www.cnblogs.com/g2thend/p/1…
