sealer实践

529 阅读5分钟

操作手册见:Quick start | sealer

sealer简介:

简介

sealer是阿里巴巴开源的集群镜像的一个实现方式,项目地址:github.com/sealerio/se…

sealer[ˈsiːlər]是一款分布式应用打包交付运行的解决方案,通过把分布式应用及其数据库中间件等依赖一起打包以解决应用整个集群整体交付问题。 sealer构建出来的产物我们称之为"集群镜像", 集群镜像里内嵌了一个kubernetes, 解决了分布式应用的交付一致性问题。 集群镜像可以push到registry中共享给其他用户使用,也可以在官方仓库中找到非常通用的分布式软件直接使用。

Docker可以把一个操作系统的rootfs+应用 build成一个容器镜像,sealer把kubernetes看成操作系统,在这个更高的抽象纬度上做出来的镜像就是集群镜像。 实现整个集群的Build Share Run !!!

注意事项

sealer会把k8s、docker等基础组件都打进镜像里面,所以,待部署的环境最好是干净的环境,会避免一些版本冲突导致的不必要bug。

sealer只支持比较高版本的linux环境,centos7以上最好。建议centos 7.6-8之间版本

sealer支持helm,sealer 与helm是协作关系,helm负责应用的编码,sealer负责集群的打包。

sealer支持集群机器的扩缩容,应用的扩缩容通过yaml文件,或者kubectl去修改就行

目前sealer支持的命令包括:

Available Commands:
  apply       apply a kubernetes cluster
  build       build an cloud image from a Kubefile
  cert        update k8s API server cert
  check       check the state of cluster
  completion  generate autocompletion script for bash
  debug       Create debugging sessions for pods and nodes
  delete      delete a cluster
  exec        exec a shell command or script on all node.
  gen         Generate a Clusterfile to take over a normal cluster which not deployed by sealer
  gen-doc     generate document for sealer CLI with MarkDown format
  help        Help about any command
  images      list all cluster images
  inspect     print the image information or clusterFile
  join        join node to cluster
  load        load image from a tar file
  login       login image repository
  merge       merge multiple images into one
  prune       prune sealer data dir
  pull        pull cloud image to local
  push        push cloud image to registry
  rmi         remove local images by name
  run         run a cluster with images and arguments
  save        save image to a tar file
  search      sealer search kubernetes
  tag         tag a image as a new one
  upgrade     upgrade your kubernetes cluster
  version     show sealer version

sealer基础操作

安装 Kubernetes 集群

wget https://github.com/sealerio/sealer/releases/download/v0.8.5/sealer-v0.8.5-linux-amd64.tar.gz && \
tar zxvf sealer-v0.8.5-linux-amd64.tar.gz && mv sealer /usr/bin

# run a kubernetes cluster

sealer run kubernetes:v1.19.8 \

--masters 192.168.0.2,192.168.0.3,192.168.0.4 \

--node 192.168.0.5,192.168.0.6,192.168.0.7 --passwd xxx
# kubectl get node

NAME STATUS ROLES AGE VERSION

izm5e42unzb79kod55hehvz Ready master 18h v1.19.8

izm5ehdjw3kru84f0kq7r7z Ready master 18h v1.19.8

izm5ehdjw3kru84f0kq7r8z Ready master 18h v1.19.8

izm5ehdjw3kru84f0kq7r9z Ready <none> 18h v1.19.8

izm5ehdjw3kru84f0kq7raz Ready <none> 18h v1.19.8

izm5ehdjw3kru84f0kq7rbz Ready <none> 18h v1.19.8

清理集群

一些基本设置的信息将被写入Clusterfile 并存储在/root/.sealer/[cluster-name]/Clusterfile 中。

sealer delete -f /root/.sealer/my-cluster/Clusterfile

构建自己的 CloudImage

例如,构建自己的集群镜像 CloudImage:

应用的yaml文件

web-deployment.yaml

apiVersion: v1
kind: Service
metadata:
  name: web
spec:
  type: NodePort
  ports:
  - name: rest
    port: 8080
    targetPort: 8080
    nodePort: 8080
  selector:
    app: web
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  selector:
    matchLabels:
      app: web
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
      - image: registry.xxx.com/tian/web:0.0.1
        name: web
        ports:
        - containerPort: 8080
          name: web
        

mysql-pv.yaml

apiVersion: v1
kind: PersistentVolume
metadata:
  name: mysql-pv-volume
  labels:
    type: local
spec:
  storageClassName: manual
  capacity:
    storage: 20Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/mnt/data"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-pv-claim
spec:
  storageClassName: manual
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi

mysql-deployment.yaml

apiVersion: v1
kind: Service
metadata:
  name: mysql
spec:
  type: NodePort
  ports:
  - name: rest
    port: 3306
    targetPort: 3306
    nodePort: 3306
  selector:
    app: mysql
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mysql
spec:
  selector:
    matchLabels:
      app: mysql
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
      - image: registry.xxx.com/tian/mysql:5.7.30
        name: mysql
        env:
          # Use secret in real usage
        - name: MYSQL_ROOT_PASSWORD
          value: xxx
        ports:
        - containerPort: 3306
          name: mysql
        volumeMounts:
        - name: mysql-persistent-storage
          mountPath: /var/lib/mysql
      volumes:
      - name: mysql-persistent-storage
        persistentVolumeClaim:
          claimName: mysql-pv-claim

Kubefile文件:

# base CloudImage contains all the files that run a kubernetes cluster needed.
#    1. kubernetes components like kubectl kubeadm kubelet and apiserver images ...
#    2. docker engine, and a private registry
#    3. config files, yaml, static files, scripts ...
FROM registry.cn-qingdao.aliyuncs.com/sealer-io/kubernetes:v1.19.8
COPY web-deployment.yaml .
COPY mysql-deployment.yaml .
COPY mysql-pv.yaml .
CMD kubectl apply -f mysql-pv.yaml, kubectl apply -f mysql-deployment.yaml, kubectl apply -f web-deployment.yaml

构建 CloudImage:

sealer build -t iregistry.xxx.com/tian/web-mysql:0.0.1 .

使用sealer运行您的 Kubernetes 集群:

# sealer will install a kubernetes on host 192.168.0.1 then apply the dashboard manifests

sealer run registry.xxx-int.com/tian/web-mysql:0.0.1 --masters 192.168.0.1 --passwd xxx

单节点运行集群还会出现master节点污点问题,导致pod一直处于pending状态,需要去除master污点

kubectl taint nodes nodeName node-role.kubernetes.io/master=:NoSchedule-

将 CloudImage 推送到注册表

# you can push the CloudImage to docker hub, Ali ACR, or Harbor

sealer push registry.xxx.com/tian/web-mysql:0.0.1

sealer高级操作

sealer自定义集群配置

Clusterfile 支持更多配置,例如用户定义的 kubeadm 配置、helm values 配置覆盖、插件...

本地新建一个clusterfile.yaml文件:

apiVersion: sealer.cloud/v2
kind: Cluster
metadata:
  name: my-cluster
spec:
  image: registry.xxx.com/tian/web-mysql:0.0.1
  hosts:
    - ips: [ 192.168.0.1 ]
      roles: [ master ]
---
kind: ClusterConfiguration
kubernetesVersion: v1.19.8
apiServer:
  extraArgs:
    service-node-port-range: 8000-9000

上面代码自定义了service对外暴露端口范围,默认是30000以上,不符合机器端口限制外网访问端,所以需要修改。

然后执行命令

sealer apply -f Clusterfile.yaml

这样就能使用自定义的配置去覆盖默认配置,实现集群启动。

覆盖 CloudRootfs 文件

有时不想关心 CloudRootfs 上下文,但需要自定义一些配置。

您可以kubernetes:v1.19.8用作 BaseImage,并使用您自己的配置文件覆盖 CloudRootfs 中的默认文件。

例如:daemon.json 是您的 docker 引擎配置,使用它来覆盖默认配置:

FROM kubernetes:v1.19.8

COPY daemon.json etc/
sealer build -t user-defined-kubernetes:v1.19.8 .

在已有集群上部署新的服务

业务场景中经常存在在已有集群上部署新服务,这个时候就需要构建一个应用镜像,然后run到集群中。

第一步:构建应用镜像,注意不是集群镜像,应用镜像需要从openebs-localpv基础镜像去构建,这样就不包含k8s集群相关的组件了

编辑应用镜像的kubefile文件,yaml文件省略,其实就是部署一个redis的基础yaml文件配置

FROM registry.cn-qingdao.aliyuncs.com/sealer-apps/openebs-localpv:3.2.0

COPY redis-confg.yaml .

COPY redis-deployment.yaml .

COPY redis-pv.yaml .

CMD kubectl apply -f redis-confg.yaml, kubectl apply -f redis-pv.yaml, kubectl apply -f redis-deployment.yaml

执行构建命令

sealer build -f Kubefile -t registry.xxx.com/tian/redis-sealer:0.0.1 --base=false .

第二步:运行应用镜像

sealer run registry.xxx.com/tian/redis-sealer:0.0.1 --masters 192.168.0.1