持续创作，加速成长！这是我参与「掘金日新计划 · 10 月更文挑战」的第5天，点击查看活动详情

k8s集群中使用glusterfs

GlusterFS主要用来提供k8s集群持久化存储pv的动态自动供给功能。相对而言，具备部署简单、高可用、易扩展等优点。 本文主要介绍在k8s集群使用glusterfs文件系统。

glusterfs客户端安装

客户端安装glusterfs-fuse 需要挂载使用glusterfs文件系统的主机，包括k8s节点，需要安装glusterfs-fuse包，否则报错unknown filesystem type 'glusterfs'或未知的文件系统类型“glusterfs”。
yum install -y glusterfs-fuse

静态使用glusterfs卷

1. heketi创建volume
   heketi-cli --user=admin --secret=admin123 volume create --size=100 --snapshot-factor=1.5 --replica=3 --name=test
2. k8s创建glusterfs集群访问endpoint

apiVersion: v1
kind: Endpoints
metadata:
  name: glusterfs-endpoints
subsets:
- addresses:
  - ip: 10.xx.xx.12
  ports:
  - port: 1
    protocol: TCP
- addresses:
  - ip: 10.xx.xx.10
  ports:
  - port: 1
    protocol: TCP
- addresses:
  - ip: 10.xx.xx.11
  ports:
  - port: 1
    protocol: TCP

3. pod中声明volmes和挂载

    spec:
      containers:
        ...
        volumeMounts:
        - mountPath: /ep
          name: ep
      volumes:
      - glusterfs:
          endpoints: heketi-storage-endpoints
          path: test
        name: heketi-storage

4. pod 容器中查看和使用挂载的文件系统，可以看到其文件系统类型为fuer.glusterfs

[root@gs-test-75447f877b-5fh8l ep]# df -hT /host /ep
Filesystem                                       Type            Size  Used Avail Use% Mounted on
/dev/mapper/centos-home                          xfs             4.4T   25G  4.3T   1% /host
10.xx.xx.11:vol_72a1761add51f87644316676359ed801 fuse.glusterfs  100G  1.1G   99G   2% /ep

通过定义k8s storageclass动态提供pv

1. 创建storageclass 参考官方文档
   kubectl apply -f storageclass.yaml
2. 创建pvc(StatefulSet volumeClaimTemplates创建或手动创建)
   kubectl apply -f pvc.yaml
3. 在pod voumes中使用pvc和挂载

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: gs-test
  name: gs-test
spec:
  replicas: 1
  selector:
    matchLabels:
      app: gs-test
  template:
    metadata:
      labels:
        app: gs-test
    spec:
      containers:
      - image: <reg_url>/centos:7
        imagePullPolicy: IfNotPresent
        name: busybox
        volumeMounts:
        - mountPath: /replicate
          name: replicate
        - mountPath: /disperse
          name: disperse
        - mountPath: /distribute
          name: distribute
        - mountPath: /host
          name: host
        - mountPath: /ep
          name: ep
        command:
        - tail
        - -f
        - /dev/null
      volumes:
      - name: replicate
        persistentVolumeClaim:
          claimName: replicate
      - name: distribute
        persistentVolumeClaim:
          claimName: distribute
      - name: disperse
        persistentVolumeClaim:
          claimName: disperse
      - name: host
        hostPath:
          path: /home/test
          type: Directory
      - glusterfs:
          endpoints: glusterfs-endpoints
          path: vol_72a1761add51f87644316676359ed801
        name: ep