持续创作，加速成长！这是我参与「掘金日新计划 · 10 月更文挑战」的第4天，点击查看活动详情

heketi部署

heketi部署需要用到持久化存储，在生产环境中一般要求高可用的存储方案，通常我们部署gluster及heketi就是为了提供高可用存储方案，没有部署完成之前是没有高可用存储的，所以可以将heketi部署分为两个阶段，第一阶段heketi持久化存储使用本地存储，比如hostpath完成gluster集群配置及glusterfs高可用卷组生成，第二阶段将heketi持久化存储迁移到glusterfs高可用卷。

使用本地存储（hostpath）部署bootstrap heketi
k apply -f heketi-bootstrap.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: heketi-service-account
  namespace: yunwei
---
kind: Service
apiVersion: v1
metadata:
  name: heketi
  namespace: yunwei
spec:
  selector:
    app: heketi
  ports:
  - name: heketi
    port: 80
    targetPort: 8080
---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: heketi
  namespace: yunwei
spec:
  replicas: 2
  selector:
    matchLabels:
      app: heketi
  template:
    metadata:
      name: heketi
      labels:
        app: heketi
    spec:
      serviceAccountName: heketi-service-account
      nodeSelector:
        joinpay.com/yunwei: glusterfs
      tolerations:
      - effect: NoSchedule
        key: joinpay.com/yunwei
      containers:
      - image: reg-prd.joinpay.com/library/heketi
        imagePullPolicy: IfNotPresent
        name: heketi
        env:
        - name: HEKETI_EXECUTOR
          value: kubernetes
        - name: HEKETI_FSTAB
          value: "/var/lib/heketi/fstab"
        - name: HEKETI_SNAPSHOT_LIMIT
          value: '14'
        - name: HEKETI_KUBE_GLUSTER_DAEMONSET
          value: "y"
        - name: HEKETI_ADMIN_KEY
          value: "admin123"
        ports:
        - containerPort: 8080
        volumeMounts:
        - name: heketi-storage
          mountPath: "/var/lib/heketi"
        #readinessProbe:
        #  timeoutSeconds: 3
        #  initialDelaySeconds: 3
        #  httpGet:
        #    path: "/hello"
        #    port: 8080
        #livenessProbe:
        #  timeoutSeconds: 3
        #  initialDelaySeconds: 30
        #  httpGet:
        #    path: "/hello"
        #    port: 8080
      volumes:
      - name: heketi-storage
        hostPath:
          path: "/home/jpadmin/heketi-data"
      #- glusterfs:
      #    endpoints: heketi-storage-endpoints
      #    path: heketidbstorage
      #  name: heketi-storage

进入heketi容器，导入gluster节点拓扑信息创建gluster集群
heketi-cli topology load --json=topo.json

{
  "clusters": [
    {
      "nodes": [
        {
          "node": {
            "hostnames": {
              "manage": [
                "fs10dfx33.jopa.cc"
              ],
              "storage": [
                "10.xx.xx.10"
              ]
            },
            "zone": 1
          },
          "devices": [
            "/dev/sda5"
          ]
        },
        {
          "node": {
            "hostnames": {
              "manage": [
                "fs11dfx33.jopa.cc"
              ],
              "storage": [
                "10.xx.xx.11"
              ]
            },
            "zone": 1
          },
          "devices": [
            "/dev/sda5"
          ]
        },
        {
          "node": {
            "hostnames": {
              "manage": [
                "fs12dfx33.jopa.cc"
              ],
              "storage": [
                "10.xx.xx.12"
              ]
            },
            "zone": 1
          },
          "devices": [
            "/dev/sda5"
          ]
        }
      ]
    }
  ]
}

创建heketi专用gulusterfs卷(Name:heketidbstorage)
heketi-cli setup-openshift-heketi-storage
拷贝heketi容器/var/lib/heketi/heketi.db到heketidbstorage卷中
可以将heketidbstorage卷挂载到heketi的一个临时目录中，拷贝/var/lib/heketi/heketi.db到该目录，如

        volumeMounts:
        - name: heketi-storage
          mountPath: "/var/lib/heketi"
        - name: heketi-storage-temp
          mountPath: "/var/lib/heketi-temp"
      volumes:
      - name: heketi-storage
        hostPath:
          path: "/home/jpadmin/heketi-data"
      - glusterfs:
          endpoints: heketi-storage-endpoints
          path: heketidbstorage
        name: heketi-storage-temp

将heketidbstorage卷挂载为/var/lib/heketi/启动新的heketi部署

      volumes:
      #- name: heketi-storage
      #  hostPath:
      #    path: "/home/jpadmin/heketi-data"
      - glusterfs:
          endpoints: heketi-storage-endpoints
          path: heketidbstorage
        name: heketi-storage

heketi持久化信息修改

heketi db import/export --jsonfile=/json/file/path/ --dbfile=/db/file/path/将heketi db文件导出为json文件进行修改，再导入为db文件实现对heketi集群拓扑、volume、brick等的修改

k8s集群部署gluster及使用系列（四）

heketi部署

heketi部署

heketi持久化信息修改