记录一次安装elasticsearch的问题与解决方案

CVSoldier
828 阅读4分钟

安装elasticsearch-7.3.0

  • 使用statefuset安装elasticsearch
apiVersion: apps/v1
kind: StatefulSet
metadata:
  namespace: kube-system  #namespace指定安装的命名空间
spec:
  podManagementPolicy: Parallel
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: elasticsearch-master   #匹配到应用名称
  serviceName: elasticsearch-master-headless
  template:
    metadata:
      annotations:
        configchecksum: 7ba9bf72c4105cb5b3bc1d6ba18674d9d646ffa931536d3eb0dab817187fead
      labels:
        app: elasticsearch-master
      name: elasticsearch-master
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - key: app
                operator: In
                values:
                - elasticsearch-master
            topologyKey: kubernetes.io/hostname
      automountServiceAccountToken: true
      containers:
      - env:
        - name: node.name
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.name
        - name: cluster.initial_master_nodes
          value: elasticsearch-master-0
        - name: discovery.seed_hosts
          value: elasticsearch-master-headless
        - name: cluster.name
          value: elasticsearch
        - name: network.host
          value: 0.0.0.0
        #- name: cluster.deprecation_indexing.enabled  # es-7.3.0版本没有该参数
        #  value: "false"
        - name: node.data   #指定角色
          value: "true"
        - name: node.ingest #指定角色
          value: "true"
        - name: node.master #指定角色
          value: "true"
        - name: node.ml     #指定角色
          value: "true"
        #- name: node.remote_cluster_client  # es-7.3.0版本没有该参数 7.16以上该参数放开
        #  value: "true"
        image: hub.aosccs.com.cn:8888/library/elasticsearch:7.3.2  #我用的本地harbor仓库，也可用dockerhub的镜像文件
        imagePullPolicy: IfNotPresent
        name: elasticsearch
        ports:
        - containerPort: 9200
          name: http
          protocol: TCP
        - containerPort: 9300
          name: transport
          protocol: TCP
          resources:
          limits:
            cpu: "1"
            memory: 2Gi
          requests:
            cpu: "1"
            memory: 2Gi
        securityContext:
          #capabilities:
          #  drop:
          #  - ALL
          runAsNonRoot: false
          runAsUser: 0
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /usr/share/elasticsearch/data
          name: elasticsearch-master   # 定义data映射目录
        - mountPath: /usr/share/elasticsearch/config/elasticsearch.yml
          name: esconfig
          subPath: elasticsearch.yml
        - mountPath: /usr/share/elasticsearch/config/certs
          name: app-cert   #生成证书  此方法不适合生成集群所有的证书，如果集群要是用证书需要将证书用secrets或configmap挂载出来再绑定到pod上；
      lifecycle:
        postStart:
          exec:
            commond:
            - /bin/bash
            - -c
            - /usr/share/elasticsearch/config/setPass.sh
      dnsPolicy: ClusterFirst
      enableServiceLinks: true
      initContainers:
      - command:
        - sysctl
        - -w
        - vm.max_map_count=262144
        image: hub.aosccs.com.cn:8888/library/elasticsearch:7.3.2
        imagePullPolicy: IfNotPresent
        name: configure-sysctl
        resources: {}
        securityContext:
          privileged: true
          runAsUser: 0
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      - command:
        - bash
        - -c
        - |
          /usr/share/elasticsearch/bin/elasticsearch-certutil ca --days 1300 --out /appcert/elasticsearch-ca.p12 --pass "" 
          /usr/share/elasticsearch/bin/elasticsearch-certutil cert -ca /appcert/elasticsearch-ca.p12 --days 1300 --out /appcert/elasticsearch.p12 --pass "" --ca-pass ""
        image: hub.aosccs.com.cn:8888/library/elasticsearch:7.3.2
        imagePullPolicy: IfNotPresent
        name: certificate
        resources: {}
        volumeMounts:
          - name: app-cert
            mountPath: /appcert
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext:
        fsGroup: 1000
        runAsUser: 1000
      volumes:
      - configMap:
          defaultMode: 424
          name: elasticsearch-master-config
        name: esconfig
      - name: app-cert
        emptyDir: {}
      updateStrategy:
        type: RollingUpdate
      volumeClaimTemplates:
      - metadata:
          creationTimestamp: null
          name: elasticsearch-master
        spec:
          accessModes:
          - ReadWriteOnce
          resources:
            requests:
              storage: 30Gi
          storageClassName: managed-nfs-storage
          volumeMode: Filesystem
status: 
  replicas: 1
  • 配置访问service
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: elasticsearch-master
  name: elasticsearch-master
  namespace: kube-system
spec:
  externalTrafficPolicy: Cluster
  ports:
  - name: http
    nodePort: 29200
    port: 9200
    protocol: TCP
    targetPort: 9200
  - name: transport
    nodePort: 29300
    port: 9300
    protocol: TCP
    targetPort: 9300
  selector:
    app: elasticsearch-master
    chart: elasticsearch
    release: elastic
  sessionAffinity: None
  type: NodePort
status:
  loadBalancer: {}

---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: elasticsearch-master
  name: elasticsearch-master-headless
  namespace: kube-system
spec:
  clusterIP: None
  ports:
  - name: http
    port: 9200
    protocol: TCP
    targetPort: 9200
  - name: transport
    port: 9300
    protocol: TCP
    targetPort: 9300
  publishNotReadyAddresses: true
  selector:
    app: elasticsearch-master
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}
  • cm配置文件
apiVersion: v1
data:
  elasticsearch.yml: |+ 
    cluster.name: elasticsearch
    node.name: elasticsearch-master-0
    network.host: 0.0.0.0
    discovery.seed_hosts: ["elasticsearch-master-0.elasticsearch-master"]
    cluster.initial_master_nodes: elasticsearch-master-0
    transport.host: 0.0.0.0
    xpack.security.enabled: true
    xpack.security.transport.ssl.enabled: true
    xpack.security.transport.ssl.cipher_suites: "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256"
    xpack.security.transport.ssl.verification_mode: certificate
    xpack.security.transport.ssl.keystore.path: certs/elasticsearch.p12
    xpack.security.transport.ssl.truststore.path: certs/elasticsearch.p12
  setPass.sh: |+
    #!/bin/bash
    ADMIN_USER=admin
    ADMIN_PASS="123456"
    USER_PASS=""
    BIND_IP="127.0.0.1"
    ELASTIC_PORT="9200"
    USER_ARR=(elastic kibana kibana_system apm_system logstash_system beats_system remote_monitoring_user)

    UserOperation() {

      /usr/share/elasticsearch/bin/elasticsearch-users useradd ${ADMIN_USER} -p ${ADMIN_PASS} -r superuser
      echo "init users"

      for i in ${USER_ARR[@]}; do
          curl -u ${ADMIN_USER}:${ADMIN_PASS} \
            -XPUT 'http://'${BIND_IP}':'${ELASTIC_PORT}'/_xpack/security/user/'${i}'/_password?pretty' \
            -H 'Content-Type:application/json' \
            -d'{"password":"'${USER_PASS}'"}'
      done

      /usr/share/elasticsearch/bin/elasticsearch-users userdel ${ADMIN_USER}
      #check_statu "del superuser"

      echo "init job done"

    }

    UserOperation
kind: ConfigMap
metadata:
  labels:
    app: elasticsearch-master
  name: elasticsearch-master-config
  namespace: kube-system

es安装集群问题与解决方案：

  • 1.Path does not chain with any of the trust anchors异常解决

解决方案：1.将证书作为secrets挂载，所有pod容器绑定该证书如果有AccessDeniedException,需要对证书进行赋权操作(chown 1000:1000 证书文件.p12)

  • 2.AccessDeniedException解决

解决方案：需要对证书进行赋权操作(chown 1000:1000 证书文件.p12)

  • 通过secrets挂载证书文件 kubectl create secrets generic 秘钥名称 --from-file=文件路径

  • defaultMode: 755 读取权限是4，写权限是2，执行权限是1，例子：755第一位7代表文件拥有者有文件的读取、写、执行权限，5代表同组用户有读取、执行权限，第三位5代表其他组用户拥有读取、执行权限；

  • 3.unable to read from standard input; is standard input open and a tty attached

解决：在stateset文件里continer下的添加属性tty: true stdin: true

    1. ERROR: Failed to verify bootstrap password 使用elasticsearch-setup-passwords脚本修改密码失败
    1. Nodes are not aviliable使用证书通过9300端口连接es错误

确认连接端口是否是9300或9300的映射端口(例如：9300 >> 29300), 确保clusterName和es的clusterName一致；

    1. elasticsearch-users add admin -p 123456 -r sueruser建立超级用户，更改elastic用户的密码提示401错误；

使用root用户执行命令，创建超级用户后使用命令修改elastic的密码 curl -u ${ADMIN_USER}:${ADMIN_PASS} \ -XPUT 'http://'${BIND_IP}':'${ELASTIC_PORT}'/_xpack/security/user/elastic/_password?pretty' \ -H 'Content-Type:application/json' \ -d'{"password":"'${USER_PASS}'"}'

  • 7.使用elasticsearch自带的工具产生证书；

进入es的pod容器内部 kubectl exec -it <容器名> /bin/bash 执行./bin/elasticsearch-certutil ca --days 3650 --out /usr/local/share/elasticsearch/config/elasticsearch.p12 --pass "" 执行完毕会生成elastic-stack-ca.p12,将容器内部的文件拷贝到本地kubectl cp -n <es服务所在namespace> <容器名>:/usr/share/elasticsearch/elastic-stack-ca.p12 /tmp/elastic-certification.p12

avatar
文章
阅读
粉丝