01｜搭建elasticsearch8.x实验集群

地铁林黛玉
276 阅读4分钟

开启掘金成长之旅！这是我参与「掘金日新计划 · 12 月更文挑战」的第4天，点击查看活动详情

1.本篇概述

本小节通过Docker搭建Elasticsarch的集群开发环境，为什么用docker呢，主要是考虑到用传统的通过Vmware安装虚拟机的方式比较耗费时间。通过docker的话相对来说轻松一点。

本篇主要任务是通过docker-compose编写集群编译环境，然后快速验证功能。

1.本小节通过Docker搭建elasticsearch的集群环境

1.1实验环境的准备

  • 一台内存大于等于16GB的电脑

  • Docker：Docker Desktop 4.13.1 (90346) 下载地址

1.2 镜像版本：

  • Elasticsearch:8.3.3
  • Cabana:8.3.3

1.3 准备工作

  • 在安装完Docker以后，我们如果直接拉去镜像会比较慢，这是因为此时仓库的地址是国外的，就跟github类似，需要从国外地址拉，所以速度十分缓慢，所以这边我们把镜像仓库地址改成网易的，点击设置->Docker Engine进行修改
image-20221113212349677.png 配置如下 
 {
   "builder": {
     "gc": {
       "defaultKeepStorage": "20GB",
       "enabled": true
     }
   },
   "experimental": false,
   "features": {
     "buildkit": true
   },
   "registry-mirrors": [
     "http://hub-mirror.c.163.com"
   ]
 }
  • 如果您的环境是Linux，注意要做以下修改，否则es可能会启动失败，用编辑工具打开文件/etc/sysctl.conf，在尾部添加一行配置vm.max_map_count = 262144，如果已存在就修改，数值不能低于262144，修改保存，然后执行命令sudo sysctl -p使其立即生效

2.编写配置文件

接下来我们编写docker-compose配置文件，在这之前，我们要新建.env文件，里边主要存放一些docker-compose.yaml的文件配置信息,大致内容如下

 # elastic账号的密码 (数字+字母+字符，大小大于6)
 ELASTIC_PASSWORD=Aa@123456
 ​
 # kibana_system账号的密码 (至少六个字符)，该账号仅用于一些kibana的内部设置，不能用来查询es
 KIBANA_PASSWORD=kibana
 ​
 # elasticsearch和kibana的版本
 STACK_VERSION=8.3.3
 ​
 # 集群名字
 CLUSTER_NAME=es-cluster
 ​
 # x-pack安全设置，这里选择basic，基础设置，如果选择了trail，则会在30天后到期
 LICENSE=basic
 #LICENSE=trial
 ​
 # es映射到宿主机的的端口
 ES_PORT=9200
 ​
 # kibana映射到宿主机的的端口
 KIBANA_PORT=5601
 ​
 # es容器的内存大小，请根据自己硬件情况调整
 MEM_LIMIT=1073741824
 ​
 # 命名空间，会体现在容器名的前缀上
 COMPOSE_PROJECT_NAME=mydemo
 ​
  • 接下里编写docker-compose.yaml文件,该文件的目的是创建一个启用了身份验证和网络加密的三节点安全 Elasticsearch 集群，以及一个与其安全连接的 Kibana 实例
 version: "1.0"
 ​
 services:
   setup:
     image: elasticsearch:${STACK_VERSION}
     volumes:
       - certs:/usr/share/elasticsearch/config/certs
     user: "0"
     command: >
       bash -c '
         if [ x${ELASTIC_PASSWORD} == x ]; then
           echo "Set the ELASTIC_PASSWORD environment variable in the .env file";
           exit 1;
         elif [ x${KIBANA_PASSWORD} == x ]; then
           echo "Set the KIBANA_PASSWORD environment variable in the .env file";
           exit 1;
         fi;
         if [ ! -f config/certs/ca.zip ]; then
           echo "Creating CA";
           bin/elasticsearch-certutil ca --silent --pem -out config/certs/ca.zip;
           unzip config/certs/ca.zip -d config/certs;
         fi;
         if [ ! -f config/certs/certs.zip ]; then
           echo "Creating certs";
           echo -ne \
           "instances:\n"\
           "  - name: es01\n"\
           "    dns:\n"\
           "      - es01\n"\
           "      - localhost\n"\
           "    ip:\n"\
           "      - 127.0.0.1\n"\
           "  - name: es02\n"\
           "    dns:\n"\
           "      - es02\n"\
           "      - localhost\n"\
           "    ip:\n"\
           "      - 127.0.0.1\n"\
           "  - name: es03\n"\
           "    dns:\n"\
           "      - es03\n"\
           "      - localhost\n"\
           "    ip:\n"\
           "      - 127.0.0.1\n"\
           > config/certs/instances.yml;
           bin/elasticsearch-certutil cert --silent --pem -out config/certs/certs.zip --in config/certs/instances.yml --ca-cert config/certs/ca/ca.crt --ca-key config/certs/ca/ca.key;
           unzip config/certs/certs.zip -d config/certs;
         fi;
         echo "Setting file permissions"
         chown -R root:root config/certs;
         find . -type d -exec chmod 750 {} ;;
         find . -type f -exec chmod 640 {} ;;
         echo "Waiting for Elasticsearch availability";
         until curl -s --cacert config/certs/ca/ca.crt https://es01:9200 | grep -q "missing authentication credentials"; do sleep 30; done;
         echo "Setting kibana_system password";
         until curl -s -X POST --cacert config/certs/ca/ca.crt -u elastic:${ELASTIC_PASSWORD} -H "Content-Type: application/json" https://es01:9200/_security/user/kibana_system/_password -d "{"password":"${KIBANA_PASSWORD}"}" | grep -q "^{}"; do sleep 10; done;
         echo "All done!";
       '
     healthcheck:
       test: ["CMD-SHELL", "[ -f config/certs/es01/es01.crt ]"]
       interval: 1s
       timeout: 5s
       retries: 120
 ​
   es01:
     depends_on:
       setup:
         condition: service_healthy
     image: elasticsearch:${STACK_VERSION}
     volumes:
       - certs:/usr/share/elasticsearch/config/certs
       - esdata01:/usr/share/elasticsearch/data
     ports:
       - ${ES_PORT}:9200
     environment:
       - node.name=es01
       - cluster.name=${CLUSTER_NAME}
       - cluster.initial_master_nodes=es01,es02,es03
       - discovery.seed_hosts=es02,es03
       - ELASTIC_PASSWORD=${ELASTIC_PASSWORD}
       - bootstrap.memory_lock=true
       - xpack.security.enabled=true
       - xpack.security.http.ssl.enabled=true
       - xpack.security.http.ssl.key=certs/es01/es01.key
       - xpack.security.http.ssl.certificate=certs/es01/es01.crt
       - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
       - xpack.security.http.ssl.verification_mode=certificate
       - xpack.security.transport.ssl.enabled=true
       - xpack.security.transport.ssl.key=certs/es01/es01.key
       - xpack.security.transport.ssl.certificate=certs/es01/es01.crt
       - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
       - xpack.security.transport.ssl.verification_mode=certificate
       - xpack.license.self_generated.type=${LICENSE}
     mem_limit: ${MEM_LIMIT}
     ulimits:
       memlock:
         soft: -1
         hard: -1
     healthcheck:
       test:
         [
           "CMD-SHELL",
           "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
         ]
       interval: 10s
       timeout: 10s
       retries: 120
 ​
   es02:
     depends_on:
       - es01
     image: elasticsearch:${STACK_VERSION}
     volumes:
       - certs:/usr/share/elasticsearch/config/certs
       - esdata02:/usr/share/elasticsearch/data
     environment:
       - node.name=es02
       - cluster.name=${CLUSTER_NAME}
       - cluster.initial_master_nodes=es01,es02,es03
       - discovery.seed_hosts=es01,es03
       - bootstrap.memory_lock=true
       - xpack.security.enabled=true
       - xpack.security.http.ssl.enabled=true
       - xpack.security.http.ssl.key=certs/es02/es02.key
       - xpack.security.http.ssl.certificate=certs/es02/es02.crt
       - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
       - xpack.security.http.ssl.verification_mode=certificate
       - xpack.security.transport.ssl.enabled=true
       - xpack.security.transport.ssl.key=certs/es02/es02.key
       - xpack.security.transport.ssl.certificate=certs/es02/es02.crt
       - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
       - xpack.security.transport.ssl.verification_mode=certificate
       - xpack.license.self_generated.type=${LICENSE}
     mem_limit: ${MEM_LIMIT}
     ulimits:
       memlock:
         soft: -1
         hard: -1
     healthcheck:
       test:
         [
           "CMD-SHELL",
           "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
         ]
       interval: 10s
       timeout: 10s
       retries: 120
 ​
   es03:
     depends_on:
       - es02
     image: elasticsearch:${STACK_VERSION}
     volumes:
       - certs:/usr/share/elasticsearch/config/certs
       - esdata03:/usr/share/elasticsearch/data
     environment:
       - node.name=es03
       - cluster.name=${CLUSTER_NAME}
       - cluster.initial_master_nodes=es01,es02,es03
       - discovery.seed_hosts=es01,es02
       - bootstrap.memory_lock=true
       - xpack.security.enabled=true
       - xpack.security.http.ssl.enabled=true
       - xpack.security.http.ssl.key=certs/es03/es03.key
       - xpack.security.http.ssl.certificate=certs/es03/es03.crt
       - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
       - xpack.security.http.ssl.verification_mode=certificate
       - xpack.security.transport.ssl.enabled=true
       - xpack.security.transport.ssl.key=certs/es03/es03.key
       - xpack.security.transport.ssl.certificate=certs/es03/es03.crt
       - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
       - xpack.security.transport.ssl.verification_mode=certificate
       - xpack.license.self_generated.type=${LICENSE}
     mem_limit: ${MEM_LIMIT}
     ulimits:
       memlock:
         soft: -1
         hard: -1
     healthcheck:
       test:
         [
           "CMD-SHELL",
           "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
         ]
       interval: 10s
       timeout: 10s
       retries: 120
 ​
   kibana:
     depends_on:
       es01:
         condition: service_healthy
       es02:
         condition: service_healthy
       es03:
         condition: service_healthy
     image: kibana:${STACK_VERSION}
     volumes:
       - certs:/usr/share/kibana/config/certs
       - kibanadata:/usr/share/kibana/data
     ports:
       - ${KIBANA_PORT}:5601
     environment:
       - SERVERNAME=kibana
       - ELASTICSEARCH_HOSTS=https://es01:9200
       - ELASTICSEARCH_USERNAME=kibana_system
       - ELASTICSEARCH_PASSWORD=${KIBANA_PASSWORD}
       - ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES=config/certs/ca/ca.crt
     mem_limit: ${MEM_LIMIT}
     healthcheck:
       test:
         [
           "CMD-SHELL",
           "curl -s -I http://localhost:5601 | grep -q 'HTTP/1.1 302 Found'",
         ]
       interval: 10s
       timeout: 10s
       retries: 120
 ​
 volumes:
   certs:
     driver: local
   esdata01:
     driver: local
   esdata02:
     driver: local
   esdata03:
     driver: local
   kibanadata:
     driver: local
 ​

我们在文件新建一个文件夹，存放上边两个编写完毕的文件

3.执行docker-compose.yaml文件

在docker-compose.yaml所在的文件夹执行如下命令 在docker-compose.yaml所在的文件夹执行如下命令

 docker-compose up -d

出现如下信息表示执行成功

image-20221113214513593.png 通过docker desktop查看集群情况，显示为Running说明启动成功 image.png

4.验证

4.1验证es集群

在浏览器输入https://localhost:9200/ ，输入用户名跟密码，用户名：elastic,密码就是我们之前在配置文件输入的密码

image-20221113215107110.png 点击登陆，现实如下，表示es集群启动成功 image.png
  • 在Chrome安装es集群监控工具：Multi Elasticsearch Head

压缩包已经转备好，请自行下载安装。安装完成后会在浏览器出现如下标志

image-20221113220758232.png

点击会弹出界面，在弹出的界面点击New,输入https://localhost:9200

image.png

完成后现实如下，es01节点前面显示五角星，说明它是主节点

image.png ES功能验证正常后，我们看下Kibana启动是否正常,在浏览器地址栏输入[http://localhost:5601](http://localhost:5601/)，在弹出的界面输入账户密码，账户:elastic，密码就是我们在.env文件配置的密码 image-20221113221414471.png ### 4. 创建一个简单的索引

点击kibana左侧三道小横杠，然后点击Dev tools

image-20221113221625072.png

在弹出的界面输入如下内容创建索引 
 PUT index_demo
 {
   "mappings": {
     "properties": {
       "name": {
         "type": "keyword"
       },
       "age": {
         "type": "integer"
       }
     }
   }
 }

image-20221113222001107.png 在这个索引中插入几条测试数据

 PUT index_demo/_bulk
 {
   "index": {
     "_id": 1
   }
 }
 {
   "name": "xiaoming",
   "age": 22
 }
 {
   "index": {
     "_id": 2
   }
 }
 {
   "name": "xiaohong",
   "age": 23
 }

image-20221113222300718.png 至此，我们的实验环境搭建完毕，接下来的章节我们将进行es的实践环节。

avatar
文章
阅读
粉丝