安装elasticsearch-7
官方文档
环境
| IP | 节点 |
|---|---|
| 192.168.20.32 | node-32 |
| 192.168.20.33 | node-33 |
| 192.168.20.34 | node-34 |
发现和集群形成设置
-
discovery.seed_hosts -
(静态) 提供集群中符合主控条件的节点的地址列表。也可以是包含用逗号分隔的地址的单个字符串。每个地址的格式为
host:port或host。这host是要由 DNS 解析的主机名、IPv4 地址或 IPv6 地址。IPv6 地址必须用方括号括起来。如果一个主机名通过 DNS 解析为多个地址，Elasticsearch 会使用所有这些地址。DNS 查找受 JVM DNS 缓存的约束。如果port未给出，则通过依次检查以下设置来确定：transport.profiles.default.porttransport.port
如果这些都没有设置，那么默认端口是
9300.discovery.seed_hosts的默认值为["127.0.0.1", "[::1]"]。见discovery.seed_hosts。此设置以前称为
discovery.zen.ping.unicast.hosts. 它的旧名称已被弃用，但会继续工作以保持向后兼容性。在未来的版本中将删除对旧名称的支持。 -
discovery.seed_providers -
（静态）指定使用哪种类型的种子主机提供程序来获取用于启动发现过程的种子节点的地址。默认情况下， 基于设置的种子主机提供程序从设置中获取种子节点地址
discovery.seed_hosts。此设置以前称为discovery.zen.hosts_provider. 它的旧名称已被弃用，但会继续工作以保持向后兼容性。在未来的版本中将删除对旧名称的支持。 -
discovery.type -
(静态) 指定 Elasticsearch 是否应该形成一个多节点集群。默认情况下，Elasticsearch 在形成集群时会发现其他节点，并允许其他节点稍后加入集群。如果
discovery.type设置为single-node，则 Elasticsearch 形成一个单节点集群并抑制 和 设置的cluster.publish.timeout超时cluster.join.timeout。有关何时可以使用此设置的更多信息，请参阅单节点发现。 -
cluster.initial_master_nodes -
在全新集群中设置初始的符合主节点的节点集。默认情况下，这个列表是空的，这意味着这个节点希望加入一个已经被引导的集群。见
cluster.initial_master_nodes。
配置集群
node-32
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-32
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /path/to/data
#
# Path to log files:
#
path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 192.168.20.32
#
# Set a custom port for HTTP:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.seed_hosts: ["192.168.20.32", "192.168.20.33", "192.168.20.34"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
cluster.initial_master_nodes: ["node-32", "node-33", "node-34"]
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
#
#是否可作为主节点
node.master: true
#是否存储数据
node.data: true
#启用或禁用跨域资源共享
http.cors.enabled: true
##允许哪些来源
http.cors.allow-origin: "*"
##允许哪些标题。默认为`X-Requested-With, Content-Type, Content-Length`
http.cors.allow-headers: Authorization,X-Requested-With,Content-Type,Content-Length
#是否锁住内存,避免交换(swapped)带来的性能损失,默认值是: *false*
bootstrap.memory_lock: false
node-33
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-33
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /path/to/data
#
# Path to log files:
#
path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 192.168.20.33
#
# Set a custom port for HTTP:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.seed_hosts: ["192.168.20.32", "192.168.20.33","192.168.20.34"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-32", "node-33", "node-34"]
cluster.initial_master_nodes: ["node-32", "node-33", "node-34"]
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
#是否可作为主节点
node.master: true
#是否存储数据
node.data: true
#启用或禁用跨域资源共享
http.cors.enabled: true
##允许哪些来源
http.cors.allow-origin: "*"
##允许哪些标题。默认为`X-Requested-With, Content-Type, Content-Length`
http.cors.allow-headers: Authorization,X-Requested-With,Content-Type,Content-Length
#是否锁住内存,避免交换(swapped)带来的性能损失,默认值是: *false*
bootstrap.memory_lock: false
node-34
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-34
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /path/to/data
#
# Path to log files:
#
path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 192.168.20.34
#
# Set a custom port for HTTP:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.seed_hosts: ["192.168.20.32", "192.168.20.33", "192.168.20.34"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-32", "node-33", "node-34"]
cluster.initial_master_nodes: ["node-32", "node-33", "node-34"]
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
#是否可作为主节点
node.master: true
#是否存储数据
node.data: true
#启用或禁用跨域资源共享
http.cors.enabled: true
##允许哪些来源
http.cors.allow-origin: "*"
##允许哪些标题。默认为`X-Requested-With, Content-Type, Content-Length`
http.cors.allow-headers: Authorization,X-Requested-With,Content-Type,Content-Length
#是否锁住内存,避免交换(swapped)带来的性能损失,默认值是: *false*
bootstrap.memory_lock: false
启动
# 依次启动3个节点
./bin/elasticsearch
#任意节点执行校验
[root@localhost software]# curl -XGET 'http://192.168.20.33:9200/_cat/nodes?v'
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
192.168.20.33 7 97 0 0.08 0.05 0.06 dilmrt - node-33
192.168.20.34 11 97 0 0.05 0.07 0.09 dilmrt - node-34
192.168.20.32 7 97 0 0.01 0.02 0.05 dilmrt * node-32
至此，没有安全认证的es集群已安装好
保护集群
安全性通过以下方式保护 Elasticsearch 集群：
-
通过密码保护、基于角色的访问控制和 IP 过滤来 防止未经授权的访问。
-
使用 SSL/TLS 加密 保护数据的完整性。
-
维护审计跟踪 ，以便您了解谁在对您的集群及其存储的数据执行什么操作
使用 SSL/TLS 加密
验证xpack.security.enabled设置是否为true。有关详细信息，请参阅安全设置
-
将每个节点配置为：
- 必需：在传输层上启用 TLS。
- 推荐：在 HTTP 层启用 TLS。
在传输层上启用 TLS
任意节点执行，本次在node-32
# 1,生成证书
#该命令输出一个默认名称为`elastic-stack-ca.p12`. 此文件是一个 PKCS#12 密钥库，其中包含 CA 的公共证书##和用于签署每个节点的证书的私钥
## 假设这里密码为123456
[ela@localhost elasticsearch-7.9.3]$ ./bin/elasticsearch-certutil ca
xxx
Please enter the desired output file [elastic-stack-ca.p12]:
Enter password for elastic-stack-ca.p12 : 123456
# 2,为集群中的每个节点生成证书和私钥
# 该命令会生成elastic-certificates.p12文件
[ela@localhost elasticsearch-7.9.3]$ bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12
xxx
#输入123456
Enter password for CA (elastic-stack-ca.p12) : 123456
Please enter the desired output file [elastic-certificates.p12]:
Enter password for elastic-certificates.p12 : 123456
Certificates written to /home/software/elasticsearch-7.9.3/elastic-certificates.p12
# 3,设置权限
# 移动config
[ela@localhost elasticsearch-7.9.3]$ mv elastic-certificates.p12 ./config/
#同步到node-33
[ela@localhost elasticsearch-7.9.3]$ scp config/elastic-certificates.p12 ela@192.168.20.33:/home/software/elasticsearch-7.9.3/config/
ela@192.168.20.33's password:
elastic-certificates.p12 100% 3581 3.5KB/s 00:00
#同步到node-34
[ela@localhost elasticsearch-7.9.3]$ scp config/elastic-certificates.p12 ela@192.168.20.34:/home/software/elasticsearch-7.9.3/config/
ela@192.168.20.34's password:
elastic-certificates.p12 100% 3581 3.5KB/s 00:00
# 依次在3个节点执行
[ela@localhost elasticsearch-7.9.3]$ chmod 777 config/elastic-certificates.p12
配置elasticsearch.yml
# 依次在3个节点上add
# 设置为`true`在节点上启用 Elasticsearch 安全功能
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
将密码添加到您的 Elasticsearch 密钥库
#依次在3个节点执行1/2
[ela@localhost elasticsearch-7.9.3]$ bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
future versions of Elasticsearch will require Java 11; your Java version from [/home/software/jdk1.8.0_333/jre] does not meet this requirement
Enter value for xpack.security.transport.ssl.keystore.secure_password: 123456
#依次在3个节点执行2/2
[ela@localhost elasticsearch-7.9.3]$ bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password
future versions of Elasticsearch will require Java 11; your Java version from [/home/software/jdk1.8.0_333/jre] does not meet this requirement
Enter value for xpack.security.transport.ssl.truststore.secure_password: 123456
重启
依次重启3个节点
./bin/elasticsearch -d
至此，在传输层上启用 TLS集群配置完成
此时打开ES
为所有内置用户设置密码
Elasticsearch 安全功能提供 内置用户来帮助您启动和运行。该elasticsearch-setup-passwords命令是首次设置内置用户密码的最简单方法
#任意一个节点执行
#假设所有用户密码都为654321
[root@localhost elasticsearch-7.9.3]# ./bin/elasticsearch-setup-passwords interactive
future versions of Elasticsearch will require Java 11; your Java version from [/home/software/jdk1.8.0_333/jre] does not meet this requirement
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y
Enter password for [elastic]:
Reenter password for [elastic]:
Enter password for [apm_system]:
Reenter password for [apm_system]:
Enter password for [kibana_system]:
Reenter password for [kibana_system]:
Enter password for [logstash_system]:
Reenter password for [logstash_system]:
Enter password for [beats_system]:
Reenter password for [beats_system]:
Enter password for [remote_monitoring_user]:
Reenter password for [remote_monitoring_user]:
Changed password for user [apm_system]
Changed password for user [kibana_system]
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [remote_monitoring_user]
Changed password for user [elastic]
# 校验 ，任一节点执行
[root@localhost elasticsearch-7.9.3]# curl -XGET --user elastic:654321 'http://192.168.20.32:9200/_cat/nodes?pretty'
192.168.20.32 10 97 4 0.01 0.16 0.17 dilmrt - node-32
192.168.20.34 10 95 4 0.02 0.14 0.14 dilmrt - node-34
192.168.20.33 12 97 4 0.09 0.15 0.15 dilmrt * node-33
再次打开ES，输入elastic：654321 登录成功
至此，安全认证的es集群已安装好
【可选 】在 HTTP 层启用 TLS
Set up basic security plus HTTPS traffic
# 1,生成证书
#此命令会生成一个 zip 文件，其中包含用于 Elasticsearch 和 Kibana 的证书和密钥
#将`http.p12`文件`elasticsearch`夹中的文件复制到每个节点上 Elasticsearch 配置目录中的一个目录中
[ela@localhost elasticsearch-7.9.3]$ bin/elasticsearch-certutil http
future versions of Elasticsearch will require Java 11; your Java version from [/home/software/jdk1.8.0_333/jre] does not meet this requirement
## Elasticsearch HTTP Certificate Utility
The 'http' command guides you through the process of generating certificates
for use on the HTTP (Rest) interface for Elasticsearch.
This tool will ask you a number of questions in order to generate the right
set of files for your needs.
## Do you wish to generate a Certificate Signing Request (CSR)?
A CSR is used when you want your certificate to be created by an existing
Certificate Authority (CA) that you do not control (that is, you don't have
access to the keys for that CA).
If you are in a corporate environment with a central security team, then you
may have an existing Corporate CA that can generate your certificate for you.
Infrastructure within your organisation may already be configured to trust this
CA, so it may be easier for clients to connect to Elasticsearch if you use a
CSR and send that request to the team that controls your CA.
If you choose not to generate a CSR, this tool will generate a new certificate
for you. That certificate will be signed by a CA under your control. This is a
quick and easy way to secure your cluster with TLS, but you will need to
configure all your clients to trust that custom CA.
Generate a CSR? [y/N]n
## Do you have an existing Certificate Authority (CA) key-pair that you wish to use to sign your certificate?
If you have an existing CA certificate and key, then you can use that CA to
sign your new http certificate. This allows you to use the same CA across
multiple Elasticsearch clusters which can make it easier to configure clients,
and may be easier for you to manage.
If you do not have an existing CA, one will be generated for you.
Use an existing CA? [y/N]y
## What is the path to your CA?
Please enter the full pathname to the Certificate Authority that you wish to
use for signing your new http certificate. This can be in PKCS#12 (.p12), JKS
(.jks) or PEM (.crt, .key, .pem) format.
CA Path: /home/software/elasticsearch-7.9.3/elastic-stack-ca.p12
Reading a PKCS12 keystore requires a password.
It is possible for the keystore's password to be blank,
in which case you can simply press <ENTER> at the prompt
Password for elastic-stack-ca.p12:
## How long should your certificates be valid?
Every certificate has an expiry date. When the expiry date is reached clients
will stop trusting your certificate and TLS connections will fail.
Best practice suggests that you should either:
(a) set this to a short duration (90 - 120 days) and have automatic processes
to generate a new certificate before the old one expires, or
(b) set it to a longer duration (3 - 5 years) and then perform a manual update
a few months before it expires.
You may enter the validity period in years (e.g. 3Y), months (e.g. 18M), or days (e.g. 90D)
For how long should your certificate be valid? [5y] 50y
## Do you wish to generate one certificate per node?
If you have multiple nodes in your cluster, then you may choose to generate a
separate certificate for each of these nodes. Each certificate will have its
own private key, and will be issued for a specific hostname or IP address.
Alternatively, you may wish to generate a single certificate that is valid
across all the hostnames or addresses in your cluster.
If all of your nodes will be accessed through a single domain
(e.g. node01.es.example.com, node02.es.example.com, etc) then you may find it
simpler to generate one certificate with a wildcard hostname (*.es.example.com)
and use that across all of your nodes.
However, if you do not have a common domain name, and you expect to add
additional nodes to your cluster in the future, then you should generate a
certificate per node so that you can more easily generate new certificates when
you provision new nodes.
Generate a certificate per node? [y/N]n
## Which hostnames will be used to connect to your nodes?
These hostnames will be added as "DNS" names in the "Subject Alternative Name"
(SAN) field in your certificate.
You should list every hostname and variant that people will use to connect to
your cluster over http.
Do not list IP addresses here, you will be asked to enter them later.
If you wish to use a wildcard certificate (for example *.es.example.com) you
can enter that here.
Enter all the hostnames that you need, one per line.
When you are done, press <ENTER> once more to move on to the next step.
localhost.localdomain
You entered the following hostnames.
- localhost.localdomain
Is this correct [Y/n]y
## Which IP addresses will be used to connect to your nodes?
If your clients will ever connect to your nodes by numeric IP address, then you
can list these as valid IP "Subject Alternative Name" (SAN) fields in your
certificate.
If you do not have fixed IP addresses, or not wish to support direct IP access
to your cluster then you can just press <ENTER> to skip this step.
Enter all the IP addresses that you need, one per line.
When you are done, press <ENTER> once more to move on to the next step.
192.168.20.32
192.168.20.33
192.168.20.34
You entered the following IP addresses.
- 192.168.20.32
- 192.168.20.33
- 192.168.20.34
Is this correct [Y/n]y
## Other certificate options
The generated certificate will have the following additional configuration
values. These values have been selected based on a combination of the
information you have provided above and secure defaults. You should not need to
change these values unless you have specific requirements.
Key Name: localhost.localdomain
Subject DN: CN=localhost, DC=localdomain
Key Size: 2048
Do you wish to change any of these options? [y/N]n
## What password do you want for your private key(s)?
Your private key(s) will be stored in a PKCS#12 keystore file named "http.p12".
This type of keystore is always password protected, but it is possible to use a
blank password.
If you wish to use a blank password, simply press <enter> at the prompt below.
Provide a password for the "http.p12" file: [<ENTER> for none]123456
Repeat password to confirm: 123456
## Where should we save the generated files?
A number of files will be generated including your private key(s),
public certificate(s), and sample configuration options for Elastic Stack products.
These files will be included in a single zip archive.
What filename should be used for the output zip file? [/home/software/elasticsearch-7.9.3/elasticsearch-ssl-http.zip]
Zip file written to /home/software/elasticsearch-7.9.3/elasticsearch-ssl-http.zip
# 同步其他两个节点
[ela@localhost elasticsearch-7.9.3]$ scp config/http.p12 ela@192.168.20.33:/home/software/elasticsearch-7.9.3/config/
ela@192.168.20.33's password:
http.p12 100% 3653 3.6KB/s 00:00
[ela@localhost elasticsearch-7.9.3]$ scp config/http.p12 ela@192.168.20.34:/home/software/elasticsearch-7.9.3/config/
ela@192.168.20.34's password:
http.p12
# 3,启用 TLS 并指定访问节点证书所需的信息
# 依次在3个节点add
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: "http.p12"
# 4, 将该密码添加到 Elasticsearch 中的安全设置中
# 依次在3个节点执行，输入你的密码123456
./bin/elasticsearch-keystore add xpack.security.http.ssl.keystore.secure_password
重启
依次重启3个节点
./bin/elasticsearch -d
# 校验 ，任一节点执行
[ela@localhost elasticsearch-7.9.3]$ curl -XGET --user elastic:654321 'https://192.168.20.32:9200/_cat/nodes?pretty' -k
192.168.20.33 9 98 1 0.15 0.21 0.16 dilmrt * node-33
192.168.20.32 11 98 1 0.07 0.18 0.17 dilmrt - node-32
192.168.20.34 7 95 1 0.23 0.32 0.22 dilmrt - node-34
至此，在 HTTP 层启用 TLS集群配置完成
