环境准备
一、安装docker环境
yum remove docker \
docker-client \
docker-client-latest \
docker-common \
docker-latest \
docker-latest-logrotate \
docker-logrotate \
docker-engine
curl -sSL https://get.daocloud.io/docker | sh
二、安装docker-compose
curl -L "https://github.com/docker/compose/releases/download/1.23.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
docker-compose -v
##docker-compose 常用命令
#启动yml文件定义的 container
docker-compose up
#后台运行
docker-compose up -d
#查看up帮助
docker-compose up --help
#-f 指定yml文件
docker-compose -f docker-compose.yml up
#停止
docker-compose stop
docker-compose start
#查看
docker-compose ls
#停止删除
docker-compose down
docker-compose ps
docker-compose images
docker-compose exec {service_name} {bash}
三、修改系统配置文件
#修改系统配置文件
cat >> /etc/security/limits.conf << EOF
* soft nofile 65536
* hard nofile 131072
* soft nproc 2048
* hard nproc 4096
EOF
cat >> /etc/sysctl.conf << EOF
vm.max_map_count=655360
EOF
四、系统备份文件
#创建es日志和数据文件
mkdir elasticsearch-7.11.2
cd elasticsearch-7.11.2
mkdir -p ./es01/data
mkdir -p ./es01/logs
mkdir -p ./es02/data
mkdir -p ./es02/logs
mkdir -p ./es03/data
mkdir -p ./es03/logs
#防止出现es启动报权限不足问题
chmod 777 es* -R
sysctl -p
五、获取对应docker镜像
#es镜像
$ docker pull docker.elastic.co/elasticsearch/elasticsearch:7.11.2
#或者
$ docker pull elasticsearch:7.11.2
#kibana镜像
$ docker pull kibana:7.11.2
证书elastic-certificates.p12
es提供了生成证书的工具elasticsearch-certutil，我们可以在docker实例中生成它，然后复制出来，后面统一使用。
##docker运行elasticsearch实例
$ docker run -dit --name=es elasticsearch:7.11.2 /bin/bash
##进入docker实例总
$ docker exec -it es /bin/bash
##代码片段
[root@d796c81da263 elasticsearch]$ ./bin/elasticsearch-certutil ca
This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL/TLS in the Elastic stack.
The 'ca' mode generates a new 'certificate authority'
This will create a new X.509 certificate and private key that can be used
to sign certificate when running in 'cert' mode.
Use the 'ca-dn' option if you wish to configure the 'distinguished name'
of the certificate authority
By default the 'ca' mode produces a single PKCS#12 output file which holds:
* The CA certificate
* The CA's private key
If you elect to generate PEM format certificates (the -pem option), then the output will
be a zip file containing individual files for the CA certificate and private key
Please enter the desired output file [elastic-stack-ca.p12]:
Enter password for elastic-stack-ca.p12 :
### 注意：elastic-stack-ca.p12处输入密码时直接回车即可，否则在开启Xpack配置后，指定证书会报秘钥解析不了错误，原因是设置密码后无法解析。
[root@d796c81da263 elasticsearch]$ ./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12
This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL/TLS in the Elastic stack.
The 'cert' mode generates X.509 certificate and private keys.
* By default, this generates a single certificate and key for use
on a single instance.
* The '-multiple' option will prompt you to enter details for multiple
instances and will generate a certificate and key for each one
* The '-in' option allows for the certificate generation to be automated by describing
the details of each instance in a YAML file
* An instance is any piece of the Elastic Stack that requires an SSL certificate.
Depending on your configuration, Elasticsearch, Logstash, Kibana, and Beats
may all require a certificate and private key.
* The minimum required value for each instance is a name. This can simply be the
hostname, which will be used as the Common Name of the certificate. A full
distinguished name may also be used.
* A filename value may be required for each instance. This is necessary when the
name would result in an invalid file or directory name. The name provided here
is used as the directory name (within the zip) and the prefix for the key and
certificate files. The filename is required if you are prompted and the name
is not displayed in the prompt.
* IP addresses and DNS names are optional. Multiple values can be specified as a
comma separated string. If no IP addresses or DNS names are provided, you may
disable hostname verification in your SSL configuration.
* All certificates generated by this tool will be signed by a certificate authority (CA)
unless the --self-signed command line option is specified.
The tool can automatically generate a new CA for you, or you can provide your own with
the --ca or --ca-cert command line options.
By default the 'cert' mode produces a single PKCS#12 output file which holds:
* The instance certificate
* The private key for the instance certificate
* The CA certificate
If you specify any of the following options:
* -pem (PEM formatted output)
* -keep-ca-key (retain generated CA key)
* -multiple (generate multiple certificates)
* -in (generate certificates from an input file)
then the output will be be a zip file containing individual certificate/key files
Enter password for CA (elastic-stack-ca.p12) :
Please enter the desired output file [elastic-certificates.p12]:
Enter password for elastic-certificates.p12 :
### 注意：elastic-stack-ca.p12处输入密码时直接回车即可，否则在开启Xpack配置后，指定证书会报秘钥解析不了错误，原因是设置密码后无法解析。此三处都直接回车即可
Certificates written to /usr/share/elasticsearch/elastic-certificates.p12
This file should be properly secured as it contains the private key for
your instance.
This file is a self contained file and can be copied and used 'as is'
For each Elastic product that you wish to configure, you should copy
this '.p12' file to the relevant configuration directory
and then follow the SSL configuration instructions in the product guide.
For client applications, you may only need to copy the CA certificate and
configure the client to trust this certificate.
#退出docker容器
$ exit
#将生成好的证书复制到当前文件夹
$ docker cp es:/usr/share/elasticsearch/elastic-certificates.p12 .
#停止docker
$ docker stop es
#删除docker
$ docker rm es
# 修改文件权限
$ chmod 777 elastic-certificates.p12
配置文件
创建 docker-compose.yml
我们已三个节点为例，进行docker-compose.yml 编写
version: '2.2'
services:
es01:
image: docker.elastic.co/elasticsearch/elasticsearch:7.11.2 ##此处取决docker pull xxxx[名字]
container_name: es01 ##docker名称
environment:
- node.name=es01 ##节点名称
- cluster.name=es-docker-cluster ##集群名称
- discovery.seed_hosts=es02,es03 ##执行发现的主机的初始列表
- cluster.initial_master_nodes=es01,es02,es03 ##初始的主合格节点集引导群集
- bootstrap.memory_lock=true ##锁住内存，不被使用到交换分区去
- "ES_JAVA_OPTS=-Xms512m -Xmx512m" ##此处根据实际情况设定Java所占内存大小
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- /etc/localtime:/etc/localtime ## 设置docker时钟
- ./es01/data:/usr/share/elasticsearch/data ##es数据存储路径
- ./es01/logs:/usr/share/elasticsearch/logs ##es日志存储路径
- ./elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml ##es配置yaml
- ./elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12 ##es证书
ports:
- 9200:9200 ##映射端口
networks:
- elastic ##单一网络使用bridge，swarm集群使用overlay
es02:
image: docker.elastic.co/elasticsearch/elasticsearch:7.11.2
container_name: es02
environment:
- node.name=es02
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es01,es03
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- /etc/localtime:/etc/localtime
- ./es02/data:/usr/share/elasticsearch/data
- ./es02/logs:/usr/share/elasticsearch/logs
- ./elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
- ./elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12
ports:
- 9201:9200
networks:
- elastic
es03:
image: docker.elastic.co/elasticsearch/elasticsearch:7.11.2
container_name: es03
environment:
- node.name=es03
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es01,es02
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- /etc/localtime:/etc/localtime
- ./es03/data:/usr/share/elasticsearch/data
- ./es03/logs:/usr/share/elasticsearch/logs
- ./elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
- ./elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12
ports:
- 9202:9200
networks:
- elastic
kib01:
depends_on: ##设置启动顺序
- es01
image: kibana:7.11.2
container_name: kib01
ports:
- 5601:5601
environment:
ELASTICSEARCH_URL: http://es01:9200
ELASTICSEARCH_HOSTS: http://es01:9200
volumes:
- ./kibana.yml:/usr/share/kibana/config/kibana.yml
networks:
- elastic
networks:
elastic:
driver: bridge ##单一网络使用bridge，swarm集群使用overlay
创建 elasticsearch.yml
network.host: 0.0.0.0
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.keystore.type: PKCS12
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /usr/share/elasticsearch/config/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /usr/share/elasticsearch/config/elastic-certificates.p12
xpack.security.transport.ssl.truststore.type: PKCS12
xpack.security.audit.enabled: true
elasticsearch.yml 官网详解：Configuring Elasticsearch | Elasticsearch Guide [7.13] | Elastic
创建 kibana.yml
#
# ** THIS IS AN AUTO-GENERATED FILE **
#
# Default Kibana configuration for docker target
server.name: kibana
server.host: "0"
elasticsearch.hosts: [ "http://10.132.5.192:9200" ]
elasticsearch.username: "elastic"
elasticsearch.password: "ZcPg1CyTW46hbPR2MXgq"
monitoring.ui.container.elasticsearch.enabled: true
#设置中文，只有在7.0以后才会有
i18n.locale: "zh-CN"
启动ES并生成密码
启动ES
#进入到你创建es文件夹
$ cd elk/elasticsearch-7.11.2/
$ ls
#包括如下文件
# docker-compose.yaml docker-compose配置文件
# elastic-certificates.p12 es证书
# elasticsearch.yml es配置文件
# kibana.yml kibana配置文件
#通过docker-compose构建docker环境
$ docker-compose up
#启动成功后查看
[root@k1 elasticsearch-7.11.2]$ docker-compose ps
Name Command State Ports
------------------------------------------------------------------------------------
es01 /bin/tini -- /usr/local/bi ... Up 0.0.0.0:9200->9200/tcp, 9300/tcp
es02 /bin/tini -- /usr/local/bi ... Up 0.0.0.0:9201->9200/tcp, 9300/tcp
es03 /bin/tini -- /usr/local/bi ... Up 0.0.0.0:9202->9200/tcp, 9300/tcp
kibana01 /bin/tini -- /usr/local/bi ... Up 0.0.0.0:5601->5601/tcp
生产密码
#进入其中一个docker
$ docker exec -it es01 /bin/bash
$ ./bin/elasticsearch-setup-passwords -h
Sets the passwords for reserved users
Commands
--------
auto - Uses randomly generated passwords
interactive - Uses passwords entered by a user
Non-option arguments:
command
Option Description
------ -----------
-E <KeyValuePair> Configure a setting
-h, --help Show help
-s, --silent Show minimal output
-v, --verbose Show verbose output
$ ./bin/elasticsearch-setup-passwords auto
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.
The passwords will be randomly generated and printed to the console.
Please confirm that you would like to continue [y/N]y
Changed password for user apm_system
PASSWORD apm_system = TFqrXCcm9rLVOuh23ocH
Changed password for user kibana_system
PASSWORD kibana_system = 8bmmcrP3eGX3vuoTJGlJ
Changed password for user kibana
PASSWORD kibana = 8bmmcrP2eGX3v6oTJGlJ
Changed password for user logstash_system
PASSWORD logstash_system = iEZva6r2pyN0VQmZouba
Changed password for user beats_system
PASSWORD beats_system = m5idcc848JOE44FRuXEB
Changed password for user remote_monitoring_user
PASSWORD remote_monitoring_user = IEM5LkxzTPXLDyVbGKvA
Changed password for user elastic
PASSWORD elastic = ZcPg1CyTW46hbPR2123gq
配置kibana.yml
#修改原有kibana.yml，配置用户名和密码
elasticsearch.username: "elastic"
elasticsearch.password: "ZcPg1CyTW46hbPR2123gq"
浏览器访问localhost:9200/9201/9202 需要输入账号
输入对应的elastic/password就好
浏览器访问localhost:5601
