ES安装

1.jdk 安装

1.安装jdk

sudo yum update
sudo yum search java | grep jdk
sudo yum install java-11-openjdk-devel

2.修改配置文件

vi /etc/profile
----------------------------------
JAVA_HOME=/usr/lib/jvm/java-11-openjdk-11.0.22.0.7-1.el7_9.x86_64
JRE_HOME=$JAVA_HOME/jre
CLASS_PATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib
PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin
export JAVA_HOME JRE_HOME CLASS_PATH PATH
----------------------------------
source /etc/profile

2.es 安装

1.下载es包，并解压

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.17.20-linux-x86_64.tar.gz
tar -zxvf elasticsearch-7.17.20-linux-x86_64.tar.gz
mv elasticsearch-7.17.20 /opt

2.修改配置文件

vi /etc/profile
------------------------------------
export ES_HOME=/opt/elasticsearch-7.17.20
export PATH=$ES_HOME/bin:$PATH
------------------------------------
source /etc/profile

3.设置启动参数

1.修改文件创建数量
vi /etc/security/limits.conf
-----------------------------
es soft nofile 65536
es hard nofile 65536
-----------------------------
2.设置文件的大小参数
vi /etc/security/limits.d/20-nproc.conf
-----------------------------
es soft nofile 65536
es hard nofile 65536
-----------------------------
3.设置最大内存的分配
vi /etc/sysctl.conf
-----------------------------
vm.max_map_count=655360
-----------------------------
完成后重新加载配置
sysctl -p

4.设置用户并授权

useradd es;
passwd es;
chown -R es /opt/elasticsearch-7.20.0/
chown -R es /opt/es/

5.登录es用户

su es

6.编辑配置文件

cd /opt/elasticsearch-7.17.20/
vi config/elasticsearch.yml;
--------------------------------------
node.name: node-1

cluster.initial_master_nodes: ["node-1"]

path.data: /opt/es/data
path.logs: /opt/es/logs

network.host: 0.0.0.0
http.port: 9200

http.cors.enabled: true
http.cors.allow-origin: "*"
--------------------------------------

7.启动es

elasticsearch -d

3.安装Kibana

1.下载kibana

wget https://artifacts.elastic.co/downloads/kibana/kibana-7.17.20-linux-x86_64.tar.gz
tar -zxvf kibana-7.17.20-linux-x86_64.tar.gz
mv kibana-7.17.20-linux-x86_64 kibana
cd kibana

2.修改配置文件

vi config/kibana.yml
----------------------------------------
server.port: 5602
server.host: "0.0.0.0"

elasticsearch.hosts: ["http://0.0.0.0:9200"]
elasticsearch.requestTimeout: 60000
i18n.locale: "zh-CN"
-----------------------------------------

启动
nohup bin/kibana > kibana.log 2>&1 &

4.安装logstash

1.下载logstash

wget https://artifacts.elastic.co/downloads/logstash/logstash-7.17.20-linux-x86_64.tar.gz
tar xzvf logstash-7.17.20-linux-x86_64.tar.gz
cd logstash-7.17.20

2.修改配置并启动

vi config/ngx_access.conf
----------------------------------------
input {
  file {
    path => "/var/log/ngx_access.log"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}
filter {
  json {
    source => "message"
  }

  mutate {
    convert => ["body_bytes_sent", "integer"]
    convert => ["bytes_sent", "integer"]
    convert => ["request_end_time", "integer"]
    convert => ["request_time", "float"]
    convert => ["upstream_response_time", "float"]
    
    remove_field => ["message", "@version", "path"]
  }
}
output {
  elasticsearch {
    hosts => ["http://0.0.0.0:9200"]
    user => "es"
    password => "eeeeeeeeeeeee"
    ilm_rollover_alias => "ls_ngx_log"
  }
}
----------------------------------------
nohup ./bin/logstash -f config/ngx_access.conf &

没有配置 ilm_patternedit 默认 {now/d}-000001 别名有默认的过期策略

eg: ls_ngx_log-2024.05.06-000001

别名： ls_ngx_log 查询直接使用别名就可以

filter: 对数据的简单处理。 json：把原始的json格式字符串解析出来 mutate: convert 修改字段类型，支持转换integer,float,string 和bollean remove_field：删除原始数据中不需要的字段

ES和logstash安装部署

ES安装

1.jdk 安装

2.es 安装

3.安装Kibana

4.安装logstash