基于openeuler22.3的ELk日志平台搭建

ELK由ElasticSearch，Logstash和Kiabana三个开源工具组成。

一，ELK概述

1，基础概念不再赘述，官网有详尽的文档

elasticsearch下载地址：Elasticsearch 8.11.3 | Elastic

logstash下载地址：Logstash 8.11.3 | Elastic

kibana下载地址：Kibana 8.11.3 | Elastic

filebeat下载地址：Filebeat 8.11.3 | Elastic

注：全部下载 RPM X86_64安装包

注：为了避免不必要的麻烦，es，logstash和kibana的版本最好保持一致

二，服务器优化以及基础组件安装

1，服务器内核参数优化

vim /etc/sysctl.conf
#修改参数
vm.swappiness=0
vm.overcommit_memory=1
 
vim /etc/security/limits.conf
#修改参数
root soft nofile 65535
root hard nofile 65535
* soft nofile 65535
* hard nofile 65535
* soft memlock unlimited
* hard memlock unlimited
* soft nproc 65535
* hard nproc 65535

2，安装jdk

下载链接：www.oracle.com/java/techno…

选择jdk8的rpm包，然后安装

rpm -ivh jdk-8u381-linux-x64.rpm
 
#验证版本信息
java -version
 
#输出信息，表示安装成功
java version "1.8.0_202"
Java(TM) SE Runtime Environment (build 1.8.0_202-b08)
Java HotSpot(TM) 64-Bit Server VM (build 25.202-b08, mixed mode)

3，安装kibana

#安装
rpm -ivh kibana-8.11.3-x86_64.rpm

#修改配置文件
/etc/kibana/kibana.yml

server.port: 5601
server.host: "192.168.0.156"
elasticsearch.hosts: ["http://192.168.0.156:9200"]
i18n.locale: "zh-CN"

#启动kibana
systemctl daemon-reload
systemctl enable kibana
systemctl start kibana

#查看状态
systemctl status kibana

4，安装elasticsearch

#安装
rpm -ivh elasticsearch-8.11.3-x86_64.rpm
 
#创建es存储目录，赋予权限
mkdir -p /data/elasticsearch/{data,logs}
chown -R elasticsearch.elasticsearch /data/elasticsearch/
 
#修改配置文件
vim /etc/elasticsearch/elasticsearch.yml
   
path.data: /data/elasticsearch/data                   #数据存储路径
path.logs: /data/elasticsearch/logs                   #存储路径
cluster.name: elasticsearch                           #集群名程
node.name: elasticsearch                              #节点名程
network.host: 192.168.80.8                            #更改为本机IP
http.cors.enabled: true                               #跨域设置
http.cors.allow-origin: "*"                           #跨域设置
http.port: 9200                                       #服务端口
 
#修改jvm配置
vim /etc/elasticsearch/jvm.options
 
-Xms2g
-Xmx2g
 
#启动服务
systemctl daemon-reload
systemctl enable elasticsearch
systemctl start elasticsearch
systemctl status elasticsearch

5，安装logstash

#安装
rpm -ivh logstash-8.11.3-x86_64.rpm

#修改配置文件
vim /etc/logstash/jvm.options
-Xms1g
-Xmx1g

#新增logstash配置文件
tee /etc/logstash/conf.d/logstash.conf <<-'EOF'
input {

  beats {
    port => 4514
    codec => json
    ssl  => false
  }

}

filter {
      json {
         source => "message"
      }
      date {
        match => ["timestamp", "dd/MMM/yyyy:HH:mm:ss Z"] #匹配timestamp字段
        target => "@timestamp"  #将匹配到的数据写到@timestamp字段中
      }
}

output {
if [type] == "index" {
    elasticsearch {
    hosts => ["192.168.0.156:9200"]
    index => ["index-%{+YYYY.MM.dd}"]
    }
    }

}
EOF

#启动服务
systemctl daemon-reload
systemctl enable logstash
systemctl start logstash

#查看状态
systemctl status logstash

6，安装filebeat日志收集工具

#安装
rpm -ivh filebeat-8.11.3-x86_64.rpm

#修改配置
vim /etc/filebeat/filebeat.yml


filebeat.inputs:
- type: log
  paths:
    - /var/log/messages
  fields:
    type: "flow"
  fields_under_root: true

output.logstash:
  hosts: ["192.168.0.156:4514"]
 
#启动服务
systemctl daemon-reload
systemctl enable filebeat
systemctl start filebeat

#查看状态
systemctl status filebeat

三，配置kibana

1，服务器默认开启firewalld，因此需要开放端口，5601，浏览器方可访问

firewall-cmd --zone=public --add-port=5601/tcp --permanent
firewall-cmd --reload

2，浏览器访问http://192.168.0.156:5601