1.环境

架构图

2.ELK以及Logstash安装

elasticsearch安装

安装java

yum install java-1.8.0-openjdk

下载rpm包

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.17.12-x86_64.rpm

安装

rpm -ivh elasticsearch-7.17.12-x86_64.rpm

修改配置文件/etc/elasticsearch/elasticsearch.yml

network.host: 0.0.0.0
node.name: node-1
cluster.initial_master_nodes: ["node-1"]
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-headers: Authorization
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true

执行命令

 sudo systemctl daemon-reload
 sudo systemctl enable elasticsearch.service
 sudo systemctl start elasticsearch.service

设置密码

/usr/share/elasticsearch/bin/elasticsearch-setup-passwords interactive

测试

[root@vms12 bin]# curl 127.0.0.1:9200 -u elastic:elastic
{
  "name" : "node-1",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "IsSsthG-TNGAKkjFlk6BPg",
  "version" : {
    "number" : "7.17.12",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "e3b0c3d3c5c130e1dc6d567d6baef1c73eeb2059",
    "build_date" : "2023-07-20T05:33:33.690180787Z",
    "build_snapshot" : false,
    "lucene_version" : "8.11.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Kibana 安装

下载rpm包

wget https://artifacts.elastic.co/downloads/kibana/kibana-7.17.12-x86_64.rpm

安装

 rpm -ivh kibana-7.17.12-x86_64.rpm 

修改配置

server.host: "0.0.0.0"
elasticsearch.hosts: ["http://192.168.26.12:9200"]
elasticsearch.username: "kibana_system"
elasticsearch.password: "elastic"

启动

systemctl enable kibana.service
systemctl start kibana.service

浏览器打开 http://ip:5601/app/integrations/browse 地址是否正常

Logstash 安装

下载

wget https://artifacts.elastic.co/downloads/logstash/logstash-7.17.12-x86_64.rpm

安装

rpm -ivh logstash-7.17.12-x86_64.rpm 

测试

./logstash -e 'input { stdin { } } output { stdout {} }'

输入hello有具体的返回就行了
hello
{
       "message" => "hello",
          "host" => {
        "hostname" => "packetbeat"
    },
      "@version" => "1",
         "event" => {
        "original" => "hello"
    },
    "@timestamp" => 2023-08-19T10:14:12.048557096Z
}

新增配置文件/etc/logstash/conf.d/pipeline.conf

input {
    beats {
        port => "5044"
    }
}
filter {
    geoip {
        source => "[destination][ip]"
        tag_on_failure => ["geoip-destination-failed"]
    }
}
output {
    elasticsearch {
        hosts => [ "192.168.26.12:9200" ]
    }
}

测试配置文件

./logstash -f pipeline.conf --config.test_and_exit

Configuration OK
[2023-08-19T18:27:57,554][INFO ][logstash.runner          ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash

启动

./logstash -f pipeline.conf

PacketBeat 安装

建议吐到es这种最简单

安装

curl -L -O https://artifacts.elastic.co/downloads/beats/packetbeat/packetbeat-7.17.12-x86_64.rpm
sudo rpm -vi packetbeat-7.17.12-x86_64.rpm

修改配置文件 /etc/packetbeat/packetbeat.yml

建议最简单的还是直接吐到es
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["192.168.26.12:9200"]

  # Protocol - either `http` (default) or `https`.
  #protocol: "https"

  # Authentication credentials - either API key or username/password.
  #api_key: "id:api_key"
  username: "elastic"
  password: "elastic"

-------------
下面要重新弄图形就有点麻烦了
output.logstash:
  # The Logstash hosts
  hosts: ["192.168.26.13:5044"]

setup.kibana:
  host: "192.168.26.10:5601"

packetbeat.interfaces.device: ens32

该命令可以列出所有的网卡

[root@packetbeat ~]# packetbeat devices
0: ens32 (No description available) (192.168.26.13 fe80::20c:29ff:fe34:a4c9)
1: any (Pseudo-device that captures on all interfaces) (Not assigned ip address)
2: lo (No description available) (127.0.0.1 ::1)
3: nflog (Linux netfilter log (NFLOG) interface) (Not assigned ip address)
4: nfqueue (Linux netfilter queue (NFQUEUE) interface) (Not assigned ip address)

安装libpcap包

sudo yum install libpcap

加载仪表板以输出日志存储

packetbeat setup -e \
  -E output.logstash.enabled=false \
  -E output.elasticsearch.hosts=['192.168.26.12:9200'] \
  -E setup.kibana.host=192.168.26.10:5601

启动

 systemctl start packetbeat.service 

利用地理 IP 信息丰富活动

PUT _ingest/pipeline/geoip-info
{
  "description": "Add geoip info",
  "processors": [
    {
      "geoip": {
        "field": "client.ip",
        "target_field": "client.geo",
        "ignore_missing": true
      }
    },
    {
      "geoip": {
        "field": "source.ip",
        "target_field": "source.geo",
        "ignore_missing": true
      }
    },
    {
      "geoip": {
        "field": "destination.ip",
        "target_field": "destination.geo",
        "ignore_missing": true
      }
    },
    {
      "geoip": {
        "field": "server.ip",
        "target_field": "server.geo",
        "ignore_missing": true
      }
    },
    {
      "geoip": {
        "field": "host.ip",
        "target_field": "host.geo",
        "ignore_missing": true
      }
    }
  ]
}

查看图形（直接吐的es的）