Filebeat收集SpringBoot日志信息

一、部署PV和PVC挂载资源目录

1. 配置NFS

所有节点安装nfs

yum install -y nfs-utils & systemctl start nfs

服务端

# 将指定路径挂载出去(挂载路径需要提前创建)
vi /etc/exports
	/data/nfs *(rw,no_root_squash)

验证是否挂载目录，测试POD

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-dep1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      namespace: test
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        volumeMounts:
        - name: wwwroot
          mountPath: /usr/share/nginx/html
        ports:
        - containerPort: 80
      volumes:
        - name: wwwroot
          nfs:
            server: 10.2.11.216
            path: /data/nfs

部署后，在挂载目录创建一个index.htm，使用命令进入容器中查看目录

# 部署
kubectl apply -f xxx.yaml
# 进入POD
kubectl exec -it pod-name bash
# 查看挂载目录文件
ls /usr/share/nginx/html

2. 创建PV和PVC

pv是服务资源目录的映射，pvc用于消费pv

pv文件

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-name
  labels:
     name: pv-name-label
spec:
# 限制容量大小
  capacity:
    storage: 5Gi
# 访问模式
  accessModes:
    - ReadWriteMany
# pv访问nfs目录映射
  nfs:
    path: /data/nfs
    server: 10.2.11.216

pvc文件

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-name
spec:
  accessModes:
   - ReadWriteMany
  resources:
    requests:
      storage: 5Gi

使用命令部署并查看绑定情况

# 安装
kubectl apply -f pv.yaml
kubectl apply -f pvc.yaml

# 查看部署绑定情况
kubectl get pv
kubectl get pvc  -n namespace

注意：pvc和对应pod需要在同一名称空间下

示例：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: test-springboot
  namespace: test
spec:
  replicas: 1
  selector:
    matchLabels:
      app: test
  template:
    metadata:
      namespace: test
      labels:
        app: test
    spec:
      containers:
      - name: spring-boot-test-1
        image: harbor.test.com:5000/test/springboot-test-1:1.0.6
        volumeMounts:
        - name: log
          mountPath: /opt/test/logs
        ports:
        - containerPort: 10030
      volumes:
       - name: log
         persistentVolumeClaim:
           claimName: test-springboot-test1-pvc

二、创建一个java-demo配置filebeat收集数据

1.  配置文件

apiVersion: v1
kind: Service
metadata:
  name: spring-boot-test-2
  namespace: test
spec:
  selector:
    project: test
    app: test-springboot-2
  ports:
  - name: spring-boot-test-2
    port: 8080
    targetPort: 8080
  type: NodePort

apiVersion: apps/v1
kind: Deployment
metadata:
   name: test-spring-boot-test-2
   namespace: test
spec:
 replicas: 1
 selector:
    matchLabels:
      project: test
      app: test-springboot-2
 template:
    metadata:
      labels:
        project: test
        app: test-springboot-2
    spec:
      containers:
      - name: spring-boot-test-2
        image: harbor.test.com:5000/test/springboot-test-2:1.0.0
        imagePullPolicy: Always
        ports:
        - containerPort: 8080
        volumeMounts:
        - name: springboot-logs
          mountPath: /opt/test/logs
        - name: springboot-data
          mountPath: /data/test
      - name: filebeat
        image: harbor.test.com:5000/test/elastic/filebeat:7.6.0
        args: [
          "-c", "/etc/filebeat.yml",
          "-e"
        ]
        securityContext:
          runAsUser: 0
        volumeMounts:
         - name: filebeat-config
           mountPath: /etc/filebeat.yml
           subPath: filebeat.yml
         - name: springboot-logs
           mountPath: /opt/test/logs
      volumes:
      - name: springboot-logs
        emptyDir: {}
      - name: filebeat-config
        configMap:
          name: springboot-test2-filebeat-config
      - name: springboot-data
        persistentVolumeClaim:
          claimName: springboot-test-pvc

apiVersion: v1
kind: ConfigMap
metadata:
  name: springboot-test2-filebeat-config
  namespace: test
data:
  filebeat.yml: |-
    filebeat.inputs:
    - type: log
      enabled: true
      paths:
        - /opt/test/logs/status.log
      fields:
        product_type: springboot-test
        module: springboot-test-2
      fields_under_root: true
      multiline:
        pattern: '^[0-2][0-9][0-2][0-9]-[0-1][0-2]-[0-3][0-9]'
        negate: true
        match: after
    output.logstash:
      hosts: ['10.96.70.176:5044']

2. 部署流程

首先创建filebeatConfigMap用于，部署pod时使用

kubectl apply -f filebeat-configMap.yaml
kubectl get cm -n namspace

随后创建pod信息，并暴露服务

kubectl apply -f demo-dep.yaml
kubectl get pod,deployment -n namespace

kubectl apply -f demo-svc.yaml
kubectl get svc