Prometheus & Grafana 监控 Linux , MySQL , Redis , Nginx

1 在机器上安装 docker

安装之前前移除现有docker

sudo yum remove docker \
                docker-client \
                docker-client-latest \
                docker-common \
                docker-latest \
                docker-latest-logrotate \
                docker-logrotate \
                docker-selinux \
                docker-engine-selinux \
                docker-engine

或

sudo yum remove docker-ce

安装工具

sudo yum install -y yum-utils device-mapper-persistent-data lvm2

设置阿里云镜像

sudo yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

更新 yum 缓存

sudo yum makecache fast

查看可用的 docker 版本

yum list docker-ce --showduplicates | sort -r

设置docker开机自启动等

systemctl start docker
systemctl enable docker
systemctl status docker

2 安装 Prometheus & Grafana (用 docker 的方式)

编写 docker-compose 文件



version: '2.0'

services:

  prometheus:
    image: prom/prometheus:latest
    restart: always
    container_name: prometheus
    ports:
      - 10000:9090
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml

  grafana:
    image: grafana/grafana
    restart: always
    container_name: grafana
    ports:
      - 10001:3000
    volumes:
      - ./data/grafana:/var/lib/grafana
    depends_on:
      - prometheus

编写 prometheus.yml 文件

global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
#alerting:
#  alertmanagers:
#    - static_configs:
#        - targets: ['10.10.170.161:9093']
          # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
#rule_files:
#  - "node_down.yml"
  # - "first_rules.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'Linux'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
      - targets: ['host:9100' , 'host:9100' , 'host:9100']

  - job_name: 'MySQL'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
      - targets: ['host:9104' , 'host:9104']

  - job_name: 'Redis'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
      - targets: ['host:9121' , 'host:9121'] 

  - job_name: 'JVM'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.
    metrics_path: '/actuator/prometheus'
    static_configs:
      - targets: ['host:8080']

  - job_name: 'Nginx'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.
    static_configs:
      - targets: ['host:9113']

启动 prometheus 和 Grafana

docker-compose -f docker-compose-prometheus.yaml up -d

3 安装 mysqld-exporter

下载 docker 镜像

docker pull prom/mysqld-exporter:latest

创建 mysql 账户用于监控

GRANT SELECT, PROCESS, SUPER, REPLICATION CLIENT, RELOAD ON *.* TO 'exporter'@'%' IDENTIFIED BY 'exporterpw';
flush privileges;

运行 mysqld-exporter

docker run -d --name mysql_exporter --restart always -p 9104:9104 -e DATA_SOURCE_NAME="exporter:exporterpw@(host:3306)/" prom/mysqld-exporter

安装 redis_exporter

下载 docker 镜像

docker pull oliver006/redis_exporter:latest

运行 redis_exporter

docker run -d --name redis_exporter --restart always -p 9121:9121 oliver006/redis_exporter --redis.addr redis://host:6379 --redis.password 'password'

安装 node-exporter (监控 linux)

下载 docker 镜像

docker pull prom/node-exporter:latest

运行 node-exporter

docker run -d --restart always -p 9100:9100 -v "/proc:/host/proc:ro"  -v "/sys:/host/sys:ro"  -v "/:/rootfs:ro"  --net="host" prom/node-exporter

启用 nginx_stub_status 模块

./nginx -V 2>&1 | grep -o with-http_stub_status_module

如果在终端输出with-http_stub_status_module，说明nginx已启用tub_status模块,如果没有就需要去安装 ngx_http_stub_status_module 模块

nginx.org/en/docs/htt…

安装 nginx-prometheus-exporter

下载 docker 镜像

docker pull nginx/nginx-prometheus-exporter:latest

运行 nginx-prometheus-exporter

docker run -d --restart always -p 9113:9113 nginx/nginx-prometheus-exporter:latest -nginx.scrape-uri http://host/nginx_status

访问 Prometheus 服务

到 Grafana Labs 网站上需要 Dashboards

grafana.com/grafana/das…

很容易找，找到后复制一下 Dashboard id , 在 grafana 界面中选择 import ，数据源选择 Prometheus 就好了。

访问 Grafana 服务

MYSQL

LINUX

REDIS

NGINX

JVM