prometheus+grafana监控服务器

安装node_exporter

wget https://github.com/prometheus/node_exporter/releases/download/v1.9.1/node_exporter-1.9.1.linux-amd64.tar.gz
tar -xf node_exporter-1.9.1.linux-amd64.tar.gz -C /usr/local/
mv /usr/local/node_exporter-1.9.1.linux-amd64/ /usr/local/node_exporter


cat > /usr/lib/systemd/system/node_exporter.service  << EOF
[Service]
ExecStart=/usr/local/node_exporter/node_exporter

[Install]
WantedBy=multi-user.target

[Unit]
Description=node_exporter
After=network.target
EOF

systemctl daemon-reload
systemctl enable node_exporter
systemctl start node_exporter.service 
systemctl status  node_exporter.service

安装完之后查看node exporter实例采集的metrics指标：

curl 127.0.0.1:9100/metrics

如果需要监控的服务器比较多，可以使用ansible 剧本进行安装，示例：

cat node_exporter.yaml 
- hosts: all         
  remote_user: root 
  tasks:        
    - name: scp node_exporter
      unarchive:  
        copy: yes
        src: /root/node_exporter-1.9.1.linux-amd64.tar.gz
        dest: /usr/local
    - name: change dir
      shell: mv /usr/local/node_exporter-1.9.1.linux-amd64/ /usr/local/node_exporter
    - name: touch node_exporter.service
      copy: 
        content: |
          [Service]
          ExecStart=/usr/local/node_exporter/node_exporter

          [Install]
          WantedBy=multi-user.target

          [Unit]
          Description=node_exporter
          After=network.target
        dest: /usr/lib/systemd/system/node_exporter.service
    - name: systemctl daemon-reload
      ansible.builtin.systemd:
        daemon_reload: yes
    - name: start node_exporter
      systemd:
        name: node_exporter
        state: started
        enabled: yes

prometheus收集node_exporter数据

我们使用基于文件的动态配置：

vim /data/prometheus/prometheus.yml
scrape_configs:
  - job_name: "node_exporter"
    scrape_interval: 15s # 表示 Prometheus 每隔 15 秒会对目标服务（如 node_exporter）进行一次数据采集
    scrape_timeout: 5s  # 定义了每次采集的超时时间
    file_sd_configs:
      - files:
        - /data/prometheus/target/system.yml
        refresh_interval: 15s #代表15秒加载一次配置文件


mkdir /data/prometheus/target
cat /data/prometheus/target/system.yml 
- targets: ['10.0.0.71:9100']
  labels:
    group: ruoyi
    name: ruoyi-mysql-master
- targets: ['10.0.0.72:9100']
  labels:
    group: ruoyi
    name: ruoyi-mysql-slave
- targets: ['10.0.0.73:9100']
  labels:
    group: ruoyi
    name: ruoyi-redis-master
- targets: ['10.0.0.74:9100']
  labels:
    group: ruoyi
    name: ruoyi-redis-slave
- targets: ['10.0.0.75:9100']
  labels:
    group: ruoyi
    name: ruoyi-backend01
- targets: ['10.0.0.76:9100']
  labels:
    group: ruoyi
    name: ruoyi-backend02
- targets: ['10.0.0.77:9100']
  labels:
    group: ruoyi
    name: ruoyi-nginx01
- targets: ['10.0.0.78:9100']
  labels:
    group: ruoyi
    name: ruoyi-nginx02
- targets: ['10.0.0.61:9100']
  labels:
    group: ruoyi
    name: ruoyi-docker
- targets: ['10.0.0.81:9100']
  labels:
    group: eladmin
    name: eladmin-mysql-master
- targets: ['10.0.0.82:9100']
  labels:
    group: eladmin
    name: eladmin-mysql-slave
- targets: ['10.0.0.83:9100']
  labels:
    group: eladmin
    name: eladmin-redis-master
- targets: ['10.0.0.84:9100']
  labels:
    group: eladmin
    name: eladmin-redis-slave
- targets: ['10.0.0.85:9100']
  labels:
    group: eladmin
    name: eladmin-backend01
- targets: ['10.0.0.86:9100']
  labels:
    group: eladmin
    name: eladmin-backen02
- targets: ['10.0.0.87:9100']
  labels:
    group: eladmin
    name: eladmin-nginx01
- targets: ['10.0.0.88:9100']
  labels:
    group: eladmin
    name: eladmin-nginx02
- targets: ['10.0.0.62:9100']
  labels:
    group: eladmin
    name: eladmin-docker
- targets: ['10.0.0.60:9100']
  labels:
    group: ops
    name: ops
- targets: ['10.0.0.63:9100']
  labels:
    group: ops
    name: prometheus

热加载prometheus：

curl -X POST http://localhost:9090/-/reload

检查：

配置grafana面板

直接导入面板代码8919即可