Prometheus 监控进程

Process-exporter

　　process-exporter可以用来检测所选进程的存活状态

下载process-exporter

　　下载地址：https://github.com/ncabatoff/process-exporter/releases/tag/v0.4.0　　

安装部署process-exporter

1	tar -zxvf process-exporter-0.4.0.linux-amd64.tar.gz -C /usr/local/process-exporter

　　编写配置文件

cat process-exporter.yaml
process_names:
 
  - name: "{{.Matches}}"
    cmdline:
    - 'dbbakup'
 
  - name: "{{.Matches}}"
    cmdline:
    - 'mysql'
 
  - name: "{{.Matches}}"
    cmdline:
    - 'pushgateway'

　　

注 cmdline:  所选进程的唯一标识，ps -ef 可以查询到。如果改进程不存在，则不会有该进程的数据采集到。

　　编写启动脚本

cat /usr/lib/systemd/system/process_exporter.service
[Unit]
Description=Prometheus exporter for processors metrics, written in Go with pluggable metric collectors.
Documentation=https://github.com/ncabatoff/process-exporter
After=network.target
 
[Service]
Type=simple
User=root
WorkingDirectory=/usr/local/process-exporter
ExecStart=/usr/local/process-exporter/process-exporter -config.path=/usr/local/process-exporter/process-exporter.yaml
Restart=on-failure
 
[Install]
WantedBy=multi-user.target

　　

启动process-exporter

1 2 3

systemctl daemon-reload systemctl start process_exporter systemctl enable process_exporter

 

验证

1	curl localhost:9256/metrics

 

修改Prometheus配置文件

1 2 3 4 5

增加    - job_name: 'process'     static_configs:     - targets: ['172.16.8.187:9256']

　　重启Prometheus

　　

 

添加告警规则

 

cat process.yml 
groups:
- name: process
  rules:
  - alert: backup-mysql-Down
    expr: absent(namedprocess_namegroup_states{groupname="map[:mysql]"})
    for: 1m
    labels:
      severity: critical
    annotations:
      summary: backup slave mysql docker process Down (instance {{ $labels.instance }})
      description: "backup slave mysql process is down\n LABELS = {{ $labels }}"


  - alert: pushgateway-Down
    expr: absent(namedprocess_namegroup_states{groupname="map[:pushgateway]"})
    for: 1m
    labels:
      severity: critical
    annotations:
      summary: pushgateway process Down (instance {{ $labels.instance }})
      description: "pushgateway process is down\n LABELS = {{ $labels }}"

  - alert: dbback-process-Down
    expr: absent(namedprocess_namegroup_states{groupname="map[:dbbakup]"})
    for: 1m
    labels:
      severity: critical
    annotations:
      summary: dbbackup docker process Down (instance {{ $labels.instance }})
      description: "dbbackup docker process is down\n LABELS = {{ $labels }}"