Prometheus的各种exporter实现MySQL-Redis-Nginx-Consul服务的监控
各种 Exporter
Prometheus 指供了大量的 Exporter 实现各种应用的监控功能
Exporter 分类
应用内置: 软件内就内置了Exporter,比如: Prometheus,Grafana,Gitlab,Zookeeper,MinIO等
应用外置: 应用安装后,还需要单独安装对应的 Exporter,比如: MySQL,Redis,MongoDB,PostgreSQL等
定制开发: 如有特殊需要,用户自行开发
Exporter 官方文档
https://prometheus.io/docs/instrumenting/exporters/
Node Exporter 监控服务
实战案例
修改 node_exporter 的配置文件
#在node1节点修改node_exporter配置文件
[root@node1 ~]# cat /lib/systemd/system/node_exporter.service [Unit] Description=Prometheus Node Exporter After=network.target [Service] Type=simple #no-collector.uname:禁止收集uname信息,会导致grafana无法显示此主机信息 #ExecStart=/usr/local/node_exporter/bin/node_exporter --no-collector.uname --collector.cgroups #ExecStart=/usr/local/node_exporter/bin/node_exporter --collector.cgroups ExecStart=/usr/local/node_exporter/bin/node_exporter --collector.cgroups --collector.systemd --collector.systemd.unit-include=".*(ssh|mysql|node_exporter|nginx).*" ExecReload=/bin/kill -HUP $MAINPID Restart=on-failure User=prometheus Group=prometheus [Install] WantedBy=multi-user.target [root@node1 ~]#
#只收集指定服务
#属性解析:
如果没有提前安装的服务,是不会被查看到的
服务名称的正则符号必须解析正确,否则无法匹配要现实的服务名称
#重启node_exporter服务
systemctl daemon-reload
systemctl restart node_exporter.service
[root@node1 ~]# curl http://node1.org:9100/metrics|grep ssh % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 3922 0 3922 0 0 22540 0 --:--:-- --:--:-- --:--:-- 22411node_systemd_unit_state{name="sshd-keygen.target",state="activating",type=""} 0 node_systemd_unit_state{name="sshd-keygen.target",state="active",type=""} 1 node_systemd_unit_state{name="sshd-keygen.target",state="deactivating",type=""} 0 node_systemd_unit_state{name="sshd-keygen.target",state="failed",type=""} 0 node_systemd_unit_state{name="sshd-keygen.target",state="inactive",type=""} 0 node_systemd_unit_state{name="sshd-keygen@ecdsa.service",state="activating",type="oneshot"} 0 node_systemd_unit_state{name="sshd-keygen@ecdsa.service",state="active",type="oneshot"} 0 node_systemd_unit_state{name="sshd-keygen@ecdsa.service",state="deactivating",type="oneshot"} 0 node_systemd_unit_state{name="sshd-keygen@ecdsa.service",state="failed",type="oneshot"} 0 node_systemd_unit_state{name="sshd-keygen@ecdsa.service",state="inactive",type="oneshot"} 1 node_systemd_unit_state{name="sshd-keygen@ed25519.service",state="activating",type="oneshot"} 0 node_systemd_unit_state{name="sshd-keygen@ed25519.service",state="active",type="oneshot"} 0 node_systemd_unit_state{name="sshd-keygen@ed25519.service",state="deactivating",type="oneshot"} 0 node_systemd_unit_state{name="sshd-keygen@ed25519.service",state="failed",type="oneshot"} 0 node_systemd_unit_state{name="sshd-keygen@ed25519.service",state="inactive",type="oneshot"} 1 node_systemd_unit_state{name="sshd-keygen@rsa.service",state="activating",type="oneshot"} 0 node_systemd_unit_state{name="sshd-keygen@rsa.service",state="active",type="oneshot"} 0 node_systemd_unit_state{name="sshd-keygen@rsa.service",state="deactivating",type="oneshot"} 0 node_systemd_unit_state{name="sshd-keygen@rsa.service",state="failed",type="oneshot"} 0 node_systemd_unit_state{name="sshd-keygen@rsa.service",state="inactive",type="oneshot"} 1 node_systemd_unit_state{name="sshd.service",state="activating",type="notify"} 0
#稍等几秒钟,到浏览器中查看监控目标
#结果显示:
每个服务都有五种状态,只有成功的状态才会显示值为1,其他状态为0
只有已安装的服务才会在这里显示,否则不显示
MySQL 监控
MySQL 监控说明
prometheus提供了专属于 MySQL 的服务监控工具 mysqld_exporter,可以借助于该模块,来实现数据库的
基本监控。
#下载链接
https://prometheus.io/download/
#使用说明
https://github.com/prometheus/mysqld_exporter
案例:二进制安装
MySQL 数据库环境准备
#更新mysql配置,如果MySQL和MySQL exporter 不在同一个主机,需要修改如下配置
sed -i 's#127.0.0.1#0.0.0.0#' /etc/mysql/mysql.conf.d/mysqld.cnf
systemctl restart mysql
#为mysqld_exporter配置获取数据库信息的用户并授权
mysql
mysql> CREATE USER 'exporter'@'localhost' IDENTIFIED BY '123456' WITH MAX_USER_CONNECTIONS 3;
mysql> GRANT PROCESS, REPLICATION CLIENT, SELECT ON *.* TO 'exporter'@'localhost';
mysql> flush privileges;
mysqld_exporter 安装
#获取软件
wget https://github.com/prometheus/mysqld_exporter/releases/download/v0.19.0/mysqld_exporter-0.19.0.linux-amd64.tar.gz
#解压软件
tar -xf mysqld_exporter-0.19.0.linux-amd64.tar.gz
mv mysqld_exporter-0.19.0.linux-amd64 mysqld_exporter
mv mysqld_exporter /usr/local/
cd /usr/local/mysqld_exporter/
mkdir bin
mv mysqld_exporter bin/
#在mysqld_exporter的服务目录下,创建 .my.cnf 隐藏文件,为mysqld_exporter配置获取数据库信息的基本属性
vim /usr/local/mysqld_exporter/.my.cnf
[root@www mysqld_exporter]# cat /usr/local/mysqld_exporter/.my.cnf [client] host=127.0.0.1 port=3306 user=exporter password=123456
useradd -r -s /sbin/nologin prometheus
chown -R prometheus:prometheus /usr/local/mysqld_exporter
#修改mysql_exporter的服务启动文件
[root@www local]# cat /lib/systemd/system/mysqld_exporter.service [Unit] Description=mysqld exporter project After=network.target [Service] Type=simple WorkingDirectory=/usr/local/mysqld_exporter ExecStart=/usr/local/mysqld_exporter/bin/mysqld_exporter --config.my-cnf=/usr/local/mysqld_exporter/.my.cnf Restart=on-failure RestartSec=5s User=prometheus Group=prometheus [Install] WantedBy=multi-user.target
#配置解析:在原来的配置文件基础上,添加了 --config.my-cnf属性
#重载并重启服务
systemctl daemon-reload
systemctl enable --now mysqld_exporter.service
systemctl start mysqld_exporter
systemctl status mysqld_exporter.service
#检查是否打开9104/tcp端口
ss -tnulp | grep mysql
#结果显示:该组件对外暴露的端口是 9104 端口
把mysqld_exporter. 9104 注册到consul
[root@master1 prometheus]# cat node2.json { "service": { "id": "192.168.3.52-mysql_exporter", "name": "192.168.3.52-mysql_exporter", "address": "192.168.3.52", "port": 9104, "tags": ["mysql_exporter"], "checks": [{ "http": "http://192.168.3.52:9104/metrics", "interval": "5s" }] } }
consul services register node2.json


Grafana 图形展示
#导入grafana的镜像模板文件
https://grafana.com/grafana/dashboards/14057,7362,11323,13106,17320(中文版)
17320 中文版
Redis 监控
prometheus提供了专属于Redis 的服务监控工具 redis_exporter,我们可以借助于该模块,来实现Redis的
基本监控。
https://github.com/oliver006/redis_exporter
基于二进制实现 redis_exporter
sed -i.bak -e '/^bind.*/c bind 0.0.0.0' -e '$a requirepass 123456'
/etc/redis/redis.conf
vim /etc/redis/redis.conf
bind 0.0.0.0
VERSION=v1.83.0
wget https://github.com/oliver006/redis_exporter/releases/download/${VERSION}/redis_exporter-${VERSION}.linux-amd64.tar.gz
tar -xf redis_exporter-v1.83.0.linux-amd64.tar.gz
mv redis_exporter-v1.83.0.linux-amd64 redis_exporter
mv redis_exporter /usr/local/
cd /usr/local/redis_exporter/
mkdir bin
mv redis_exporter bin/
#修改node_exporter的服务启动文件
[root@www redis_exporter]# cat /lib/systemd/system/redis_exporter.service [Unit] Description=redis exporter project After=network.target [Service] Type=simple ExecStart=/usr/local/redis_exporter/bin/redis_exporter -redis.addr redis://localhost:8712 -redis.password 5twiuzQ1uTu2odFd Restart=on-failure User=prometheus Group=prometheus [Install] WantedBy=multi-user.target
[root@www redis_exporter]# cat /lib/systemd/system/redis_exporter.service [Unit] Description=redis exporter project After=network.target [Service] Type=simple ExecStart=/usr/local/redis_exporter/bin/redis_exporter -redis.addr redis://localhost:8712 -redis.password 5twiuzQ1uTu2odFd --web.listen-address=:9122 Restart=on-failure User=prometheus Group=prometheus [Install] WantedBy=multi-user.target
systemctl daemon-reload
systemctl enable --now redis_exporter
systemctl start redis_exporter
systemctl status redis_exporter
#测试访问
curl -s 127.0.0.1:9122/metrics
注册到consul
[root@master1 prometheus]# cat redis_exporter.json { "id": "192.168.3.52-redis_exporter", "name": "192.168.3.52-redis_exporter", "address": "192.168.3.52", "port": 9122, "tags": ["redis_exporter"], "checks": [{ "http": "http://192.168.3.52:9122/metrics", "interval": "5s" }] }
curl -X PUT \ -H 'Content-Type: application/json' \ --data-binary @redis_exporter.json \ http://192.168.3.60:8500/v1/agent/service/register
Grafana 展示
grafana 模板 763
Nginx 监控
Nginx 默认自身没有提供 Json 格式的指标数据,可以通过下三种方式实现 Prometheus 监控
方法1:
通过容器方式 nginx/nginx-prometheus-exporter 容器配合nginx的stub状态页实现nginx的监控
方法2:Prometheus metric library for Nginx
https://prometheus.io/docs/instrumenting/exporters/
https://github.com/knyar/nginx-lua-prometheus
方法3
需要先编译安装一个第三方模块nginx-vts,将状态页转换为Json格式
再利用nginx-vts-exporter采集数据到Prometheus
部署nginx
dnf install nginx
systemctl start nginx
nginx-prometheus-exporter 容器实现
https://hub.docker.com/r/nginx/nginx-prometheus-exporter
范例:基于 docker 实现
docker pull nginx/nginx-prometheus-exporter:1.5.1
#方法1
cat /etc/nginx/conf.d/status.conf
[root@node1 ~]# cat /etc/nginx/conf.d/status.conf server { listen 8888; location /basic_status { stub_status; } }
curl 127.0.0.1:8888/basic_status
#后台启动
#方法1:
docker run -p 9113:9113 \ --name nginx-prometheus-exporter \ --restart always \ -d nginx/nginx-prometheus-exporter:1.5.1 \ --nginx.scrape-uri=http://192.168.3.63:8888/basic_statu
curl 192.168.3.63:9113/metrics
[root@node1 ~]# cat nginx_exporter.json { "id": "node1.org-nginx_exporter", "name": "node1.org-nginx_exporter", "address": "node1.org", "port": 9113, "tags": ["nginx_exporter"], "checks": [{ "http": "http://node1.org:9113/metrics", "interval": "5s" }] }
curl -X PUT -H 'Content-Type: application/json' --data-binary @nginx_exporter.json http://192.168.3.60:8500/v1/agent/service/register
Grafana 模板
11199
14900
Consul 监控
Consul Exporter 可以实现对 Consul 的监控
需要为每个Consul实例部署consul-exporter,它负责将Consul的状态信息转为Prometheus兼容的指标格式并予以暴露。
下载程序包和展开程序包consul-exporter
https://github.com/prometheus/consul_exporter
范例:
#先安装consul.再部署consul_exporter
wget https://github.com/prometheus/consul_exporter/releases/download/v0.13.0/consul_exporter-0.13.0.linux-amd64.tar.gz
tar -xf consul_exporter-0.13.0.linux-amd64.tar.gz
mv consul_exporter-0.13.0.linux-amd64 consul_exporter
mv consul_exporter /usr/local/
mkdir /usr/local/consul_exporter/bin/
mv /usr/local/consul_exporter/consul_exporter /usr/local/consul_exporter/bin/
创建service文件
保存于/lib/systemd/system/consul_exporter.service文件中
[root@master1 prometheus]# cat /lib/systemd/system/consul_exporter.service [Unit] Description=consul_exporter Documentation=https://prometheus.io/docs/introduction/overview/ After=network.target [Service] Type=simple User=consul EnvironmentFile=-/etc/default/consul_exporter ExecStart=/usr/local/consul_exporter/bin/consul_exporter \ --consul.server="http://localhost:8500" \ --web.listen-address=":9107" \ --web.telemetry-path="/metrics" \ --log.level=info \ $ARGS ExecReload=/bin/kill -HUP $MAINPID TimeoutStopSec=20s Restart=always [Install] WantedBy=multi-user.target
#启动服务
systemctl daemon-reload
systemctl enable --now consul_exporter.service
systemctl start consul_exporter
systemctl status consul_exporter
验证监听的端口,并测试访问其暴露的指标
ss -tnlp | grep '9107'
curl localhost:9107/metrics
[root@master1 prometheus]# cat consul_exporter.json { "id": "master1.org-consul_exporter", "name": "master1.org-consul_exporter", "address": "master1.org", "port": 9107, "tags": ["consul_exporter"], "checks": [{ "http": "http://master1.org:9107/metrics", "interval": "5s" }] }
curl -X PUT -H 'Content-Type: application/json' --data-binary @consul_exporter.json http://192.168.3.60:8500/v1/agent/service/register
Grafana 展示
模板 12049
浙公网安备 33010602011771号