K8S 搭建 Prometheus (四) 钉钉告警

打开电脑版钉钉创建机器人

1.创建钉钉机器人
打开电脑版钉钉，创建一个群，创建自定义机器人，按如下步骤创建
https://ding-doc.dingtalk.com/doc#/serverapi2/qf2nxq

我创建的机器人如下：
群设置-->智能群助手-->添加机器人-->自定义-->添加

机器人名称：kube-event
接收群组：钉钉报警测试

安全设置：
自定义关键词：cluster1

上面配置好之后点击完成即可，这样就会创建一个kube-event的报警机器人，创建机器人成功之后怎么查看webhook，按如下：

点击智能群助手，可以看到刚才创建的kube-event这个机器人，点击kube-event，就会进入到kube-event机器人的设置界面
出现如下内容：
机器人名称：kube-event
接受群组：钉钉报警测试
消息推送：开启
webhook：https://oapi.dingtalk.com/robot/send?access_token=cc7e93b8ac6e53f941dc8bef815d2d2f2775c8b793fe75141309c9b1253b755f
安全设置：
自定义关键词：cluster1

配置 prometheus-webhook-dingtalk

# 在 master 上操作
mkdir -p /data/yaml/monitor-sa/prometheus-webhook-dingtalk

cd /data/yaml/monitor-sa/prometheus-webhook-dingtalk

cat deployment.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: prometheus-webhook-dingtalk
  namespace: monitor-sa
  labels:
    app: prometheus-webhook-dingtalk
spec:
  replicas: 1
  minReadySeconds: 100
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
  selector:
    matchLabels:
      app: prometheus-webhook-dingtalk
  template:
    metadata:
      labels:
        app: prometheus-webhook-dingtalk
    spec:
      terminationGracePeriodSeconds: 60
      restartPolicy: Always # 只要退出就重启
      containers:
      - name: prometheus-webhook-dingtalk
        image: harbor.junengcloud.com/monitor/prometheus-webhook-dingtalk:0.3.0
        imagePullPolicy: IfNotPresent
        args:
          - 'cc7e93b8ac6e53f941dc8bef815d2d2f2775c8b793fe75141309c9b1253b755f' # 钉钉 token
        ports:
        - containerPort: 8060
          protocol: TCP
          name: http
        resources:
          requests:
            cpu: 100m
            memory: 1024Mi
          limits:
            cpu: 2000m
            memory: 2048Mi
        livenessProbe:
          tcpSocket:
            port: 8060
          initialDelaySeconds: 30
          periodSeconds: 90

kubectl apply -f deployment.yaml 

cat svc.yaml 
apiVersion: v1
kind: Service
metadata:
  name: prometheus-webhook-dingtalk
  namespace: monitor-sa
  labels:
    app: prometheus-webhook-dingtalk
spec:
  ports:
    - port: 8060
      targetPort: 8060
  selector:
    app: prometheus-webhook-dingtalk

kubectl apply -f svc.yaml 

更新 alertmanager-cm.yaml 文件

cd /data/yaml/monitor-sa/alertmanager

cp alertmanager-cm.yaml  alertmanager-cm.yaml.email

cat alertmanager-cm.yaml
kind: ConfigMap
apiVersion: v1
metadata:
  name: alertmanager
  namespace: monitor-sa
data:
  alertmanager.yml: |-
    global:
      resolve_timeout: 1m
    route:
      group_by: [alertname]
      group_wait: 10s
      group_interval: 10s
      repeat_interval: 10m
      receiver: cluster1
    receivers:
    - name: cluster1
      webhook_configs:
      - url: 'http://prometheus-webhook-dingtalk.monitor-sa:8060/dingtalk/cluster1/send'
        send_resolved: true

kubectl apply -f alertmanager-cm.yaml