grafana/lock-stack 日志 Pipeline 配置 - 实践

前言

本文使用的是 grafana/loki-stack chart 抓取的 k8s 日志。其他 chart 配置都差不多。

日志问题

docker 容器运行时 pod 内原始日志

[cpu-4] Hello, 第 9788 次报时，时间：2025-08-01T06:35:42+0000
{
"HOSTNAME":"cpu-4","level":"info","count":9788,"time":"2025-08-01T06:35:42+0000","msg":"Hello from cpu-4"
}
{
"HOSTNAME":"cpu-4","log":"{"HOSTNAME":"cpu-4","level":"info","count":9788,"time":"2025-08-01T06:35:42+0000","msg":"Hello from cpu-4"}"
}
{
"log":"Hi, 第 71461 次报时，Node: cpu-4, The current time is: 2025-08-01 14:47:47 +0800"
}

docker 容器运行时被 docker 存放的日志，是一个带 json 格式（带 log stream time 字段）

{
"log":"[cpu-4] Hello, 第 1061 次报时，时间：2025-08-01T08:14:58+0000\n","stream":"stdout","time":"2025-08-01T08:14:58.490561319Z"
}
{
"log":"{\"HOSTNAME\":\"cpu-4\",\"level\":\"info\",\"count\":1061,\"time\":\"2025-08-01T08:14:58+0000\",\"msg\":\"Hello from cpu-4\"}\n","stream":"stdout","time":"2025-08-01T08:14:58.490600455Z"
}
{
"log":"{\"HOSTNAME\":\"cpu-4\",\"log\":\"{\"HOSTNAME\":\"cpu-4\",\"level\":\"info\",\"count\":1061,\"time\":\"2025-08-01T08:14:58+0000\",\"msg\":\"Hello from cpu-4\"}\"}\n","stream":"stdout","time":"2025-08-01T08:14:58.490611277Z"
}
{
"log":"{\"log\":\"Hi, 第 76992 次报时，Node: cpu-4, The current time is: 2025-08-01 16:20:45 +0800\"}\n","stream":"stdout","time":"2025-08-01T08:20:45.91951057Z"
}

如果是 containerd 容器运行时，会在原来的日志上加前缀 2025-08-01T11:20:10.111114661+08:00 stdout F 这里不再展示日志对比。

解决

编辑 values.yaml 文件添加如下配置

promtail:
config:
snippets:
pipelineStages:
# 参考 https://grafana.com/docs/loki/latest/send-data/promtail/configuration/?utm_source=chatgpt.com#docker
- docker: {
}
# 参考 https://grafana.com/docs/loki/latest/send-data/promtail/configuration/?utm_source=chatgpt.com#cri
- cri: {
}

配置说明：
docker: {} 相当于如下配置：

- json:
expressions:
output: log
stream: stream
timestamp: time
- labels:
stream:
- timestamp:
source: timestamp
format: RFC3339Nano
- output:
source: output

cri: {} 相当于如下配置：

- regex:
expression: "^(?s)(?P<time>\\S+?) (?P<stream>stdout|stderr) (?P<flags>\\S+?) (?P<content>.*)$"
  - labels:
  stream:
  - timestamp:
  source: time
  format: RFC3339Nano
  - output:
  source: content

具体配置参考官网

docker: {}
cri: {}