从 DeepFlow Dashboard 提取原始 API 与 SQL 语句

在某些场景下，我们需要通过 API 或直接在 ClickHouse 中执行 SQL 查询，从而将追踪、监控能力集成到自动化脚本或第三方 GUI 中。为了避免重复编写繁琐的查询语句，通常希望复用 DeepFlow Dashboard 面板中已有的过滤逻辑，但直接使用时往往无法执行成功。这是因为 Dashboard 展示的查询语句是经过 Grafana Plugin 渲染后的结果，而非真正的原始查询。

本文将介绍如何绕过插件渲染，直接获取 DeepFlow 面板背后的真实查询语句，实现 Dashboard 查询的无缝迁移与复用。

通过 Dashboard 获取 Server API 查询语句

  1. 在 Dashboard 中通过 Variables 或 Edit 进行查询，获取生成的 API 语句；

    img_v3_02ui_51e1c5bd-6d61-4dac-8961-0225e2a3ed9g

  2. 通过 edit 进入编辑页面，查询语句使用的 db 库；

    img_v3_02ui_bd3bfb3d-9683-428b-b2b1-d4269d26b13g

  3. 调用 Server API

    注意事项：

    • 复制的 SQL 语句包含许多反引号，需要通过反斜杠去除他们的特殊含义，否则会被当成命令执行；
    • 结尾的时间范围转换成时间戳，精确到秒即可，类型是 float64，所以不需要使用单引号。
    ## 请求结尾使用了 | jq 输出
curl -sXPOST "http://${deepflow-server_svc_ip}:20416/v1/query/" \
--data-urlencode "db=flow_log" \
--data-urlencode "sql=SELECT \`response_duration\` AS \`Response Delay\`, toString(start_time) AS \`start_time\`, auto_instance_0, auto_instance_1, Enum(observation_point), Enum(l7_protocol), l7_protocol_str, request_type, request_domain, request_resource, Enum(response_status), response_code, auto_instance_id_0, auto_instance_id_1 FROM l7_flow_log WHERE \`request_type\` = 'GET' AND \`request_resource\` = '/shop/full-test' AND (\`pod_cluster_0\` = 'k8s-d-5yDbJ9wKVx' OR \`pod_cluster_1\` = 'k8s-d-5yDbJ9wKVx') AND (\`pod_ns_0\` = 'deepflow-otel-spring-demo' OR \`pod_ns_1\` = 'deepflow-otel-spring-demo') AND (\`pod_group_0\` = 'web-shop' OR \`pod_group_1\` = 'web-shop') AND \`response_status\` IN ('3') AND \`l7_protocol\` IN ('20') AND time>=1770073200 AND time<=1770073800 LIMIT 2" | jq

## 输出结果
{
  "OPT_STATUS": "SUCCESS",
  "DESCRIPTION": "",
  "result": {
	"columns": [
	  "Response Delay",
	  "start_time",
	  "auto_instance_0",
	  "auto_instance_type_0",
	  "auto_instance_1",
	  "auto_instance_type_1",
	  "Enum(observation_point)",
	  "Enum(l7_protocol)",
	  "l7_protocol_str",
	  "request_type",
	  "request_domain",
	  "request_resource",
	  "Enum(response_status)",
	  "response_code",
	  "auto_instance_id_0",
	  "auto_instance_id_1"
	],
	"schemas": [
	  {
		"label_type": "",
		"pre_as": "response_duration",
		"type": 0,
		"unit": "",
		"value_type": "UInt64"
	  },
	  {
		"label_type": "",
		"pre_as": "toString(start_time)",
		"type": 0,
		"unit": "",
		"value_type": "String"
	  },
	  {
		"label_type": "",
		"pre_as": "",
		"type": 0,
		"unit": "",
		"value_type": "Nullable(String)"
	  },
	  {
		"label_type": "",
		"pre_as": "",
		"type": 0,
		"unit": "",
		"value_type": "UInt8"
	  },
	  {
		"label_type": "",
		"pre_as": "",
		"type": 0,
		"unit": "",
		"value_type": "Nullable(String)"
	  },
	  {
		"label_type": "",
		"pre_as": "",
		"type": 0,
		"unit": "",
		"value_type": "UInt8"
	  },
	  {
		"label_type": "",
		"pre_as": "",
		"type": 0,
		"unit": "",
		"value_type": "String"
	  },
	  {
		"label_type": "",
		"pre_as": "",
		"type": 0,
		"unit": "",
		"value_type": "String"
	  },
	  {
		"label_type": "",
		"pre_as": "",
		"type": 0,
		"unit": "",
		"value_type": "LowCardinality(String)"
	  },
	  {
		"label_type": "",
		"pre_as": "",
		"type": 0,
		"unit": "",
		"value_type": "LowCardinality(String)"
	  },
	  {
		"label_type": "",
		"pre_as": "",
		"type": 0,
		"unit": "",
		"value_type": "String"
	  },
	  {
		"label_type": "",
		"pre_as": "",
		"type": 0,
		"unit": "",
		"value_type": "String"
	  },
	  {
		"label_type": "",
		"pre_as": "",
		"type": 0,
		"unit": "",
		"value_type": "String"
	  },
	  {
		"label_type": "",
		"pre_as": "",
		"type": 0,
		"unit": "",
		"value_type": "Nullable(Int32)"
	  },
	  {
		"label_type": "",
		"pre_as": "",
		"type": 0,
		"unit": "",
		"value_type": "UInt32"
	  },
	  {
		"label_type": "",
		"pre_as": "",
		"type": 0,
		"unit": "",
		"value_type": "UInt32"
	  }
	],
	"values": [
	  [
		36716,
		"2026-02-03 07:00:11.159516",
		"ingress-nginx-controller-8l9qk",
		10,
		"web-shop-7c48fd68dc-szchh",
		10,
		"Server Process",
		"HTTP",
		"HTTP",
		"GET",
		"nginx.webshop.demo",
		"/shop/full-test",
		"Server Error",
		500,
		71,
		269
	  ],
	  [
		37061,
		"2026-02-03 07:00:11.159190",
		"ingress-nginx-controller-8l9qk",
		10,
		"web-shop-7c48fd68dc-szchh",
		10,
		"Server NIC",
		"HTTP",
		"HTTP",
		"GET",
		"nginx.webshop.demo",
		"/shop/full-test",
		"Server Error",
		500,
		71,
		269
	  ],
	]
  },
  "debug": null
}

通过 Dashboard 获取 ClickHouse 查询语句

  1. 在 Dashboard 中通过 Variables 或 Edit 进行查询；

    img_v3_02ui_e4adb498-7833-4bcf-b38d-1047fcd1bb1g

  2. 通过 View 进入指定模块；

    img_v3_02ui_b711af7f-04e8-4b98-ad20-885fb7d4ce4g

  3. 通过在 uri 中添加 debug 获取 SQL 查询语句；

    img_v3_02ui_e1398e1a-2e5e-4793-b82a-b25b656e898g

  4. 在 ClickHouse 中查询：

WITH
    if(auto_instance_type_0 IN (0, 255), if(is_ipv4 = 1, ip4_0, NULL), NULL) AS auto_instance_ip4_0,
    if(auto_instance_type_0 IN (0, 255), if(is_ipv4 = 0, ip6_0, NULL), NULL) AS auto_instance_ip6_0,
    if(auto_instance_type_1 IN (0, 255), if(is_ipv4 = 1, ip4_1, NULL), NULL) AS auto_instance_ip4_1,
    if(auto_instance_type_1 IN (0, 255), if(is_ipv4 = 0, ip6_1, NULL), NULL) AS auto_instance_ip6_1,
    dictGetOrDefault('flow_tag.string_enum_map', 'name_en', ('observation_point', observation_point), observation_point) AS `Enum(observation_point)`,
    dictGetOrDefault('flow_tag.int_enum_map', 'name_en', ('l7_protocol', toUInt64(l7_protocol)), l7_protocol) AS `Enum(l7_protocol)`,
    dictGetOrDefault('flow_tag.int_enum_map', 'name_en', ('response_status', toUInt64(response_status)), response_status) AS `Enum(response_status)`
SELECT
    response_duration AS `Response Delay`,
    toString(start_time) AS start_time,
    if(auto_instance_type_0 IN (0, 255), if(is_ipv4 = 1, IPv4NumToString(auto_instance_ip4_0), IPv6NumToString(auto_instance_ip6_0)), dictGet('flow_tag.device_map', 'name', (toUInt64(auto_instance_type_0), toUInt64(auto_instance_id_0)))) AS auto_instance_0,
    auto_instance_type_0,
    if(auto_instance_type_1 IN (0, 255), if(is_ipv4 = 1, IPv4NumToString(auto_instance_ip4_1), IPv6NumToString(auto_instance_ip6_1)), dictGet('flow_tag.device_map', 'name', (toUInt64(auto_instance_type_1), toUInt64(auto_instance_id_1)))) AS auto_instance_1,
    auto_instance_type_1,
    `Enum(observation_point)`,
    `Enum(l7_protocol)`,
    l7_protocol_str,
    request_type,
    request_domain,
    request_resource,
    `Enum(response_status)`,
    response_code,
    if(auto_instance_type_0 IN (0, 255), subnet_id_0, auto_instance_id_0) AS auto_instance_id_0,
    if(auto_instance_type_1 IN (0, 255), subnet_id_1, auto_instance_id_1) AS auto_instance_id_1
FROM flow_log.l7_flow_log
WHERE (request_type = 'GET') AND (request_resource = '/shop/full-test') AND ((toUInt64(pod_cluster_id_0) GLOBAL IN (
    SELECT id
    FROM flow_tag.pod_cluster_map
    WHERE name = 'k8s-d-5yDbJ9wKVx'
)) OR (toUInt64(pod_cluster_id_1) GLOBAL IN (
    SELECT id
    FROM flow_tag.pod_cluster_map
    WHERE name = 'k8s-d-5yDbJ9wKVx'
))) AND ((toUInt64(pod_ns_id_0) GLOBAL IN (
    SELECT id
    FROM flow_tag.pod_ns_map
    WHERE name = 'deepflow-otel-spring-demo'
)) OR (toUInt64(pod_ns_id_1) GLOBAL IN (
    SELECT id
    FROM flow_tag.pod_ns_map
    WHERE name = 'deepflow-otel-spring-demo'
))) AND ((toUInt64(pod_group_id_0) GLOBAL IN (
    SELECT id
    FROM flow_tag.pod_group_map
    WHERE name = 'web-shop'
)) OR (toUInt64(pod_group_id_1) GLOBAL IN (
    SELECT id
    FROM flow_tag.pod_group_map
    WHERE name = 'web-shop'
))) AND (response_status IN ('3')) AND (l7_protocol IN ('20')) AND (time >= 1770073998) AND (time <= 1770074898)
LIMIT 5

Query id: 6508955a-3c02-49c4-8990-d044ae7a24f4

┌─Response Delay─┬─start_time─────────────────┬─auto_instance_0────────────────┬─auto_instance_type_0─┬─auto_instance_1───────────┬─auto_instance_type_1─┬─Enum(observation_point)─┬─Enum(l7_protocol)─┬─l7_protocol_str─┬─request_type─┬─request_domain─────┬─request_resource─┬─Enum(response_status)─┬─response_code─┬─auto_instance_id_0─┬─auto_instance_id_1─┐
│          67019 │ 2026-02-03 07:14:11.735603 │ ingress-nginx-controller-hfm6h │                   10 │ web-shop-7c48fd68dc-szchh │                   10 │ Client Process          │ HTTP              │ HTTP            │ GET          │ nginx.webshop.demo │ /shop/full-test  │ Server Error          │           500 │                 72 │                269 │
│          65864 │ 2026-02-03 07:14:11.736547 │ ingress-nginx-controller-hfm6h │                   10 │ web-shop-7c48fd68dc-szchh │                   10 │ Server Process          │ HTTP              │ HTTP            │ GET          │ nginx.webshop.demo │ /shop/full-test  │ Server Error          │           500 │                 72 │                269 │
│          66788 │ 2026-02-03 07:14:11.735655 │ ingress-nginx-controller-hfm6h │                   10 │ web-shop-7c48fd68dc-szchh │                   10 │ Server NIC              │ HTTP              │ HTTP            │ GET          │ nginx.webshop.demo │ /shop/full-test  │ Server Error          │           500 │                 72 │                269 │
│          66853 │ 2026-02-03 07:14:11.735626 │ ingress-nginx-controller-hfm6h │                   10 │ web-shop-7c48fd68dc-szchh │                   10 │ Client NIC              │ HTTP              │ HTTP            │ GET          │ nginx.webshop.demo │ /shop/full-test  │ Server Error          │           500 │                 72 │                269 │
│          67606 │ 2026-02-03 07:14:11.736000 │ ingress-nginx-controller-hfm6h │                   10 │ web-shop-7c48fd68dc-szchh │                   10 │ Server Application      │ HTTP              │ http            │ GET          │ nginx.webshop.demo │ /shop/full-test  │ Server Error          │           500 │                 72 │                269 │
└────────────────┴────────────────────────────┴────────────────────────────────┴──────────────────────┴───────────────────────────┴──────────────────────┴─────────────────────────┴───────────────────┴─────────────────┴──────────────┴────────────────────┴──────────────────┴───────────────────────┴───────────────┴────────────────────┴────────────────────┘

12 rows in set. Elapsed: 0.291 sec. Processed 156.69 thousand rows, 3.78 MB (324.34 thousand rows/s., 13.62 MB/s.)
Peak memory usage: 261.38 KiB.
posted @ 2026-02-03 10:35  怎么还在写代码  阅读(19)  评论(0)    收藏  举报