spark写入doris
1.获取kafka数据,直接写入doris的主键表
https://www.doubao.com/chat/collection/36860041743623426?type=Thread
2.获取kafka数据,统计用户点击数,写入doris的主键表
https://www.doubao.com/chat/collection/37022619812871426?type=Thread
通过doris.sync.write.upsert.sql来实现
PROPERTIES (
"replication_allocation" = "tag.location.default: 3",
-- 核心配置:指定Upsert更新规则,实现旧值+新值(全局累积的关键)
"doris.sync.write.upsert.sql" = "INSERT INTO stream_db.user_click_global_sum (user_id, total_click) VALUES (?, ?) ON DUPLICATE KEY UPDATE total_click = total_click + VALUES(total_click)"
);
浙公网安备 33010602011771号