|NO.Z.00017|——————————|BigDataEnd|——|Hadoop&OLAP_ClickHouse.V14|——|ClickHouse.v14|MergeTree家族表引擎|AggregateMergeTree|
一、MergeTree家族表引擎:AggregateMergeTree
### --- AggregateMergeTree
~~~ 说明: 该引擎继承自 MergeTree,并改变了数据片段的合并逻辑。
~~~ ClickHouse 会将相同主键的所有行(在一个数据片段内)替换为单个存储一系列聚合函数状态的行。
~~~ 可以使用 AggregatingMergeTree 表来做增量数据统计聚合,包括物化视图的数据聚合。
~~~ 引擎需使用 AggregateFunction 类型来处理所有列。
~~~ 如果要按一组规则来合并减少行数,则使用 AggregatingMergeTree 是合适的。
~~~ 对于AggregatingMergeTree不能直接使用insert来查询写入数据。
~~~ 一般是用insert select。但更常用的是创建物化视图
二、AgrgregateMergeTree示例
### --- 先创建一个MergeTree引擎的基表
~~~ # 创建表
hadoop01 :) create table arr_table_base (id String, city String, code String,value UInt32) engine=MergeTree partition by city order by (id,city);
CREATE TABLE arr_table_base
(
`id` String,
`city` String,
`code` String,
`value` UInt32
)
ENGINE = MergeTree
PARTITION BY city
ORDER BY (id, city)
Ok.
### --- 往基表写入数据
### --- 创建一个AggregatingMergeTree的物化视图
~~~ # 创建一个AggregatingMergeTree的物化视图
hadoop01 :) create materialized view agg_view engine=AggregatingMergeTree() partition by city order by(id,city) as select id,city,uniqState(code) as code, sumState(value) as value from arr_table_base group by id,city;
CREATE MATERIALIZED VIEW agg_view
ENGINE = AggregatingMergeTree()
PARTITION BY city
ORDER BY (id, city) AS
SELECT
id,
city,
uniqState(code) AS code,
sumState(value) AS value
FROM arr_table_base
GROUP BY
id,
city
Ok.
### --- 根据b往基表写数据的方法重写写一次将数据填充到物化视图amt_tab_view中并查询
~~~ # 加载物化视图表
hadoop01 :) insert into table arr_table_base values ('A000','wuhan','code1',1), ('A000','wuhan','code2',200), ('A000','zhuhai','code1',200);
### --- 通过optimize命令手动Merge后查询
hadoop01 :) select id,sumMerge(value),uniqMerge(code) from agg_view group by id,city;
SELECT
id,
sumMerge(value),
uniqMerge(code)
FROM agg_view
GROUP BY
id,
city
┌─id───┬─sumMerge(value)─┬─uniqMerge(code)─┐
│ A000 │ 200 │ 1 │
│ A000 │ 201 │ 2 │
└──────┴─────────────────┴─────────────────┘
### --- 使用场景
~~~ 可以使用AggregatingMergeTree表来做增量数据统计聚合,包括物化视图的数据聚合。
Walter Savage Landor:strove with none,for none was worth my strife.Nature I loved and, next to Nature, Art:I warm'd both hands before the fire of life.It sinks, and I am ready to depart
——W.S.Landor
浙公网安备 33010602011771号