随笔分类 -  Query Optimization

1 2 下一页

Calcite分析 -- LinQ4j
摘要:Calcite Version:1.26.0 Linq是一种对Query的抽象,任意Query都可以抽象成ExpressionTree,那么只要bind不同的implementor,就可以转化成不同的查询语言, 比如LINQ to Objects、LINQ to SQL、LINQ to Datase 阅读全文

posted @ 2021-10-13 20:03 fxjwind 阅读(2184) 评论(0) 推荐(0)

Calcite分析 -- Implement
摘要:Calcite Version:1.26.0 CsvTest select "EMPNO", "JOINTIMES" from "DATE" order by "JOINTIMES" 到优化前的调用栈, 其中avatica,是calcite项目中一个独立的子项目,用于封装和server之间的交互,参 阅读全文

posted @ 2021-10-13 19:36 fxjwind 阅读(814) 评论(0) 推荐(0)

TPC-H 分析
摘要:TCPH的表结构 最小的且不变的, - Region,Nation(FK,REGION记录属于哪个区域) 1万级别, - Supplier(供应商),Skey,Name,地址,NKey(FK,记录所属国家key),电话,acctbal(Account Balance,账户余额) 10万级别, - C 阅读全文

posted @ 2021-09-28 17:29 fxjwind 阅读(643) 评论(0) 推荐(0)

论文解析 -- TPC-H Analyzed: Hidden Messages and Lessons Learned from an Influential Benchmark
摘要:该论文主要的目的是,讨论TCPH中的Choke points, Chokepoints are those technological challenges underlying a benchmark, whose resolution will significantly improve the 阅读全文

posted @ 2021-09-28 17:13 fxjwind 阅读(681) 评论(0) 推荐(0)

Calcite分析 -- Cost
摘要:Calcite Version:1.26.0 初始的cost 加入RelNode的时候,需要更新subset的cost RelSubset 找出traits想匹配的subsets, 然后逐一更新cost, propagateCostImprovements0 getCost 叠加所有input的co 阅读全文

posted @ 2021-09-09 19:17 fxjwind 阅读(413) 评论(0) 推荐(0)

Calcite分析 -- ConverterRule
摘要:Calcite Version:1.26.0 EnumerableRule是一种ConverterRule,用于将逻辑算子转化为物理算子,并改变其convention。 EnumerableSortedAggregateRule 将LogicalAggregate转换成物理算子, Converter 阅读全文

posted @ 2021-09-08 11:18 fxjwind 阅读(650) 评论(0) 推荐(0)

Calcite分析 -- TopDownRuleDriver
摘要:Calcite Version:1.26.0 Debug TopDownOptTest, 加入 @Test void testMY() { final String sql = "select mgr, count(*) from sales.emp\n" + "group by mgr order 阅读全文

posted @ 2021-09-08 11:17 fxjwind 阅读(537) 评论(0) 推荐(0)

Calcite分析 -- SortRemoveRule
摘要:Calcite Version:1.26.0 Rule执行之前的情况, 对#19,apply SortRemoveRule #19的input是rel#13:RelSubset#2.NONE.[] 这里就是将Sort中的,collation,放到RelSubset#2.NONE.[]的input上去 阅读全文

posted @ 2021-09-02 20:03 fxjwind 阅读(210) 评论(0) 推荐(0)

Calcite分析 -- Convention&Converter
摘要:Calcite,version:1.26.0 什么是Convention 1. 是一种RelTrait,而且是默认必须有的RelTraitDef 2. 逻辑算子的convention为None,表示未被实现;反过来说物理算子的convention一定不会是None 在CBO优化前会把root进行co 阅读全文

posted @ 2021-08-31 19:56 fxjwind 阅读(843) 评论(0) 推荐(0)

Calcite分析 -- Register
摘要:Calcite Version:1.26.0 ensureRegistered getSubset,从mapRel2Subset中查询一下是否注册过 无注册调用register equivRel不为空,需要merge的case,参考,https://www.cnblogs.com/fxjwind/p 阅读全文

posted @ 2021-08-30 19:30 fxjwind 阅读(459) 评论(0) 推荐(0)

论文解析 -- QuickSel: Quick Selectivity Learning with Mixture Models (SIGMOD’20)
摘要:INTRODUCTION Estimating the selectivity of a query—the fraction of input tuples that satisfy the query’s predicate—is a fundamental component in cost- 阅读全文

posted @ 2021-05-10 14:32 fxjwind 阅读(388) 评论(0) 推荐(0)

Is Query Optimization a “Solved” Problem? (2014, Guy Lohman)
摘要:25年前提出的问题仍然没有得到解决,学者们总是在攻克他们能攻克的问题, 而不是他们应该去攻克的问题,有点意思! 说到Cardinality是查询优化的阿基琉斯之踵,但是之前很多研究都关注在新的结构来解决individual local predicates的估计问题,作者认为完全就是无用功 并提出应 阅读全文

posted @ 2021-04-29 14:45 fxjwind 阅读(202) 评论(0) 推荐(0)

Adaptive Statistics in Oracle 12c(PVLDB 2017)
摘要:Oracle的SQL引擎大体分为这几部分, (1)首先要进入Prepare的阶段,Parser,SA和TC A SQL statement goes through the Parser, Semantic Analysis (SA), and Type-Check (TC) first befor 阅读全文

posted @ 2021-04-28 17:18 fxjwind 阅读(270) 评论(0) 推荐(0)

论文解析 -- A Survey on Advancing the DBMS Query Optimizer: Cardinality Estimation, Cost Model, and Plan Enumeration (Data Science and Engineering 2021)
摘要:Why Key Components in Optimizer are Still Not Accurate? In this section, we summarize the reasons why the cardinality estimation, cost model, and plan 阅读全文

posted @ 2021-04-23 17:47 fxjwind 阅读(426) 评论(0) 推荐(0)

DeepDB: Learn from Data, not from Queries! (2020,PVLDB)
摘要:主导的学习组件,是query-based,workload-driven,以一堆有代表性的queryset去实际运行,产生训练集。 The predominant「占主导的」 approach for learned DBMS components is that they capture the 阅读全文

posted @ 2021-03-11 14:19 fxjwind 阅读(538) 评论(0) 推荐(0)

Self-Tuning, GPU-Accelerated Kernel Density Models for Multidimensional Selectivity Estimation
摘要:Histogram和KDE的分别 参考,https://www.jianshu.com/p/428ae3658f85 Histogram的问题比较明显,首先不平滑,再者分布和区间大小和偏移有很大关系,比如底下两个分布图差别非常大 所以Histogram误差是比较大的 KDE的基本思想比较简单 首先K 阅读全文

posted @ 2021-01-21 11:36 fxjwind 阅读(278) 评论(0) 推荐(0)

论文解析 -- Selectivity Estimation for Range Predicates using Lightweight Models (2019)
摘要:摘要 (点题)Query optimizers depend on selectivity estimates of query predicates to produce a good execution plan. (当前的方案)When a query contains multiple pr 阅读全文

posted @ 2021-01-19 15:45 fxjwind 阅读(827) 评论(0) 推荐(0)

Join Reorder优化 - 论文摘要
摘要:Query Simplification: Graceful Degradation for Join-Order Optimization 这篇的related work可以参考,列的比较全面, Query图分为下面几种, Graph Simplification算法, Heuristic Opt 阅读全文

posted @ 2019-09-29 20:34 fxjwind 阅读(1511) 评论(0) 推荐(0)

查询优化 In Oracle
摘要:Cost-based query transformation in Oracle Enhanced Subquery Optimizations in Oracle Cost-based query transformation in Oracle 本文介绍Oracle的查询优化框架, 先描述,O 阅读全文

posted @ 2019-08-27 20:16 fxjwind 阅读(2439) 评论(0) 推荐(0)

子查询优化 - Hyper
摘要:Unnesting Arbitrary Queries - T Neumann, A KemperThe Complete Story of Joins (in HyPer) - Thomas Neumann, Viktor Leis, Alfons Kemper Unnesting Arbitra 阅读全文

posted @ 2019-08-21 15:24 fxjwind 阅读(813) 评论(0) 推荐(0)

1 2 下一页