MatrixOne从入门到实践09——MO-Tester
MatrixOne从入门到实践——MO-Tester
从 0.5.0 版本开始，MatrixOne 引入了一个自动测试框架 MO-Tester。
MO-Tester 测试框架，也可以称作为测试器，是通过 SQL 测试 MatrixOne 或其他数据库功能的。
简介
MO-Tester 是基于 Java 语言进行开发，用于 MatrixOne 的测试套件。MO-Tester 构建了一整套完整的工具链来进行 SQL 自动测试。它包含测试用例和运行结果。MO-Tester 启动后，MO-Tester 将使用 MatrixOne 运行所有 SQL 测试用例，并将所有输出 SQL 测试结果与预期结果进行比较。所有案例的结果无论成功或者失败，都将记录在报告中。
MO-Tester 相关用例、结果和报告的链接如下：
- Cases: https://github.com/matrixorigin/mo-tester/tree/main/cases
- Result: https://github.com/matrixorigin/mo-tester/tree/main/result
- Report: 运行结束后，本地目录自动生成
mo-tester/report。
测试用例和测试结果一一对应。如需添加新的测试用例和测试结果请进入右侧所示 MatrixOne 仓库路径中进行添加：https://github.com/matrixorigin/matrixone/tree/main/test
MO-Tester 测试用例如下表所示：
| 测试用例 | 描述 |
|---|---|
| Benchmark/TPCH | DDL and 22 Queries of TPCH Benchmark |
| Database | DDL Statements, creation/drop databases |
| Table | DDL Statements, creation/drop tables |
| DML | DML Statements, including insert, select, show statements |
| dtype | Data Types and type conversion test cases |
| Expression | Case when, With(CTE), Temporal Interval |
| Function | Aggregate function, built-in function |
| Explain | Explain statement |
| Join | Join statement, including Left/Right/Inner/Outer/Natural Join |
| Operator | Including +,-,*,/,MOD,%,=, >, <, IS, LIKE etc |
| Subquery | Including Select/From/Where subquery |
| Transaction | Test of isolation level, atomicity |
使用MO-Tester
环境准备
-
安装jdk 8
# 下载jdk压缩包 下载地址：http://www.oracle.com/technetwork/java/javase/downloads/index.html 下载版本为java 8 的： jdk-8u131-linux-x64.tar.gz # 解压 tar -zxf jdk-8u131-linux-x64.tar.gz # 解压后 的目录假设为 /home/java/jdk1.8.0_131 # 编辑/etc/profile，文件的末尾添加 export JAVA_HOME=/home/java/jdk1.8.0_131 export PATH=$PATH:$JAVA_HOME/bin # 刷新profile source /etc/profile # 验证 jdk [root@motest java]# java -version java version "1.8.0_131" Java(TM) SE Runtime Environment (build 1.8.0_131-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode) -
启动 MatrixOne
请参考MatrixOne部署
-
克隆 MO-Tester 仓库
git clone https://github.com/matrixorigin/mo-tester.git
配置 MO-Tester
MO-tester 基于 Java 语言进行开发，因此 Mo-tester 所依赖的 Java 数据库连接（JDBC，Java Database Connectivity） 驱动程序需要配置 mo.yml 文件里的参数信息：进入到 mo-tester 本地仓库，打开 mo.yml 文件，配置服务器地址、默认的数据库名称、用户名和密码等。
以下是本地独立版本 MatrixOne 的默认示例。
#jdbc
jdbc:
driver: "com.mysql.cj.jdbc.Driver"
server:
- addr: "127.0.0.1:6001"
#- addr: "127.0.0.1:3306"
#- addr: "127.0.0.1:3306"
database:
default: ""
paremeter:
characterSetResults: "utf8"
continueBatchOnError: "false"
useServerPrepStmts: "true"
alwaysSendSetIsolation: "false"
useLocalSessionState: "true"
zeroDateTimeBehavior: "CONVERT_TO_NULL"
failoverReadOnly: "false"
serverTimezone: "Asia/Shanghai"
socketTimeout: 30000
#users
user:
name: "dump"
password: "111"
运行MO-Tester
运行以下所示命令行，SQL 所有测试用例将自动运行，并将报告和错误消息生成至 report/report.txt 和 report/error.txt 文件中。
./run.sh
如果你想调整测试范围，你可以修改 run.yml 文件中的 path 参数。或者，在执行 run.sh 命令时，你也可以指定一些参数，参数解释如下:
| 参数 | 参数定义 |
|---|---|
| -p | 设置由 MO-tester 执行的测试用例的路径。默认值可以参见 run.yml 文件中 path 的配置参数 |
| -m | 设置 MO-tester 测试的方法，即直接运行或者生成新的测试结果。默认值可以参见 run.yaml 文件中 method 的配置参数 |
| -t | 设置 MO-tester 执行 SQL 命令的格式类型。默认值可以参见 run.yml 文件中 type 的配置参数。 |
| -r | 设置测试用例应该达到的成功率。默认值可以参见 run.yml 文件中 rate 的配置参数。 |
| -i | 设置包含列表，只有路径中名称包含其中一个列表的脚本文件将被执行，如果有多个，如果没有指定，用'，'分隔，指的是包含的所有情况set the including list, and only script files in the path whose name contains one of the lists will be executed, if more than one, separated by ,, if not specified, refers to all cases included |
| -e | 设置排除列表，如果路径下的脚本文件的名称包含一个排除列表，则不会被执行，如果有多个，用'，'分隔，如果没有指定，表示不排除任何情况set the excluding list, and script files in the path whose name contains one of the lists will not be executed, if more than one, separated by ,, if not specified, refers to none of the cases excluded |
| -g | 表示带有[-- @bvt:issue#{issueNO.}]标志的 SQL 命令将不会被执行，该标志以 [-- @bvt:issue#{issueNO.}]开始，以 [-- @bvt:issue]结束。例如， -- @bvt:issue#3236 select date_add("1997-12-31 23:59:59",INTERVAL "-10000:1" HOUR_MINUTE); select date_add("1997-12-31 23:59:59",INTERVAL "-100 1" YEAR_MONTH); -- @bvt:issue 这两个 SQL 命令与问题 #3236 相关联，它们将不会在 MO-tester 测试中执行，直到问题 #3236 修复后标签移除才可以在测试中执行。 |
| -n | 表示在比较结果时将忽略结果集的元数据 |
run.yml:
#test scritps path
path: "cases/"
#method
#run: means execute test scripts configured by para[path] completely,and generate the test reprot,execution logs
#debug: means execute test scripts configured by para[path] completely, only print the result to console,do not generate the test reprot,execution logs
#genrs: means generate the expected result for the test scripts only
method: "run"
#rate: means the execution success rate,if the actual rate is less than this,the programe will exit with status 1
rate: 100
示例：
./run.sh -p case -m run -t script -r 100 -i select,subquery -e substring -g
使用MO-Tester完成TPCH
注意： 这里的tpch只是一个测试用例，仅用来测试数据库能够跑通tpch的22条查询语句，包含的数据条数较少。
-
获取MatrixOne源码
git clone https://github.com/matrixorigin/matrixone.git -
复制test目录到MO-Tester
cp -r matrixone/test/cases/benchmark /home/motest/mo-tester/cases # 复制后，可以在MO-Tester中看到相关case 如下所示： root@motest cases]# pwd /home/motest/mo-tester/cases [root@motest cases]# cd benchmark/ [root@motest benchmark]# ll total 0 drwxr-xr-x. 6 root root 71 Oct 1 09:06 tpch [root@motest benchmark]# pwd /home/motest/mo-tester/cases/benchmark同理，将MatrixOne源码目录中test下的result也复制过来
cp -r matrixone/test/result/benchmark /home/motest/mo-tester/result -
运行MO-Tester进行TPCH测试
./run.sh -p ./cases/benchmark/tpch -m run -t script -r 100最终结果显示：
The path of the cases that need to be executed by mo-tester : ./cases/benchmark/tpch The method that mo-tester will run with :run The type of the format that mo-tester execute the sqlcommand in : script The success rate that test cases should reach : 100 2022-10-01 09:18:23 INFO Tester:134 - The method is [run],now start to run the scripts in the path[./cases/benchmark/tpch]. 2022-10-01 09:18:23 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/01_DDL/01_create_table.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:23 INFO Executor:186 - The script file[./cases/benchmark/tpch/01_DDL/01_create_table.sql] has been executed, and cost: 0.02s, total:11, success:11, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:23 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/02_LOAD/02_insert_customer.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:23 INFO Executor:186 - The script file[./cases/benchmark/tpch/02_LOAD/02_insert_customer.sql] has been executed, and cost: 0.032s, total:16, success:16, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:23 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/02_LOAD/03_insert_lineitem.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:25 INFO Executor:186 - The script file[./cases/benchmark/tpch/02_LOAD/03_insert_lineitem.sql] has been executed, and cost: 1.584s, total:602, success:602, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:25 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/02_LOAD/04_insert_nation.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:25 INFO Executor:186 - The script file[./cases/benchmark/tpch/02_LOAD/04_insert_nation.sql] has been executed, and cost: 0.008s, total:4, success:4, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:25 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/02_LOAD/05_insert_orders.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:25 INFO Executor:186 - The script file[./cases/benchmark/tpch/02_LOAD/05_insert_orders.sql] has been executed, and cost: 0.288s, total:151, success:151, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:25 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/02_LOAD/06_insert_part.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:25 INFO Executor:186 - The script file[./cases/benchmark/tpch/02_LOAD/06_insert_part.sql] has been executed, and cost: 0.031s, total:21, success:21, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:25 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/02_LOAD/07_insert_partsupp.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/02_LOAD/07_insert_partsupp.sql] has been executed, and cost: 0.114s, total:81, success:81, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/02_LOAD/08_insert_region.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/02_LOAD/08_insert_region.sql] has been executed, and cost: 0.004s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/02_LOAD/09_insert_supplier.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/02_LOAD/09_insert_supplier.sql] has been executed, and cost: 0.004s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/02_LOAD/10_select_count.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/02_LOAD/10_select_count.sql] has been executed, and cost: 0.016s, total:8, success:8, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q1.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q1.sql] has been executed, and cost: 0.028s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q10.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q10.sql] has been executed, and cost: 0.007s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q11.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q11.sql] has been executed, and cost: 0.005s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q12.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q12.sql] has been executed, and cost: 0.005s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q13.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q13.sql] has been executed, and cost: 0.006s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q14.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q14.sql] has been executed, and cost: 0.005s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q15.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q15.sql] has been executed, and cost: 0.004s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q16.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q16.sql] has been executed, and cost: 0.008s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q17.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q17.sql] has been executed, and cost: 0.005s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q18.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q18.sql] has been executed, and cost: 0.006s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q19.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q19.sql] has been executed, and cost: 0.01s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q2.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q2.sql] has been executed, and cost: 0.006s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q20.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q20.sql] has been executed, and cost: 0.005s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q21.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q21.sql] has been executed, and cost: 0.006s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q22.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q22.sql] has been executed, and cost: 0.007s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q3.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q3.sql] has been executed, and cost: 0.008s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q4.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q4.sql] has been executed, and cost: 0.005s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q5.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q5.sql] has been executed, and cost: 0.007s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q6.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q6.sql] has been executed, and cost: 0.006s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q7.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q7.sql] has been executed, and cost: 0.011s, total:3, success:3, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q8.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q8.sql] has been executed, and cost: 0.008s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/03_QUERIES/q9.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/03_QUERIES/q9.sql] has been executed, and cost: 0.011s, total:2, success:2, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Executor:25 - Start to execute the script file[./cases/benchmark/tpch/04_CLEANUP/01_DROP.sql] now, and it will take a few moment,pleas wait...... 2022-10-01 09:18:26 INFO Executor:186 - The script file[./cases/benchmark/tpch/04_CLEANUP/01_DROP.sql] has been executed, and cost: 0.015s, total:9, success:9, failed:0, ignored:0, abnoraml:0 2022-10-01 09:18:26 INFO Tester:136 - All the scripts in the path[./cases/benchmark/tpch] have been excuted.Now start to create the test report. 2022-10-01 09:18:26 INFO TestReport:49 - [SUMMARY] COST : 1s, TOTAL :952, SUCCESS : 952, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/01_DDL/01_create_table.sql] COST : 0.02s, TOTAL :11, SUCCESS :11, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/02_LOAD/02_insert_customer.sql] COST : 0.032s, TOTAL :16, SUCCESS :16, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/02_LOAD/03_insert_lineitem.sql] COST : 1.584s, TOTAL :602, SUCCESS :602, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/02_LOAD/04_insert_nation.sql] COST : 0.008s, TOTAL :4, SUCCESS :4, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/02_LOAD/05_insert_orders.sql] COST : 0.288s, TOTAL :151, SUCCESS :151, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/02_LOAD/06_insert_part.sql] COST : 0.031s, TOTAL :21, SUCCESS :21, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/02_LOAD/07_insert_partsupp.sql] COST : 0.114s, TOTAL :81, SUCCESS :81, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/02_LOAD/08_insert_region.sql] COST : 0.004s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/02_LOAD/09_insert_supplier.sql] COST : 0.004s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/02_LOAD/10_select_count.sql] COST : 0.016s, TOTAL :8, SUCCESS :8, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q1.sql] COST : 0.028s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q10.sql] COST : 0.007s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q11.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q12.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q13.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q14.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q15.sql] COST : 0.004s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q16.sql] COST : 0.008s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q17.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q18.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q19.sql] COST : 0.01s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q2.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q20.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q21.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q22.sql] COST : 0.007s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q3.sql] COST : 0.008s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q4.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q5.sql] COST : 0.007s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q6.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q7.sql] COST : 0.011s, TOTAL :3, SUCCESS :3, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q8.sql] COST : 0.008s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/03_QUERIES/q9.sql] COST : 0.011s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO TestReport:54 - [./cases/benchmark/tpch/04_CLEANUP/01_DROP.sql] COST : 0.015s, TOTAL :9, SUCCESS :9, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% 2022-10-01 09:18:26 INFO Tester:138 - The test report has been generated in files[report.txt,report.xml]. 2022-10-01 09:18:26 INFO Tester:144 - The execution success rate is 100%, and not less than config value 100%,this test succeed.我们可以看一下report.txt 观察测试用例结果
cd ./report/ [root@motest report]# cat report.txt [SUMMARY] COST : 1s, TOTAL :952, SUCCESS : 952, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/01_DDL/01_create_table.sql] COST : 0.02s, TOTAL :11, SUCCESS :11, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/02_LOAD/02_insert_customer.sql] COST : 0.032s, TOTAL :16, SUCCESS :16, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/02_LOAD/03_insert_lineitem.sql] COST : 1.584s, TOTAL :602, SUCCESS :602, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/02_LOAD/04_insert_nation.sql] COST : 0.008s, TOTAL :4, SUCCESS :4, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/02_LOAD/05_insert_orders.sql] COST : 0.288s, TOTAL :151, SUCCESS :151, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/02_LOAD/06_insert_part.sql] COST : 0.031s, TOTAL :21, SUCCESS :21, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/02_LOAD/07_insert_partsupp.sql] COST : 0.114s, TOTAL :81, SUCCESS :81, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/02_LOAD/08_insert_region.sql] COST : 0.004s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/02_LOAD/09_insert_supplier.sql] COST : 0.004s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/02_LOAD/10_select_count.sql] COST : 0.016s, TOTAL :8, SUCCESS :8, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q1.sql] COST : 0.028s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q10.sql] COST : 0.007s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q11.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q12.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q13.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q14.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q15.sql] COST : 0.004s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q16.sql] COST : 0.008s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q17.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q18.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q19.sql] COST : 0.01s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q2.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q20.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q21.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q22.sql] COST : 0.007s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q3.sql] COST : 0.008s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q4.sql] COST : 0.005s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q5.sql] COST : 0.007s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q6.sql] COST : 0.006s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q7.sql] COST : 0.011s, TOTAL :3, SUCCESS :3, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q8.sql] COST : 0.008s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/03_QUERIES/q9.sql] COST : 0.011s, TOTAL :2, SUCCESS :2, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100% [./cases/benchmark/tpch/04_CLEANUP/01_DROP.sql] COST : 0.015s, TOTAL :9, SUCCESS :9, FAILED :0, IGNORED :0, ABNORAML :0, SUCCESS RATE : 100%同理，我们如果需要执行其他的测试用例，只需要将对应的cases复制过来，运行时指定对应的路径即可
