PG-基准测试

数据库基准测试

​ 在数据库服务器的硬件、软件环境中建立已知的性能基准线称为基准测试。根据测试目的的不同，可针对压力、性能、最大负载进行专门测试，或综合测试。通过定量的、可复现的、能对比的方法衡量系统的吞吐量，也可以对新硬件的实际性能和可靠性进行测试，或在生产环境遇到问题时，在测试环境复现问题。

衡量指标

​ 通常数据库的基准测试最关键的衡量指标有：吞吐量（Throughput），响应时间（RT)或延迟（Latency）和并发量。

• 吞吐量衡量数据库的单位时间内的事务处理能力，常用的单位是TPS（每秒事务数）。

• 响应时间或延迟，描述操作过程里用来响应服务的时间，根据不同的应用可以使用分钟、秒、毫秒和微秒作为单位。通常还会根据响应时间的最大值、最小值以及平均值做分组统计，例如90%的运行周期内的响应时间是1毫秒，10%的运行周期内相应时间是5毫秒，这样可以得出相对客观的测试结果。

• 并发量是指同时工作的连接数。

在不同的测试场景，需要关注的指标也会不同，分析测试结果时，吞吐量、响应时间、并发量是必须关注的三个基本要素。

测试工具

​ TPC（事务处理性能委员会：Transaction Processing Performance Council,http://www.tpc.org）已经推出了TPC-A、TPC-B、TPC-C、TPC-D、TPC-E、TPC-W等基准程序的标准规范，其中TPC-C是经典的衡量在线事务处理（OLTP）系统性能和可伸缩性的基准测试规范，还有比较新的OLTP测试规范TPC-E。常见的开源数据库的基准测试工具有benchmarksql、sysbench等，PostgreSQL自带运行基准测试的简单程序pgbench。pgbench是一个类TPC-B的基准测试工具，可以执行内置的测试脚本，也可以自定义脚本文件。

使用pgbench进行测试

初始化测试数据

pgbench 的内嵌脚本需要 4 张表： pgbench_branches 、 pgbench_tellers 、 pgbench_accounts
和 pgbench_history。 使用 pgbench 初始化测试数据， pgbench 会自动去创建这些表并生成测试数据。

pgbench -i -s 2 -F 80 -U dev -d devdb

dropping old tables...
NOTICE:  table "pgbench_accounts" does not exist, skipping
NOTICE:  table "pgbench_branches" does not exist, skipping
NOTICE:  table "pgbench_history" does not exist, skipping
NOTICE:  table "pgbench_tellers" does not exist, skipping
creating tables...
generating data...
100000 of 200000 tuples (50%) done (elapsed 0.29 s, remaining 0.29 s)
200000 of 200000 tuples (100%) done (elapsed 0.66 s, remaining 0.00 s)
vacuuming...
creating primary keys...
done.

内置脚本

pgbench -b list
Available builtin scripts:
	tpcb-like
	simple-update
	select-only

在源码包中postgresql-12.0/src/bin/pgbench/pgbench.c可看到对应的内建脚本执行SQL

static const BuiltinScript builtin_script[] =
{
        {
                "tpcb-like",
                "<builtin: TPC-B (sort of)>",
                "\\set aid random(1, " CppAsString2(naccounts) " * :scale)\n"
                "\\set bid random(1, " CppAsString2(nbranches) " * :scale)\n"
                "\\set tid random(1, " CppAsString2(ntellers) " * :scale)\n"
                "\\set delta random(-5000, 5000)\n"
                "BEGIN;\n"
                "UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid = :aid;\n"
                "SELECT abalance FROM pgbench_accounts WHERE aid = :aid;\n"
                "UPDATE pgbench_tellers SET tbalance = tbalance + :delta WHERE tid = :tid;\n"
                "UPDATE pgbench_branches SET bbalance = bbalance + :delta WHERE bid = :bid;\n"
                "END;\n"
        },
        {
                "simple-update",
                "<builtin: simple update>",
                "\\set aid random(1, " CppAsString2(naccounts) " * :scale)\n"
                "\\set bid random(1, " CppAsString2(nbranches) " * :scale)\n"
                "\\set tid random(1, " CppAsString2(ntellers) " * :scale)\n"
                "\\set delta random(-5000, 5000)\n"
                "BEGIN;\n"
                "UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid = :aid;\n"
                "SELECT abalance FROM pgbench_accounts WHERE aid = :aid;\n"
                "INSERT INTO pgbench_history (tid, bid, aid, delta, mtime) VALUES (:tid, :bid, :aid, :delta, CURRENT_TIMESTAMP);\n"
                "END;\n"
        },
        {
                "select-only",
                "<builtin: select only>",
                "\\set aid random(1, " CppAsString2(naccounts) " * :scale)\n"
                "SELECT abalance FROM pgbench_accounts WHERE aid = :aid;\n"
        }
};

简单测试

1) simple-update测试

pgbench -b simple-update -U dev devdb

starting vacuum...end.
transaction type: <builtin: simple update>
scaling factor: 2
query mode: simple
number of clients: 1
number of threads: 1
number of transactions per client: 10
number of transactions actually processed: 10/10
latency average = 1.975 ms
tps = 506.261467 (including connections establishing)
tps = 611.965929 (excluding connections establishing)

2）混合脚本测试

选择 3 种内 置脚本混合进行测试，并在脚本名称后面加上＠符号， ＠符号后面加一个脚本运行比例的权重 的整数值

pgbench -b simple-update@2 -b select-only@8 -b tpcb@0 -U dev devdb

[postgres@progs pg_root]$ pgbench -b simple-update@2 -b select-only@8 -b tpcb@0 -U dev devdb
starting vacuum...end.
transaction type: multiple scripts
scaling factor: 2
query mode: simple
number of clients: 1
number of threads: 1
number of transactions per client: 10
number of transactions actually processed: 10/10
latency average = 1.465 ms
tps = 682.708822 (including connections establishing)
tps = 994.907467 (excluding connections establishing)
SQL script 1: <builtin: simple update>
 - weight: 2 (targets 20.0% of total)
 - 2 transactions (20.0% of total, tps = 136.541764)
 - latency average = 3.454 ms
 - latency stddev = 1.169 ms
SQL script 2: <builtin: select only>
 - weight: 8 (targets 80.0% of total)
 - 8 transactions (80.0% of total, tps = 546.167058)
 - latency average = 0.385 ms
 - latency stddev = 0.320 ms
SQL script 3: <builtin: TPC-B (sort of)>
 - weight: 0 (targets 0.0% of total)
 - 0 transactions (0.0% of total, tps = 0.000000)
[postgres@progs pg_root]$

自定义脚本进行测试

1) 创建测试表

CREATE TABLE tb1( id serial primary key, ival int);

2) 运行自定义脚本

# 1. 查询语句测试脚本
echo "SELECT id, ival FROM tb1 ORDER BY id DESC LIMIT 10;" > bench_script_for_select.sql

# 2. 执行基准测试脚本
pb_bench -f bench_script_for_select.sql -h 192.168.10.181 -p 1921 -U dev devdb

# 3. 插入语句测试脚本
cat > bench_script_for_insert.sql <<-EOF
\set ival randon(1,100000)
INSERT INTO tb1(ival) VALUES(:ival);
EOF

# 4. 执行基准测试脚本
pb_bench -f bench_script_for_insert.sql -h 192.168.10.181 -p 1921 -U dev devdb

# 5. 按照指定权重测试
pg_bench -T 60 -f bench_script_for_select.sql@3 -f bench_script_for_insert.sql@10 -h 192.168.10.181 -p 1921 -U dev devdb