服务器性能测试利器之sysbench

 

前言

sysbench是一个开源的、模块化的、跨平台的多线程性能测试工具，可以用来进行CPU、内存、磁盘I/O、线程、数据库的性能测试。sysbench是基于LuaJIT的可编写脚本的多线程基准测试工具。

 

sysbench附带以下捆绑的基准测试：

 

  • oltp_*.lua：类似OLTP的数据库基准测试的集合
  • fileio：文件系统级基准
  • cpu：简单的CPU基准测试
  • memory：内存访问基准
  • threads：基于线程的调度程序基准
  • mutex：POSIX互斥基准测试

 

特征

 

  • 提供有关速率和延迟的大量统计信息，包括延迟百分位数和直方图；
  • 即使有数千个并发线程，开销也很低。sysbench每秒能够生成和跟踪数亿个事件。
  • 通过在用户提供的Lua脚本中实现预定义的挂钩，可以轻松创建新的基准测试；
  • 可以也可以作为一个通用的Lua解释，只需更换#!/usr/bin/lua用#!/usr/bin/sysbench在你的脚本。

安装

    安装以Centos为例，其他版本或者mac请参考sysbench说明

1.下载安转

复制代码
wget https://github.com/akopytov/sysbench/archive/1.0.zip -O "sysbench-1.0.zip"
  
unzip sysbench-1.0.zip
 
 
cd sysbench-1.0

//安装依赖
yum install -y automake libtool
 
复制代码

2.编译安装

1
2
3
4
5
./autogen.sh
./configure
export LD_LIBRARY_PATH=/usr/include/mysql
make
make install

3.监测是否安装成功

1
2
[root@guanbin-k8s-master ~]# sysbench --version
sysbench 1.0.19　　

测试

  1. cpu测试

   cpu测试主要是进行素数的加法运算，我们的例子中，指定了最大的质数发生器数量为 20000

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
[root@guanbin-k8s-master ~]# sysbench --test=cpu --cpu-max-prime=20000 run
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
sysbench 1.0.19 (using bundled LuaJIT 2.1.0-beta2)
 
Running the test with following options:
Number of threads: 1
Initializing random number generator from current time
 
 
Prime numbers limit: 20000
 
Initializing worker threads...
 
Threads started!
 
CPU speed:
    events per second:   168.59
 
General statistics:
    total time:                          10.0027s
    total number of events:              1687
 
Latency (ms):
         min:                                    5.63
         avg:                                    5.93
         max:                                   66.09
         95th percentile:                        6.43
         sum:                                 9999.10
 
Threads fairness:
    events (avg/stddev):           1687.0000/0.00
    execution time (avg/stddev):   9.9991/0.00

执行时间 大约为10秒（这个测试最好是两台机器进行对比测试，才能发现那个机器有问题）

2.对内存（memory）进行测试 
   测试是在内存中传输 10G 的数据量，每个 block 大小为 8K
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
[root@guanbin-k8s-master ~]# sysbench --test=memory --memory-block-size=8k --memory-total-size=10G run
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
sysbench 1.0.19 (using bundled LuaJIT 2.1.0-beta2)
 
Running the test with following options:
Number of threads: 1
Initializing random number generator from current time
 
 
Running memory speed test with the following options:
  block size: 8KiB
  total size: 10240MiB
  operation: write
  scope: global
 
Initializing worker threads...
 
Threads started!
 
Total operations: 1310720 (724931.70 per second)
 
10240.00 MiB transferred (5663.53 MiB/sec)
 
 
General statistics:
    total time:                          1.8045s
    total number of events:              1310720
 
Latency (ms):
         min:                                    0.00
         avg:                                    0.00
         max:                                   12.44
         95th percentile:                        0.00
         sum:                                 1441.61
 
Threads fairness:
    events (avg/stddev):           1310720.0000/0.00
    execution time (avg/stddev):   1.4416/0.00

totaltime为1.8045s

3.对disk 的IO测试

新建一个临时目录，并确保磁盘剩余空间足够（--file-total-size不要超过剩余容量）

测试场景：16个线程，创建128个文件，总共占用磁盘空间10GB，测试磁盘读写效率

运行结束后要删除临时测试文件： rm -rf test_file.*

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
root@guanbin-k8s-master tmp]# sysbench --test=fileio --num-threads=16 --file-total-size=1G --file-test-mode=rndrw prepare
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
WARNING: --num-threads is deprecated, use --threads instead
sysbench 1.0.19 (using bundled LuaJIT 2.1.0-beta2)
 
128 files, 8192Kb each, 1024Mb total
Creating files for the test...
Extra file open flags: (none)
Creating file test_file.0
Creating file test_file.1
Creating file test_file.2
Creating file test_file.3
Creating file test_file.4
Creating file test_file.5
Creating file test_file.6
Creating file test_file.7
Creating file test_file.8
Creating file test_file.9
Creating file test_file.10
Creating file test_file.11
Creating file test_file.12
Creating file test_file.13
Creating file test_file.14
Creating file test_file.15
Creating file test_file.16
Creating file test_file.17
Creating file test_file.18
Creating file test_file.19
Creating file test_file.20
Creating file test_file.21
Creating file test_file.22
Creating file test_file.23
Creating file test_file.24
Creating file test_file.25
Creating file test_file.26
Creating file test_file.27
Creating file test_file.28
Creating file test_file.29
Creating file test_file.30
Creating file test_file.31
Creating file test_file.32
Creating file test_file.33
Creating file test_file.34
Creating file test_file.35
Creating file test_file.36
Creating file test_file.37
Creating file test_file.38
Creating file test_file.39
Creating file test_file.40
Creating file test_file.41
Creating file test_file.42
Creating file test_file.43
Creating file test_file.44
Creating file test_file.45
Creating file test_file.46
Creating file test_file.47
Creating file test_file.48
Creating file test_file.49
Creating file test_file.50
Creating file test_file.51
Creating file test_file.52
Creating file test_file.53
Creating file test_file.54
Creating file test_file.55
Creating file test_file.56
Creating file test_file.57
Creating file test_file.58
Creating file test_file.59
Creating file test_file.60
Creating file test_file.61
Creating file test_file.62
Creating file test_file.63
Creating file test_file.64
Creating file test_file.65
Creating file test_file.66
Creating file test_file.67
Creating file test_file.68
Creating file test_file.69
Creating file test_file.70
Creating file test_file.71
Creating file test_file.72
Creating file test_file.73
Creating file test_file.74
Creating file test_file.75
Creating file test_file.76
Creating file test_file.77
Creating file test_file.78
Creating file test_file.79
Creating file test_file.80
Creating file test_file.81
Creating file test_file.82
Creating file test_file.83
Creating file test_file.84
Creating file test_file.85
Creating file test_file.86
Creating file test_file.87
Creating file test_file.88
Creating file test_file.89
Creating file test_file.90
Creating file test_file.91
Creating file test_file.92
Creating file test_file.93
Creating file test_file.94
Creating file test_file.95
Creating file test_file.96
Creating file test_file.97
Creating file test_file.98
Creating file test_file.99
Creating file test_file.100
Creating file test_file.101
Creating file test_file.102
Creating file test_file.103
Creating file test_file.104
Creating file test_file.105
Creating file test_file.106
Creating file test_file.107
Creating file test_file.108
Creating file test_file.109
Creating file test_file.110
Creating file test_file.111
Creating file test_file.112
Creating file test_file.113
Creating file test_file.114
Creating file test_file.115
Creating file test_file.116
Creating file test_file.117
Creating file test_file.118
Creating file test_file.119
Creating file test_file.120
Creating file test_file.121
Creating file test_file.122
Creating file test_file.123
Creating file test_file.124
Creating file test_file.125
Creating file test_file.126
Creating file test_file.127
1073741824 bytes written in 110.77 seconds (9.24 MiB/sec).
[root@guanbin-k8s-master tmp]# sysbench --test=fileio --num-threads=16 --file-total-size=1G --file-test-mode=rndrw run
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
WARNING: --num-threads is deprecated, use --threads instead
sysbench 1.0.19 (using bundled LuaJIT 2.1.0-beta2)
 
Running the test with following options:
Number of threads: 16
Initializing random number generator from current time
 
 
Extra file open flags: (none)
128 files, 8MiB each
1GiB total file size
Block size 16KiB
Number of IO requests: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Initializing worker threads...
 
Threads started!
 
 
File operations:
    reads/s:                      157.44
    writes/s:                     104.96
    fsyncs/s:                     480.79
 
Throughput:
    read, MiB/s:                  2.46
    written, MiB/s:               1.64
 
General statistics:
    total time:                          13.3355s
    total number of events:              7865
 
Latency (ms):
         min:                                    0.00
         avg:                                   20.61
         max:                                 1115.85
         95th percentile:                      142.39
         sum:                               162071.40
 
Threads fairness:
    events (avg/stddev):           491.5625/141.20
    execution time (avg/stddev):   10.1295/0.03　　
吞吐量：读2.46MiB/s，写1.64MiB/s; 95%延迟 142.39ms
 
4.测试mysql性能
       测试时使用的脚本为lua脚本，可以使用sysbench自带脚本，也可以自己开发。对于大多数应用，使用sysbench自带的脚本就足够了。不同版本的sysbench中，lua脚本的位置可能不同，可以自己在sysbench路径下使用find命令搜索oltp.lua。
     P.S.：大多数数据服务都是oltp类型的，如果你不了解什么是oltp，那么大概率你的数据服务就是oltp类型的。MySQL OLTP（On-Line Transaction Processing联机事务处理过程）；
  • （1）尽量不要在MySQL服务器运行的机器上进行测试，一方面可能无法体现网络（哪怕是局域网）的影响，另一方面，sysbench的运行（尤其是设置的并发数较高时）会影响MySQL服务器的表现。
  • （2）可以逐步增加客户端的并发连接数（--thread参数），观察在连接数不同情况下，MySQL服务器的表现；如分别设置为10,20,50,100等。
  • （3）一般执行模式选择complex即可，如果需要特别测试服务器只读性能，或不使用事务时的性能，可以选择simple模式或nontrx模式。
  • （4）如果连续进行多次测试，注意确保之前测试的数据已经被清理干净。

下面是sysbench使用的一个例子：

 注意：先准备数据库

1
mysql> create database sbtest;

 其中，执行模式为complex，使用了10个表，每个表有10万条数据，客户端的并发线程数为10，执行时间为120秒，每10秒生成一次报告。

1
sysbench ./tests/include/oltp_legacy/oltp.lua --mysql-host={yourhostip} --mysql-port=3306 --mysql-user={username} --mysql-password={password} --oltp-test-mode=complex --mysql-table-engine=innodb --oltp-tables-count=10 --oltp-table-size=100000 --threads=10 --time=120 --report-interval=10 prepare

其中的hostip、username、password请自行切换为自己真实的环境数据

查看执行结果

1
sysbench ./tests/include/oltp_legacy/oltp.lua --mysql-host={yourhostip} --mysql-port=3306 --mysql-user={username} --mysql-password={password} --oltp-test-mode=complex --oltp-tables-count=10 --oltp-table-size=100000 --threads=10 --time=120 --report-interval=10 run
复制代码
Running the test with following options:
Number of threads: 10
Report intermediate results every 10 second(s)
Initializing random number generator from current time


Initializing worker threads...

Threads started!

[ 10s ] thds: 10 tps: 17.19 qps: 362.78 (r/w/o: 254.64/72.76/35.38) lat (ms,95%): 1589.90 err/s: 0.00 reconn/s: 0.00
[ 20s ] thds: 10 tps: 33.50 qps: 670.08 (r/w/o: 469.06/134.02/67.01) lat (ms,95%): 530.08 err/s: 0.00 reconn/s: 0.00
[ 30s ] thds: 10 tps: 39.30 qps: 786.00 (r/w/o: 550.20/157.20/78.60) lat (ms,95%): 458.96 err/s: 0.00 reconn/s: 0.00
[ 40s ] thds: 10 tps: 38.80 qps: 775.45 (r/w/o: 543.16/154.69/77.59) lat (ms,95%): 634.66 err/s: 0.00 reconn/s: 0.00
[ 50s ] thds: 10 tps: 30.10 qps: 602.54 (r/w/o: 421.43/120.91/60.20) lat (ms,95%): 590.56 err/s: 0.00 reconn/s: 0.00
[ 60s ] thds: 10 tps: 33.90 qps: 678.00 (r/w/o: 474.60/135.60/67.80) lat (ms,95%): 419.45 err/s: 0.00 reconn/s: 0.00
[ 70s ] thds: 10 tps: 29.50 qps: 589.98 (r/w/o: 412.99/118.00/59.00) lat (ms,95%): 787.74 err/s: 0.00 reconn/s: 0.00
[ 80s ] thds: 10 tps: 39.90 qps: 798.02 (r/w/o: 558.62/159.60/79.80) lat (ms,95%): 569.67 err/s: 0.00 reconn/s: 0.00
[ 90s ] thds: 10 tps: 38.10 qps: 761.99 (r/w/o: 533.40/152.40/76.20) lat (ms,95%): 484.44 err/s: 0.00 reconn/s: 0.00
[ 100s ] thds: 10 tps: 43.90 qps: 877.99 (r/w/o: 614.60/175.60/87.80) lat (ms,95%): 383.33 err/s: 0.00 reconn/s: 0.00
[ 110s ] thds: 10 tps: 37.80 qps: 756.00 (r/w/o: 529.20/151.20/75.60) lat (ms,95%): 530.08 err/s: 0.00 reconn/s: 0.00
[ 120s ] thds: 10 tps: 44.40 qps: 888.01 (r/w/o: 621.60/177.60/88.80) lat (ms,95%): 419.45 err/s: 0.00 reconn/s: 0.00
SQL statistics:
    queries performed:
        read:                            59836
        write:                           17096
        other:                           8548
        total:                           85480
    transactions:                        4274   (35.59 per sec.)
    queries:                             85480  (711.77 per sec.)
    ignored errors:                      0      (0.00 per sec.)
    reconnects:                          0      (0.00 per sec.)

General statistics:
    total time:                          120.0919s
    total number of events:              4274

Latency (ms):
         min:                                   50.07
         avg:                                  280.96
         max:                                 2486.17
         95th percentile:                      559.50
         sum:                              1200810.10

Threads fairness:
    events (avg/stddev):           427.4000/4.20
    execution time (avg/stddev):   120.0810/0.01
复制代码

其中，对于我们比较重要的信息包括：

queries：查询总数及qps

transactions：事务总数及tps

Latency-95th percentile：前95%的请求的最大响应时间，本例中是344毫秒，这个延迟非常大，是因为我用的MySQL服务器性能很差；在正式环境中这个数值是绝对不能接受的。

最后清理数据：

sysbench ./tests/include/oltp_legacy/oltp.lua --mysql-host={yourhostip} --mysql-port=3306 --mysql-user={username} --mysql-password={password} --oltp-tables-count=10  cleanup

清理的日志输出

1
2
3
4
5
6
7
8
9
10
11
12
sysbench 1.0.19 (using bundled LuaJIT 2.1.0-beta2)
 
Dropping table 'sbtest1'...
Dropping table 'sbtest2'...
Dropping table 'sbtest3'...
Dropping table 'sbtest4'...
Dropping table 'sbtest5'...
Dropping table 'sbtest6'...
Dropping table 'sbtest7'...
Dropping table 'sbtest8'...
Dropping table 'sbtest9'...
Dropping table 'sbtest10'...
posted @ 2024-12-17 08:51  心冰之海  阅读(1007)  评论(0)    收藏  举报