JMH 使用指南 - java 性能测试
JMH 篇
JMH，即Java Microbenchmark Harness 翻译：java 微基准测试 工具套件。
## 1.添加依赖
```
<dependency>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-core</artifactId>
<version>1.19</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-generator-annprocess</artifactId>
<version>1.19</version>
<scope>provided</scope>
</dependency>
```
## 2.第一个例子
请参加： JMHFirstBenchmark.java
请参加(晋级)： SecondBenchmark.java
请参加(晋级)： ThirdBenchmark.java
## 3.常用注解说明
###3.1 @BenchmarkMode(Mode.All)
Mode有：- Throughput: 整体吞吐量，例如“1秒内可以执行多少次调用” (thrpt,参加第5点)
- AverageTime: 调用的平均时间，例如“每次调用平均耗时xxx毫秒”。（avgt）
- SampleTime: 随机取样，最后输出取样结果的分布，例如“99%的调用在xxx毫秒以内，99.99%的调用在xxx毫秒以内”（simple）
- SingleShotTime: 以上模式都是默认一次 iteration 是 1s，唯有 SingleShotTime 是只运行一次。往往同时把 warmup 次数设为0，用于测试冷启动时的性能。（ss）
### 3.2 @OutputTimeUnit(TimeUnit.MILLISECONDS)
统计单位， 微秒、毫秒 、分、小时、天
### 3.3 @State
可参：JMHFirstBenchmark.java
类注解，JMH测试类必须使用@State注解，State定义了一个类实例的生命周期，可以类比Spring Bean的Scope。由于JMH允许多线程同时执行测试，不同的选项含义如下：
```
Scope.Thread：默认的State，每个测试线程分配一个实例；
Scope.Benchmark：所有测试线程共享一个实例，用于测试有状态实例在多线程共享下的性能；
Scope.Group：每个线程组共享一个实例；
```
### 3.4 @Benchmark
很重要的方法注解，表示该方法是需要进行 benchmark 的对象。和@test 注解一致
### 3.5 @Setup
方法注解，会在执行 benchmark 之前被执行，正如其名，主要用于初始化。
### 3.6 @TearDown (Level)
方法注解，与@Setup 相对的，会在所有 benchmark 执行结束以后执行，主要用于资源的回收等。
(Level) 用于控制 @Setup，@TearDown 的调用时机，默认是 Level.Trial。
Trial：每个benchmark方法前后；
Iteration：每个benchmark方法每次迭代前后；
Invocation：每个benchmark方法每次调用前后，谨慎使用，需留意javadoc注释；
### 3.7 @Param
@Param注解接收一个String数组 ，
可以用来指定某项参数的多种情况。特别适合用来测试一个函数在不同的参数输入的情况下的性能。
可参：JMHFirstBenchmark.java
## 4 Options常用选项
### 4.1 include
benchmark 所在的类的名字，这里可以使用正则表达式对所有类进行匹配。
参考：SecondBenchmark.java
### 4.2 fork
JVM因为使用了profile-guided optimization而“臭名昭著”，这对于微基准测试来说十分不友好，因为不同测试方法的profile混杂在一起，“互相伤害”彼此的测试结果。对于每个@Benchmark方法使用一个独立的进程可以解决这个问题，这也是JMH的默认选项。注意不要设置为0，设置为n则会启动n个进程执行测试（似乎也没有太大意义）。
fork选项也可以通过方法注解以及启动参数来设置。
### 4.3 warmupIterations
预热次数，每次默认1秒。
### 4.4 measurementIterations
实际测量的迭代次数，每次默认1秒。
### 4.5 Group
方法注解，可以把多个 benchmark 定义为同一个 group，则它们会被同时执行，譬如用来模拟生产者－消费者读写速度不一致情况下的表现。
### 4.6 Threads
每个fork进程使用多少条线程去执行你的测试方法，默认值是Runtime.getRuntime().availableProcessors()。
## 5 输出结果
```
# @BenchmarkMode(Mode.All)
# JMH version: 1.19
# VM version: JDK 1.7.0_80, VM 24.80-b11
# VM invoker: C:\Program Files\Java\jdk1.7.0_80\jre\bin\java.exe
# VM options: -javaagent:D:\Program Files\JetBrains\IntelliJ IDEA 2018.1\lib\idea_rt.jar=51664:D:\Program Files\JetBrains\IntelliJ IDEA 2018.1\bin -Dfile.encoding=UTF-8
# Warmup: 2 iterations, single-shot each
# Measurement: 2 iterations, single-shot each
# Timeout: 10 min per iteration
# Threads: 10 threads
# Benchmark mode: Single shot invocation time
# Benchmark: com.gemantic.wealth.yunmatong.service.jmh.SecondBenchmark.singleThreadBench
# Parameters: (length = 100000)
# Run progress: 99.98% complete, ETA 00:00:00
# Fork: 1 of 1
# Warmup Iteration 1: 34.641 ±(99.9%) 33.844 ms/op
# Warmup Iteration 2: 7.129 ±(99.9%) 9.238 ms/op
Iteration 1: 7.573 ±(99.9%) 4.581 ms/op
Iteration 2: 6.235 ±(99.9%) 4.150 ms/op
# Run complete. Total time: 00:00:36
Benchmark (length) Mode Cnt Score Error Units
SecondBenchmark.multiThreadBench 100000 thrpt 2 147.758 ops/ms
SecondBenchmark.singleThreadBench 100000 thrpt 2 0.983 ops/ms
SecondBenchmark.multiThreadBench 100000 avgt 2 0.068 ms/op
SecondBenchmark.singleThreadBench 100000 avgt 2 10.510 ms/op
SecondBenchmark.multiThreadBench 100000 sample 295532 0.068 ± 0.001 ms/op
SecondBenchmark.multiThreadBench:multiThreadBench·p0.00 100000 sample 0.010 ms/op
SecondBenchmark.multiThreadBench:multiThreadBench·p0.50 100000 sample 0.066 ms/op
SecondBenchmark.multiThreadBench:multiThreadBench·p0.90 100000 sample 0.095 ms/op
SecondBenchmark.multiThreadBench:multiThreadBench·p0.95 100000 sample 0.104 ms/op
SecondBenchmark.multiThreadBench:multiThreadBench·p0.99 100000 sample 0.126 ms/op
SecondBenchmark.multiThreadBench:multiThreadBench·p0.999 100000 sample 0.172 ms/op
SecondBenchmark.multiThreadBench:multiThreadBench·p0.9999 100000 sample 1.729 ms/op
SecondBenchmark.multiThreadBench:multiThreadBench·p1.00 100000 sample 4.309 ms/op
SecondBenchmark.singleThreadBench 100000 sample 2036 10.196 ± 0.581 ms/op
SecondBenchmark.singleThreadBench:singleThreadBench·p0.00 100000 sample 6.201 ms/op
SecondBenchmark.singleThreadBench:singleThreadBench·p0.50 100000 sample 8.020 ms/op
SecondBenchmark.singleThreadBench:singleThreadBench·p0.90 100000 sample 10.355 ms/op
SecondBenchmark.singleThreadBench:singleThreadBench·p0.95 100000 sample 38.443 ms/op
SecondBenchmark.singleThreadBench:singleThreadBench·p0.99 100000 sample 41.943 ms/op
SecondBenchmark.singleThreadBench:singleThreadBench·p0.999 100000 sample 73.498 ms/op
SecondBenchmark.singleThreadBench:singleThreadBench·p0.9999 100000 sample 74.973 ms/op
SecondBenchmark.singleThreadBench:singleThreadBench·p1.00 100000 sample 74.973 ms/op
SecondBenchmark.multiThreadBench 100000 ss 2 0.223 ms/op
SecondBenchmark.singleThreadBench 100000 ss 2 6.904 ms/op
```
6.第一个例子
package com.gemantic.wealth.yunmatong.service.jmh;
import lombok.extern.slf4j.Slf4j;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import org.openjdk.jmh.runner.options.TimeValue;
import java.util.concurrent.TimeUnit;
@Slf4j
@BenchmarkMode(Mode.AverageTime)// 测试方法平均执行时间
@OutputTimeUnit(TimeUnit.MICROSECONDS)// 输出结果的时间粒度为微秒
@State(Scope.Benchmark) // 每个测试线程一个实例
public class JMHFirstBenchmark {
/*
* Most of the time, you need to maintain some state while the benchmark is
* running. Since JMH is heavily used to build concurrent benchmarks, we
* opted for an explicit notion of state-bearing objects.
*
* Below are two state objects. Their class names are not essential, it
* matters they are marked with @State. These objects will be instantiated
* on demand, and reused during the entire benchmark trial.
*
* The important property is that state is always instantiated by one of
* those benchmark threads which will then have the access to that state.
* That means you can initialize the fields as if you do that in worker
* threads (ThreadLocals are yours, etc).
*/
@State(Scope.Benchmark)
public static class BenchmarkState {
volatile double x = Math.PI;
}
@State(Scope.Thread)
public static class ThreadState {
volatile double x = Math.PI;
}
@Benchmark
public void measureUnshared(ThreadState state) {
// All benchmark threads will call in this method.
//
// However, since ThreadState is the Scope.Thread, each thread
// will have it's own copy of the state, and this benchmark
// will measure unshared case.
state.x++;
try {
Thread.sleep(500);
} catch (InterruptedException e) {
e.printStackTrace();
}
log.info("measureUnshared:"+ state.x);
}
@Benchmark
public void measureShared(BenchmarkState state) {
// All benchmark threads will call in this method.
//
// Since BenchmarkState is the Scope.Benchmark, all threads
// will share the state instance, and we will end up measuring
// shared case.
state.x++;
try {
Thread.sleep(500);
} catch (InterruptedException e) {
e.printStackTrace();
}
log.info("measureShared:"+ state.x);
}
/*
* ============================== HOW TO RUN THIS TEST: ====================================
*
* You are expected to see the drastic difference in shared and unshared cases,
* because you either contend for single memory location, or not. This effect
* is more articulated on large machines.
*
* You can run this test:
*
* a) Via the command line:
* $ mvn clean install
* $ java -jar target/benchmarks.jar JMHSample_03 -wi 5 -i 5 -t 4 -f 1
* (we requested 5 measurement/warmup iterations, with 4 threads, single fork)
*
* b) Via the Java API:
* (see the JMH homepage for possible caveats when running from IDE:
* http://openjdk.java.net/projects/code-tools/jmh/)
*/
public static void main(String[] args) throws RunnerException {
// 可以通过注解
Options opt = new OptionsBuilder()
.include(JMHFirstBenchmark.class.getSimpleName())
.warmupIterations(3) // 预热3次
.measurementIterations(2).measurementTime(TimeValue.valueOf("1s")) // 运行5次，每次10秒
.threads(10) // 10线程并发
.forks(2)
.build();
new Runner(opt).run();
}
}
第二个例子setup&TearDown&Param
package com.gemantic.wealth.yunmatong.service.jmh;
import com.gemantic.wealth.yunmatong.service.jmh.service.Calculator;
import com.gemantic.wealth.yunmatong.service.jmh.service.MultithreadCalculator;
import com.gemantic.wealth.yunmatong.service.jmh.service.SinglethreadCalculator;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import java.util.concurrent.TimeUnit;
@BenchmarkMode(Mode.All)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@State(Scope.Benchmark)
public class SecondBenchmark {
@Param({"100000"})
private int length;
private int[] numbers;
private Calculator singleThreadCalc;
private Calculator multiThreadCalc;
public static void main(String[] args) throws RunnerException {
Options opt = new OptionsBuilder()
.include(SecondBenchmark.class.getSimpleName()) // .include("JMHF.*") 可支持正则
.forks(0)
.warmupIterations(2)
.measurementIterations(2).threads(10)
.build();
new Runner(opt).run();
}
@Benchmark
public long singleThreadBench() {
return singleThreadCalc.sum(numbers);
}
@Benchmark
public long multiThreadBench() {
return multiThreadCalc.sum(numbers);
}
@Setup(Level.Trial)
public void prepare() {
int n = length;
numbers =new int[n];
for (int i=0;i<n;i++){
numbers[i]=i;
}
singleThreadCalc = new SinglethreadCalculator();
multiThreadCalc = new MultithreadCalculator(Runtime.getRuntime().availableProcessors());
}
@TearDown
public void shutdown() {
singleThreadCalc.shutdown();
multiThreadCalc.shutdown();
}
}
第三个例子group
package com.gemantic.wealth.yunmatong.service.jmh;
import com.gemantic.wealth.yunmatong.service.jmh.service.Calculator;
import com.gemantic.wealth.yunmatong.service.jmh.service.MultithreadCalculator;
import com.gemantic.wealth.yunmatong.service.jmh.service.SinglethreadCalculator;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import org.openjdk.jmh.runner.options.TimeValue;
import java.util.concurrent.TimeUnit;
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@State(Scope.Benchmark)
public class ThirdBenchmark {
@State(Scope.Group)
public static class BenchmarkState {
volatile double x = Math.PI;
}
@Benchmark
@Group("custom")
@GroupThreads(10)
public void read(BenchmarkState state) {
state.x++;
try {
Thread.sleep(5);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("ThirdBenchmark.read: "+ state.x);
}
@Benchmark
@Group("custom")
public void book(BenchmarkState state) {
state.x++;
try {
Thread.sleep(5);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("ThirdBenchmark.book: "+ state.x);
}
public static void main(String[] args) throws RunnerException {
Options opt = new OptionsBuilder()
.include(ThirdBenchmark.class.getSimpleName()) // .include("JMHF.*") 可支持正则
.forks(0)
.warmupIterations(0)
.measurementIterations(2).measurementTime(TimeValue.valueOf("10ms")).threads(5)
.build();
new Runner(opt).run();
}
}
