开源jvm性能基准测试工具之renaissance

JVM标准的性能测试工具是SPECjbb 2015，

SPECjbb2015是SPEC组织的一个用于评估服务器端Java应用性能的基准测试程序，其官方主页为 https://www.spec.org/jbb2015 。在其之前还有SPECjbb2013、SPECjbb2005等版本。该基准测试主要测试Java虚拟机（JVM）、JIT编译器、垃圾回收、Java线程等方面，也可对CPU、缓存、内存结构的性能进行度量。

SPECjbb2015既是CPU密集型也是内存密集型的基准测试程序，它利用Java应用能够比较真实地反映Java程序在某个系统上的运行性能。

只不过它是商业工具，且没有试用版，包括JEP 333 ZGC都是用specjbb 2015测的。最近为了对比下jdk 8，jdk 11以及jdk 17下各个GC的性能，找到了一个可参考的性能测试工具Renaissance Benchmark，虽然生成的报告比较简单，但覆盖大数据、并发、web等方面，测试还是比较有参考性。

[zjh@hs-10-20-30-193 ~]$ time java -Xms16g -Xmx16g -XX:+UseG1GC -XX:MaxGCPauseMillis=5 -jar renaissance-mit-0.14.1.jar -r 3 dec-tree
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
NOTE: 'dec-tree' benchmark uses Spark local executor with 6 (out of 32) threads.
====== dec-tree (apache-spark) [default], iteration 0 started ======
GC before operation: completed in 41.340 ms, heap usage 128.745 MB -> 42.149 MB.
====== dec-tree (apache-spark) [default], iteration 0 completed (3485.129 ms) ======
====== dec-tree (apache-spark) [default], iteration 1 started ======
GC before operation: completed in 61.878 ms, heap usage 833.410 MB -> 69.500 MB.
====== dec-tree (apache-spark) [default], iteration 1 completed (969.746 ms) ======
====== dec-tree (apache-spark) [default], iteration 2 started ======
GC before operation: completed in 59.005 ms, heap usage 643.616 MB -> 91.783 MB.
====== dec-tree (apache-spark) [default], iteration 2 completed (740.158 ms) ======

real 0m9.893s
user 0m37.843s
sys 0m2.825s

覆盖的用例包括：

List of benchmarks

The following is the complete list of benchmarks, separated into groups.

apache-spark

• als - Runs the ALS algorithm from the Spark ML library.
  Default repetitions: 30; APACHE2 license, MIT distribution; Supported JVM: 1.8 and later

• chi-square - Runs the chi-square test from Spark MLlib.
  Default repetitions: 60; APACHE2 license, MIT distribution; Supported JVM: 1.8 and later

• dec-tree - Runs the Random Forest algorithm from the Spark ML library.
  Default repetitions: 40; APACHE2 license, MIT distribution; Supported JVM: 1.8 and later

• gauss-mix - Computes a Gaussian mixture model using expectation-maximization.
  Default repetitions: 40; APACHE2 license, MIT distribution; Supported JVM: 1.8 and later

• log-regression - Runs the Logistic Regression algorithm from the Spark ML library.
  Default repetitions: 20; APACHE2 license, MIT distribution; Supported JVM: 1.8 and later

• movie-lens - Recommends movies using the ALS algorithm.
  Default repetitions: 20; APACHE2 license, MIT distribution; Supported JVM: 1.8 and later

• naive-bayes - Runs the multinomial Naive Bayes algorithm from the Spark ML library.
  Default repetitions: 30; APACHE2 license, MIT distribution; Supported JVM: 1.8 and later

• page-rank - Runs a number of PageRank iterations, using RDDs.
  Default repetitions: 20; APACHE2 license, MIT distribution; Supported JVM: 1.8 and later

concurrency

• akka-uct - Runs the Unbalanced Cobwebbed Tree actor workload in Akka.
  Default repetitions: 24; MIT license, MIT distribution; Supported JVM: 1.8 and later

• fj-kmeans - Runs the k-means algorithm using the fork/join framework.
  Default repetitions: 30; APACHE2 license, MIT distribution; Supported JVM: 1.8 and later

• reactors - Runs benchmarks inspired by the Savina microbenchmark workloads in a sequence on Reactors.IO.
  Default repetitions: 10; MIT license, MIT distribution; Supported JVM: 1.8 and later

database

• db-shootout - Executes a shootout test using several in-memory databases.
  Default repetitions: 16; APACHE2 license, MIT distribution; Supported JVM: 1.8 - 11

• neo4j-analytics - Executes Neo4J graph queries against a movie database.
  Default repetitions: 20; GPL3 license, GPL3 distribution; Supported JVM: 11 - 15

functional

• future-genetic - Runs a genetic algorithm using the Jenetics library and futures.
  Default repetitions: 50; APACHE2 license, MIT distribution; Supported JVM: 1.8 and later

• mnemonics - Solves the phone mnemonics problem using JDK streams.
  Default repetitions: 16; MIT license, MIT distribution; Supported JVM: 1.8 and later

• par-mnemonics - Solves the phone mnemonics problem using parallel JDK streams.
  Default repetitions: 16; MIT license, MIT distribution; Supported JVM: 1.8 and later

• rx-scrabble - Solves the Scrabble puzzle using the Rx streams.
  Default repetitions: 80; GPL2 license, GPL3 distribution; Supported JVM: 1.8 and later

• scrabble - Solves the Scrabble puzzle using JDK Streams.
  Default repetitions: 50; GPL2 license, GPL3 distribution; Supported JVM: 1.8 and later

scala

• dotty - Runs the Dotty compiler on a set of source code files.
  Default repetitions: 50; BSD3 license, MIT distribution; Supported JVM: 1.8 and later

• philosophers - Solves a variant of the dining philosophers problem using ScalaSTM.
  Default repetitions: 30; BSD3 license, MIT distribution; Supported JVM: 1.8 and later

• scala-doku - Solves Sudoku Puzzles using Scala collections.
  Default repetitions: 20; MIT license, MIT distribution; Supported JVM: 1.8 and later

• scala-kmeans - Runs the K-Means algorithm using Scala collections.
  Default repetitions: 50; MIT license, MIT distribution; Supported JVM: 1.8 and later

• scala-stm-bench7 - Runs the stmbench7 benchmark using ScalaSTM.
  Default repetitions: 60; BSD3, GPL2 license, GPL3 distribution; Supported JVM: 1.8 and later

web

• finagle-chirper - Simulates a microblogging service using Twitter Finagle.
  Default repetitions: 90; APACHE2 license, MIT distribution; Supported JVM: 1.8 and later

• finagle-http - Sends many small Finagle HTTP requests to a Finagle HTTP server and awaits response.
  Default repetitions: 12; APACHE2 license, MIT distribution; Supported JVM: 1.8 and later

The suite also contains a group of benchmarks intended solely for testing purposes:

dummy

• dummy-empty - A dummy benchmark which only serves to test the harness.
  Default repetitions: 20; MIT license, MIT distribution; Supported JVM: 1.8 and later

• dummy-failing - A dummy benchmark for testing the harness (fails during iteration).
  Default repetitions: 20; MIT license, MIT distribution; Supported JVM: 1.8 and later

• dummy-param - A dummy benchmark for testing the harness (test configurable parameters).
  Default repetitions: 20; MIT license, MIT distribution; Supported JVM: 1.8 and later

• dummy-setup-failing - A dummy benchmark for testing the harness (fails during setup).
  Default repetitions: 20; MIT license, MIT distribution; Supported JVM: 1.8 and later

• dummy-teardown-failing - A dummy benchmark for testing the harness (fails during teardown).
  Default repetitions: 20; MIT license, MIT distribution; Supported JVM: 1.8 and later

• dummy-validation-failing - A dummy benchmark for testing the harness (fails during validation).
  Default repetitions: 20; MIT license, MIT distribution; Supported JVM: 1.8 and later

•  

spec jbb2015报告参考 https://blog.csdn.net/don_chiang709/article/details/100576653

https://www.spec.org/jbb2015/docs/userguide.pdf

https://renaissance.dev/resources/docs/renaissance-suite.pdf