Topling CSPP MemTable 开源啦

开源地址：GitHub - topling/cspp-memtable

编译

CSPP MemTable 是 ToplingDB 的组件，跟 ToplingDB 是松耦合的，所以未使用 submodule，而是在编译 ToplingDB 的时候，会自动 clone 并获取最新版。

clone 并编译 ToplingDB(with CSPP MemTable)

sudo yum -y install git libaio-devel gcc-c++ gflags-devel zlib-devel bzip2-devel
git clone https://github.com/topling/toplingdb
cd toplingdb
make -j`nproc` db_bench DEBUG_LEVEL=0
cp sideplugin/rockside/src/topling/web/{style.css,index.html} ${/path/to/dbdir}
cp sideplugin/rockside/sample-conf/lcompact_community.yaml .
export LD_LIBRARY_PATH=`find sideplugin -name lib_shared`

 

性能测试

memtablerep_bench 性能测试

./memtablerep_bench -benchmarks=fillrandom,readrandom,readwrite   \
                    -memtablerep='cspp:{"mem_cap":"16G","use_hugepage":false}' \
                    -write_buffer_size=536870912 \
                    -item_size=0 -num_operations=10000000

 

memtablerep_bench 测试的是 MemTable 本身的性能，虽然相比直接测试 CSPP 算法损失了不少性能，但相比 SkipList 仍体现出巨大的优势。

db_bench 性能指标

# change ./lcompact_community.yaml
# 1. path items (search /dev/shm), if you have no fast disk(such as on a cloud server), use /dev/shm
# 2. change max_background_compactions to your cpu core num
# command option -json can accept json and yaml files, here use yaml file for more human readable
./db_bench -json lcompact_community.yaml -num 10000000 -disable_wal=true \
           -value_size10 -batch_size=1000 -benchmarks=fillrandom,readrandom
# you can access http://127.0.0.1:8081 to see webview
# you can see this db_bench is much faster than RocksDB

 

db_bench 包含了完整的读写流程，CSPP MemTable 的性能优势会被拉平很多，通过把 batch_size 改大，value_size 改小，CSPP MemTable 的优势会体现得更突出：

cspp     fillrandom   :      13.751 micros/op 4654214 ops/sec;  106.5 MB/s
skiplist fillrandom   :      39.414 micros/op 1623731 ops/sec;   37.2 MB/s