Compressor Microbenchmark: WiredTiger LSM

Symas Corp., February 2015

Summary

Most of the benchmark results showed very little variation in throughput or data compression effectiveness across all of the compression libraries. One which showed a significant difference was the random overwrite test:

These charts show the final stats at the end of the run, after all compactions completed. The RSS shows the maximum size of the process for a given run. The times are the total User and System CPU time, and the total Wall Clock time to run all of the test operations for a given compressor.

While bzip2 and lzma usually achieve greater compression, they also have the greatest performance cost. zlib actually shows the smallest size at the end of the test. Overall there's very little variation in the DB sizes, because WiredTiger writes in block-sized chunks and pads out any minor differences in the compressors.

Files

The files used to perform these tests are all available for download.

The command script: cmd-wired.sh. Raw output: out.wiredl.txt. OpenOffice spreadsheet WiredLSM.ods.