  With the wide application of time series databases (TSDBs) in big data fields
like cluster monitoring and industrial IoT, there have been developed a number
of TSDBs for time series data management. Different TSDBs have test reports
comparing themselves with other databases to show their advantages, but the
comparisons are typically based on their own tools without using a common
well-recognized test framework. To the best of our knowledge, there is no
mature TSDB benchmark either. With the goal of establishing a standard of
evaluating TSDB systems, we present the IoTDB-Benchmark framework, specifically
designed for TSDB and IoT application scenarios. We pay close attention to some
special data ingestion scenarios and summarize 10 basic queries types. We use
this benchmark to compare four TSDB systems: InfluxDB, OpenTSDB, KairosDB and
TimescaleDB. Our benchmark framework/tool not only measures performance metrics
but also takes system resource consumption into consideration.
