Reference
M3DB, a distributed time series database
About M3DB, inspired by Gorilla and Cassandra, is a distributed time series database released as open source by Uber Technologies. It can be used for storing realtime metrics at long retention.
Here are some attributes of the project:
Distributed time series storage, single nodes use a WAL commit log and persists time windows per shard independently Cluster management built on top of etcd Built-in synchronous replication with configurable durability and read consistency (one, majority, all, etc) M3TSZ float64 compression inspired by Gorilla TSZ compression, configurable as lossless or lossy Arbitrary time precision configurable from seconds to nanoseconds precision, able to switch precision with any write Configurable out of order writes, currently limited to the size of the configured time window’s block size Current Limitations Due to the nature of the requirements for the project, which are primarily to reduce the cost of ingesting and storing billions of timeseries and providing fast scalable reads, there are a few limitations currently that make M3DB not suitable for use as a general purpose time series database.
Read more
M3 Coordinator, API for reading/writing metrics and management
M3 Coordinator is a service that coordinates reads and writes between upstream systems, such as Prometheus, and downstream systems, such as M3DB.
It also provides management APIs to setup and configure different parts of M3.
The coordinator is generally a bridge for read and writing different types of metrics formats and a management layer for M3.
Note: M3DB by default includes the M3 Coordinator accessible on port 7201. For production deployments it is recommended to deploy it as a dedicated service to ensure you can scale the write coordination role separately and independently to database nodes as an isolated application separate from the M3DB database role.
Read more
M3 Query, a stateless query server for M3DB and Prometheus
M3 Query is a service that exposes all metrics query endpoints along with metrics time series metadata APIs that return dimensions and labels of metrics that reside in a M3DB cluster.
Note: M3 Coordinator, and by proxy M3DB, by default includes the M3 Query endpoints accessible on port 7201. For production deployments it is recommended to deploy it as a dedicated service to ensure you can scale the memory heavy query role separately from the metrics ingestion write path of writes through M3 Coordinator to M3DB database role nodes.
Read more