ClickHouse as the analytics database for User Behavior Metrics
Timeline
| Time | Description |
|---|---|
| 2021-11-30 12:00:01 | Implementation |
Status: Accepted
Context
We need to store and later on process user generated behavior metrics in an efficient way, so that we can:
- ship large quantity of user generated metrics to the database
- store the shipped metrics for a longer period of time, potentially more than 6 months
- support large number of users when the Archipelo platform will become a success
- support analytics on the user behavior metrics in real time to support the context
Decision
We decided to introduce ClickHouse as the data store for the user behavior metrics, which will initially include:
- A single ClickHouse node
- A single ZooKeeper node
- Terraform changes to be able to create the infrastructure elements
Consequences
The following consequences will follow introduction of ClickHouse:
- single ClickHouse node in all environments, with potentially more needed for HA/FT in production
- single ZooKeeper server in all environments, with production having the 3 node ensemble eventually
- Terraform changes to spin all the necessary infrastructure elements
- additional costs for infrastructure elements
- maintenance for self-managed infrastructure elements