$ lexprog.com

// notes from an old coder -- php, databases, and the occasional rant

[September 24, 2024] ClickHouse

ClickHouse Real-Time Dashboard

ClickHouse Real-Time Dashboard

────────────────────────────────────────────────────────

ClickHouse Real-Time Dashboard

Tip: Live Query

SELECT count() FROM events WHERE created_at >= now() - INTERVAL 5 MINUTE;

Gotcha: Refresh Rate

Polling every second adds load. Use 5-30 second intervals.

Tip: Materialized Views

Pre-compute dashboard metrics with materialized views for instant reads.

Gotcha: Concurrent Queries

Multiple dashboard panels = multiple concurrent queries. Monitor server load.

Tip: Approximate Counts

SELECT uniq(user_id) FROM events;

Faster than uniqExact() for dashboard metrics.

Gotcha: Data Freshness

Real-time dashboards show data that's a few seconds old due to insert buffering.

Tip: Order of Columns in ORDER BY Matters Massively

ClickHouse's primary key is defined by ORDER BY. Put high-cardinality columns first for better data skipping. ORDER BY (timestamp, user_id) is very different from ORDER BY (user_id, timestamp) in query performance.

Tip: Use LowCardinality for Enum-Like Strings

Strings like status, country, browser benefit from LowCardinality(String) — it's stored as a dictionary internally, reducing storage 10x and speeding up scans.

Gotcha: Mutations Are Heavy

ALTER TABLE ... UPDATE and DELETE in ClickHouse create new parts instead of modifying in place. A single mutation on a large table can take hours and block merges. Design for append-only from day one.

Senior Insight

Building real-time dashboards with ClickHouse requires a different architecture than batch-oriented BI tools. I use AggregatingMergeTree for pre-aggregated metrics, updating every minute via materialized views. The dashboard queries are simple SELECTs on the pre-aggregated tables — they return in milliseconds even with years of data. The front end polls every 30 seconds. This pattern serves dashboards with sub-second load times and can handle thousands of concurrent users.

Source: ClickHouse Blog (https://clickhouse.com/blog), Altinity Blog (https://altinity.com/blog), Altinity Knowledge Base (https://kb.altinity.com/)

────────────────────────────────────────────────────────
<-- back to posts