Only read the columns the query needs
A latency aggregation can scan timestamps, service and duration without pulling every attribute or payload through memory
Columnar observability architecture
Stateless by design
ClickHouse® database underneath
PulsusDB gives each observability signal a purpose-built ClickHouse data model, optimised for the queries it serves and the bytes it stores. Its stateless compiler lowers complex PromQL, LogQL and TraceQL into ClickHouse SQL, pushing work to the data and returning reduced results
PulsusDB ArchitectureStateless application tier · Stateful database tierINGEST AND QUERY SCALE INDEPENDENTLY
01INGEST PATHtelemetry in
PRODUCERSOTel Collectorapps · agents · infrastructure
OTLP / HTTP
STATELESSPulsusDB application tierscale horizontally
API SURFACEIngest API/v1/logs · /v1/metrics
/v1/traces · /v1development/profiles
/v1/traces · /v1development/profiles
VALIDATED BATCHES
INGEST PROCESSWriternormalise · batch · route
NATIVE INSERT
STATEFULClickHouse database tierdurable state
STORAGE MODELSignal-aware tablesindexes · compression · rollups · object storage
02QUERY PATHanswers out
CONSUMERSGrafana · Persesdashboards · explore · API clients
QUERY LANGUAGES
STATELESSPulsusDB application tierscale horizontally
API SURFACEQuery APIsPromQL · LogQL · TraceQL
TYPED EXPRESSION
QUERY PROCESSQuery language compilerparse → resolve → plan → lower
SQL PLAN →
← REDUCED RESULTS
← REDUCED RESULTS
STATEFULClickHouse database tierdurable state
SQL EXECUTIONDatabase executionprune · scan · join · aggregate
all durable state lives in the ClickHouse databaseheavy compute stays beside the stored data
COLLECTPLANPRUNESCANAGGREGATETIER
STATEFUL DATABASE TIER
Why we chose the ClickHouse database
Observability combines sustained batched ingestion with analytical queries across enormous time ranges. ClickHouse software gives PulsusDB columnar and vectorised execution, high compression, sparse and skipping indexes, materialised-view rollups, object-storage tiering and horizontal scale in one database
The execution model
Read fewer bytes
Use every core
Observability queries are analytical: filter a time window, scan selected columns and aggregate many values. ClickHouse software is built around exactly that shape
How columnar engines work ↗Process values in CPU-friendly batches
ClickHouse software evaluates filters and aggregations over contiguous column vectors, using cache locality, parallel pipelines and SIMD-friendly loops
Skip granules before they are scanned
MergeTree sorting keys, sparse primary indexes and data-skipping indexes let PulsusDB constrain reads by tenant, series, stream, trace and time
Scale from cores to shards
Queries run across CPU cores on one node and across distributed tables in a cluster, merging partial results at the coordinator
Compression shaped around time series
Extreme storage efficiency
Without losing precision
PulsusDB chooses codecs by column shape. Regular timestamps compress as changes between changes, floating-point metric values use Gorilla encoding, and compact integer counts use T64 before ZSTD handles the resulting byte stream
Read about ClickHouse codecs ↗schema / metric_samplesLOSSLESS
TIMESTAMP
unix_milli Int64DoubleDelta + ZSTD1700000000000+1000+1000Δ² 0
METRIC VALUE
value Float64Gorilla + ZSTD42.0142.0342.02XOR
COUNTS
count UInt32T64 + ZSTDsame values in · fewer bytes at rest · exact values out
The data model is the performance feature
Model for the query
Compress for the shape
PulsusDB does not pour every signal into one generic table. Each model is designed backwards from its dominant read path, helping ClickHouse software prune more data, scan fewer columns and store repeated values efficiently
Metrics
Query shapeMetric name + series + time window
Physical modelName-first samples, fingerprinted series and no strings on the hot sample path
Storage efficiencyDoubleDelta timestamps, Gorilla values and multi-resolution materialised rollups
Logs
Query shapeStream labels + time + whole-word or pipeline filters
Physical modelService-led samples with separate stream and label indexes
Storage efficiencyLabels stored off the hot scan, compressed bodies and word-level skipping
Traces
Query shapeAttributes + trace structure + time
Physical modelSpan payloads, typed attribute index and service/time projections
Storage efficiencyDelta-encoded time fields, compact durations and compressed payload columns
Profiles
Query shapeProfile labels + time + stack merge
Physical modelFingerprint index, service-led samples and precomputed flamegraph trees
Storage efficiencyCompact trees are merged without repeatedly parsing original profiles
No generic telemetry table
Each signal keeps the columns and indexes its queries need, with physical order chosen for the dominant read path rather than compatibility convenience
Automatic rollup routing
ClickHouse materialised views build a ladder of metric resolutions. PulsusDB automatically selects the most efficient eligible level for the PromQL range and step, scanning fewer samples while preserving the requested response grid
Hot-to-cold tiering
TTL rules can move older raw data and rollup parts from fast local volumes to a configured cold volume such as S3-backed object storage, then expire whole partitions by retention policy
Horizontal ClickHouse database scale
Replicated MergeTree tables and Distributed wrappers extend the same signal-aware schemas across shards. Sharding keys keep traces together and route other families around their read paths
PulsusDB is stateless
Scale the work
Not another state tier
All durable state lives in the ClickHouse database. Reader caches are rebuildable, so PulsusDB processes can be restarted, replaced and load-balanced without moving ownership or replaying a separate database
Read the process model ↗INGEST PRESSURE
writer 01writer 02writer N
QUERY PRESSURE
reader 01reader 02reader N
DURABLE STATE + DISTRIBUTED COMPUTEClickHouse databaseshards · replicas · rollups · object storage
Run all mode beside one ClickHouse node when the deployment is small
Separate writer and reader processes when ingest and query pressure diverge
Rebuild caches from the ClickHouse database and put any healthy process behind a load balancer
The PulsusDB query compiler
Complex queries in
Reduced results out
The compiler lowers complex PromQL, LogQL and TraceQL into one or more ClickHouse SQL stages. Filters, joins, grouping and eligible aggregations run beside the data, minimising traffic between the stateless and database tiers
Query languages
PromQLLogQLTraceQL
Stateless compiler parse → plan → lower preserve language semantics
SQL plan → ← reduced results narrow tier boundary
ClickHouse execution prune → scan → join → aggregate compute runs beside the stored columns
PromQL, LogQL or TraceQL becomes a typed plan
Find series, streams, spans and the required time window
Automatically choose raw data, an index or the most efficient eligible rollup
Push exact filters, joins, grouping and eligible aggregation into ClickHouse SQL
Finish language semantics the database should not decide
Go deeper in the sourceThe schema and architecture documents are the implementation authority
Read schema design ↗ Storage configuration