Telemetry dataspaces

A telemetry dataspace stores history: one row per sample, never updated in place. Use it for sensor streams, device datapoints, traces, counters, and any reading you will graph or aggregate later. The API modality is timeseries. Edge configs and conversation often say telemetry or append; those map to this modality. You cannot PUT or DELETE a sample through row CRUD. There is no current-row identity — only (source_id, event_ts). → All modalities

When to use it

Use telemetry when

The row is a reading, not a record a UI will edit.
You have (or will freeze) a named column set: temperature, humidity, line_voltage.
Dashboards, rollups, and time-range scans matter.
The device is the only writer. A late sample is another point, not a conflict.

Use point instead when each sample is a sparse bag of metric names that changes per firmware or device, and you do not want to declare columns. Use state or mutable instead when the row must be updated or deleted.

Backing store

Telemetry lives in ClickHouse as a MergeTree (or ReplicatedMergeTree in clustered deployments). You never name the engine on create.

Physical detail	Behavior
Table engine	`MergeTree` — append only
Sort key	`(source_id, event_ts)`
Partitioning	By month of `event_ts` (unless you override `partition_by`)
Updates	Not supported. A new sample is a new row.
Deletes	Not a hot-path operation. Rare erasure (for example device purge) is a background mutation.
Dedup on retry	Native block deduplication for an identical replayed batch

Reads go through a read view that projects typed columns and, while the layout still has overflow, JSON paths from the overflow blob. Query the dataspace name (QueryScript) or the catalog read_view (raw SQL). Do not hardcode physical hot/cold table names.

Meta columns

Every telemetry row carries:

Column	Meaning
`source_type`	Producer class (`device`, `edge`, `api`, …)
`source_id`	Producer UUID (device id on the edge path)
`event_ts`	Event time from the payload, UTC, millisecond precision. Drives partitioning, age-out, and time filters. If the payload has no timestamp, the write plane falls back to `ingest_ts`.
`ingest_ts`	Server receive time

User fields sit after these. version and is_deleted are not telemetry columns.

Schema and layouts

Default layout is jsonb: undeclared fields land in an overflow JSON blob and stay readable. As a path is observed at a stable type, you can promote it to a real column (hybrid, then typed).

Layout	What you get
`jsonb` (default)	Fast to start; all user data in overflow until promotion
`hybrid`	Hot fields as columns; the rest still in overflow
`typed`	Frozen schema, no overflow. Required before `hot_cold` age-out. Surprise columns orphan the batch.

Promotion is how telemetry reaches a cheap cold tier: you cannot age a still-evolving jsonb / hybrid schema to immutable Parquet. If you request tiering: hot_cold at create while the layout is still evolving, the platform keeps the dataspace hot and records the request. Age-out turns on only after every field is typed and overflow is drained. Pass fields at create if you already know the column list. That is the honest way to start closer to typed. A project-wide cap of 1000 columns applies. Over-cap or type-mismatch rows on a typed layout go to the orphan table for review — they are not discarded.

Time window and late data

The write plane accepts a row when event_ts is not more than one day in the future. Future-dated rows go to dropped-rows (retained, with reason future). Late history (older than the hot window) is diverted only when the dataspace is hot_cold. That protects the immutable cold range. A hot-only telemetry dataspace accepts arbitrarily old backfill. Edge devices that buffer offline should stay untiered (hot) unless you explicitly accept dropped late batches.

Tiering and storage class

Setting	Meaning
`storage_class: fast` (default)	Hot MergeTree. Fast queries.
`storage_class: cheap`	Parquet on object storage, no hot table. Requires a frozen/uniform layout. Rare for telemetry until the schema is typed.
`tiering: hot` (default)	Everything stays in the hot table.
`tiering: hot_cold`	Age-out moves old partitions to an S3/Parquet cold table. The read view unions hot and cold. Requires `typed` (or a completed promotion).

Cold data is immutable. You cannot rewrite a sample that has aged out. That is why evolving overflow layouts cannot tier.

Events and APIs

Telemetry does not emit a platform event per insert. Subscribe to derived signals or query the dataspace. There is no GET/PUT/PATCH/DELETE …/rows surface. List and filter history with QueryScript or POST /projects/{project_id}/dataspaces/query. Edge sync is upsync only. You cannot attach an edge_sqlite sync target to a telemetry dataspace (rejected). Devices send samples; the cloud does not push telemetry rows back.

Create

curl -sS -X POST "$API/projects/$PROJECT_ID/dataspaces" \
  -H "Authorization: Bearer $TOKEN" \
  -H "ORG-ID: $ORG_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "device_readings",
    "modality": "timeseries"
  }'

With known fields and a future cold tier (stays hot until promotion finishes):

curl -sS -X POST "$API/projects/$PROJECT_ID/dataspaces" \
  -H "Authorization: Bearer $TOKEN" \
  -H "ORG-ID: $ORG_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "line_voltage",
    "modality": "timeseries",
    "tiering": "hot_cold",
    "fields": [
      {"path": "volts", "type": "float8"},
      {"path": "phase", "type": "string"}
    ]
  }'

Dataspaces overview
Schema and layouts
Storage
Point — sparse metric bags instead of named columns
Edge capture strategies
API introduction

Getting started

Dataspaces

Console

Dashboards

Telemetry dataspaces

When to use it

Backing store

Meta columns

Schema and layouts

Time window and late data

Tiering and storage class

Events and APIs

Create

​When to use it

​Backing store

​Meta columns

​Schema and layouts

​Time window and late data

​Tiering and storage class

​Events and APIs

​Create

​Related

When to use it

Backing store

Meta columns

Schema and layouts

Time window and late data

Tiering and storage class

Events and APIs

Create

Related