> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ilyama.golain.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Telemetry dataspaces

> Append-only named-field time series (API modality timeseries) — ClickHouse MergeTree, schema promotion, and hot/cold storage.

A **telemetry** dataspace stores **history**: one row per sample, never updated in place. Use it for sensor streams, device datapoints, traces, counters, and any reading you will graph or aggregate later.

The API modality is **`timeseries`**. Edge configs and conversation often say `telemetry` or `append`; those map to this modality.

You cannot `PUT` or `DELETE` a sample through row CRUD. There is no current-row identity — only `(source_id, event_ts)`.

→ [All modalities](/dataspaces/overview)

## When to use it

**Use telemetry when**

* The row is a reading, not a record a UI will edit.
* You have (or will freeze) a **named column set**: `temperature`, `humidity`, `line_voltage`.
* Dashboards, rollups, and time-range scans matter.
* The device is the only writer. A late sample is another point, not a conflict.

**Use [point](/dataspaces/point) instead** when each sample is a sparse bag of metric names that changes per firmware or device, and you do not want to declare columns.

**Use [state](/dataspaces/state) or [mutable](/dataspaces/mutable) instead** when the row must be updated or deleted.

## Backing store

Telemetry lives in **ClickHouse** as a `MergeTree` (or `ReplicatedMergeTree` in clustered deployments). You never name the engine on create.

| Physical detail | Behavior                                                                                    |
| --------------- | ------------------------------------------------------------------------------------------- |
| Table engine    | `MergeTree` — append only                                                                   |
| Sort key        | `(source_id, event_ts)`                                                                     |
| Partitioning    | By month of `event_ts` (unless you override `partition_by`)                                 |
| Updates         | Not supported. A new sample is a new row.                                                   |
| Deletes         | Not a hot-path operation. Rare erasure (for example device purge) is a background mutation. |
| Dedup on retry  | Native block deduplication for an identical replayed batch                                  |

Reads go through a **read view** that projects typed columns and, while the layout still has overflow, JSON paths from the overflow blob. Query the dataspace **name** (QueryScript) or the catalog `read_view` (raw SQL). Do not hardcode physical hot/cold table names.

## Meta columns

Every telemetry row carries:

| Column        | Meaning                                                                                                                                                                              |
| ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `source_type` | Producer class (`device`, `edge`, `api`, …)                                                                                                                                          |
| `source_id`   | Producer UUID (device id on the edge path)                                                                                                                                           |
| `event_ts`    | Event time from the payload, UTC, millisecond precision. Drives partitioning, age-out, and time filters. If the payload has no timestamp, the write plane falls back to `ingest_ts`. |
| `ingest_ts`   | Server receive time                                                                                                                                                                  |

User fields sit after these. `version` and `is_deleted` are **not** telemetry columns.

## Schema and layouts

Default layout is **`jsonb`**: undeclared fields land in an overflow JSON blob and stay readable. As a path is observed at a stable type, you can **promote** it to a real column (`hybrid`, then `typed`).

| Layout            | What you get                                                                                           |
| ----------------- | ------------------------------------------------------------------------------------------------------ |
| `jsonb` (default) | Fast to start; all user data in overflow until promotion                                               |
| `hybrid`          | Hot fields as columns; the rest still in overflow                                                      |
| `typed`           | Frozen schema, no overflow. Required before `hot_cold` age-out. Surprise columns **orphan** the batch. |

Promotion is how telemetry reaches a cheap cold tier: you cannot age a still-evolving `jsonb` / `hybrid` schema to immutable Parquet. If you request `tiering: hot_cold` at create while the layout is still evolving, the platform keeps the dataspace **hot** and records the request. Age-out turns on only after every field is typed and overflow is drained.

Pass `fields` at create if you already know the column list. That is the honest way to start closer to `typed`.

A project-wide cap of **1000 columns** applies. Over-cap or type-mismatch rows on a typed layout go to the **orphan** table for review — they are not discarded.

## Time window and late data

The write plane accepts a row when `event_ts` is not more than **one day in the future**. Future-dated rows go to **dropped-rows** (retained, with reason `future`).

**Late** history (older than the hot window) is diverted only when the dataspace is **`hot_cold`**. That protects the immutable cold range. A **hot-only** telemetry dataspace accepts arbitrarily old backfill.

Edge devices that buffer offline should stay **untiered** (`hot`) unless you explicitly accept dropped late batches.

## Tiering and storage class

| Setting                         | Meaning                                                                                                                                   |
| ------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| `storage_class: fast` (default) | Hot MergeTree. Fast queries.                                                                                                              |
| `storage_class: cheap`          | Parquet on object storage, **no** hot table. Requires a frozen/uniform layout. Rare for telemetry until the schema is typed.              |
| `tiering: hot` (default)        | Everything stays in the hot table.                                                                                                        |
| `tiering: hot_cold`             | Age-out moves old partitions to an S3/Parquet cold table. The read view unions hot and cold. Requires `typed` (or a completed promotion). |

Cold data is **immutable**. You cannot rewrite a sample that has aged out. That is why evolving overflow layouts cannot tier.

## Events and APIs

Telemetry does **not** emit a platform event per insert. Subscribe to derived signals or query the dataspace.

There is no `GET/PUT/PATCH/DELETE …/rows` surface. List and filter history with QueryScript or `POST /projects/{project_id}/dataspaces/query`.

Edge sync is **upsync only**. You cannot attach an `edge_sqlite` sync target to a telemetry dataspace (rejected). Devices send samples; the cloud does not push telemetry rows back.

## Create

```bash theme={null}
curl -sS -X POST "$API/projects/$PROJECT_ID/dataspaces" \
  -H "Authorization: Bearer $TOKEN" \
  -H "ORG-ID: $ORG_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "device_readings",
    "modality": "timeseries"
  }'
```

With known fields and a future cold tier (stays hot until promotion finishes):

```bash theme={null}
curl -sS -X POST "$API/projects/$PROJECT_ID/dataspaces" \
  -H "Authorization: Bearer $TOKEN" \
  -H "ORG-ID: $ORG_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "line_voltage",
    "modality": "timeseries",
    "tiering": "hot_cold",
    "fields": [
      {"path": "volts", "type": "float8"},
      {"path": "phase", "type": "string"}
    ]
  }'
```

## Related

* [Dataspaces overview](/dataspaces/overview)
* [Schema and layouts](/dataspaces/schema)
* [Storage](/dataspaces/storage)
* [Point](/dataspaces/point) — sparse metric bags instead of named columns
* [Edge capture strategies](/edge/data-sync/capture-strategies)
* [API introduction](/api-reference/introduction)