> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ilyama.golain.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Mutable dataspaces

> Last-value-wins current tables in ClickHouse — ReplacingMergeTree, natural keys, tombstones, and when to use this instead of state.

A **mutable** dataspace is a **table of current rows**: many live records per device, last-write-wins on `version`, deletes as tombstones. It is the ClickHouse sibling of [state](/dataspaces/state) — same idea, different scale and guarantees.

Use it for edge relational mirrors (work orders, inventory, child rows) and cloud catalogs you upsert and scan at analytics volume.

The API modality is **`mutable`**.

→ [All modalities](/dataspaces/overview)

## When to use it

**Use mutable when**

* Each source holds **many current rows**, keyed by a business primary key.
* You upsert and delete as a bulk replica, not as one document per device.
* Columnar scans and filters matter more than GET-after-PUT.
* Last-write-wins on a monotonic `version` is enough. A missed change event on crash is acceptable.

**Use [state](/dataspaces/state) instead** when you need HTTP row CRUD, read-your-write, downsync of a small keyset, or a change event that **commits with the row** (shadow, e-stop, settings).

**Use [telemetry](/dataspaces/timeseries) instead** when rows are samples you will never update.

**Do not use mutable** for compare-and-set (`UPDATE … WHERE status = 'running'`). ReplacingMergeTree has no row locks. Job/claim state belongs in ordinary platform tables.

## Backing store

Mutable lives in **ClickHouse** as **`ReplacingMergeTree(version, is_deleted)`** (replicated variant in clustered deployments).

| Physical detail | Behavior                                                                                                                                                                           |
| --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Table engine    | `ReplacingMergeTree(version, is_deleted)`                                                                                                                                          |
| Sort / identity | `(source_id, natural_key)`                                                                                                                                                         |
| Partitioning    | **Single partition** (`PARTITION BY tuple()`). ReplacingMergeTree only collapses versions **inside a partition**. Splitting one key across partitions would break last-value-wins. |
| Update          | Insert a **new row** with the same identity and a **higher** `version`. Nothing is overwritten in place.                                                                           |
| Delete          | Insert a row with `is_deleted = 1` at a higher `version` (tombstone).                                                                                                              |
| Merge           | Background merges keep the max-`version` row per key. Until then, multiple versions can exist on disk.                                                                             |
| Tiering         | **Not allowed.** Cold Parquet cannot collapse versions or honor tombstones. `hot_cold` is rejected.                                                                                |
| Storage class   | Hot MergeTree only (`fast`).                                                                                                                                                       |

## How reads collapse versions

A raw `SELECT` on the hot table is **not** last-value-wins while extra versions still sit unmerged. The **read view** collapses for you:

* Group by `(source_id, natural_key)`
* Take `argMax(column, version)` for each projected field
* Hide keys whose latest version is a tombstone (`HAVING argMax(is_deleted, version) = 0`)

Query through that view (QueryScript dataspace **name**, or catalog `read_view`). Do not scan the physical table and assume one row per key.

<Warning>
  **Read-your-write is not guaranteed** the instant after an insert. The view is correct once it aggregates versions, but this is not Postgres. That is why mutable has **no HTTP row CRUD**. An API whose GET after PUT is *usually* right is worse than no API.
</Warning>

## Identity and `natural_key`

Mutable **requires** a natural key at create. A keyed table with no identity is rejected.

* `source_id` — which producer (device id on the edge path).
* `natural_key` — the row’s business identity. You declare which source fields compose it (`natural_key: ["sku"]` or a composite, in order). The table stores a single `natural_key` column plus `source_id`; the catalog remembers the field list.

Edge mirrors typically hash the SQLite primary key into `natural_key` and set `source_id` to the device. Cloud producers should send a stable encoding of the same fields on every write and retry.

**`version` must be stable across redeliveries** — the producer’s logical version (`commit_seq`, `source_commit_seq`), **never** wall clock. A clock default would turn last-write-wins into last-arrival-wins and emit duplicate events.

You can **rekey** only if the dataspace currently has **no** key. Mutable always has a key, so identity is fixed at create.

## Meta columns

In addition to the ClickHouse meta set (`source_type`, `source_id`, `event_ts`, `ingest_ts`):

| Column        | Meaning                           |
| ------------- | --------------------------------- |
| `natural_key` | Row identity besides the producer |
| `version`     | Last-write-wins token (`Int64`)   |
| `is_deleted`  | `0` live, `1` tombstone           |

`event_ts` is present but **not** the sort or partition key. Mutable rows are not a time series.

## Schema and layouts

| Layout             | Allowed? | Notes                                                                                        |
| ------------------ | -------- | -------------------------------------------------------------------------------------------- |
| `hybrid` (default) | Yes      | Undeclared columns land in overflow and stay readable. Promotion to typed columns is opt-in. |
| `typed`            | Yes      | Strict. Surprise columns orphan the batch.                                                   |
| `jsonb`            | No       | No identity columns to reason about.                                                         |
| `map`              | No       | That layout is [point](/dataspaces/point).                                                   |

Default `hybrid` so a mirrored SQLite table can ingest before every column is approved. Sync-tagged dataspaces force **review** before promotion.

## Events and edge

Mutable **does** emit row-changed events (full row) so cloud-authoritative tables can downsync. The ClickHouse insert and the Postgres outbox row are **not one transaction**. A crash between them can miss an event or require the poll sweep. Treat delivery as at-least-once and **best-effort** relative to state.

Consumers must be idempotent on identity + `version`.

Edge:

* Policy strategies `rows`, `mutable`, `relational`, `row_batch` provision a **mutable** dataspace.
* **Upsync** always. **Downsync** only if the table’s authority is `cloud`.
* Device-authoritative mirrors still upsync; the cloud does not push those rows down.
* Tombstones replicate: a source `DELETE` is an `is_deleted` insert, so a stale replay stays a read-time no-op.

## Mutable vs state

|               | State                             | Mutable                        |
| ------------- | --------------------------------- | ------------------------------ |
| Store         | Postgres                          | ClickHouse ReplacingMergeTree  |
| Shape         | One (or few) documents per source | Many rows per source           |
| HTTP CRUD     | Yes, with `409` on stale version  | No                             |
| GET after PUT | Yes                               | Not guaranteed                 |
| Events        | Same transaction as the row       | Two systems; best-effort       |
| Cold storage  | No                                | No                             |
| Typical use   | Shadow, settings, e-stop          | Mirrors, catalogs, inventories |

If you are unsure and the table has a primary key and `UPDATE`/`DELETE` in SQLite, it is almost always **mutable**. If it is “the device’s current JSON blob”, it is **state**.

## Create

```bash theme={null}
curl -sS -X POST "$API/projects/$PROJECT_ID/dataspaces" \
  -H "Authorization: Bearer $TOKEN" \
  -H "ORG-ID: $ORG_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "work_orders",
    "modality": "mutable",
    "natural_key": ["order_id"],
    "fields": [
      {"path": "order_id", "type": "string"},
      {"path": "status", "type": "string"}
    ]
  }'
```

`$API` is the [platform base URL](/api-reference/introduction).

## Related

* [Dataspaces overview](/dataspaces/overview)
* [Schema and layouts](/dataspaces/schema)
* [Storage](/dataspaces/storage)
* [State](/dataspaces/state) — ACID current documents
* [Edge capture strategies](/edge/data-sync/capture-strategies)
* [API introduction](/api-reference/introduction)