> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ilyama.golain.io/llms.txt
> Use this file to discover all available pages before exploring further.

# State dataspaces

> Last-value-wins current documents in Postgres — versioned upserts, tombstones, HTTP row CRUD, and change events that commit with the row.

A **state** dataspace is a keyed table of **what is true now**: device shadow, settings, e-stop, enrollment, one live document per source (or per source plus a business key).

Writes are last-write-wins on a monotonic **`version`**. Deletes are **tombstones**, not physical removal. A stale write is a no-op in storage and a **`409`** on the HTTP API.

The API modality is **`state`**.

→ [All modalities](/dataspaces/overview)

## When to use it

**Use state when**

* You have **one current document** per device (or a small keyset), not thousands of child rows.
* Cloud-authored changes must **reach the device** (downsync).
* You need **GET after PUT**, `If-Match` / ETag, or a **change event that cannot be lost** if the process crashes.
* Safety-adjacent values (shadow, interlock, “armed”) must be recorded and propagated. The local device loop still owns hard-real-time trip; the platform owns durable record + fan-out.

**Use [mutable](/dataspaces/mutable) instead** when each device holds a **table** of current rows (work orders, SKUs, child records). Postgres will hold high cardinality; it is the wrong engine for it.

**Use [telemetry](/dataspaces/timeseries) instead** when the data is history you will not edit.

Do not put a compare-and-set job (`UPDATE … WHERE status = 'running'`) in a dataspace. That belongs in ordinary platform tables.

## Backing store

State lives in **Postgres** as an ordinary table in the project dataspace schema. It is **never** ClickHouse and **never** tiered to object storage. You cannot set `hot_cold` or `storage_class` on state (rejected).

| Physical detail | Behavior                                                                                               |
| --------------- | ------------------------------------------------------------------------------------------------------ |
| Engine          | Postgres                                                                                               |
| Identity        | `source_id` only, or `UNIQUE (source_id, natural_key)` when you declare a key                          |
| Upsert          | Insert on conflict, update only when the stored `version` is lower than the incoming `version`         |
| Stale write     | Zero rows updated — skipped, no event                                                                  |
| Delete          | Version-guarded **tombstone** (`is_deleted`). The read view hides tombstones. A newer write undeletes. |
| Transactions    | The row change and the outbox event commit **together**                                                |

That shared transaction is why state stays on Postgres. A ClickHouse-backed “state” would put the row in one system and the event in another. For values you will act on, that regression is not acceptable.

## Identity

`source_id` is the **producer** (device id on the edge path, user/connection on the API path). It is a UUID.

A business key — `machine_id`, `(line_id, station_id)`, SKU — belongs in **`natural_key`**, declared at create as an ordered list of field paths. Order is significant; reordering is a different identity.

* No `natural_key` → one live row per `source_id` (classic shadow).
* With `natural_key` → one live row per `(source_id, natural_key)`. HTTP `{row_key}` is the natural key, not the UUID.

You can **rekey** only a dataspace that currently has **no** key (`POST …/dataspaces/{id}/rekey`). Changing an existing key can merge rows that are distinct today; that is refused. Create a new dataspace instead.

## Meta columns

| Column        | Meaning                                                                                             |
| ------------- | --------------------------------------------------------------------------------------------------- |
| `source_type` | Producer class                                                                                      |
| `source_id`   | Producer UUID                                                                                       |
| `event_ts`    | Event time                                                                                          |
| `version`     | Monotonic last-write-wins token. Exposed as `ETag` on HTTP. Send it as `If-Match` on update/delete. |
| `updated_at`  | Last change time                                                                                    |
| `natural_key` | Present when declared                                                                               |

State **does not** expose `ingest_ts`. Overflow JSON **is** readable on a `hybrid` state dataspace (the only modality where the overflow blob is a query surface).

**`version` must be stable across retries.** Use the producer’s logical version (for example edge `commit_seq`). Do not default to wall-clock `now`: a redelivery would look newer and last-write-wins would become last-arrival-wins.

## Schema and layouts

Default layout is **`hybrid`**: declared columns are typed; undeclared keys land in overflow and stay readable. That default exists because Postgres schema does **not** auto-grow. Nothing infers new columns from overflow the way ClickHouse telemetry promotion does.

| Layout             | When to use it                                                                                                      |
| ------------------ | ------------------------------------------------------------------------------------------------------------------- |
| `hybrid` (default) | You do not have a full field list yet. Surprises are kept.                                                          |
| `typed`            | You pass `fields` at create and want undeclared columns rejected (HTTP `400` on row CRUD; firehose batches orphan). |

Schema changes go through the **control API** (proposals / review when sync is attached). The write plane will not `ALTER` state tables on its own. Column cap is **1000**. Over-cap or typed mismatch → **orphan** + error to the caller.

## HTTP row CRUD

State is the only modality with synchronous JSON row APIs:

```
GET    /projects/{pid}/dataspaces/{dsid}/rows
GET    /projects/{pid}/dataspaces/{dsid}/rows/{row_key}
PUT    /projects/{pid}/dataspaces/{dsid}/rows/{row_key}
PATCH  /projects/{pid}/dataspaces/{dsid}/rows/{row_key}
DELETE /projects/{pid}/dataspaces/{dsid}/rows/{row_key}

POST   /projects/{pid}/dataspaces/{dsid}/rows/batch
PATCH  /projects/{pid}/dataspaces/{dsid}/rows          # predicate update
DELETE /projects/{pid}/dataspaces/{dsid}/rows          # predicate tombstone
```

`{row_key}` is the declared natural key, or `source_id` when there is none.

| Behavior                     | Contract                                                                  |
| ---------------------------- | ------------------------------------------------------------------------- |
| Stale `version` / `If-Match` | **`409`**, never a silent `200`                                           |
| Undeclared column on `typed` | **`400`** (not an orphan row — wrong for a synchronous request)           |
| Delete                       | Tombstone; a later higher-version PUT restores the row                    |
| Reads                        | Served from the state **read view** with `can_read_data` in SQL           |
| Writes                       | Executed by the write plane. The HTTP API does not write Postgres itself. |

Predicate `PATCH`/`DELETE` use a structured filter (equality, `in`, ranges on **declared** columns). Raw SQL `WHERE` is not accepted. An empty filter is rejected; whole-dataspace updates must set `"where": {"all": true}`. Each changed row gets its own version bump and its own outbox event. `max_rows` caps fan-out.

## Change events and edge

On a successful commit the write plane emits a row-changed event with the **full row** (deletes carry identity + version). Delivery is **at-least-once**. Idempotency key for consumers: dataspace + source + version (and natural key when present).

Edge:

* **Upsync and downsync.** A cloud write to a sync-tagged state dataspace fans out to devices.
* Authority is per table: `device` (cloud is a replica) or `cloud` (cloud is source of truth).
* Offline upsync is safe: an older `version` is a no-op.
* Attaching any sync target forces schema **promotion policy = review**.

## Joining state into analytics queries

State lives in Postgres and your history lives in the analytics engine, but an analytics query can read a state dataspace directly, in the same query, **without copying anything**.

That closes the gap that used to force you to duplicate current values into a telemetry dataspace just so a report could see them. You can now write the rollups people actually ask for:

* Average reading per machine, **labelled by that machine's current mode** rather than by an id.
* Yesterday's throughput per line, restricted to the lines whose current state is `running`.
* A daily report that pairs history with the settings in force when it runs.

Three things to know:

|                          |                                                                                                                                                                                                  |
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Read-only**            | Analytics queries read state; they never write it. Every change still goes through row CRUD, the edge path, or the write plane, so versioning, tombstones, and change events cannot be bypassed. |
| **Always the live rows** | The query sees the same read view your API reads. Tombstoned rows are excluded, and rows quarantined for schema or timestamp problems are not visible at all.                                    |
| **No copy, no lag**      | There is no mirror to keep in sync. The values a rollup reads are the values in the table at the moment it runs.                                                                                 |

Reference the dataspace by **name**, as you would anywhere else. Physical table names are not a supported query surface.

<Note>
  On a self-hosted deployment this connection is configured once at install time. If a query that joins state against history returns rows from the history side and nothing from the state side, that connection is the first thing to check.
</Note>

## Create

One row per device (shadow-style):

```bash theme={null}
curl -sS -X POST "$API/projects/$PROJECT_ID/dataspaces" \
  -H "Authorization: Bearer $TOKEN" \
  -H "ORG-ID: $ORG_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "device_shadow",
    "modality": "state"
  }'
```

Business key, declared up front:

```bash theme={null}
curl -sS -X POST "$API/projects/$PROJECT_ID/dataspaces" \
  -H "Authorization: Bearer $TOKEN" \
  -H "ORG-ID: $ORG_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "machine_status",
    "modality": "state",
    "natural_key": ["machine_id"],
    "fields": [
      {"path": "machine_id", "type": "string"},
      {"path": "mode", "type": "string"}
    ]
  }'
```

## Related

* [Dataspaces overview](/dataspaces/overview)
* [Schema and layouts](/dataspaces/schema)
* [Mutable](/dataspaces/mutable) — many current rows per device, ClickHouse
* [Document](/dataspaces/document) · [Vector](/dataspaces/vector) — the two content modalities built on this same keyed-row shape
* [Edge data sync](/edge/data-sync/overview)
* [API introduction](/api-reference/introduction)