> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ilyama.golain.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Document dataspaces

> A collection of files — register, upload to a presigned URL, list and download what you have, and opt individual files into semantic search.

A **document** dataspace is a collection of **files**: manuals, SOPs, spec sheets, drawings, signed purchase orders, calibration certificates. It holds one row per file, recording where the bytes are and what the file describes. The bytes themselves live in object storage, not in the row.

The API modality is **`document`**.

Storing a file is the whole job by default. You can additionally opt an individual file into chunking and embedding with `vectorize: true`, which makes its text semantically searchable through a paired [vector dataspace](/dataspaces/vector). That choice is per file, not per dataspace, so one collection can hold manuals worth searching and delivery notes that are not.

→ [All modalities](/dataspaces/overview)

## When to use it

**Use document when**

* You have files that describe your equipment and you want them findable by what they describe: asset, model, manufacturer, revision.
* You want somewhere to keep the attachments a record accumulates — a signed PO against an order, a calibration certificate against an instrument — and to hand someone a link to the bytes later.
* You want some of those files searchable by meaning. Pair the dataspace with a [vector](/dataspaces/vector) dataspace and register the files worth searching with `vectorize: true`.
* The same file is attached to many assets. Documents are content-addressed, so uploading identical bytes twice stores one copy.

**Do not use document** for the extracted text itself. That is a [vector](/dataspaces/vector) dataspace.

**Do not use document** as general blob storage for firmware, images, or backups. It is a catalogue of files that describe things, with an optional parse pipeline attached — not a bucket.

## The dataspace is the collection; a document is one file in it

This is the distinction most likely to trip you up, because both are named with the word "document" and both have an id.

|               | What it is                                                                                 | How you name it                        | Its id                                                                                             |
| ------------- | ------------------------------------------------------------------------------------------ | -------------------------------------- | -------------------------------------------------------------------------------------------------- |
| **Dataspace** | The collection. One per project per body of files: `equipment_manuals`, `quality_records`. | You choose the name at create.         | `dataspace_id` — the UUID in every path below.                                                     |
| **Document**  | One file inside that collection.                                                           | You choose a key when you register it. | `document_id` — derived by the platform, stable across re-registrations, and what chunks point at. |

You do not create a dataspace per file. A project typically has a handful of document dataspaces and thousands of documents inside them.

### The document's key has three spellings

One value, three places, because it is a request field, a URL segment, and a stored column:

| Where you meet it                 | Spelling                                    |
| --------------------------------- | ------------------------------------------- |
| Register request body             | `natural_key`                               |
| Read paths                        | `{document_key}`                            |
| Read responses and the stored row | `natural_key`, held in the `doc_key` column |

Pick something stable and human-meaningful: `hpu-450-service-manual`, `sop/lockout-tagout`, `po/2026-00814`. Percent-encode it in paths if it contains slashes or spaces.

Re-registering the **same** key is a **new revision of that document**, not a second document. Do not key on the file's hash: that would make every revision a new document and leave the superseded one live and searchable beside the current one, which is the failure mode revisions exist to prevent.

## Vectorizing is opt-in

`vectorize` defaults to **`false`**. Register a file without it and the platform stores the bytes and writes the row, and does nothing else: no conversion, no chunking, no embedding. The document's `parse_state` is **`skipped`**.

That default is about cost, and about what a document dataspace actually holds. Parsing a file costs a format conversion plus one embedding call per chunk — hundreds of provider calls for a 500-page manual. Most files in a document dataspace are attachments nobody will ever ask a question of: a signed PO, a calibration certificate, a photo of a nameplate. Indexing those buys nothing, and a project that uses documents purely as an attachment store never needs a vector dataspace at all.

Set `vectorize: true` on the files worth searching, and only those:

```json theme={null}
{ "natural_key": "hpu-450-service-manual", "sha256": "…", "content_type": "application/pdf", "vectorize": true }
```

<Note>
  **`skipped` is a distinct state from `pending`, deliberately.** `skipped` means nobody asked for this file to be indexed. `pending` means somebody did and it has not happened yet. Collapsing them would make every stored attachment look exactly like a stalled pipeline, and you would have no way to tell which one you were looking at.
</Note>

<Warning>
  **`vectorize: true` needs a chunk corpus, and is rejected without one.** If the dataspace has no `chunk_dataspace_id`, registration fails with a message saying so rather than accepting a file that could never become searchable. Create the [vector dataspace](/dataspaces/vector) first and pair it at create — `chunk_dataspace_id` is fixed for the life of the document dataspace, so there is no repointing it afterwards.
</Warning>

## Backing store

Document dataspaces are **Postgres**, using the same keyed-row shape as [state](/dataspaces/state): one live row per document, last-write-wins on `version`, deletes as tombstones.

| Physical detail         | Behavior                                                                                                    |
| ----------------------- | ----------------------------------------------------------------------------------------------------------- |
| Engine                  | Postgres                                                                                                    |
| Identity                | The document key — your stable name for the file                                                            |
| Layout                  | **`hybrid` by default**, so your own fields are kept. `typed` is allowed if you want the stricter contract. |
| Bytes                   | Object storage, keyed by the file's SHA-256                                                                 |
| Storage class / tiering | **Not applicable.** `storage_class` and `hot_cold` are rejected — the bytes already live in object storage. |
| Row CRUD                | Read and write, same surface as state                                                                       |

## The schema

The platform declares the columns it reads itself. You do not pass `fields` for those — anything you send is replaced — and you cannot remove them.

| Group               | Column                                                      | What it holds                                                                                     |
| ------------------- | ----------------------------------------------------------- | ------------------------------------------------------------------------------------------------- |
| Identity            | `doc_key`                                                   | Your stable name for this file. The row key.                                                      |
|                     | `document_id`                                               | Derived from the dataspace and the key. Stable across re-registrations, and what chunks point at. |
| Where the bytes are | `object_key`, `content_sha256`, `content_type`, `byte_size` |                                                                                                   |
| What it is          | `title`, `language`                                         |                                                                                                   |
| Which version       | `revision`, `effective_from`                                | Which edition of the document these bytes are, and from when it applies.                          |
| What it describes   | `asset_id`, `equipment_model`, `manufacturer`               | Copied onto every chunk, so search can filter on them.                                            |
| Indexing            | `vectorize`                                                 | Whether this file was opted into chunking and embedding.                                          |
| Progress            | `upload_state`, `parse_state`, `parse_error`                | See below.                                                                                        |

Every column is nullable, because the row is written in stages — it exists before the bytes land.

### Your own fields

The built-in columns cover what the platform itself reads. What you need to record alongside a file is yours to decide, and you attach it with a **`metadata`** object on the register call:

```json theme={null}
{
  "natural_key": "ubx-m10-datasheet",
  "sha256": "…",
  "content_type": "application/pdf",
  "title": "MAX-M10S data sheet",
  "metadata": {
    "series": "MAX-M10",
    "silicon_revision": "B2",
    "distributor_part_number": "DK-1893-1121-1-ND"
  }
}
```

Those keys come back under `metadata` on every read. There is no special metadata column and no separate metadata store: your keys go into the row like any other field, and the dataspace's ordinary overflow storage holds the ones that are not built-in columns. That is exactly why `document` defaults to `hybrid` — on a `typed` layout an undeclared column orphans the whole row, so your own field would not merely be untyped, it would take the document with it.

`metadata` is an envelope rather than loose top-level keys because the register call also carries arguments that are not row columns (`sha256`, `vectorize`, `multipart`). Nesting your fields means a field of yours named `multipart` can never be mistaken for a platform flag.

Keys must be made of letters, digits, underscores, and dots.

<Warning>
  **A `metadata` key that collides with a built-in field is rejected.** `metadata.title` and the real `title` would be two values for one thing with no rule about which wins, so registration fails with a message naming the key. Set the built-in directly instead — `title`, `revision`, `asset_id`, `equipment_model`, `manufacturer`, `language`, and the rest of the table above are all top-level fields on the register call. Row meta columns (`source_id`, `source_type`, `event_ts`, `ingest_ts`, `overflow`) and `doc_key` are refused for the same reason.
</Warning>

On a `typed` document dataspace there is no overflow, so `metadata` comes back empty and undeclared keys are refused. Choose `typed` only when you have declared every field you intend to send.

## The two states

Upload and parse are tracked separately because they are two different things that fail for different reasons.

| Column         | Values                             | Advanced by                                                                              |
| -------------- | ---------------------------------- | ---------------------------------------------------------------------------------------- |
| `upload_state` | `awaiting_upload` → `uploaded`     | The object store, when your bytes actually land. Never by your client claiming they did. |
| `parse_state`  | `skipped`, or `pending` → `parsed` | The platform, after it has read and chunked the file.                                    |

| `parse_state` | Meaning                                                                                       |
| ------------- | --------------------------------------------------------------------------------------------- |
| `skipped`     | You did not ask for this file to be indexed. The normal, final state for a stored attachment. |
| `pending`     | You asked, and it has not happened yet.                                                       |
| `parsed`      | Chunks have been written to the paired vector dataspace.                                      |
| `failed`      | Reserved for a parse that ran and produced nothing, with the reason in `parse_error`.         |

A document sitting at `awaiting_upload` means the bytes never arrived — usually an upload that was abandoned, or a presigned URL that expired before the client got round to using it.

<Note>
  **A parse that does not complete leaves the document at `pending`, not `failed`.** So `pending` is the state to watch: a document that has been `uploaded` for a long time and is still `pending` is the symptom of a parse that did not run or did not finish, and the usual cause is a missing or wrong `content_type` at registration. The row is safe to re-drive once that is fixed. List with `?parse_state=pending` to find them.
</Note>

## Registering a document

```
POST /projects/{project_id}/dataspaces/{dataspace_id}/documents
```

You send the document's identity, its SHA-256, and whatever you know about it. You get back an upload target.

| Field                                         | Required             | Notes                                                                                            |
| --------------------------------------------- | -------------------- | ------------------------------------------------------------------------------------------------ |
| `natural_key`                                 | Yes                  | The document's key.                                                                              |
| `sha256`                                      | Yes                  | Hex or base64, and it must decode to 32 bytes. Both the storage key and the integrity guarantee. |
| `content_type`                                | Strongly recommended | See below — parsing and download both depend on it.                                              |
| `vectorize`                                   | No, default `false`  | Opt this file into chunking and embedding. Requires a paired vector dataspace.                   |
| `metadata`                                    | No                   | Your own fields, as an object.                                                                   |
| `title`, `language`, `byte_size`              | No                   | What it is.                                                                                      |
| `revision`, `effective_from`                  | No                   | Which edition, and from when.                                                                    |
| `asset_id`, `equipment_model`, `manufacturer` | No                   | What it describes. Copied onto every chunk for filtering.                                        |
| `multipart`                                   | No                   | Opt in for large files. Returns part URLs instead of one PUT, and costs extra round trips.       |

The row is written **before** your bytes exist. A client that vanishes mid-upload leaves a visible row at `awaiting_upload` rather than nothing at all.

<Warning>
  **Send `content_type`.** Files are stored under their hash, so the stored object has no filename and no extension. The content type you declare is how the platform knows what kind of file it is parsing, and it is what gives a download its extension. Register a PDF without one and the upload succeeds, the parse runs, and it produces no text — a failure that surfaces late and points nowhere.
</Warning>

### Content addressing and deduplication

The object key is the file's SHA-256, so **identical bytes are stored once**. The same service manual attached to forty assets is one object and forty rows.

When the bytes are already present, the response comes back with `already_uploaded: true` and **no presigned URL**. Skip the upload entirely.

### Send `put_headers` verbatim

<Warning>
  The response's `put_headers` are **part of the upload's signature**. Send every one of them, exactly as returned, with nothing added, dropped, or changed. Alter or omit any and the object store rejects the upload with a signature mismatch.
</Warning>

That is deliberate rather than fussy. The digest and the content type are signed into the URL, so the store itself refuses bytes that do not hash to what you declared and refuses to store the file under a content type you did not declare. It is the store enforcing your claim, not the platform checking afterwards and hoping.

Presigned upload URLs are short-lived — the response tells you how long, in `expires_in_seconds`. If yours expires, register the document again to mint a new one; re-registering the same key is not a duplicate.

## Reading a document dataspace

Three endpoints cover the collection: list it, read one document, and get a link to the bytes.

```
GET /projects/{project_id}/dataspaces/{dataspace_id}/documents
GET /projects/{project_id}/dataspaces/{dataspace_id}/documents/{document_key}
GET /projects/{project_id}/dataspaces/{dataspace_id}/documents/{document_key}/download
```

All three need `can_read_data` on the dataspace. Without it — or if the dataspace is not `document` — you get a **`404`**, not a `403`, so a caller who may not read the dataspace cannot learn that it exists.

### List the documents

| Query parameter | Effect                                                                                                                                           |
| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| `asset_id`      | Only documents describing this asset. The grouping filter — see [one asset, many documents](#one-asset-many-documents).                          |
| `parse_state`   | One of `skipped`, `pending`, `parsed`, `failed`. This is what makes "show me everything still waiting" answerable without reading every row.     |
| `q`             | Substring of your document key or its title. A picker, not a search — the semantic surface is the paired [vector](/dataspaces/vector) dataspace. |
| `limit`         | Page size. Defaults to `50`. Anything above `500` falls back to the default rather than being clamped, so ask for a page you actually want.      |
| `offset`        | Where to start. Defaults to `0`.                                                                                                                 |

Newest change first: results are ordered by `updated_at` descending, then by key.

```bash theme={null}
curl -sS "$API/projects/$PROJECT_ID/dataspaces/$DOC_DS_ID/documents?asset_id=$ASSET_ID&limit=20" \
  -H "Authorization: Bearer $TOKEN" \
  -H "ORG-ID: $ORG_ID"
```

```json theme={null}
{
  "ok": 1,
  "data": {
    "items": [
      {
        "document_id": "3f8b1c2a-9d4e-5f60-8a71-2b3c4d5e6f70",
        "natural_key": "hpu-450-service-manual",
        "title": "HPU-450 Service Manual",
        "content_type": "application/pdf",
        "content_sha256": "9f2c…",
        "byte_size": 4812233,
        "revision": "C",
        "effective_from": "2026-01-15T00:00:00Z",
        "asset_id": "b21c8f30-77a4-4a11-9c5e-0f2d6a8b4c19",
        "equipment_model": "HPU-450",
        "manufacturer": "Acme",
        "object_key": "projects/…/documents/9f2c…",
        "vectorize": true,
        "upload_state": "uploaded",
        "parse_state": "parsed",
        "metadata": { "binder": "hydraulics", "reviewed_by": "j.okafor" },
        "updated_at": "2026-08-28T09:14:22Z"
      }
    ],
    "total": 4
  }
}
```

`total` counts everything matching your filters, not just the page, so you can drive a pager from one call.

### Read one document

```bash theme={null}
curl -sS "$API/projects/$PROJECT_ID/dataspaces/$DOC_DS_ID/documents/hpu-450-service-manual" \
  -H "Authorization: Bearer $TOKEN" \
  -H "ORG-ID: $ORG_ID"
```

The response is `{"item": …}` with the same fields as a list row. This is what you poll while a document moves through its two states.

### Download the bytes

```bash theme={null}
curl -sS "$API/projects/$PROJECT_ID/dataspaces/$DOC_DS_ID/documents/hpu-450-service-manual/download" \
  -H "Authorization: Bearer $TOKEN" \
  -H "ORG-ID: $ORG_ID"
```

```json theme={null}
{
  "ok": 1,
  "data": {
    "url": "https://…?X-Amz-Algorithm=…&X-Amz-Signature=…",
    "expires_in_seconds": 3600,
    "content_type": "application/pdf",
    "byte_size": 4812233,
    "filename": "hpu-450-service-manual.pdf"
  }
}
```

You get a **short-lived presigned link, not a proxied stream**. The object store serves the bytes directly, so a 400 MB manual never travels through the API, and the link opens in a browser tab with no `Authorization` header attached. Treat it the way you would treat the bytes: it is a bearer capability to read one object, which is why it expires.

A document whose bytes have not landed has nothing to link to, so this returns **`404`** while `upload_state` is still `awaiting_upload`.

<Note>
  **Use `filename`.** Objects are stored under their content hash, with no extension. Hand the browser the `url` alone and the download lands as a 64-character hex blob that nothing will open. `filename` is built from your document key plus the extension implied by `content_type` — another reason to send one.
</Note>

<Note>
  Document dataspaces also expose the generic row surface (`…/dataspaces/{id}/rows`) that [state](/dataspaces/state) uses, where `{row_key}` is the document key. Prefer the `documents` endpoints: they return the derived `document_id` and your `metadata`, and they leave out row meta columns that mean nothing to a caller holding a document key.
</Note>

## One asset, many documents

This is the grouping the modality exists for, and it is worth being concrete.

A u-blox positioning module has a **datasheet**, a **reference design**, a set of **application notes**, and a **command reference**. That is four separate documents. It is one asset. It is one dataspace.

|                                       |                                                                                                                                    |
| ------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- |
| One `equipment_manuals` **dataspace** | The collection. You create it once.                                                                                                |
| Four **documents** in it              | `ubx-m10-datasheet`, `ubx-m10-reference-design`, `ubx-m10-app-notes`, `ubx-m10-command-reference`. Each has its own `document_id`. |
| One `asset_id` on all four            | What ties them together.                                                                                                           |
| `metadata` per document               | What distinguishes them: series, silicon revision, distributor part number.                                                        |

Register each one with the same `asset_id`:

```bash theme={null}
curl -sS -X POST "$API/projects/$PROJECT_ID/dataspaces/$DOC_DS_ID/documents" \
  -H "Authorization: Bearer $TOKEN" \
  -H "ORG-ID: $ORG_ID" \
  -H "Content-Type: application/json" \
  -d "{
    \"natural_key\": \"ubx-m10-command-reference\",
    \"sha256\": \"$SHA_HEX\",
    \"content_type\": \"application/pdf\",
    \"title\": \"MAX-M10S interface description\",
    \"asset_id\": \"$ASSET_ID\",
    \"manufacturer\": \"u-blox\",
    \"equipment_model\": \"MAX-M10S\",
    \"vectorize\": true,
    \"metadata\": { \"series\": \"MAX-M10\", \"silicon_revision\": \"B2\", \"doc_class\": \"interface\" }
  }"
```

Then pull the whole set back with one call:

```bash theme={null}
curl -sS "$API/projects/$PROJECT_ID/dataspaces/$DOC_DS_ID/documents?asset_id=$ASSET_ID" \
  -H "Authorization: Bearer $TOKEN" \
  -H "ORG-ID: $ORG_ID"
```

Put what the documents have in common in the built-in columns, and what distinguishes them in `metadata`. `asset_id`, `equipment_model`, `manufacturer`, and `revision` are built in precisely because they are the filters that also have to work on [search](/dataspaces/vector#filters): they are copied onto every chunk, and a `metadata` key is not.

## End to end: from an API key to a searchable PDF

This is the whole flow for a file you **do** want to search. It assumes `$API`, `$TOKEN`, `$ORG_ID`, and `$PROJECT_ID` are set — see the [API introduction](/api-reference/introduction).

<Steps>
  <Step title="Connect an embedding provider">
    Register an integration account for your organization. Its provider must declare the `embed` capability.

    ```bash theme={null}
    curl -sS -X POST "$API/organizations/integrations/accounts" \
      -H "Authorization: Bearer $TOKEN" \
      -H "ORG-ID: $ORG_ID" \
      -H "Content-Type: application/json" \
      -d '{
        "provider_kind": "openai",
        "provider_version": 1,
        "name": "Embeddings (OpenAI)",
        "capabilities": ["embed"],
        "auth_config": { "secrets": { "api_key": "REPLACE_WITH_YOUR_API_KEY" } }
      }'
    ```

    ```json theme={null}
    {
      "ok": 1,
      "data": {
        "account_id": "6a1f0a4c-8b21-4a55-9a0e-2b7c1f9d3e41",
        "status": "active",
        "auth_ref": { "scope_type": "account", "scope_id": "6a1f0a4c-8b21-4a55-9a0e-2b7c1f9d3e41", "names": ["api_key"] }
      }
    }
    ```

    Keep `account_id`. The key is sealed in the platform's secret store and is never readable again.
  </Step>

  <Step title="Create the vector dataspace">
    The corpus your chunks land in. The model and dimension are permanent — see [vector dataspaces](/dataspaces/vector#two-choices-you-make-once).

    ```bash theme={null}
    curl -sS -X POST "$API/projects/$PROJECT_ID/dataspaces" \
      -H "Authorization: Bearer $TOKEN" \
      -H "ORG-ID: $ORG_ID" \
      -H "Content-Type: application/json" \
      -d '{
        "name": "manual_chunks",
        "modality": "vector",
        "embedding_model": "text-embedding-3-small",
        "embedding_dims": 1536,
        "embedding_account_id": "6a1f0a4c-8b21-4a55-9a0e-2b7c1f9d3e41"
      }'
    ```

    ```json theme={null}
    {
      "ok": 1,
      "data": {
        "item": {
          "id": "c4d2e9a1-5f37-4b8c-9e10-7a6b5c4d3e2f",
          "name": "manual_chunks",
          "modality": "vector",
          "engine": "pgvector",
          "layout": "typed",
          "embedding_model": "text-embedding-3-small",
          "embedding_dims": 1536,
          "schema_version": 1
        }
      }
    }
    ```
  </Step>

  <Step title="Create the document dataspace, paired to it">
    `chunk_dataspace_id` is the vector dataspace's `id` from the previous step. It must be a vector dataspace in the same project, and it is fixed for the life of this dataspace.

    ```bash theme={null}
    curl -sS -X POST "$API/projects/$PROJECT_ID/dataspaces" \
      -H "Authorization: Bearer $TOKEN" \
      -H "ORG-ID: $ORG_ID" \
      -H "Content-Type: application/json" \
      -d '{
        "name": "equipment_manuals",
        "modality": "document",
        "chunk_dataspace_id": "c4d2e9a1-5f37-4b8c-9e10-7a6b5c4d3e2f"
      }'
    ```

    ```json theme={null}
    {
      "ok": 1,
      "data": {
        "item": {
          "id": "9e7f6a5b-4c3d-42e1-b0a9-8f7e6d5c4b3a",
          "name": "equipment_manuals",
          "modality": "document",
          "engine": "postgres",
          "layout": "hybrid",
          "chunk_dataspace_id": "c4d2e9a1-5f37-4b8c-9e10-7a6b5c4d3e2f",
          "schema_version": 1
        }
      }
    }
    ```

    Skip `chunk_dataspace_id` and the dataspace still works as a file store — but `vectorize: true` is then rejected on every registration, and nothing in it can ever become searchable.
  </Step>

  <Step title="Register the PDF, asking for it to be indexed">
    Hash the file first. The API takes hex or base64; the upload header wants base64, and the response gives you both in the right places.

    `vectorize: true` is the part that matters here. Leave it out and you get a stored PDF and nothing to search.

    ```bash theme={null}
    SHA_HEX=$(shasum -a 256 hpu-450-manual.pdf | cut -d' ' -f1)

    curl -sS -X POST "$API/projects/$PROJECT_ID/dataspaces/9e7f6a5b-4c3d-42e1-b0a9-8f7e6d5c4b3a/documents" \
      -H "Authorization: Bearer $TOKEN" \
      -H "ORG-ID: $ORG_ID" \
      -H "Content-Type: application/json" \
      -d "{
        \"natural_key\": \"hpu-450-service-manual\",
        \"sha256\": \"$SHA_HEX\",
        \"content_type\": \"application/pdf\",
        \"vectorize\": true,
        \"title\": \"HPU-450 Service Manual\",
        \"revision\": \"C\",
        \"effective_from\": \"2026-01-15T00:00:00Z\",
        \"equipment_model\": \"HPU-450\",
        \"manufacturer\": \"Acme\",
        \"asset_id\": \"b21c8f30-77a4-4a11-9c5e-0f2d6a8b4c19\",
        \"metadata\": { \"binder\": \"hydraulics\", \"reviewed_by\": \"j.okafor\" }
      }"
    ```

    ```json theme={null}
    {
      "ok": 1,
      "data": {
        "item": {
          "document_id": "3f8b1c2a-9d4e-5f60-8a71-2b3c4d5e6f70",
          "object_key": "projects/…/documents/9f2c…",
          "already_uploaded": false,
          "presigned_put_url": "https://…?X-Amz-Algorithm=…&X-Amz-Signature=…",
          "put_headers": {
            "Content-Type": "application/pdf",
            "x-amz-checksum-sha256": "nyxg…="
          },
          "expires_in_seconds": 900
        }
      }
    }
    ```

    If `already_uploaded` is `true`, there is no `presigned_put_url` and no upload to do — jump to the next step.
  </Step>

  <Step title="Upload the bytes">
    Send every header from `put_headers`, exactly as returned.

    ```bash theme={null}
    curl -sS -X PUT "$PRESIGNED_PUT_URL" \
      -H "Content-Type: application/pdf" \
      -H "x-amz-checksum-sha256: nyxg…=" \
      --data-binary @hpu-450-manual.pdf
    ```

    A `403` with a signature message here almost always means a header was changed, dropped, or added by your HTTP client. Many clients set their own `Content-Type` on a PUT unless you stop them.

    You do not tell the platform the upload finished. The object store does.
  </Step>

  <Step title="Wait for it to become searchable">
    Poll the document. It moves `awaiting_upload` → `uploaded`, then `pending` → `parsed`.

    ```bash theme={null}
    curl -sS "$API/projects/$PROJECT_ID/dataspaces/9e7f6a5b-4c3d-42e1-b0a9-8f7e6d5c4b3a/documents/hpu-450-service-manual" \
      -H "Authorization: Bearer $TOKEN" \
      -H "ORG-ID: $ORG_ID"
    ```

    ```json theme={null}
    {
      "ok": 1,
      "data": {
        "item": {
          "document_id": "3f8b1c2a-9d4e-5f60-8a71-2b3c4d5e6f70",
          "natural_key": "hpu-450-service-manual",
          "title": "HPU-450 Service Manual",
          "revision": "C",
          "content_type": "application/pdf",
          "vectorize": true,
          "upload_state": "uploaded",
          "parse_state": "parsed",
          "metadata": { "binder": "hydraulics", "reviewed_by": "j.okafor" },
          "updated_at": "2026-08-28T09:14:22Z"
        }
      }
    }
    ```

    `parse_state: parsed` means the chunks have been written. They are searchable by keyword immediately, and semantically once the background job has embedded them — usually a minute or two for one document. See [how chunks get their embeddings](/dataspaces/vector#how-chunks-get-their-embeddings).

    If it is still `pending` long after `upload_state` reached `uploaded`, the parse has not completed; check `content_type` first. If it is `skipped`, you did not pass `vectorize: true`.
  </Step>

  <Step title="Search it">
    Search the **vector** dataspace, not the document one.

    ```bash theme={null}
    curl -sS -X POST "$API/projects/$PROJECT_ID/dataspaces/c4d2e9a1-5f37-4b8c-9e10-7a6b5c4d3e2f/search" \
      -H "Authorization: Bearer $TOKEN" \
      -H "ORG-ID: $ORG_ID" \
      -H "Content-Type: application/json" \
      -d '{
        "query": "hydraulic pump will not build pressure",
        "mode": "hybrid",
        "top_k": 3,
        "equipment_model": "HPU-450",
        "revision": "C"
      }'
    ```

    ```json theme={null}
    {
      "ok": 1,
      "data": {
        "mode": "hybrid",
        "items": [
          {
            "document_id": "3f8b1c2a-9d4e-5f60-8a71-2b3c4d5e6f70",
            "chunk_ordinal": 42,
            "chunk_text": "If the pump does not build pressure, verify the relief valve setting before replacing the cartridge…",
            "context_prefix": "Acme — HPU-450 — HPU-450 Service Manual — revision C — 4.3 Pressure faults",
            "revision": "C",
            "effective_from": "2026-01-15T00:00:00Z",
            "asset_id": "b21c8f30-77a4-4a11-9c5e-0f2d6a8b4c19",
            "score": 0.0325
          }
        ]
      }
    }
    ```

    A result carries the `document_id` it came from, so you can close the loop: look the document up, then call [download](#download-the-bytes) to hand the reader the PDF itself.

    Check `mode` on the response. If it comes back `lexical` when you asked for `hybrid`, a `degraded` field says why — see [when search quietly falls back](/dataspaces/vector#when-search-quietly-falls-back).
  </Step>
</Steps>

## Storing a file you will never search

The common case is shorter, needs no vector dataspace, and costs nothing to index. Register the file, upload it, hand out a link.

<Steps>
  <Step title="Register it without vectorize">
    ```bash theme={null}
    SHA_HEX=$(shasum -a 256 po-2026-00814-signed.pdf | cut -d' ' -f1)

    curl -sS -X POST "$API/projects/$PROJECT_ID/dataspaces/$DOC_DS_ID/documents" \
      -H "Authorization: Bearer $TOKEN" \
      -H "ORG-ID: $ORG_ID" \
      -H "Content-Type: application/json" \
      -d "{
        \"natural_key\": \"po/2026-00814\",
        \"sha256\": \"$SHA_HEX\",
        \"content_type\": \"application/pdf\",
        \"title\": \"PO 2026-00814 (signed)\",
        \"asset_id\": \"b21c8f30-77a4-4a11-9c5e-0f2d6a8b4c19\",
        \"metadata\": { \"supplier\": \"Acme\", \"po_value_inr\": 412000 }
      }"
    ```

    Upload the bytes to `presigned_put_url` exactly as above. When the object store reports them, `upload_state` becomes `uploaded` and `parse_state` stays **`skipped`** — final, and correct. Nothing was parsed, chunked, or embedded, and no provider call was made.
  </Step>

  <Step title="Hand someone the file">
    ```bash theme={null}
    curl -sS "$API/projects/$PROJECT_ID/dataspaces/$DOC_DS_ID/documents/po%2F2026-00814/download" \
      -H "Authorization: Bearer $TOKEN" \
      -H "ORG-ID: $ORG_ID"
    ```

    ```json theme={null}
    {
      "ok": 1,
      "data": {
        "url": "https://…?X-Amz-Algorithm=…&X-Amz-Signature=…",
        "expires_in_seconds": 3600,
        "content_type": "application/pdf",
        "byte_size": 184220,
        "filename": "po2026-00814.pdf"
      }
    }
    ```

    Put `url` behind a button in your own UI and save it as `filename`. The key contains a slash, so it is percent-encoded as `po%2F2026-00814` in the path — and dropped from the suggested filename, which has no directories to make.
  </Step>
</Steps>

To find every attachment on an asset later, list with `?asset_id=`. To audit what you have never indexed, list with `?parse_state=skipped`.

## Loading a revision

To publish a new edition of a document you already have, register the **same** key with the new file's hash, a new `revision`, and a new `effective_from`. The document row updates in place, the new bytes are uploaded, and if the document is registered with `vectorize: true` its chunks are overwritten rather than added to.

Then pass `revision` on your searches so a superseded procedure cannot surface as a current one. `effective_at` alone will not do it — it hides revisions that are not yet in force, not ones that have been replaced. See [filters](/dataspaces/vector#filters).

<Warning>
  **Chunks are overwritten by position, and surplus ones are not swept.** If the new edition produces fewer chunks than the edition it replaces, the extra chunks from the previous parse stay in the corpus, still carrying the old `revision`. A search filtered to the new `revision` will not return them. A search that is not filtered by revision can.
</Warning>

If that matters — a procedure was removed rather than reworded, say — clear the document's chunks before re-registering it. Tombstone them by `document_id` on the **vector** dataspace, then register the new edition:

```bash theme={null}
curl -sS -X DELETE "$API/projects/$PROJECT_ID/dataspaces/$VECTOR_DS_ID/rows" \
  -H "Authorization: Bearer $TOKEN" \
  -H "ORG-ID: $ORG_ID" \
  -H "Content-Type: application/json" \
  -d '{"where": {"document_id": "3f8b1c2a-9d4e-5f60-8a71-2b3c4d5e6f70"}}'
```

## Create

```bash theme={null}
curl -sS -X POST "$API/projects/$PROJECT_ID/dataspaces" \
  -H "Authorization: Bearer $TOKEN" \
  -H "ORG-ID: $ORG_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "equipment_manuals",
    "modality": "document",
    "chunk_dataspace_id": "c4d2e9a1-5f37-4b8c-9e10-7a6b5c4d3e2f"
  }'
```

Drop `chunk_dataspace_id` for a pure file store. Add `"layout": "typed"` if you want undeclared `metadata` keys refused instead of absorbed.

`$API` is the [platform base URL](/api-reference/introduction) (production: `https://api.ilyama.golain.io/core/api/v1`).

## Related

* [Vector dataspaces](/dataspaces/vector) — the chunks, and how search works
* [Dataspaces overview](/dataspaces/overview)
* [State](/dataspaces/state) — the keyed-row shape document dataspaces share
* [Schema and layouts](/dataspaces/schema) — what `hybrid` and `typed` mean
* [API introduction](/api-reference/introduction)