> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ilyama.golain.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Alarm notifications and work orders

> Send alarm events to email, Slack, or Microsoft Teams, re-notify until someone acknowledges, and raise work orders automatically.

An alarm definition decides **when** an alarm fires. A **notification subscription** decides **who hears about it, and where**. You set the two up separately: the definition lives under **Alarms**, the subscription lives under **Integrations**.

## How alarm notifications work

```mermaid theme={null}
flowchart LR
  Def[Alarm definition] -->|condition met| Inst[Alarm instance]
  Inst -->|fires, clears, acknowledged| Evt[Alarm event]
  Evt --> Sub{Notification subscriptions}
  Sub -->|matches event + alarm| Ch[Email / Slack / Teams]
  Evt -->|auto_create_wo on| WO[Work order]
```

1. An alarm instance changes state — it fires, it resolves, or an operator acknowledges it.
2. Golain records an **alarm event** for that change.
3. Every enabled notification subscription in scope checks the event:
   * Is the event key in its `subscribed_events`?
   * If it has `alarm_def_ids`, is the alarm's definition in that list? An empty list means **every alarm**.
4. Each matching subscription delivers a card on its channel.

An alarm with no subscriptions still fires and still shows on the **Board**. Nobody is told.

## Alarm events

You can subscribe to three alarm events.

| Event key | Display name | Console label | When it fires |
| - | - | - | - |
| `alarm.alarm_instance.activated.v1` | Alarm Fired | **Alarm opens** | An instance becomes active (firing). Also sent again on every [re-notify](#re-notify-and-escalation). |
| `alarm.alarm_instance.cleared.v1` | Alarm Resolved | **Alarm resolves** | An instance returns to normal. |
| `alarm.alarm_instance.acknowledged.v1` | Alarm Acknowledged | **Someone acknowledges** | An operator acknowledges an instance. |

### Card fields

Cards show names, not IDs. A field is left out when Golain could not resolve it.

| Field | Alarm Fired | Alarm Resolved | Alarm Acknowledged | Shows |
| - | :-: | :-: | :-: | - |
| **Alarm** | Yes | Yes | Yes | The definition's name |
| **Where** | Yes | Yes | Yes | The asset's place in your hierarchy, for example `Zone_3 › Temperatures` |
| **Asset** | Yes | Yes | Yes | The definition's asset |
| **Signal** | Yes | Yes | Yes | The instance's labels — what this firing is about |
| **Value** | Yes | — | — | The value that triggered the alarm |
| **Priority** | Yes | — | — | The alarm's priority |

To see the full list of notifiable events, with sample payloads and template variables, call:

```bash theme={null}
curl -sS \
  -H "Authorization: Bearer $TOKEN" \
  -H "ORG-ID: $ORG_ID" \
  "$API/notification-event-registry"
```

## Create a subscription

**Console**

**1. Open the definition**

Go to **Alarms** → **Definitions** and open the alarm. Save it first if it is new.

**2. Open Notifications**

The **Notifications** card lists the subscriptions that already listen to this alarm. Click **Open Notifications**. The **Integrations** page opens with a new subscription form, already targeted at this alarm.

**3. Fill in the form**

| Field | What to set |
| - | - |
| **Send to** | A connected channel — Slack, email, Teams, Telegram, or SMS — and where on it to deliver |
| **When** | One or more of **Alarm opens**, **Alarm resolves**, **Someone acknowledges** |
| **Which alarms** | The definitions this subscription covers. Leave empty for every alarm in the project. |

**4. Save**

Click **Create subscription**.

> [!NOTE]
>
> Per-alarm targeting only applies to project-scoped subscriptions. Organization- and fleet-scoped subscriptions receive every alarm.

**API**

Create a project-scoped subscription:

```
POST /core/api/v1/projects/{project_id}/notification-subscriptions
```

**Permission:** `can_manage_integrations` on the project.

```bash theme={null}
curl -sS -X POST \
  -H "Authorization: Bearer $TOKEN" \
  -H "ORG-ID: $ORG_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "subscribed_events": [
      "alarm.alarm_instance.activated.v1",
      "alarm.alarm_instance.cleared.v1"
    ],
    "channel_ref": {
      "kind": "email",
      "to": ["oven-team@example.com"],
      "subject": "Oven alarm"
    },
    "alarm_def_ids": ["'"$ALARM_DEF_ID"'"]
  }' \
  "$API/projects/$PROJECT_ID/notification-subscriptions"
```

A successful create returns `201` with `{"ok": 1, "data": { ... }}`, where `data` is the subscription.

**Request body:**

| Field | Type | Required | Description |
| - | - | - | - |
| `subscribed_events` | string\[] | Yes | Event keys from the [alarm events](#alarm-events) table |
| `channel_ref` | object | Yes | Where to deliver — see [Channel shapes](#channel-shapes) |
| `account_id` | uuid | Slack and Teams | The connected integration account. Not used for email. |
| `alarm_def_ids` | uuid\[] | No | Limit to these alarm definitions. Empty means every alarm. |
| `enabled` | boolean | No | Default `true` |
| `template_id` | uuid | No | Pin a notification template. Empty picks one by scope. |

The same routes exist at organization scope (`/organizations/notification-subscriptions`) and fleet scope (`/projects/{project_id}/fleets/{fleet_id}/notification-subscriptions`).

### Channel shapes

The shape of `channel_ref` depends on the channel.

```json Email theme={null}
{
  "kind": "email",
  "to": ["oven-team@example.com", "shift-lead@example.com"],
  "subject": "Oven alarm"
}
```

```json Slack (webhook) theme={null}
{
  "kind": "webhook",
  "webhook_url": "https://hooks.slack.com/services/…"
}
```

```json Slack (bot) theme={null}
{
  "kind": "bot",
  "channel": "#bakery-ops"
}
```

```json Teams (bot) theme={null}
{
  "kind": "bot",
  "team_id": "…",
  "channel_id": "…"
}
```

```json Teams (webhook) theme={null}
{
  "webhook_url": "https://…"
}
```

| Channel | Notes |
| - | - |
| Email | `to` is a fixed list of addresses. `subject` is optional. No `account_id`. |
| Slack | Send `account_id`. `kind` is `webhook` (or omitted) with an optional `webhook_url`, or `bot` with `channel`. |
| Microsoft Teams | Send `account_id`. `kind` is `bot` with `team_id` and `channel_id`, or omit `kind` and send `webhook_url`. |

> [!NOTE]
>
> `recipient_roles` works with email only, and only for events that declare roles. Alarm events declare none, so give an alarm email subscription a fixed `to` list.

### Change or delete a subscription

| Method | Path | Body |
| - | - | - |
| `GET` | `/projects/{project_id}/notification-subscriptions` | — |
| `PATCH` | `/projects/{project_id}/notification-subscriptions/{id}` | Any create field. An absent field is left alone. `"alarm_def_ids": []` resets targeting to every alarm. |
| `DELETE` | `/projects/{project_id}/notification-subscriptions/{id}` | — |

### Test a subscription

Send a synthetic test notification through the channel:

```bash theme={null}
curl -sS -X POST \
  -H "Authorization: Bearer $TOKEN" \
  -H "ORG-ID: $ORG_ID" \
  "$API/projects/$PROJECT_ID/notification-subscriptions/$SUBSCRIPTION_ID/test"
```

The call waits up to 10 seconds for the delivery result.

| Status | Meaning |
| - | - |
| `200` | Delivered |
| `400` | The subscription is disabled or suspended, or it is an email subscription with no fixed `to` |
| `404` | Subscription not found |
| `502` | The channel rejected the test |

The test checks the channel only. It does not simulate an alarm.

## Card actions

| Action | Events | Channels | What it does |
| - | - | - | - |
| **View Alarm** | All three | All | Opens the alarm instance in the Console |
| **Acknowledge** | Alarm Fired | Microsoft Teams | Acknowledges the instance as the user who clicks it |

> [!NOTE]
>
> The Teams **Acknowledge** button is only sent when the Teams account can sign card actions. Otherwise the card arrives with **View Alarm** only — the alert itself is never dropped.

## Re-notify and escalation

Set these on the alarm definition. In the Console, **Re-notify every** is in the definition's timing settings.

| Field | Type | Default | Description |
| - | - | - | - |
| `repeat_interval_seconds` | integer | `0` | Re-send **Alarm Fired** at this interval while the instance stays firing and unacknowledged. `0` sends once. |
| `escalation_after_seconds` | integer | `0` | Once the instance has been firing and unacknowledged this long, escalate. `0` never escalates. |
| `escalation_receivers` | string\[] | — | Receivers added when the alarm escalates |
| `receivers` | string\[] | — | Base receivers for the alarm |

### Re-notify

* The first reminder goes out one full interval after the alarm fires.
* The interval runs in whole minutes. Values under 60 seconds become 1 minute; other values round down to the minute.
* Each reminder is an **Alarm Fired** event, so every subscription on that event receives it again. The event carries `renotify: true` and a `renotify_count` that counts up from 1.

Reminders stop for good when any of these happens:

* An operator acknowledges the instance.
* The instance resolves, latches, or closes.
* The definition is deleted, or `repeat_interval_seconds` is set back to `0`.

Reminders **pause** while the instance is shelved or matched by an active silence. They resume on the next interval once the shelve or silence lifts.

### Escalation

Escalation is checked on each re-notify, so it needs `repeat_interval_seconds` above `0`.

* The clock starts when the current firing episode began. A re-fire restarts it.
* Once `escalation_after_seconds` has passed without an acknowledgement, the event's receivers become `receivers` plus `escalation_receivers`, and `escalation_tier` becomes `1`.
* Once escalated, the instance stays escalated until the episode ends. There is one escalation step.

> [!WARNING]
>
> `receivers` and `escalation_receivers` are recorded on the alarm event for your own downstream use. They do **not** pick who gets notified. Delivery is controlled by notification subscriptions: every subscription that matches the event and alarm receives it, whether escalated or not.

## Silences and shelving

Silences and inhibitions do not stop an alarm from opening. The instance still opens and its event is still recorded, but notification delivery for it is suppressed. Re-notify reminders are skipped while an instance is shelved or silenced.

See [Silences and shelving](/alarms/silences-and-shelving).

## Automatic work orders

An alarm definition can open a work order in [Ticketing](/ticketing/overview) when an instance fires. In the Console, turn on **Work order on activation** in the definition editor.

| Field | Type | Default | Description |
| - | - | - | - |
| `auto_create_wo` | boolean | `false` | Open a work order when an instance fires |
| `wo_workflow_def_id` | uuid | — | Ticket workflow to open it in (**Workflow** in the Console). Empty uses the project's active work-order workflow. |
| `wo_dedup_key_cel` | string | Per asset | CEL expression that returns a string (**Dedup key** in the Console) |
| `wo_cooldown` | string | `24 hours` | How long a dedup key blocks a new work order |

How it behaves:

1. When the alarm fires, Golain computes a dedup key. With no expression, the key is the alarm's asset, so one asset gets at most one work order per cooldown.
2. If the same key already raised a work order inside the cooldown, no new work order is created.
3. Otherwise a work order opens on the active version of the chosen workflow, tagged with the asset and the alarm instance.

Write `wo_cooldown` as a duration such as `30m` or `4h`, or as an interval such as `2 hours` or `1 day`.

The dedup expression can read the alarm event as `event.payload` and the project as `project.id`:

```
event.payload.asset_id + ":" + event.payload.alarm_def_id
```

> [!WARNING]
>
> No work order is created when:
>
> * The alarm instance has no asset.
> * The chosen workflow has no active version, or the project has no work-order workflow.

## Next steps

* [Silences and shelving](/alarms/silences-and-shelving) — quiet alarms during maintenance
* [Console guide](/alarms/console-guide) — Board, Definitions, and Silences
* [API reference](/alarms/api-reference) — every alarm endpoint
* [Troubleshooting](/alarms/troubleshooting) — when notifications don't arrive


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.