# Alerts URL: /integrate/alerts Route findings to the people who act on them. Alert rules, starting from the six defaults R1 to R6, set the severity and the routes, and each destination receives only its data class. Turn on a destination, route a rule to it and read every firing. Alert rules decide who hears about a finding, how fast and with how much detail. Detection stays with the [detection rules](/rules), which produce the findings. An alert rule reads the same signals, adds a severity and routes a message. Every program starts from six alert rules, R1 to R6, and each one can be edited. **New alert rule** adds your own. Each edit saves the rule as its next version, and firings already recorded keep theirs. ## The Alerts page [#the-alerts-page] **Alerts** sits in the **Connect** group of the sidebar, at `/app/alerts`. It has two tabs: **Alert rules** and **Recent firings**, at `/app/alerts/firings`. The **Alert rules** tab opens with five figures. The values below come from the example dataset. | Figure | Example | What it counts | | -------------------- | ---------------------- | ---------------------------------- | | **Rules on** | 6 of 6 | Alert rules switched on | | **Live since** | 23 Sep 2026, 14:09 UTC | When the rules were turned on | | **Fired since then** | 8 | Firings recorded since that time | | **Messages sent** | 9, all delivered | Deliveries made for those firings | | **Pager** | None configured | Whether a pager destination exists | The rules are a table with five columns: **Alert rule** (its ID, name and version, and whether it is a program default or who created it), **Built on** (the detection rule, such as `vibration_high 1.0`), **Delivers to** (its first route and how many more), **Fired** and **On**, the switch. Select a row to open the rule's drawer. It gives **Signals**, **Default condition** and **Severity and routing**, with a chip for each route, then the **Basis**, the **Sources** and **This program**, which names the detection rule it builds on. **Firings** at its foot opens the rule's firings, and **Edit** opens its editor. Below the table the console states the default: "An alert message is L0: rule, severity, vehicle alias, UTC time and a link to the case. Evidence stays in Foxborne, where access is audited." ## Before you start [#before-you-start] * An admin is available to turn on egress. Admins confirm with a hardware security key. * The destination is reachable. A destination marked **Needs a network path out** or **Needs a VPC endpoint** also needs that route. * The credential sits in your secret store. Foxborne takes a reference to it, never the secret. * For L1, your program has authorized the destination for CUI. ## Choose a destination [#choose-a-destination] The **Notify** group on the **Destinations** tab holds the alert destinations. The **Protocol** list and the target field in each destination's settings offer these choices. | Destination | Protocol choices | Target field | Class | | --------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------- | -------- | | Email (SMTP relay) | SMTP submission, 587 STARTTLS; SMTP submission, 465 implicit TLS | **Sender and recipients** | L0 or L1 | | Mattermost | Incoming webhook, HTTPS POST; REST API with a bot token | **Channels** | L0 or L1 | | Microsoft Teams (GCC High, DoD) | Teams bot on Azure Bot Service, Azure Government; Microsoft Graph, GCC High (graph.microsoft.us); Microsoft Graph, DoD (dod-graph.microsoft.us) | **Team and channel** | L0 or L1 | | Microsoft Teams (commercial, GCC) | Workflows webhook, HTTPS POST | **Team and channel** | L0 or L1 | | GovSlack | Incoming webhook on slack-gov.com; Web API chat.postMessage on slack-gov.com | **Channel** | L0 or L1 | | PagerDuty | Events API v2, POST events.pagerduty.com/v2/enqueue | **Service** | L0 only | | AWS SNS (GovCloud) | SNS Publish, us-gov-west-1; SNS Publish, us-gov-east-1 | **Topic ARN** | L0 or L1 | | Generic webhook (HMAC) | Standard Webhooks v1, HMAC-SHA256; Standard Webhooks v1a, Ed25519 | **Endpoint and events** | L0 or L1 | | Alertmanager and Grafana | Alertmanager API v2, /api/v2/alerts; Metrics endpoint for Grafana to query | **Receiver** | L0 or L1 | L1 needs the box **The program has authorized this destination for CUI**. No alert carries L2: evidence moves only through [Export](/integrate/exports), with approval. **Email.** The sender goes in **Sender and recipients**, and "Recipients are set per alert rule." Exchange Online turns off Basic authentication for SMTP AUTH by default at the end of December 2026. A relay that signs in there then needs OAuth. **Mattermost and GovSlack.** A Mattermost incoming webhook is a JSON POST to `https:///hooks/`, and the URL itself is the secret. GovSlack uses `slack-gov.com` instead of `slack.com`, including for generated URLs such as webhooks and `response_url`. **Microsoft Teams.** Office 365 connectors, including connector-based incoming webhooks, were switched off between 18 and 22 May 2026. Workflows is marked not available in GCC High and DoD, so use a Teams bot on Azure Bot Service or Microsoft Graph there. **PagerDuty and AWS SNS.** The FedRAMP Marketplace lists PagerDuty at Class B (Low), below the Moderate-equivalent bar DFARS 252.204-7012 sets, so it takes L0 only. In GovCloud, SNS must carry no export-controlled data to email, HTTP or push subscribers, or in topic names and subjects. **Alertmanager and Grafana.** Grafana's built-in Alertmanager handles only Grafana-managed alerts. Foxborne posts to a Prometheus Alertmanager that Grafana uses as an external Alertmanager, and re-sends firing alerts until they resolve. ## Set up a destination and route a rule to it [#set-up-a-destination-and-route-a-rule-to-it] Go to **Integrations**. On the **Destinations** tab, select the destination's row in the **Notify** panel, then select **Set up** in its drawer. Enter the **Host allowlist**, one host:port per line, and choose the **Protocol**. Fill in the target field, then the **Credential reference** and the **Data class**. The example dataset's Mattermost destination uses these values: chat.hfr.internal:443 Incoming webhook, HTTPS POST test-conductor, reliability-alerts secretsmanager:foxborne/int/mattermost-webhook L0 Select **Send test**, and Foxborne sends "One message with no data, to the first host". The dialog reports **Delivered at** a UTC time with the response, such as `200 OK, body "ok"`. **View** opens the delivery in the ledger, and the audit log records **Sent test delivery**. Select **Save**; the toast reads, for example, **Mattermost saved. Egress stays off until an admin turns it on.** The audit log records **Configured integration** with the first host, the protocol and the class. Select the switch in the destination's **Egress** column, then **Turn on egress**. An admin confirms with a hardware security key; anyone else picks the confirming admin under **Admin confirming**. The audit log records **Enabled egress**. Go to **Alerts**, select the rule's row, then **Edit** in its drawer. Under **Delivers to**, tick the routes the rule should use; each shows where it sends, such as "L0 to chat.hfr.internal:443". Select **Save**, and the rule saves as its next version. ## Default alert rules [#default-alert-rules] The six defaults, as their drawers give them. Words in code are PX4 fields and parameters. | Rule | Default condition | Severity and routing | | -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------ | | R1 Vibration above threshold (tracked-UGV profile) | `accel_vibration_metric` above 24 m/s² continuously for 1.0 s or more while armed. Escalate if any `accel_clipping` counter increments in the window. | Warning, or High with clipping. L0 to the test conductor channel. | | R2 Telemetry gap | Gap between consecutive `HEARTBEAT` messages from the vehicle's system ID above 3.0 s while armed. Escalate when the gap reaches that flight's `COM_DL_LOSS_T`. | Medium; High at `COM_DL_LOSS_T`. | | R3 Failsafe entered | Any transition of `vehicle_status.failsafe` into failsafe while armed; repeats within 30 s merge into one finding. | High. Opens a case and ticket; L0 page. | | R4 Memory pressure (companion computer) | Any process RSS at or above 6,144 MiB for 3 consecutive samples. Critical if the kernel logs "Out of memory: Killed process" or "Memory cgroup out of memory". | Warning, then Critical on an OOM kill. | | R5 Device disconnect | A device in the vehicle's registry (USB port path or serial number, or a CAN interface) disconnects or goes bus-off while armed. 2 or more re-enumerations within 60 s flag as flapping. | High for flight-critical devices (GNSS, radio, autopilot link, CAN ESC bus); Medium for payload. | | R6 GNSS jamming | `jamming_state` = 3 (Detected) for 2 s or more, or `jamming_indicator` at 80 or above for 5 s. Confidence rises when `satellites_used` is under 12 or `eph` is above 1 m. Downgrade to Info inside scheduled range GPS-test windows. | High; L0 to range safety. | The **Basis** line traces each default. R1's 24 m/s² "is a program choice for tracked vehicles, not a PX4 number." R2's 3 s gives warning before the failsafe, because `COM_DL_LOSS_T` defaults to 10 s (range 5 to 300). For reference, PX4 Flight Review shades the vibration metric red above 9.81 m/s² for air vehicles. See [PX4 parameters Foxborne reads](/reference/px4-parameters). In the example dataset, the routes read as follows. A route whose destination is not configured shows **not configured**. | Rule | **Delivers to** in the example dataset | | ---- | ------------------------------------------------------------------------------------------------------- | | R1 | Mattermost, test-conductor | | R2 | Mattermost, reliability-alerts | | R3 | A case, as a draft for the investigator; Jira Data Center, project REL; PagerDuty page (not configured) | | R4 | Mattermost, reliability-alerts; Email, reliability on-call, Critical only | | R5 | Mattermost, test-conductor | | R6 | Email, range safety | ## Turn a rule off or change it [#turn-a-rule-off-or-change-it] The switch in a rule's row turns it on or off. Turned off, the toast reads "Findings are still produced and recorded; nobody is notified." The audit log records **Enabled alert rule** or **Disabled alert rule**, with the name and version. **Edit**, in the rule's drawer, opens **Edit** *rule*. For a rule at 1.0, its note reads "Saves as version 1.1. Firings already recorded keep version 1.0." You can change **Default condition**, **Severity and routing** and **Delivers to**. **Signals and basis** stay "Fixed per rule, so every firing cites the same signals and sources." **Save** confirms with, for example, **R1 saved as version 1.1**. The audit log records **Changed alert rule** with what changed, such as `Version 1.1: condition`. ## Create an alert rule [#create-an-alert-rule] Select **New alert rule** at the top right of the **Alerts** page. The dialog's note: "Decides who hears about a detection, how fast and with how much detail. It carries L0 metadata unless a destination is cleared for more." | Field | What to enter | | --------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | | **Name** and **Short name** | The short name shows in firing lists and message subjects, such as **Low pack voltage** | | **Built on** | The detection rule whose results the alert reads, from every rule on **Rules** | | **Condition** | When the alert fires, in your words, such as "While armed, the detection holds for 2 s or longer" | | **Severity** | **Info**, **Warning**, **Medium**, **High** or **Critical** | | **Delivers to** | The routes, each with where it sends. A route whose destination is not configured says "Not configured, so nothing is sent". | Select **Create alert rule**. The rule takes the next ID, such as **R7**, saves at version 1.0 and starts on. Its **Signals** name the detection rule it reads, and its **Basis** records that it is a program choice, by whom and when, to review against known-good runs. The toast reads, for example, **R7 Low pack voltage created and on. It runs when the next sortie closes out.**, and the audit log records **Created alert rule**. ## How alert rules differ from detection rules [#how-alert-rules-differ-from-detection-rules] | Alert rule default | Detection rule default | What differs | | --------------------------------------- | ------------------------------------------------------------------------------------------ | -------------------------------------------------- | | R1: above 24 m/s² for 1.0 s while armed | [`vibration_high`](/rules/vibration-high) 1.0: above 24 m/s² for 1 s (T4), 12 m/s² (Q4) | Tracked profile, armed only, escalates on clipping | | R2: above 3.0 s while armed | [`telemetry_gap`](/rules/telemetry-gap) 1.3: more than 3 s without a HEARTBEAT | Armed only, escalates at `COM_DL_LOSS_T` | | R3: any transition while armed | [`failsafe_entered`](/rules/failsafe-entered) 1.0: any failsafe | Armed only, repeats within 30 s merged | | R4: 6,144 MiB for 3 samples | [`memory_pressure`](/rules/memory-pressure) 1.1: RSS at or above 6,144 MiB for 3 samples | Critical on an OOM kill | | R5: registry device or CAN interface | [`device_disconnect`](/rules/device-disconnect) 1.0: any disconnect on an allowlisted port | Adds bus-off and flapping | | R6: `jamming_state` 3 for 2 s | [`gnss_jamming`](/rules/gnss-jamming) 1.0: `jamming_state` 3 for 1 s | Alert at 2 s, and also on the indicator | In the example dataset, R4 follows the detection rule's T4 override of 24,576 MiB on T4 vehicles. See [Thresholds, scopes and overrides](/rules/configure). ## Check the result [#check-the-result] **Recent firings** lists each firing with two times, because rules run when a sortie closes out. The first is when the condition held on the vehicle, the second when Foxborne evaluated it after ingest. Filter by **All**, **Critical**, **High** or **Medium and Warning**, and by rule. The **Delivered** column names the destinations and reads **Delivered** or **Not all delivered**. Open a firing for **When**, **What held**, the rule and version, and each message under **Delivered**, which opens its ledger entry. In the example dataset, R4 fired Critical for UAS-04 on run R-0931. The Mattermost post to reliability-alerts was delivered on the first attempt. The email to reliability on-call got `421 4.3.2 Service not available, closing transmission channel` and was delivered on attempt 2, 30 s later. The L0 message R4 posted for UAS-06 on run R-0929 shows what an alert carries: ```json title="Payload, as sent" caption="Example dataset: R4 at Warning, Mattermost, L0" { "channel": "reliability-alerts", "username": "Foxborne", "text": "R4 Memory pressure, Warning. UAS-06 at 16:20:22.534 UTC, 23 Sep 2026. Run R-0929: https://foxborne.hfr.internal/app/runs/R-0929", "props": { "foxborne": { "rule": "R4", "severity": "warning", "vehicle": "UAS-06", "time": "2026-09-23T16:20:22.534Z", "link": "https://foxborne.hfr.internal/app/runs/R-0929" } } } ``` With no case, the link points to the run. With a case, it points to the incident. ## Troubleshoot [#troubleshoot] **A route chip reads not configured.** Its destination is not configured or its egress is off: "Not configured, so nothing is sent." Set it up and turn on egress. **R3 fired, and no page went out.** The firing reads "The L0 page was not sent: no pager is configured." Set up PagerDuty, which carries L0 only. **A finding produced no alert.** The alert rule may be off. Findings are still produced and recorded; turn the rule on to notify again. **Findings from before the go-live were never routed.** The **Recent firings** tab notes that firings before the **Live since** date "were never routed, because no alert rule was on." **Exchange Online refuses the relay's login.** Basic authentication for SMTP AUTH is off by default from the end of December 2026. Configure the relay for OAuth. **A destination keeps failing.** After 5 attempts, the delivery moves to the dead-letter queue and the **Integrations** count turns amber. Fix the destination, then select **Replay** on the **Delivery ledger** tab. ## Next [#next] * [Webhooks](/integrate/webhooks) covers the signed webhook and how to verify it. * [Automations](/integrate/automations) shows what follows a firing: cases, tickets and evidence packs. * [Egress, data classes and the delivery ledger](/integrate/egress-controls) explains the classes, egress and the ledger. * [How rules work](/rules) covers the detection rules behind every finding.