Automations
Four automations act when a sortie closes out or an alert fires. They triage the sortie, pack failsafe evidence, catch a repeat hardware fault and seal an approved case, and anything that would leave the enclave waits for a named approver.
On this page13
Detection rules produce findings, and alert rules decide who hears about them. Automations do the work that follows: "What Foxborne does on its own when a sortie closes out or an alert fires." The program starts with four, A1 to A4, New automation adds your own, and each one is switched on or off.
The guardrail holds for all four: "Anything that would move content out of the enclave waits for a named approver." Every delivery an automation makes goes into the delivery ledger.
The Automations page#
Automations sits in the Connect group of the sidebar, at /app/automations. It has two tabs: Automations and Run history, at /app/automations/runs. The sidebar count shows how many runs are held for approval.
The Automations tab opens with four figures. In the example dataset, On reads 3 of 4 and Runs since 23 Sep reads 10. Held for approval reads 1, and Named approvers lists the program's two approvers.
The automations are a table with four columns: Automation (its ID, its name, and who turned it on and when), Trigger, Last ran (with its number of runs and any still waiting) and On, the switch. Select a row to open its drawer. The drawer gives Trigger, Conditions, Actions and Guardrails, then This program, which says how your program applies it and who turned it on. Run history at its foot opens the automation's runs.
Before you start#
- The destinations each automation uses have egress on. See the first step below.
- Each sortie has a test card that lists its expected sources and any planned failsafe tests. A1 and A2 read it.
- Your program has named approvers. On the Exports tab of Integrations, Approval policy lists them under Named approvers.
Turn an automation on#
Turn on the destinations its actions use
Automation Destinations it uses in the example dataset A1 Jira Data Center (project REL) and Mattermost (test-conductor) A2 Email, to assign the draft case file; PagerDuty, if the program has one A3 Jira Data Center (project REL) and email for the weekly digest A4 S3 with Object Lock, the case's ticket and the SIEM over syslog with TLS See Alerts, Ticketing and Forward logs to your SIEM.
Give A4 its write-once storage
A4 cannot run without WORM storage. Until S3 with Object Lock is configured with egress on, its row reads Needs S3 with Object Lock, and its drawer warns: "A4 writes each closed case to WORM storage, and none is configured." Select Set up S3 with Object Lock at the foot of the drawer to open that destination's settings.
Switch the automation on
Select the switch in the automation's row. The toast reads, for example, A1 is on, and This program adds who turned it on and when. The audit log records Enabled automation with the automation's ID and name.
Watch its runs
Open Run history. Each run lists its start time, the automation and its trigger, the run and vehicle, the result and a status.
Create an automation#
Open New automation
Select New automation at the top right of the Automations page. The dialog notes: "What Foxborne does on its own. Anything that would move content out of the enclave still waits for a named approver."
Say when it runs
Give it a Name. Under When, choose A sortie closes out, or an alert rule firing, such as R3 Failsafe entered fires. Only if takes the conditions in your words; left empty, the automation runs every time the trigger fires.
Choose what it does
Tick one or more actions under Then:
Action What it does Open a draft case Opens a draft case for the investigator on the sortie window Post an L0 summary to Mattermost Posts an L0 summary to the test-conductor channel Open a Jira issue Opens a Jira Data Center issue in project REL that holds the case link Draft an evidence pack Freezes the window across all sources and drafts the PDF case file, marked DRAFT, for a named approver The guardrails are fixed and the dialog says so: notifications carry L0 metadata only, and nothing leaves the enclave until a named approver signs off.
Create it
Select Create automation. It takes the next ID, such as A5, and starts turned on. The toast reads, for example, A5 Jamming report to range safety created and on, and the audit log records Created automation. An action reaches its destination only while that destination's egress is on.
Sortie closeout triage (A1)#
- Trigger
- Ingest complete for a sortie: S3 ObjectCreated to SQS, drop folder or removable-media import, or an API "bundle complete" call.
- Conditions
- Every source on the test card has arrived (autopilot log, ground-station
.tlog, companion journal, ROS 2 bag), or 30 minutes have passed with nothing new. - Actions
- Verify the sender's SHA-256 manifest, align clocks and run the rule pack. If any finding is High, open a case and a Jira Data Center or GitLab issue that holds the case link, and post an L0 summary to Mattermost.
- Guardrails
- A missing source becomes its own finding. Notifications carry no technical data.
"A case the automation opens stays a draft until an investigator opens it on a window." A run that finishes parsing while A1 is on closes out through A1.
In the example dataset, A1 closed out run R-0931 and drafted INC-0142:
| Step | A1 on R-0931 |
|---|---|
| Ingest complete | 4 of 4 sources on the test card at 15:01:20 |
| Sender manifest | SHA-256 verified for 4 files |
| Clocks | Aligned, widest bound ±46 ms |
| Rule pack | 13 rules, 6 findings in 0.9 s |
| Alerts | R4 Memory pressure, Critical; R2 Telemetry gap, Medium |
| Case | Drafted INC-0142, which the investigator opened at 15:02:11 |
| Ticket | REL-418 in Jira Data Center, with the case link |
| Mattermost | L0 summary to test-conductor |
On run R-0930 the companion journal never arrived. After 30 minutes with nothing new, A1 closed the sortie out with 2 of 3 sources and recorded the missing journal as its own finding.
Failsafe evidence pack (A2)#
- Trigger
- R3 fires.
- Conditions
- Armed, and the test card does not list a planned failsafe test for that window.
- Actions
- Freeze a ±120 s window across all sources. Draft the PDF case file with the CUI banner, footer and designation indicator from program settings, marked DRAFT. Assign it to the test director. Send an L0 PagerDuty page if the program has one.
- Guardrails
- Nothing leaves the enclave until a named approver signs off.
In the example dataset, INC-0143 has the transition A2 waits for. UGV-02's flight log shows vehicle_status.failsafe turning true with gcs_connection_lost, and PX4 switched to Hold. The run's steps read:
| Step | A2 on R-0934 |
|---|---|
| Trigger | R3, for the Hold at 15:41:14.293 |
| Armed | Yes, since 15:37:40.118 |
| Test card | Breach lane 2, rehearsal 3 lists no planned failsafe test |
| Frozen window | 15:39:14.293 to 15:43:14.293 on the flight log, companion journal and ground receive log |
| Case file | Drafted with the CUI banner, footer and designation indicator from Settings, marked DRAFT |
| Assigned | The test director, by email |
| Page | Not sent. No PagerDuty service is configured |
| Release | Held. EXP-0034 has 1 of 2 approvals |
The run shows Held for approval. Its drawer adds Export request: "EXP-0034 asks to take the pack to removable media. 1 of 2 approvals so far." Removable media leaves the enclave, so both named approvers must sign.
Repeat hardware fault (A3)#
- Trigger
- R5 fires.
- Conditions
- The same device (serial, port path or CAN interface) has fired in 3 or more of the vehicle's last 10 sorties.
- Actions
- Open a maintenance ticket (ServiceNow, Jira or GitLab) listing sorties and timestamps; mark the vehicle "inspect before next sortie"; include it in the weekly email digest.
- Guardrails
- Deduplicate per device. Auto-close after 10 clean sorties.
In the example dataset, R5 fired for the mesh radio on USB port path 1-2.3 of UGV-02. A3 read Not met: "The mesh radio disconnected on 1 of UGV-02's last 10 sorties, and A3 needs 3." The same port path had disconnected on three other vehicles in 30 days, but "A3 counts per vehicle, so these do not add up."
Case closure and retention (A4)#
- Trigger
- Case moves to Approved.
- Conditions
- None.
- Actions
- Build a BagIt bag (originals, derived MCAP and Parquet, final PDF, sha256 and sha512 manifests). Write it to WORM storage: S3 Object Lock in compliance mode in GovCloud, or on-prem WORM. Record the tag-manifest SHA-256 in the ticket. Forward an audit event over syslog/TLS, Splunk HEC or Sentinel. Optionally write Parquet to the program data lake.
- Guardrails
- Two-person approval for any destination outside the enclave. Every delivery is written to the ledger.
A4 is off in the example dataset. Its switch opens A4 cannot run yet until S3 with Object Lock is configured with egress on. The dialog adds: "Two-person approval still applies to any destination outside the enclave, and every delivery is written to the ledger."
Under RFC 8493, the tag manifest lists every payload manifest, and each payload manifest lists every payload file with its checksum. The tag-manifest SHA-256 in the ticket therefore pins the whole bag.
Run history#
| Status | What it means | Example result |
|---|---|---|
| Completed | The automation ran to the end | "R4 fired at Warning. No finding was High, so no case." |
| Stopped | The run could not be triaged | "Stopped. The flight log is encrypted, so it was rejected and no rules ran." |
| Held for approval | An export waits for its named approvers | "Held for approval." |
| Waiting | The sources are still arriving or parsing | "Waiting for the flight log to finish parsing." |
| Not met | The trigger fired, and the conditions did not hold | "Not met." |
Open a run for its drawer. Result repeats the status and the result, and Steps lists each step with its value.
Export request links the held export, and Deliveries opens each delivery's ledger entry. Automation restates the actions and guardrails.
Turn an automation off#
Select the switch again. The toast reads, for example, "A1 is off. Sorties still import, and alerts still fire."
The audit log records Disabled automation.
Check the result#
- Run history lists the run with Completed, and its drawer shows each step.
- The Delivery ledger tab lists each delivery under Automations, with its class and payload SHA-256.
- After A1 drafts a case, the incident's Activity tab shows Foxborne's entry: "sortie closeout triage (A1) drafted this case".
Troubleshoot#
A run reads Waiting. Ingest is still in progress, and the result reads "Waiting for the flight log to finish parsing" with the percentage done.
A run reads Stopped, and no rules ran. A file failed validation, and the Validation step says why, such as an encrypted flight log. See Import statuses and fixes.
A sortie was not triaged. The result reads "A1 was off when this run finished parsing, so it was not triaged." Turn A1 on before the next sortie.
A2 froze no window for a failsafe. The vehicle was not armed, or the test card listed a planned failsafe test for that window.
A3 reads Not met. The same device has to fire in 3 or more of that vehicle's last 10 sorties.
A4 will not switch on. Configure S3 with Object Lock and turn on its egress first.
Next#
- Alerts defines R3 and R5, the triggers for A2 and A3.
- Ticketing connects the trackers A1, A3 and A4 write to.
- Export formats and evidence bundles covers the approvals a held export needs.