Rules and findings
Rules are fixed, versioned checks that report what the sources show. Each finding names the rule, version and threshold that produced it and the events it read, so a rerun gives the same answer.
On this page8
A rule is a deterministic check on the sources, such as a memory threshold or a gap between heartbeats. A finding is what one rule observed on one run, tied to the events it read. Neither says why the vehicle failed: the engineer does that, in the report.
The rules#
Foxborne has 13 rules. Each has an ID, a version, a threshold and a default state, and each reads specific sources.
| Rule | Detects | Threshold | Default |
|---|---|---|---|
| process_exitService exit, v1.2 | An allowlisted systemd unit leaves the active state. | 6 units on the allowlist | On |
| service_restartService restart, v1.0 | systemd schedules a restart of an allowlisted unit. | Any restart | On |
| memory_pressureMemory pressure, v1.1 | A watched process stays above its memory threshold. | RSS at or above 6,144 MiB for 3 samples | On |
| telemetry_gapTelemetry gap, v1.3 | No HEARTBEAT arrives at the ground receiver beyond the expected interval. | More than 3 s without a HEARTBEAT | On |
| device_disconnectDevice disconnect, v1.0 | The kernel reports that a USB device on the allowlist disconnected. | Any disconnect on an allowlisted port | On |
| vibration_highVibration, v1.0 | accel_vibration_metric in vehicle_imu_status stays above the platform threshold. | Above 24 m/s² for 1 s (T4), 12 m/s² (Q4) | On |
| failsafe_enteredFailsafe, v1.0 | vehicle_status reports a failsafe, with the cause from failsafe_flags. | Any failsafe | On |
| topic_stallTopic stall, v1.0 | A logged uORB topic stops updating at its expected rate. | No sample for 5 times the median period | On |
| gnss_jammingGNSS jamming, v1.0 | sensor_gps reports jamming_state 3 (detected) from the receiver. | jamming_state 3 for 1 s | On |
| gps_quality_dropGPS quality drop, v1.0 | Horizontal accuracy or satellite count crosses the vehicle threshold. | eph above 2.5 m or fewer than 8 satellites for 2 s | On |
| log_dropoutLogger dropout, v1.0 | The flight log records a dropout marker. | Dropout of 50 ms or longer | On |
| clock_uncertainClock uncertain, v1.0 | A source has no trustworthy anchor, or its estimated error exceeds the ordering bound. | Anchor missing or error above 2 s | Always on |
| battery_sagBattery sag, v0.9 | Pack voltage drops faster than the configured rate under load. | More than 2.0 V in 2 s above 70% track current | Off |
clock_uncertain is always on and cannot be edited. battery_sag is off by default, and the other 11 are on. Each rule applies to a scope, such as All vehicles or T4 vehicles, and the Rules page counts its results over the last 30 days.
What a finding records#
A finding carries what it takes to reproduce it:
- The rule ID and version, such as
telemetry_gap 1.3. - The threshold in force, such as more than 3 s without a
HEARTBEAT. - The events it read, each with its file, location and SHA-256.
- The rule's fixed wording, such as Observed: telemetry receive gap, not necessarily radio failure.
The memory finding in INC-0142 shows the values a finding keeps. These come from the example dataset:
- Rule
- memory_pressure 1.1
- Event
- perception_node memory above 6,144 MiB for 3 samples
- Samples
- 6,188, 6,402 and 6,655 MiB
- Threshold
- 6,144 MiB for 3 samples
- Confirmed at
- 14:32:03.204
- Source
- Process samples, rows 2,811 to 2,813
- Wording
- Observed: memory rose above threshold
The console shows the same record in three places. The timeline's Findings row flags each finding, and the inspector's Rule block gives the rule, what it detects, its threshold and its wording. The event page repeats the rule, version and threshold under Highlights.
In INC-0142 the rules produced six findings from five rules:
| Finding | Rule and version |
|---|---|
| Memory above threshold | memory_pressure 1.1 |
| obstacle_distance stalled | topic_stall 1.0 |
| perception.service killed | process_exit 1.2 |
| Router stopped | process_exit 1.2 |
| Telemetry gap 6.96 s | telemetry_gap 1.3 |
| Service restarted | service_restart 1.0 |
Rules observe, engineers conclude#
The Rules page puts the principle under its table: "Rules observe. They never conclude." Rule wording starts with Observed or Warning, and a report keeps rule output apart from what an engineer writes.
The wording is fixed per rule. The edit dialog shows it read-only, with the reason: "Fixed per rule, so every report describes the same detection the same way." telemetry_gap shows why that matters: it reports that the receiver heard nothing and adds "not necessarily radio failure."
The conclusion belongs to the engineer, in the report's finding, marked as observed, a hypothesis or unknown. See Evidence levels.
Dismissing a finding#
A finding can be expected in context, such as a restart during a scheduled bench test. Dismiss this finding, in the evidence inspector, asks for a Reason and will not go ahead without one.
The dialog states "The raw evidence is not changed." After you confirm, the finding's flag leaves the Findings row and the inspector shows who dismissed it and why. The audit log keeps the dismissal and its reason.
Versions and thresholds#
An admin changes a rule under Rules with Edit. The dialog shows the version it will create before anything is saved, such as Saves as telemetry_gap 1.4.
Saving a change raises the number after the dot by one: 1.2 becomes 1.3, and 0.9 becomes 0.10. Findings already produced keep the version and threshold that made them, as the dialog's hint says: "Results already produced keep version 1.3 and its threshold." If nothing changed, the dialog closes with Nothing changed and no new version.
A rule can also carry a scope and an override. memory_pressure 1.1 applies 6,144 MiB to Q4 vehicles and overrides it at 24,576 MiB for T4 vehicles. See Thresholds, scopes and overrides.
On, off and locked#
The Enabled switch takes effect on the next import, as its confirmation says: "Battery sag enabled. Applies to the next import."
The rule keeps its version, and the audit log records who changed it.
clock_uncertain cannot be switched off or edited. Its switch gives the reason: "Always on. Every ordering claim depends on it."
Running rules again#
Run rules again, on a run's page, reruns the rules at the same versions. The console reports "Rules ran again. Results are identical to the previous run." and records the rerun in the audit log.
Related#
- How rules work covers what each rule reads and when it fires.
- Thresholds, scopes and overrides walks through changing a rule.
- Telemetry gap reads one rule end to end.
- Evidence levels explains the levels a report adds on top of rule output.