# Memory pressure URL: /rules/memory-pressure Flags a watched process whose resident memory stays at or above its threshold for three samples in a row. It observes memory use on the companion computer, not a leak and not what followed. ## What it reads [#what-it-reads] The rule reads process samples from the Foxborne collector: one record per watched process per sample, with its resident memory in `rss_kib`. The collector policy names the processes and the rate. In the example dataset it samples `perception_node` and `mavlink-routerd` once per second. ```json title="uas04-orin-samples.jsonl" caption="Example dataset, INC-0142, the first of rows 2,811 to 2,813, as the evidence inspector shows it" { "t_utc": "2026-09-24T14:32:01.204Z", "mono_ns": 1149990000000, "boot_id": "9b2e4c1d", "proc": "perception_node", "pid": 2213, "rss_kib": 6336512, "cpu_pct": 187.4, "restarts": 0 } ``` `rss_kib` is in KiB, so 6,336,512 KiB is 6,188 MiB. The samples carry the companion's monotonic time and boot ID, so they share the journal's clock and its error bound. ## When it fires [#when-it-fires] A finding is produced when a watched process holds 6,144 MiB or more for 3 samples in a row. At one sample per second, the three samples span 2 s. The event sits on the first of them, and the third confirms it. | Value | Example from INC-0142 | | ------------------------ | ------------------------------------------------- | | Samples | 6,188, 6,402 and 6,655 MiB | | First sample at or above | 14:32:01.204 | | Confirmed at | 14:32:03.204 | | Threshold | 6,144 MiB for 3 samples | | Location | Rows 2,811 to 2,813 of `uas04-orin-samples.jsonl` | The finding's title names the process and the line: **perception\_node memory above 6,144 MiB for 3 samples**. ## Wording and evidence level [#wording-and-evidence-level] Every finding reads **Observed: memory rose above threshold**. The samples show the value directly, so the level is observed. ## What the finding does not mean [#what-the-finding-does-not-mean] * **It does not say the process leaked.** In INC-0142 `perception_node` logged at startup that its tile cache had no size limit. The report still asks, as an open question, why the cache grew without a limit on that build. * **It does not say the memory caused what followed.** The kernel killed `perception_node` 0.914 s after the confirming sample. The kill is a separate observation in the journal, and linking the two is the engineer's work. * **A sample is one reading at one moment.** The kill line reports `anon-rss:6823516kB` and `file-rss:10240kB`, 6,673.6 MiB resident in all. That is 18.6 MiB above the last sample, taken 0.914 s earlier, and the kernel's kB are KiB. * **Resident memory leaves out GPU buffers.** On the example's Jetson modules, the CPU and the GPU share one memory. The event page for the INC-0142 kill lists 8,212 MiB of GPU buffers (nvmap) under **Not in these counters**. * **Below the line is silent.** For the 44 samples before the crossing, from 14:31:17.204 to 14:32:00.204, `perception_node` sat between 6,117 and 6,143 MiB. None of them produced a finding, so read the **Process samples** lane as well as the flags. ## Threshold and overrides [#threshold-and-overrides] In the example dataset the threshold reads **RSS at or above 6,144 MiB for 3 samples**, applied to **Q4 vehicles**. The rule also carries the dataset's one override, **T4 vehicles: 24,576 MiB**. Under the threshold, the **Threshold** column shows both as **Q4 vehicles. T4 vehicles: 24,576 MiB**. An override covers vehicles whose hardware differs. In the example, the T4's companion computer has 64 GB of memory against 16 GB on the Q4, both shared by CPU and GPU. Change either value under **Rules** with **Edit**. The rule saves as `memory_pressure 1.2`, and findings already produced keep version 1.1 and its threshold. The collector's **Sample rate** also changes what 3 samples span: at 5 per second they cover 0.4 s, not 2 s. ## Example finding [#example-finding] In INC-0142, UAS-04 flew the return leg of Route Iron. `perception_node` read 6,122 MiB at 14:32:00.204, then 6,188, 6,402 and 6,655 MiB over the next three samples. The finding was confirmed at 14:32:03.204, and the kernel killed the process at 14:32:04.118. The **Process samples** lane draws the line as **Threshold 6,144 MiB**. The **Findings** row labels the finding **Memory above threshold**, and it was the incident's first finding. On the event page, **Memory around this event** plots 8 s before to 4 s after the first sample. Filled points are at or above the dashed threshold, and a table lists the samples within 2.1 s of the event. The event page's **Across the fleet** panel counts 6 occurrences of the same signature on 4 vehicles in the last 30 days. A single finding can be the start of a pattern, and that panel is where the pattern shows. ## Related [#related] * [process\_exit](/rules/process-exit) records the exit that followed the kill. * [service\_restart](/rules/service-restart) records systemd bringing the unit back. * [Collector JSONL records](/reference/collector-jsonl) lists every field of a process sample. * [Set the collector policy](/collect/collector/policy) chooses the processes and the sample rate. * [Inspect an event](/investigate/event) describes the kernel report and the memory chart.