Align a clock by hand

When chrony never reached a time source, the companion journal and process samples keep their own clock. Foxborne then makes no before or after claim between them and the flight log. If both logs recorded the same moment, you can propose an offset from that pair and apply it as a manual alignment.

This page walks through INC-0139 from the example dataset. On run R-0922, obstacle data stopped reaching PX4 on UAS-07 at 14:17:05 UTC and the vehicle held. On the companion, perception_node crashed with a CUDA illegal memory access. The companion clock never synchronized, so the report's finding is written as unknown. The reviewer's open question asks whether the stall started before the crash.

Before you start#

• You need the Investigator or Admin role, because an alignment edits the incident.
• The incident's clock chip reads Not aligned, and its Reconstruction tab shows The companion clock was never synchronised.
• A matching pair exists: one event on the companion clock and one in the flight log that mark the same moment. In INC-0139, mavlink-routerd opened the flight controller UART on the companion, and PX4 saw the first onboard controller heartbeat on TELEM2.

Align the clock#

1. 01

   Read the clock note

   The note on the Reconstruction tab of INC-0139 reads:

   The companion clock was never synchronised

   chrony logged no suitable source at boot, and collector 0.7.3 does not record SYSTEM_TIME, so there is no shared anchor. Its journal and process samples are shown on their own clock, and no before or after claim is made against the flight log.

   On the timeline, the companion lanes sit below a Companion clock divider marked Not aligned, own axis. Their events read Own clock in the inspector. The run page of R-0922 says the same under The companion clock is not aligned, with the button Align in INC-0139.

2. 02

   Open the alignment

   Select Align from matching events in the note. The dialog Align the companion clock opens: "Choose two events that happened at the same moment on both clocks. The alignment is recorded as a manual assumption."

3. 03

   Choose the pair

   Under Pairs found on both clocks, the AI models list events on the companion clock that have a counterpart in the flight log. Each pair gives both events with their recorded times, the offset it implies and its bound, and what the delay between the two events was learned from:

   Pair	Offset	What it rests on
   mavlink-routerd opened the flight controller UART, 03:02:52.431 own clock, and First onboard controller heartbeat on TELEM2, 14:05:57.912 UTC	+11:03:04.916, ±1.04 s	"On the 184 aligned runs with a flight log and a companion journal, the flight controller event comes 565 ms after the companion one."
   perception.service started, 03:14:13.020 own clock, and obstacle_distance updating again, 14:18:16.540 UTC	+11:04:01.130, ±1.04 s	"Seen on 1 aligned run only (R-0931, 2.390 s apart). Too few to learn the delay, so it cannot be used. It would put the offset 56.21 s away from the pair above."

   The first pair is selected. The second cannot be chosen: one aligned run is too few to learn how long perception takes to publish after it starts, and it disagrees with the first by nearly a minute. The companion times are read on their own clock, about 11 hours adrift of UTC.

   Below the list, the dialog gives the reason for the selected pair: "Both mark the companion link coming up. PX4 sees the first onboard controller heartbeat a fixed delay after the router opens the UART, and heartbeats arrive at 1 Hz, so the pair fixes the offset to within about one second."

   With the models off, the dialog offers the single pair recorded on the incident, with the same offset and bound.

4. 04

   Check the offset and the bound

   Offset

   +11:03:04.916, companion to UTC

   Bound

   ±1.04 s, from the 1 s heartbeat period and the 40 ms GPS anchor

   Ordering

   Below the 2.0 s bound, so before and after claims become possible. Each one is marked as manual.

   The bound adds two sources of doubt. The 1,000 ms heartbeat period plus the flight log's 40 ms anchor gives 1,040 ms, shown as ±1.04 s. Ordering compares it with the ordering bound in Settings, 2.0 s by default. The dialog shows the pair, the offset and the bound, with nothing to edit but the choice of pair.

   A note closes the dialog: "The report will list this alignment under Limitations, with the pair it came from. Removing the alignment restores the original state."

5. 05

   Apply it

   Select Apply alignment. Foxborne confirms with Companion clock aligned by hand, ±1.04 s.

What the alignment changes#

Read the result#

The alignment turns the reviewer's question into arithmetic. Every companion time moves by the offset, and its bound grows by the 1,040 ms of the alignment.

CUDA error, companion clock          03:14:07.884
offset, companion to UTC           + 11:03:04.916
CUDA error, UTC                      14:17:12.800   ±1.046 s   6 ms journal + 1,040 ms alignment
perception.service exit, UTC         14:17:12.857   ±1.046 s

last obstacle_distance sample, UTC   14:17:05.310   ±0.040 s   flight log

crash after stall                    14:17:12.800 − 14:17:05.310 = 7.490 s
combined bound                       1.046 s + 0.040 s = ±1.086 s

The crash lands about 7.5 s after the flight-side stall, far outside the ±1.086 s combined bound. The stall's interval ends at 14:17:05.350 and the crash's begins at 14:17:11.754, 6.404 s later. So Foxborne orders them, stall first, and the claim rests on the manual alignment.

The report's 2 Sequence now lists the stall at 14:17:05.310, then the CUDA error and the exit at 14:17:12.800 and 14:17:12.857, each with ±1.05 s. What that means for the cause is the engineer's call. The finding, still written as unknown, is where to say it: see Write and review a report.

Remove the alignment#

Select Remove alignment in the clock note. It takes effect at once, with no confirmation step. Foxborne confirms with Manual alignment removed.

The manual limitation leaves the report and the chip returns to Not aligned. Activity gains Removed the manual clock alignment, and the audit log records Removed clock alignment for the Companion clock.

Why collector 0.8.1 avoids this#

The pair in INC-0139 fixes the offset to about one second, because heartbeats arrive once a second. Collector 0.8.1 and later record MAVLink SYSTEM_TIME pairs, which give companion evidence a shared anchor with the flight log. UAS-07 ran collector 0.7.3, which does not, and the clock note says so.

The example report's Next check ends the same way: upgrade UAS-07 so that future runs carry SYSTEM_TIME. In Sources, under Vehicle collectors, older versions carry the tag No SYSTEM_TIME and an Update button. See Install and enroll the collector. The other fix sits upstream: chrony on the companion with a reachable time source.

Check the result#

• The chip reads Manual ±1.09 s.
• 4 Limitations lists the alignment with its pair, offset and bound.
• Before you export shows Uses a manual clock alignment, ±1.04 s.
• The audit log holds Aligned clock by hand with your name.

Troubleshoot#

There is no Align from matching events button. The button appears only while a companion clock has no alignment. If the note reads Companion clock aligned by hand, an alignment is already in place: remove it to start again. If the note reads All N sources are on one timeline, every source is anchored and nothing needs aligning.

Events still read Order unknown after the alignment. The manual bound is wide. Two events closer than their combined bound, about 1.09 s against the flight log here, stay unordered, as they should. A tighter anchor, such as SYSTEM_TIME from collector 0.8.1, is the fix for future runs.

Next#