# How data gets into Foxborne

URL: /collect

Evidence reaches Foxborne by three paths: import files, connect storage you already run or install a collector or receiver. Every original is hashed on arrival, matched to a vehicle and assembled into a run.



Foxborne reads evidence and never writes it back. Files arrive from an investigator's browser, from storage your program already runs or from services on the vehicle and at range control. Whatever the path, each file goes through the same import job.

<Figure caption="The import job. Each file takes these five steps whichever path it arrived by, and the same files and parser versions always give the same event IDs.">
  <ImportPipeline />
</Figure>

## Three paths in [#three-paths-in]

| Path                     | What arrives                                                                                                     | Set up by                | Where in the console                                                       |
| ------------------------ | ---------------------------------------------------------------------------------------------------------------- | ------------------------ | -------------------------------------------------------------------------- |
| Import files             | A flight log, journal export, collector samples or receive log, up to 250 MB per file by default                 | An investigator or admin | **Import files**                                                           |
| Connect storage          | Files that already land in a bucket, container, SFTP directory or network share, and bundles your test rigs push | An admin                 | **Sources**, **Connections** tab                                           |
| Collectors and receivers | Journal units and process samples from each companion computer, and the MAVLink messages range control hears     | An admin                 | **Sources**, **Vehicle collectors** tab, and a **MAVLink receiver** source |

### Import files [#import-files]

An investigator adds the files for one run on the **Import files** page. The browser computes each file's SHA-256 before the upload, and Foxborne recognizes the format from the file's content. Use it for a single run, or for files that land nowhere Foxborne can reach.

### Connect storage [#connect-storage]

An admin connects a source once, and Foxborne picks up new files as they land. It reads Amazon S3, including AWS GovCloud, and S3-compatible stores. It also reads Azure Blob Storage, including Azure Government, Google Cloud Storage, SFTP and SMB or NFS shares.

Test rigs and pipelines push signed bundles through the Ingest API instead. Each connection is read-only. Before an admin can add a source, its connection test attempts a write and a delete and confirms that both are refused.

### Collectors and receivers [#collectors-and-receivers]

The companion collector runs on each vehicle's companion computer. It reads an allowlist of journal units and processes, spools while the vehicle has no uplink and uploads over mutual TLS. The MAVLink receiver records what range control hears from each vehicle, stamped with the receiver's own clock.

Both are optional. Foxborne works from imported files alone.

## Hashed on arrival [#hashed-on-arrival]

Foxborne computes the SHA-256 of each original before it parses anything, then stores the file exactly as it arrived. Originals are written once, and Foxborne cannot modify or delete one.

The run page lists each file with its hash under **Source files**. A report quotes each excerpt with its file, its place in that file and the file's SHA-256. Anyone holding the original can check every line.

## Matching files to vehicles [#matching-files-to-vehicles]

Foxborne matches files to vehicles by the MAVLink system ID in each flight log, or by the `{vehicle}` part of a source's file pattern.

* **By system ID.** Each vehicle page shows the vehicle's MAVLink system ID. In the example dataset UAS-04 flies as system ID 4, so a flight log from system ID 4 belongs to UAS-04.
* **By file pattern.** The example fleet log archive reads `{vehicle}/{yyyy-mm-dd}/*.ulg`: a folder per vehicle, then a folder per day, with the flight logs inside.
* **By enrollment.** A collector enrolls as one vehicle and holds that vehicle's certificate, so its uploads arrive already matched.
* **By choice.** On **Import files**, the **Vehicle** field detects the vehicle from the flight log, or you pick it yourself.

## From files to runs [#from-files-to-runs]

A run is one flight or bench session on one vehicle. It has up to four sources: the flight log, the companion journal, process samples and the ground receive log. Foxborne matches new files to runs by vehicle and time, so files that took different paths land on the same run.

In the example dataset, run R-0931 was assembled from four files that came through three sources. They were the fleet log archive, the companion collector and the range control receive logs. Its **Import job** drawer shows each step and its time.

| Step                              | Time   |
| --------------------------------- | ------ |
| Validate type, size and schema    | 0.4 s  |
| Hash originals (SHA-256)          | 1.2 s  |
| Parse flight log, 11 of 13 topics | 38.1 s |
| Parse journal and samples         | 2.6 s  |
| Align clocks                      | 0.3 s  |
| Run 13 rules                      | 0.9 s  |
| Total                             | 43.5 s |

The job is deterministic. Parse the same files with the same parser versions next month, and every event keeps its ID.

## What a complete run needs [#what-a-complete-run-needs]

| Source             | Example                                  | Need        | What it gives                                                    |
| ------------------ | ---------------------------------------- | ----------- | ---------------------------------------------------------------- |
| PX4 flight log     | `.ulg`                                   | Required    | The completed ULog from the flight controller's SD card          |
| Companion journal  | `journalctl -o json --since … --until …` | Recommended | The systemd journal export covering the flight, with its boot ID |
| Ground receive log | `.jsonl`                                 | Recommended | What range control heard from the vehicle, and when              |
| Collector samples  | `.jsonl`                                 | Optional    | Process memory, CPU and restarts, if the collector was installed |

A run missing a source it should have shows **Partial**. The run page names the gap, for example **No ground receive log for this run**. Findings that depend on the missing source are not produced, and reports list the gap under **Limitations**.

Time needs one more thing: a GPS fix in the flight log. GPS time in `sensor_gps` anchors the run to UTC, and a run with no fix, such as an indoor hover, shows **Elapsed time only**.

## Next [#next]

<Cards>
  <Card title="Import files in the console" href="/collect/import-files" icon="import">
    Add one run's files from your browser, hashed before they upload.
  </Card>

  <Card title="Connect a storage source" href="/collect/sources" icon="database">
    Read new files from the buckets, containers and shares you already run.
  </Card>

  <Card title="The companion collector" href="/collect/collector" icon="cpu">
    Journal units and process samples from each vehicle, spooled while offline.
  </Card>

  <Card title="MAVLink ground receiver" href="/collect/receiver" icon="antenna">
    Record what range control heard, on the receiver's own clock.
  </Card>

  <Card title="Supported file formats" href="/collect/formats" icon="file">
    The formats Foxborne reads and how it recognizes each one.
  </Card>

  <Card title="Import statuses and fixes" href="/collect/troubleshoot-imports" icon="alert">
    What Partial, Quarantined and Rejected mean, and what to do.
  </Card>
</Cards>
