# Stream logs to Foxborne

URL: /collect/streaming

Send logs from companion computers, ground stations and buckets without a manual export: the HTTPS ingest API, an S3 bucket watch, journald, syslog, OTLP, Fluent Bit or Vector, or a MAVLink log-pull agent. Each endpoint authenticates with certificates or an IAM role.



Streaming paths deliver companion and ground logs without anyone exporting a file after the run. Each path speaks a protocol your hosts may already run: stock systemd, syslog over TLS, OpenTelemetry or the Fluent Bit and Vector agents.

Every path is an endpoint on the **Ingest** tab of **Integrations**, at `/app/integrations/ingest`: "Endpoints Foxborne listens on, and the files it imports." The connections Foxborne reads from itself, such as a log bucket or an SFTP directory, stay on **Sources**.

## Endpoint states [#endpoint-states]

Each endpoint shows its **State** for your deployment. Select a row for its drawer, where **This deployment** repeats the state and adds **Listens on** and **Used by**.

| State                       | What it means                                                                                         |
| --------------------------- | ----------------------------------------------------------------------------------------------------- |
| **Listening**               | The endpoint accepts senders. **Used by** names the source that uses it.                              |
| **On**, with a format count | File import only: the formats your deployment reads, such as **On, 4 formats**.                       |
| **Off**                     | "Nothing listens for this protocol, so no port is open for it." An admin turns it on from its drawer. |
| **Later phase**             | "A later phase. Nothing listens yet."                                                                 |

## Before you start [#before-you-start]

* You need the **Admin** role.
* The endpoint your sender needs reads **Listening** on the **Ingest** tab, or you can turn it on as below.
* Each sender has a network path to its endpoint's port.
* Each sender that connects to a receiver has a client certificate, because every machine endpoint uses mutual TLS.
* For an S3 bucket watch, Foxborne has an IAM role, and a customer managed KMS key if the queue is encrypted.

## Choose a path [#choose-a-path]

| Endpoint                   | Protocol                                                        | Auth                                                    | Air-gapped                          | State in the example dataset                    |
| -------------------------- | --------------------------------------------------------------- | ------------------------------------------------------- | ----------------------------------- | ----------------------------------------------- |
| HTTPS ingest API           | HTTPS POST, chunked and resumable, Content-Digest               | mTLS device certificates plus short-lived bearer tokens | Yes                                 | **Listening** on HTTPS 443, `/api/fx/v1/ingest` |
| S3 bucket watch            | S3 ObjectCreated to SQS                                         | IAM role; customer managed KMS key for encrypted queues | No; use a watched directory on-prem | **Listening** on an SQS queue in us-gov-west-1  |
| Fluent Bit or Vector agent | `http`, `s3` or `opentelemetry` outputs                         | Client certificate, bearer                              | Yes                                 | **Off**                                         |
| OTLP receiver              | OTLP/HTTP on 4318                                               | mTLS                                                    | Yes                                 | **Off**                                         |
| journald receiver          | systemd-journal-remote protocol, HTTPS 19532                    | Client certificates                                     | Yes                                 | **Off**                                         |
| Syslog receiver            | RFC 5425 TLS on 6514                                            | Mutual X.509                                            | Yes                                 | **Off**                                         |
| MAVLink log-pull agent     | MAVLink `LOG_*` or MAVLink FTP over the ground link, then HTTPS | Agent certificate to Foxborne                           | Yes                                 | **Off**                                         |

In the example dataset, the **Bench rig uploads** source uses the HTTPS ingest API. The **Fleet log archive** source uses the S3 bucket watch. **File import**, the last ingest path, reads **On, 4 formats** and is used by **Import files**; see [Supported file formats](/collect/formats).

### HTTPS ingest API [#https-ingest-api]

Test rigs and pipelines push signed bundles over HTTPS. Machines prove who they are with mTLS client certificates, and bearer tokens stay short-lived. [Push bundles with the Ingest API](/collect/sources/ingest-api) covers it in full.

### S3 bucket watch [#s3-bucket-watch]

S3 event notifications (`s3:ObjectCreated:*`) go to a standard SQS queue, and Foxborne reads the queue with an IAM role. A FIFO queue is not a direct destination. Delivery is at least once, typically in seconds and sometimes a minute or longer, so Foxborne deduplicates on bucket, key and version.

The watch needs a path to AWS, so it does not work air-gapped: on premises, use a watched directory. In GovCloud, use the FIPS endpoints.

### journald receiver [#journald-receiver]

`systemd-journal-upload` sends journal entries over HTTPS on port 19532 and resumes from a saved cursor, so companions need only the stock systemd tool. Today, journals reach Foxborne through the [Foxborne collector](/collect/collector). [journalctl JSON exports](/collect/formats/journald) lists the journal fields Foxborne uses.

### Syslog receiver [#syslog-receiver]

The receiver takes RFC 5424 messages over TLS on port 6514, as rsyslog's `omfwd` sends them. Both sides present X.509 certificates.

### OTLP receiver [#otlp-receiver]

Hosts that already run OpenTelemetry Collectors send logs over OTLP/HTTP on port 4318, with mutual TLS, and need no new agent.

### Fluent Bit or Vector agent [#fluent-bit-or-vector-agent]

Fluent Bit is small enough for Jetson companions. It can send to the HTTPS ingest API, the OTLP receiver or a GovCloud bucket. Vector's disk buffers suit intermittent range links.

### MAVLink log-pull agent [#mavlink-log-pull-agent]

A ground-station agent pulls logs after disarm with `LOG_REQUEST_LIST`, `LOG_REQUEST_DATA` and `LOG_REQUEST_END`, or faster with MAVLink FTP. It hands them to the HTTPS ingest API, with its own agent certificate.

The MAVLink specification warns that on some systems `LOG_REQUEST_LIST` may stop onboard logging until `LOG_REQUEST_END`. The agent therefore never pulls in flight.

## Connect a sender [#connect-a-sender]

<Steps>
  <Step title="Check the endpoint">
    Go to **Integrations** and open the **Ingest** tab. Find the endpoint the sender needs and open its drawer.

    If its **State** reads **Off**, select **Turn on**. The dialog **Turn on the** *endpoint&#x2A; &#x2A;*?** says where Foxborne will listen and that only senders with a client certificate it trusts are accepted. Confirm with **Turn on**. The endpoint then reads **Listening**, the drawer's **Listens on** gives the address, and the audit log records **Turned on ingest endpoint**.

    | Endpoint                   | **Listens on**, once on                 |
    | -------------------------- | --------------------------------------- |
    | Fluent Bit or Vector agent | HTTPS 443                               |
    | OTLP receiver              | OTLP/HTTP 4318, `/v1/logs`              |
    | journald receiver          | HTTPS 19532                             |
    | Syslog receiver            | TLS 6514                                |
    | MAVLink log-pull agent     | HTTPS 443, through the HTTPS ingest API |

    An endpoint that is on and used by no source shows **Turn off**, which closes its port; evidence already received stays. **File import** and an endpoint a source uses, such as the HTTPS ingest API for **Bench rig uploads**, have no **Turn off**.
  </Step>

  <Step title="Open the network path">
    Allow the sender to reach the endpoint's port: 443 for the HTTPS ingest API, 19532 for journald, 6514 for syslog or 4318 for OTLP. For the bucket watch, give Foxborne's IAM role access to the queue, and to the KMS key if the queue is encrypted.
  </Step>

  <Step title="Give the sender its certificate">
    Install a client certificate on the sender. The endpoint accepts only senders whose certificates it trusts, over TLS 1.2 or later.
  </Step>

  <Step title="Point the sender at Foxborne">
    Set the endpoint's address and port in the sender's configuration. The Fluent Bit or Vector agent also carries its bearer token.
  </Step>
</Steps>

## Guardrails on inbound paths [#guardrails-on-inbound-paths]

* **TLS 1.2 or later**, with TLS 1.3 supported, on every connection.
* **Mutual TLS** on every machine endpoint.
* **No program or vehicle names** in cloud resource names, tags or subjects. Name the bucket watch's queue and its tags generically.
* **SHA-256 on arrival.** "Every file is hashed with SHA-256 at receipt and checked against the sender's manifest before it is parsed."

## Check the result [#check-the-result]

Streamed evidence lands where imported evidence does. Foxborne matches it to runs by vehicle and time, and the run page lists each original under **Source files** with its SHA-256, parser and clock.

Ingest completing for a sortie is also the trigger for automation A1, which closes the sortie out and runs the rules. See [Automations](/integrate/automations).

## TENA gateway [#tena-gateway]

<Availability status="planned" /> The TENA gateway, an adapter for the DoD test-range middleware, is a later phase, and nothing listens for it yet.

## Next [#next]

* [Push bundles with the Ingest API](/collect/sources/ingest-api) covers the HTTPS path for test rigs and pipelines.
* [Connect a storage source](/collect/sources) reads the buckets and shares logs already land in.
* [Integrations overview](/integrate) lists every ingest endpoint beside the outbound catalog.