A scenario's assertions decide whether a run passed. SatLink separates two questions for every
assertion: did the condition hold (passed), and does the verdict rest on something the board
actually measured (verified)? A run passes only if its assertions pass and are verified.
Assertion types#
type | Passes when | Verified when |
|---|---|---|
metric_threshold | A metric, reduced over the run or a window, compares true against value | The metric was measured on live PL registers |
packet_seen | At least min_count (default 1) packets were recovered from the RX stream | The datapath was attached and acquired, so the RX side was really watched |
packet_seen_after_event | A packet was recovered within within after the event named by event_ref | The datapath was attached and acquired |
metric_threshold#
- name: "the_mute_actually_cut_the_link"
type: "metric_threshold"
metric: "rx.lock.max"
operator: "<="
value: 0.0
during:
step: "link_down"
settle: "3s"| Field | Default | Meaning |
|---|---|---|
metric | — | What to measure (below) |
operator | <= | <=, >=, <, >, == (or =) |
value | 0 | The threshold. Decimal or hex (value: 0xFF01), so a register value is written once |
during | whole run | The interval the metric is reduced over (see Windows) |
packet_seen and packet_seen_after_event#
- name: "the_isolated_telecommand_came_back_whole"
type: "packet_seen"
min_count: 1
- name: "tm_after_tc"
type: "packet_seen_after_event"
event_ref: "tc" # the id of a timeline step
within: "5s"packet_seen counts packets recovered from the receive stream, not packets the scenario sent
or expected. Its result distinguishes three states:
| State | Result |
|---|---|
| No datapath in the daemon | Not verified: nothing watched the link |
| Datapath present but not acquired when the run started | Not verified, and not a statement about the link. Enable the datapath (or let chain conditioning do it) |
| Datapath watched | Verified; passes or fails on the count |
Measurable metrics#
| Namespace | Metrics | Source |
|---|---|---|
rx.* | evm, snr_db, mer_db, ber, ber_asm, cfo_hz, phase_err, rssi_dbm, part_rssi_db, part_rx_gain_db, agc_gain, lock, carrier_lock | The telemetry samples collected during the run (see Telemetry and Metrics) |
chan.* | ctrl, atten, awgn_amp, nco_freq, burst_rate | The channel emulator's active bank, read back on every sample |
reg.<name> | Any register listed in [telemetry] watch_registers | Sampled on the same tick as the other metrics |
rf.* | ber, frame_error_rate, snr_db, doppler_hz | Model values computed from what the scenario asked for. Never verified |
Each measured metric takes an optional aggregate suffix:
| Suffix | Value |
|---|---|
.last (default) | The last sample in the window |
.min, .max, .mean | Over the samples in the window |
.delta | max − min: how much a cumulative counter moved during the window |
Notes on specific metrics:
rx.lockandrx.carrier_lockare booleans exposed as 0/1, so.meanis the fraction of samples locked: "stayed locked" isrx.lock.mean >= 1.0, "never locked" isrx.lock.max <= 0.lockis the timing loop;carrier_lockis the carrier loop, a different fact.- Frame counters are cumulative and never reset between profiles. Use
reg.rx_demod.deframer.frame_cnt.delta(frames delimited during the window) andreg.tx.framer.frame_cnt.delta(frames transmitted);.maxwould pass on counts from an earlier run. rx.agc_gain.max <= 15says the receiver was not starved. Without it, a starved run and a healthy one produce otherwise identical reports.rx.evmreads the QPSK or BPSK demodulator; on a GMSK profile it is meaningless. Assert on frame counters, or onreg.rx_demod.demod_gmsk.status.- A register not listed in
watch_registersis unresolvable, not zero, and the assertion is reported not verified. Scenarios cannot add registers to the watch list; the daemon's configuration does.
An unknown metric name is a scenario error: the assertion fails, not verified, and the message lists the measurable metrics.
Windows#
Without during, a metric is reduced over the whole run, and an assertion can rarely test the
condition it is named after: "the mute released the lock" becomes "the lock dropped at some
point", which a single dropout anywhere, or an inert channel, satisfies. Give assertions a window.
during: { step: "feed", settle: "4s" } # the window of a timeline step
during: { from: "3s", to: "4s" } # explicit bounds
during: { from: "30s" } # to the end of the run| Field | Meaning |
|---|---|
step | A timeline step's id. A span (from/to) gives its window; an instant step (at) opens a window that lasts until the next step starts, or to the end of the run |
from, to | Explicit bounds; to is exclusive and optional |
settle | Time to ignore after the window opens, for the effect to establish itself |
step and from/to cannot be combined. A window that names no step, is empty, or is entirely
consumed by its settle makes the assertion fail not verified, never falls back to the whole
run. A window in which no telemetry sample falls is likewise not verified.
Passed, verified, and the run verdict#
| Assertion result | Counts toward a pass? |
|---|---|
| Passed and verified | Yes |
| Failed and verified | No: the board measured a failure |
| Not verified (model metric, no sampler, stub samples, empty window, no datapath) | No: "we did not look" is not evidence |
pass_criteria.mode: all (the default) requires every assertion to pass and be verified;
any requires one. A scenario with no assertions never passes.
A telemetry sample taken while no bitstream answers (every register reads zero) is marked stub;
an assertion resting only on stub samples is not verified.
The run report#
GET /api/v1/reports/{run_id} (also GET /api/v1/scenario-runs/{run_id} for the summary)
returns the report of a finished run:
| Field | Meaning |
|---|---|
run_id, scenario_name, profile_ref | What ran |
status | running, passed, failed, cancelled |
started_at, ended_at, duration_ms | Timing (board clock, see below) |
steps_executed, assertions_passed, assertions_failed | Counts |
verified | Whether any assertion read the link rather than the script |
aborted | The run did not reach the end of its timeline |
source | Where the numbers came from: pl, stub, mixed or none |
events | The event log, in time order |
assertions | One result per assertion: name, passed, verified, message |
metric_samples | Every telemetry sample of the run (see Telemetry and Metrics) |
sample_interval_ms | The sampler's cadence; absent means no sampler was attached |
An assertion message shows what was measured, for example:
[measured, 105 samples over 7.0s..25.0s (after 4.0s of settling)] rx.agc_gain.max = 9.4580e0 <= 1.5000e1 → PASSEvents#
kind | Meaning |
|---|---|
scenario_start, scenario_end | Run boundaries |
profile_applied | The profile was applied (and the channel reset to pass-through) |
channel_set, channel_ramp, channel_curve | A channel change reached the datapath |
channel_staged | A channel change was written but not committed: it had no effect |
receiver_cleared, receiver_clear_unavailable | No longer emitted. Older reports carry them: the engine used to flush the receiver 500 ms into a traffic step. The PL now re-arms the receiver itself, at each profile commit and at each burst onset |
traffic_started, traffic_stopped | The built-in feed, with what it sent and how many packets |
traffic_refused | A feed was asked for and did not start: the run measured silence |
packet_injected | An inject_packet step ran |
packet_transmitted | Its bytes were actually queued on the datapath (with the egress state) |
packet_expected, packet_observed | An expect_packet step, and a matching packet actually recovered |
link_opened, link_unavailable | Scenario endpoints bound, or not |
tone_on, tone_refused | A tone was transmitted, or refused |
Read the event log of every new scenario's first run: a *_staged, *_refused or
*_unavailable event means the timeline did not do what the file says.
Time in reports#
Run times (started_at, frame times) come from the board's clock, which starts at 1970 at every
power-on unless it has been set (see Hardware Setup).
Event and sample times are elapsed_ms since the start of the run, which is always reliable.
Keeping reports#
The run store is on the board's RAM disk: reports are lost at reboot. Save the ones you need:
curl -s $SATLINK_API/api/v1/reports/<run-id> > report-<run-id>.jsonThe web console's report page can export a report as JSON too. See Analysis & Reporting.