Compilation, Snapshots and Locks

One compiler serves the CI, the CLI, the language server and the web editor. It reads the catalogues, the topology, the libraries of steps and procedures and the simulations of a configuration repository, checks them all, and compiles them into one typed intermediate representation (IR) that the services read. No service ever parses a source file.

What the compiler checks#

Errors are found before anything runs:

• Cross-references: measures, telecommands, enums, components, roles, targets, links, procedures called, packages and their versions.
• Types and units: inputs, arguments, expressions and conditions. Subtracting °C from volts, or passing 300 V where the range is [0 V, 60 V], is an error.
• Safety: every send of a hazardous telecommand is preceded by its confirmation; a retry only on steps whose telecommands do not change the on-board state; no recursion between procedures; alarm reactions send no hazardous telecommand. See Safety Rules and Maximum Duration.
• Bounded durations: every wait has a literal timeout, so the maximum duration of a procedure is known before launch.
• Environment policies: draft catalogues where they are refused, links and parameters by environment, gateway constraints.
• Modes and parameters: the setter and readback of each parameter of a catalogue, the value of each parameter of a target in each declared mode, the modes named by configure and param (catalogue::parameter-*, catalogue::unknown-setter, topology::unknown-mode, procedure::nothing-to-configure…). See Modes and Parameters.
• Simulations: compiled against the catalogue of their platform.

Diagnostics are written for operators: a plain message, the faulty line underlined, and a suggestion when there is one.

procedure::missing-argument

  × telecommand `set_voltage` needs `voltage`
   ╭─[test.proc:6:3]
 5 │   send set_voltage to sat.tcu[tcu]
 6 │   send set_voltage to psu
   ·   ───────────────────────
 7 │   send set_voltage to psu with voltage = 5 A
   ╰────
  help: add `with voltage = …`

A .proc file with an error only loses its faulty line: all the errors of a file are reported at once. A warning (for instance a hazardous telecommand without verify) does not prevent the compilation.

Checking a repository#

stellar check examples/config

stellar check compiles everything, prints the diagnostics, then lists what must be validated again, and ends with a summary:

To validate again (`stellar lock` records the validation):
  platform-v3-steps: procedure `Check TCU`: step `TCU answers` changed
  platform-v3-steps: step `TCU answers`: it changed
Checked 4 catalogues, 1 library, the topology, 2 simulations (4 steps, 9 procedures): 0 errors, 0 warnings.

It exits with code 1 on an error, and with --locked also on a pending validation.

JSON output#

--format json prints one JSON object per line on the standard output: first the diagnostics, then the revalidations. Positions follow the Language Server Protocol (zero-based line, column in UTF-16 code units), so that tools can place them in any editor.

{"severity":"error","code":"procedure::missing-argument","message":"telecommand `set_voltage` needs `voltage`","help":"add `with voltage = …`","file":"packages/platform-v3-steps/power.proc","range":{"start":{"line":5,"character":2},"end":{"line":5,"character":25}}}
{"revalidate":"step","library":"platform-v3-steps","name":"TCU answers","reasons":["it changed"]}

Compilation parameters#

Two values of the global configuration are part of what is compiled and validated, so the CI must use the same values as the deployment:

Pass the file with --config or STELLAR_CONFIG; the defaults apply without it.

The snapshot#

stellar compile produces the snapshot: the whole compiled configuration.

stellar compile examples/config                                   # writes <hash>.ir
stellar compile examples/config --output snapshot.ir
stellar compile examples/config --publish nats://nats.example:4222 \
    --nats-credentials ci.creds --nats-ca ca.pem

It prints the hash on the standard output and refuses to write anything when there is an error.

Content of the IR#

• The catalogues, with the standard components added: link (one instance per link of the target, see COP-1 and the Link Component), files and stream where declared.
• The topology: environments, targets, links, parameters, connectors.
• The libraries: steps and procedures, with every reference resolved (role, component, instance, measure), durations as constants, and each call carrying its roles and inputs explicitly, implicit passing included.

The IR holds no source position and no comment: reformatting a file does not change its hash.

Hash and descriptions#

The IR is encoded with postcard and addressed by the SHA-256 of its bytes. Components check an IR against its hash when they read it.

Descriptions — the description fields of catalogues and the description "…" lines of steps and procedures — travel beside the IR, in the snapshot (descriptions). Changing a description changes the configuration revision, but neither the hash of a catalogue nor the lock of a step: rewording an explanation for operators never invalidates a validated procedure, nor the binding of a driver.

Publication#

--publish does two things:

1. stores the snapshot in the stellar_ir object store, under its hash;
2. points the key current of the stellar_config bucket at it: {"hash": "…", "published_at": "…", "publisher": "<$USER>"}.

The stellar_config bucket keeps the last ten pointers: to go back to a previous configuration, publish it again.

Applying a snapshot: safe points#

The leader of the reconciler watches current, loads the snapshot and gives each target a revision, published in stellar_readiness:

• a target held by a run (a lease in stellar_leases) keeps its revision until the lease is released;
• every other target moves to the new snapshot at once.

The components bound to a target stamp its revision on their messages (Stellar-Config header): the driver decodes, and the compute stage calibrates, with the catalogue of the revision of the target. A run is resolved against the current snapshot at launch and keeps the resolved IR it was given (archived in stellar_ir) until its end, whatever is published meanwhile.

Locks and revalidation#

Each library carries a stellar.lock file next to its package.yaml, like a Cargo.lock. It records the hash of the IR of each validated step and procedure, and those of all their dependencies, direct or not: steps called, sub-procedures, catalogues of their roles.

# Generated by `stellar lock`: IR hashes of the validated steps and procedures of this
# library and of their dependencies. Updating it validates them again: review it.
version: 1
catalogues:
  lab-psu@1.0.0: e39a56b1df97b0432a111efe4903e9b3ab4a4ff61371cf52842907bf23bee3ad
  platform-v3@1.4.0: fdd1e09c64a7f12e2e85ab3d05d92999ec2767b25ef05cd17117d376ee9bb773
steps:
  TCU answers:
    hash: d2a5613044bb5efae6eeb8330d2a9de609ecef14324614379316c7e83dd921d2
    depends:
      catalogue platform-v3@1.4.0: fdd1e09c64a7f12e2e85ab3d05d92999ec2767b25ef05cd17117d376ee9bb773
procedures:
  AOS acquisition:
    hash: 6678a2786959aa08a7e730c12260ac8774a343b2b36030f5f264845bc9028cc6
    depends:
      catalogue platform-v3@1.4.0: fdd1e09c64a7f12e2e85ab3d05d92999ec2767b25ef05cd17117d376ee9bb773
      step TCU answers: d2a5613044bb5efae6eeb8330d2a9de609ecef14324614379316c7e83dd921d2

A step or procedure must be validated again when:

Changing a shared step therefore flags every procedure that uses it, directly or through other procedures. A catalogue republished in place under the same version also changes its hash and is caught.

stellar lock rewrites the lock of every library from the current sources (it refuses to while there are errors) and prints, for each library, up to date or what it validated. The lock diff goes through the review of the merge request, and that approval is the validation. In CI, stellar check --locked fails as long as a lock is not up to date. See Libraries and Validation.

JSON Schemas#

The YAML files are validated against JSON Schemas generated from the types of the compiler; the VS Code extension installs them. Print them for other tools:

stellar schema catalogue     # catalogue.yaml
stellar schema library       # package.yaml
stellar schema topology      # topology.yaml
stellar schema simulation    # simulations/*.yaml

The same command prints the schemas of the messages of the NATS contract (registration, tc-event, run-event, sample…); see NATS Contract Reference.