Libraries and Validation

Steps and procedures live in libraries: versioned packages of the configuration repository, next to the catalogue packages they depend on. A library records in a lock file which steps and procedures were validated, and against what. Changing a shared step, or a catalogue, marks every procedure that depends on it for revalidation; the review of the merge request that updates the lock is the validation. The MCS imposes no editing role: the review process of the team on the Git repository decides.

Layout#

packages/
  platform-v3/catalogue.yaml       catalogue packages
  lab-psu/catalogue.yaml
  platform-v3-steps/               a library
    package.yaml                   its manifest
    specification.proc             steps and procedures, in any number of files
    operations.proc
    transfers.proc
    stellar.lock                   validated versions, written by `stellar lock`

Every directory under packages/ is one package version. The shared steps of a platform are typically in a library versioned with its catalogue: platform-v3-steps@1.4.x for platform-v3@1.4.x.

Manifest: package.yaml#

# Steps and procedures of platform-v3, with the bench power supply.
package: platform-v3-steps
version: 1.4.2
status: released
depends:
  platform-v3: 1.4.x
  lab-psu: 1.x

A version requirement is an exact version (1.4.0), any patch of a minor version (1.4.x) or any version of a major one (1.x, or 1). The highest catalogue version of the repository that satisfies it is used (package::unresolved when none does). The platform of a uses must be one of these dependencies (procedure::unknown-platform).

stellar schema library prints the JSON Schema of the manifest, for editors and CI.

A role of a package also accepts a target whose platform imports components of that package, in the version the library was compiled against: the procedures of a subsystem run unchanged on the platform that integrates it, under the names of the package, and their lock stays valid. A procedure that uses a component the platform does not import is refused (run::not-imported). See Imports, drivers and procedures.

Names in a library#

Step names are unique in a library, and so are procedure names; do and run resolve them in the library of the caller. A run request names a procedure; when several libraries of the snapshot have a procedure of that name, it also names the library (library:, see Running Procedures).

The lock: stellar.lock#

The lock records, like a Cargo.lock, the hash of the compiled form (IR) of every validated step and procedure, and the hashes of all its dependencies, direct or not: the steps it calls, the sub-procedures it runs, the catalogues of its roles.

# Generated by `stellar lock`: IR hashes of the validated steps and procedures of this
# library and of their dependencies. Updating it validates them again: review it.
version: 1
catalogues:
  lab-psu@1.0.0: e39a56b1df97b0432a111efe4903e9b3ab4a4ff61371cf52842907bf23bee3ad
  platform-v3@1.4.0: fdd1e09c64a7f12e2e85ab3d05d92999ec2767b25ef05cd17117d376ee9bb773
steps:
  TCU answers:
    hash: d2a5613044bb5efae6eeb8330d2a9de609ecef14324614379316c7e83dd921d2
    depends:
      catalogue platform-v3@1.4.0: fdd1e09c64a7f12e2e85ab3d05d92999ec2767b25ef05cd17117d376ee9bb773
procedures:
  Check TCU:
    hash: 9d8f34a8d94a2707fdbc01eec60d7c9ac750dbe3d7bbdca5666bdef2d2c26ef5
    depends:
      catalogue platform-v3@1.4.0: fdd1e09c64a7f12e2e85ab3d05d92999ec2767b25ef05cd17117d376ee9bb773
      step TCU answers: d2a5613044bb5efae6eeb8330d2a9de609ecef14324614379316c7e83dd921d2
      step TCU in mode: f63fc53a0375547d229da6c543063ff0a40c346f33f79942a76851cf031f1563

The IR holds neither positions in the sources nor comments nor descriptions: reformatting a file, commenting it or rewording a description changes no hash, so it needs no revalidation.

What must be validated again#

A step or procedure must be validated again when:

Changing a shared step therefore signals every procedure that uses it, even through sub-procedures.

stellar check examples/config

To validate again (`stellar lock` records the validation):
  platform-v3-steps: step `TCU answers`: it changed
  platform-v3-steps: procedure `AOS acquisition`: step `TCU answers` changed
  platform-v3-steps: procedure `Check TCU`: step `TCU answers` changed
  platform-v3-steps: procedure `Hot standby test`: step `TCU answers` changed
Checked 4 catalogues, 1 library, the topology, 2 simulations (4 steps, 9 procedures): 0 errors, 0 warnings.

With --format json, each revalidation is a JSON line {"revalidate": "procedure", "library": …, "name": …, "reasons": […]} after the diagnostics.

The validation workflow#

flowchart LR
    A[Edit steps or<br/>catalogues] --> B[stellar check]
    B --> C[stellar lock]
    C --> D[Merge request:<br/>the lock diff lists<br/>what is validated]
    D --> E[CI: stellar check --locked]
    E --> F[Review and merge]
    F --> G[CI: stellar compile --publish]

1. The author changes catalogues, steps or procedures and runs stellar check until it reports no error; it lists what must be validated again.
2. Once the change is ready for review, stellar lock rewrites every stellar.lock (it refuses while there are errors).
3. The merge request shows the diff of the lock: every step and procedure it validates again, including those that only use a changed step. Approving the merge request is the validation.
4. CI runs stellar check --locked, which fails on any error and on any change whose lock was not updated: an unreviewed change cannot be merged.
5. After the merge, the CI of the default branch publishes the snapshot with stellar compile --publish; the reconciler applies it at the next safe point of each target.

See Compilation, Snapshots and Locks for the CI configuration of GitHub and GitLab.

Diagnostics of libraries#