Questions, Decisions and Control

A run sometimes waits for a person: a question of the procedure (ask operator), or a decision about a step that failed or was interrupted. An operator may also act on a run at any time: suspend it, resume it, or abort it. Every answer and command is traced in the log with the identity of whoever gave it.

What a run waits for#

stellar watch prints the command expected under the event, and stellar status shows the question or decision pending with its path.

Answering#

stellar answer <run> yes             # confirm
stellar answer <run> no              # refuse: the step fails
stellar answer <run> '27.9 V'        # a typed answer
stellar answer <run> true
stellar answer <run> replay          # a decision: replay, skip or abort

stellar answer finds the step waiting in the state of the run; --path names another step. yes and no answer accepted: true or false; any other word is the value of an accepted answer (parsed as JSON when it can be: 3, true; a string otherwise: 27.9 V, STANDBY, replay).

Through the API:

curl -X POST http://localhost:8080/v1/runs/<run>/answers -H 'X-Stellar-User: alice' \
  -d '{"path": "Hot standby test / Power the PPU", "accepted": true, "value": "27.9 V"}'

The API answers 202 once the answer is published to the executor on stellar.run.approval.<run_id>, with the identity of the caller, its roles and, for a verified token, the token itself. The executor decides whether the answer is taken.

Confirmations and typed answers#

• A confirmation answered no (accepted: false) fails the step, which is not retried.
• A typed answer is converted into the type and unit of the answer: a quantity into its unit (27900 mV for an f32 V), a number, a boolean, an enum value. An answer missing or of the wrong type fails the step.
• The answer is a name of the step from then on:

step "Supply voltage"
  uses psu: lab-psu
  ask operator "Voltage read on the bench multimeter?" as f32 V into measured
  check psu.voltage is measured +/- 0.5 V

• Without human orchestration (human_orchestration: off, typically SIM and AIT), a confirmation is acknowledged automatically: the log shows answered without by, and the run does not stop. A typed answer then fails the step: no operator is expected to give it.

Answers not taken#

The executor checks each answer with the policies of the environment of the run. An answer that does not qualify is logged as answer_refused, with who answered and why, and the run keeps waiting:

• with identity: jwt, no token, a token that does not verify, or the token of someone else;
• for a hazardous confirmation, someone without the role awaited, or someone who already confirmed this step.

  answer of bob not taken: `bob` is not supervisor

See Hazardous Confirmations.

Decisions#

A step waits for a decision (decision_required) in two cases:

• a step not eligible for retry (it sends a telecommand that changes the on-board state) failed: the reason is the step failed: …;
• a step was interrupted by a restart of the executor after a telecommand that changes the on-board state: the executor cannot know whether the telecommand acted.

A decision answered no, or with anything but replay, skip or fail, aborts the run. Decisions are asked in every environment, human orchestration or not: only a person can say whether replaying a state-changing telecommand is safe. A dry run is the exception: its targets are simulated and nobody watches it, so it answers by itself, fail by default (stellar run --dry --on-decision, see Dry runs).

Suspend, resume, abort#

stellar suspend <run>
stellar resume <run>                  # --choice replay (default), skip, fail or abort
stellar abort <run>

Details:

• Suspension and abort take effect between two statements: a telecommand already sent finishes its acknowledgement chain, and no new one leaves.
• The choices of a resumption: replay replays the step where the run stopped (a new attempt), skip counts it as passed, fail counts it as failed — the run goes on as after any failure, its if failed blocks included —, abort ends the run.
• A resumption whose targets are not ready, or held by another run, is refused, and the run stays suspended.
• A suspended run can be aborted with stellar abort, or resumed with --choice abort.
• Live hazardous confirmations of a replayed step are asked again.
• The log traces suspended (with the step and the operator) and continued (with the choice); an abort ends with finished and the reason aborted by <who>; a step failed by the choice reads suspended by <who suspended>, failed by <who resumed>.
• A run taken over by another executor while suspended stays suspended.
• An alarm reaction suspends the run holding its target (by = alarm <key>) before its own run starts; see Alarm Handling.

The API sends the command to the executor holding the run, on stellar.run.control.<run_id>, and waits for its reply:

Identity#

Answers and commands need the identity of the operator: a token (--token, STELLAR_TOKEN, Authorization: Bearer) or, where the environment allows it, a declared identity (--as, --role; X-Stellar-User, X-Stellar-Role). The environment of the run applies, as at launch:

• an answer or a command without any identity is refused (api::anonymous);
• with identity: jwt, a declared identity is refused (auth::jwt-required).

See Identity and Roles.