A step that fails can be tried again automatically, typically to acquire the link at the start of a pass. Retries are safe by construction: only steps whose telecommands have no effect on board are retried, every attempt is bounded, and every attempt keeps its own evidence in the log.
Grammar#
retry [<N> times] [every <duration>] [for <duration>]| Part | Meaning |
|---|---|
<N> times (1 time, 3 times) | At most N new attempts after the first |
for <duration> | Total duration of the attempts: no attempt starts that could not end within it |
every <duration> | Interval between the starts of two attempts; without it, the next attempt starts at once |
A count, a total duration or both are required: every alone is refused
(procedure::unbounded-retry). With both limits, the first reached stops the attempts. The
durations are literals.
A policy is written in two places:
# On the step: its own policy, wherever it is called.
step "TCU answers" retry 3 times every 2 s
uses sat: platform-v3
input tcu: tcu_id
send ping to sat.tcu[tcu] via direct
expect sat.tcu[tcu].responding is true within 5 s
# On the call: overrides the policy of the step, for this call only.
procedure "AOS acquisition"
uses sat: platform-v3
input tcu: tcu_id
do "TCU answers" with sat, tcu retry every 5 s for 3 minretry is accepted on the step line of library and inline steps, and on do and run
calls.
Eligibility#
A step is eligible for retry when every telecommand it sends is declared
changes_state: false in the catalogue; a step without telecommand is eligible too. A procedure
is eligible when every telecommand it sends, directly or through its calls, is
changes_state: false.
A retry on something not eligible is a compile error, on the step line or on the call
(procedure::retry-not-allowed):
procedure::retry-not-allowed
× this step cannot be retried: it sends telecommands that change the on-board state
╭─[test.proc:1:16]
1 │ step "Standby" retry 2 times
· ─────────────changes_state is true by default in the catalogue: only a telecommand explicitly declared
without effect is ever replayed automatically. See
Telecommands and Verification.
Which policy applies#
For a do call or an inline step:
- the
retryof the call, if any; - else the
retryof the step; - else, for an eligible step only, the default policy of the global configuration,
executor.default_retry({times: 1, every: 2s}by default: one new attempt, 2 s after the start of the first); - else no retry.
A run call is retried only with its own retry, and only if the sub-procedure is eligible;
the default policy never applies to it. A retry of a run replays the whole sub-procedure.
# stellar.yaml: the default policy of eligible steps.
executor:
default_retry: {times: 1, every: 2s} # also `for: 1 min`The default policy is a compilation parameter too: stellar check and stellar compile read it
from the global configuration (--config, STELLAR_CONFIG) to compute the maximum durations.
What triggers a new attempt#
| Failure | New attempt |
|---|---|
expect, check or wait until not satisfied | yes |
Telecommand REJECTED (a requires not met), SEND_FAILED, VERIFY_TIMEOUT | yes |
Telecommand ENCODE_FAILED or VERIFY_FAILED | no |
ask refused, transfer that cannot be created | no |
A failure that is not retried, or the last attempt failing, fails the step.
A step that is not eligible and fails does not simply fail: the run waits for an operator
decision (decision_required): replay it, skip it (the step counts as passed), or abort
the run. See Questions, Decisions and Control.
How attempts run#
- Each attempt is numbered in the log (
step_startedandstep_finishedcarryattempt). - Each attempt sends its telecommands with new identifiers; each carries
retry_of, the identifier of the telecommand at the same place in the first attempt, to relate them in the report. - Each attempt keeps its own evidence: the acknowledgements and the samples judged.
everyis measured from the start of each attempt: withevery 5 s, an attempt that took 3 s is followed by the next 2 s later; one that took 7 s by the next at once.- With
for, an attempt starts only if it can end before the limit, its maximum duration counted: the total stays bounded.
Maximum duration with retries#
For a step whose one attempt lasts at most d, and an interval e (zero without every):
| Policy | Bound |
|---|---|
N times | d + N × max(d, e) |
for T | max(d, T) |
| both | the smaller of the two |
For example, "TCU answers" sends ping, whose verification window is 5 s, then expects within
5 s: d = 10 s. With retry 3 times every 2 s, it lasts at most 10 + 3 × 10 = 40 s. Called with
retry every 5 s for 3 min, at most 3 min. See Safety Rules and Maximum
Duration for the rest of the computation.