Deployment Options

Stellar Control is a handful of stateless services around one NATS server with JetStream. The same parts make every deployment, from a test laptop without network to a platform in the cloud serving benches on several sites: what changes is where each part runs and which networks the traffic crosses.

Choosing#

What to run#

Two rules hold in every topology:

• Every connection to NATS goes out from its client. The core, the edges and the CLI connect to NATS; NATS connects to nothing (but to the hub, from a leaf node). A bench needs no inbound port.
• The configuration is published, not deployed. Each change of the configuration repository is compiled and published with stellar compile <repository> --publish <nats-url>, from a CI job, a workstation or a one-shot container; the services pick it up at their next safe point.

NATS URLs take four schemes, in the global configuration of the core (nats.url) as in the environment of the edges (STELLAR_NATS_URL):

Topologies#

100 % cloud#

Everything runs in a cloud (or a data centre): the core, NATS, the simulators and connectors. The operators reach the console through a reverse proxy over HTTPS. Suited to engineering phases on simulated targets, to training and to data processing.

            Internet                     Cloud (private network)
  ┌───────────────────┐  HTTPS 443   ┌───────────────────────────────────────────────┐
  │ browser, CLI      │─────────────►│ reverse proxy ──► stellar-api :8080           │
  └───────────────────┘              │                     │                         │
                                     │ core services ──────┴──► NATS + JetStream :4222│
                                     │ simulators, connectors ──┘   (private)         │
                                     └───────────────────────────────────────────────┘

Hybrid: the core in the cloud, the benches on site#

The core and NATS run in the cloud; the gateways and drivers run on site, next to the benches and the EGSE, and connect out to NATS over tls:// on 4222, or wss:// on 443 when the site only lets HTTPS out. Each edge has the bootstrap credentials of the platform and gets its own JWT from the reconciler (see NATS Accounts and Credentials).

  Site (bench network)                       Cloud
  ┌──────────────────────┐                   ┌────────────────────────────────┐
  │ bench ◄─► gateway    │    tls:// 4222 or │ NATS + JetStream ◄── core      │
  │ EGSE  ◄─► driver     │ wss:// 443 ─────► │ :4222, :443 (TLS)     services │
  │  (outgoing only)     │                   │ reverse proxy ──► stellar-api  │
  └──────────────────────┘                   └────────────────────────────────┘
  Operators: browser, CLI ─── HTTPS 443 ──────────────► reverse proxy

A site with many edges can run a NATS leaf node of its own: the edges connect to it on the bench network, and only the leaf node crosses the firewall (see Edges behind a firewall).

On-premise#

Everything runs on the networks of the customer: the core and NATS on a server or a cluster of the IT department, the edges on the bench and test networks, routed to NATS through the internal firewalls. Nothing leaves the site; the images or binaries come from the delivery of Stellar Systems, kept in a local registry or repository.

  Customer site
  ┌───────────────────────────────────────────────────────────────────────┐
  │  Server network                      Bench / test networks            │
  │  ┌──────────────────────────┐ tls:// ┌───────────────────────────┐    │
  │  │ NATS + JetStream :4222   │◄───────│ gateways, drivers, EGSE   │    │
  │  │ core services            │  4222  └───────────────────────────┘    │
  │  │ stellar-api :8080        │◄─── HTTP(S) ─── operator workstations   │
  │  └──────────────────────────┘                (browser, CLI)           │
  │  local registry ── images / binaries of the delivery                  │
  └───────────────────────────────────────────────────────────────────────┘

Local, on an air-gapped test laptop#

One machine without network: NATS, stellar-control and the simulators of the configuration repository, all on the loopback interface. Suited to preparing and checking procedures away from the benches, to demonstrations, and to training. The images are loaded from files of the delivery (docker load -i stellar-control-<version>.tar), or the binaries copied with a nats-server binary; nothing is pulled.

  Laptop (no network)
  ┌────────────────────────────────────────────────────────┐
  │  NATS + JetStream  127.0.0.1:4222                      │
  │        ▲                  ▲                            │
  │        │                  │                            │
  │  stellar-control     stellar-simulator                 │
  │  :8080 ◄── browser, CLI   (one per target)             │
  │                                                        │
  │  stellar run --dry  (its own nats-server, ephemeral)   │
  └────────────────────────────────────────────────────────┘

Docker Compose runs the first part in one command. Dry runs need neither of them: stellar run --dry starts its own nats-server and the services it needs in its process, on ephemeral simulated targets (see Dry runs).

Edges behind a firewall#

During integration, verification and validation (IVV), the benches often sit on a network of the customer, a test centre or a partner, behind a firewall that lets nothing in and only HTTPS out. The NATS server then opens its WebSocket port, with TLS, on 443 (or on the WebSocket port of the server, 8443 in the delivered NATS job, when 443 is taken), and the edges reach it there.

Two ways, both outgoing only from the bench network:

  Bench network (outgoing HTTPS only)          Firewall          Platform
                                                  │
  (a) every edge connects directly                │
  gateway ─┐                                      │
  driver  ─┼──── wss://nats.example.org:443 ──────┼────► NATS WebSocket :443 (TLS)
  sim     ─┘                                      │           │
                                                  │      NATS + JetStream ◄── core
  (b) one leaf node on site                       │           ▲
  gateway ─┐                                      │           │
  driver  ─┼─► nats-server (leaf node) ─ wss:// ──┼───────────┘
  sim     ─┘   nats://127.0.0.1:4222      443     │

• (a) Direct. Each edge sets STELLAR_NATS_URL=wss://nats.example.org:443, with its bootstrap credentials (STELLAR_NATS_CREDENTIALS) and, for a private CA, STELLAR_NATS_CA. The Rust SDK and the simulator speak WebSocket as they are; a component of the Python SDK needs the aiohttp package installed beside it, which the NATS client uses for ws:// and wss://. Simplest with a few edges.
• (b) Leaf node. A NATS server on the bench network connects to the platform as a leaf node, over wss:// on 443, bound to the account of the platform; the edges connect to it on the local network. Only one connection crosses the firewall, whatever the number of edges. See the leaf node documentation of NATS for its configuration.

On the server side, the WebSocket and leaf node ports always use TLS:

websocket {
  port: 8443                      # published on 443
  tls {
    cert_file: /etc/nats/tls/server.pem
    key_file: /etc/nats/tls/server-key.pem
  }
}

leafnodes {
  port: 7422
  tls {
    cert_file: /etc/nats/tls/server.pem
    key_file: /etc/nats/tls/server-key.pem
    ca_file: /etc/nats/tls/ca.pem
  }
}

The delivered Nomad job of NATS opens them with websocket_port and leafnode_port (see Nomad). The client port 4222 stays on the private network. A proxy that inspects TLS must let the WebSocket upgrade through, or be bypassed for the NATS host. See TLS, Encryption and Secrets.

Through the binaries#

Stellar Systems delivers the binaries on request, for hosts that cannot run containers: the services of the core, stellar-control, stellar-editor, the CLI stellar and stellar-simulator, and the files of the web console. NATS comes from its own release (nats-server 2.12). A supervisor runs each service and restarts it when it exits with 1 (see Exit codes).

An outline with systemd, one unit per service (here the API), all reading one global configuration:

# /etc/systemd/system/stellar-api.service
[Unit]
Description=Stellar Control API
After=network-online.target nats-server.service
Wants=network-online.target

[Service]
User=stellar
Environment=STELLAR_CONFIG=/etc/stellar/stellar.yaml
Environment=STELLAR_INSTANCE=api-1
ExecStart=/usr/local/bin/stellar-api
Restart=on-failure
RestartSec=5s
# Stops on SIGTERM, within 2 s
TimeoutStopSec=10s
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
PrivateTmp=true

[Install]
WantedBy=multi-user.target

For a small installation, a single unit runs stellar-control instead of one per service.

The keys are those of the Global Configuration. Publish the configuration once NATS is up (stellar compile <repository> --publish <nats-url>), before or after the services start.

In containers#

The image stellar-control, from the registry given with the delivery, holds every binary above and the web console; NATS runs from its official image (nats:2.12). See Docker for its content.