← 2. Getting started · Contents · Next → 4. A decision

3. Architecture¶

The whole picture¶

Two worlds, joined by a single channel.

flowchart TB
    subgraph SITE["Physical site"]
        RD["Badge reader"] -->|"OSDP / RS-485"| EDGE
        CAM["ANPR camera"] --> EDGE
        QR["QR reader"] --> EDGE
        EDGE["Cabinet<br/>(Rust)"] --> DB[("Encrypted<br/>SQLite")]
        EDGE -->|"dry contact"| BAR["Door"]
        EDGE -.->|"/metrics"| MON["Site monitoring"]
    end
    subgraph CLOUD["Cloud"]
        MQ["EMQX<br/>MQTT broker"]
        BE["Backend<br/>(NestJS)"]
        PG[("PostgreSQL")]
        KC["Keycloak"]
        S3["MinIO / S3"]
        UI["Dashboard<br/>(Angular)"]
        MQ <--> BE
        BE <--> PG
        BE <--> KC
        BE <--> S3
        UI -->|"REST"| BE
        BE -->|"WebSocket"| UI
    end
    EDGE ==>|"MQTT over mTLS, outbound only"| MQ
    EDGE -.->|"presigned HTTPS, firmware only"| S3

Two arrows leave the site, and both are outbound. Nothing ever connects into a customer's network.

The six components¶

The cabinet (Rust)¶

The box on site. It reads credentials, decides, drives the relays and journals. Written in Rust with Tokio, as two crates:

wardn-domain: the decision logic. No I/O, no clock, no database. A total, pure function from (door mode, candidate rights, instant) to a decision.
wardn-edge handles everything that touches the world: the encrypted store, the reader bus, the relay, the MQTT forwarder, telemetry, OTA, the watchdog.

Why Rust? This program runs unattended, in an enclosure, for years, on modest hardware. You want no unpredictable garbage collection, a small memory footprint, and a compiler that rejects the class of bugs that take a service down at three in the morning. A crash here is a door that no longer opens.

Its local store is encrypted SQLite (SQLCipher). Encrypted because the box is physically reachable: someone can open the enclosure and take the SD card, which would otherwise hold badge numbers and plates. It holds hashes rather than values, so even decrypted it does not hand over a list of everyone's plates.

It keeps only active rights, never history. Its database is a cache, rebuildable from the cloud. The one exception is the queue of pending passages. That is the one piece of data the cabinet is the sole holder of, for the duration of an outage.

The broker (EMQX)¶

The post office between cabinets and the cloud.

Why MQTT rather than HTTP? Three reasons. The cabinet connects outwards and keeps the connection open, so no inbound port has to be opened on the customer's router. The protocol handles reconnection and quality of service natively. And QoS 1 guarantees a message is delivered at least once, which is exactly what replaying a backlog of passages needs.

Locally it is EMQX; in production, AWS IoT Core plays the same role with the same protocol. No application code knows the broker's brand.

The backend (NestJS)¶

The brain on the cloud side. It serves the REST API over Fastify, consumes MQTT messages, writes to PostgreSQL, pushes the live feed to the dashboard, keeps the audit trails, and runs the scheduled maintenance.

It is stateless and leaderless. No instance is "the leader": start two, and MQTT shared subscriptions spread the messages between them, with each event handled exactly once.

make scale   # starts a second instance and proves it

Scheduled work runs in every instance: the offline sweep, the retention purge, and the two reconciliation passes. It coordinates through a PostgreSQL advisory lock. Whoever takes the lock does the work. The others move on.

The rights queue is drained differently, and deliberately so: tasks are claimed with FOR UPDATE SKIP LOCKED and a lease, so several instances drain the same queue concurrently instead of taking turns.

Why leaderless? A leader election adds a mechanism that can get it wrong, and a moment when nobody is leader. Shared subscriptions and advisory locks move that problem into components whose job it is.

The database (PostgreSQL)¶

The memory. Rights, passages, audit trails.

The three journal tables are partitioned by month. Deleting a month of data then becomes instant: it is just dropping a partition, instead of a DELETE over millions of rows. That is what makes retention actually enforceable.

They are also immutable at the database level. Three roles exist:

Role	Used by	Privileges
`wardn_owner`	Migrations and the seed	Owns everything
`wardn_app`	The backend's ordinary pool	`SELECT`, `INSERT` on the log tables; full access on the business tables
`wardn_maintenance`	Retention and erasure only	Adds the ability to drop partitions and to rewrite a log row

The application connects as wardn_app, which has UPDATE and DELETE revoked on activity_logs, system_audit_logs and control_plane_audit_logs.

Why at the database level? A rule enforced by code is a rule a future developer will work around without meaning to. A permission refused by PostgreSQL produces an immediate, visible error. make verify asserts it.

Identities (Keycloak)¶

Humans sign in to the dashboard through Keycloak (OpenID Connect, authorization code flow with PKCE). The realm defines two roles:

wardn-user: the default role, held by every operator;
wardn-admin adds one thing: pushing firmware to a cabinet.

The backend accepts any token the realm signed, and enforces exactly one role check: wardn-admin on the OTA endpoint. Everything else is open to any authenticated operator and recorded in the audit trail instead.

Why so few roles? A fine-grained permission matrix is a document nobody maintains and everybody works around with an exception. Here everything is open and traced, except the one gesture whose mistake is most expensive to undo: pushing firmware to a fleet.

Machines use API keys, and a key is confined to a single tenant. See 9. The API and 10. Security.

Object storage (MinIO)¶

Firmware artefacts and technical documents. In production, S3.

The backend never carries a firmware: it signs a temporary link and the cabinet downloads it itself, directly. A multi-megabyte binary has no business passing through an API's memory.

The backend speaks only the little of S3 it needs: read an object, list a prefix, presign a GET. It does this with hand-written Signature V4 rather than the AWS SDK.

The dashboard (Angular)¶

A single-page application. It calls the API over REST and receives the live feed over a WebSocket. Configured at container start, not at build: the image writes a config.js from its environment, holding the addresses a browser dials. See 8. The dashboard.

The message channels¶

Every topic lives under one tree: wardn/devices/{deviceId}/{channel}.

flowchart LR
    subgraph UP["Cabinet → Cloud"]
        E1["events<br/>passages, QoS 1"]
        E2["telemetry<br/>state + heartbeat, QoS 0"]
        E3["debug<br/>raw frames on demand, QoS 0"]
        E4["ack<br/>command and delta receipts, QoS 1"]
        E5["rights/request<br/>send me everything, QoS 1"]
    end
    subgraph DOWN["Cloud → Cabinet"]
        D1["commands<br/>open, restart, debug, OTA…, QoS 1"]
        D2["rights/sync<br/>rights deltas, QoS 1"]
    end

Topic	Direction	QoS	Payload
`wardn/devices/{id}/events`	cabinet → cloud	1	One passage. Detailed in chapter 6
`wardn/devices/{id}/telemetry`	cabinet → cloud	0	The cabinet's state. Chapter 7
`wardn/devices/{id}/debug`	cabinet → cloud	0	One raw diagnostic signal. Never stored
`wardn/devices/{id}/ack`	cabinet → cloud	1	`{refId, status, revision?, error?, timestamp}`
`wardn/devices/{id}/rights/request`	cabinet → cloud	1	Empty. "Send me my whole set"
`wardn/devices/{id}/commands`	cloud → cabinet	1	`{commandId, action, parameters}`. Chapter 7
`wardn/devices/{id}/rights/sync`	cloud → cabinet	1	`{syncId, revision, fullSync, operations[]}`. Chapter 5
`wardn/internal/broadcast`	backend ↔ backend	0	What one instance tells the others to show live

Why those quality levels. Passages, commands and deltas are QoS 1: the broker holds them for a cabinet that is away, and losing one is not recoverable. Telemetry and debug are QoS 0: a heartbeat held by the broker and delivered in a burst on reconnect would describe a cabinet as it was minutes ago, which is worse than the gap it is trying to fill.

The backend subscribes to the cabinet-to-cloud topics through a shared subscription ($share/<group>/wardn/devices/+/…), so each message reaches exactly one instance. It subscribes to wardn/internal/broadcast outside the group, because there every instance must receive it. That is how a passage recorded by instance A reaches a dashboard connected to instance B.

Idempotence, end to end¶

Every passage carries a eventId generated by the cabinet. Replaying it creates no duplicate: activity_logs has a unique index on (event_id, occurred_at) and the insert says ON CONFLICT DO NOTHING.

That is what makes MQTT's "at least once" acceptable. Rather than chasing an "exactly once" nobody knows how to guarantee simply, the duplicate is made harmless.

The same idea runs through the rest:

a rights delta carries a revision; a cabinet ignores anything not ahead of what it already holds, so a redelivery is inert;
provisioning a right is idempotent on (zoneId, externalId);
make up, the migrations and the seed are idempotent, and CI asserts it.

The principles that keep coming back¶

The cloud never decides an opening in real time. The subject of chapter 1, and the reason for the rest.

Communication is outbound only. Nothing connects into a site. No port to open, no fixed address to negotiate with a customer's IT department.

Access rights are immutable. A right is never edited: a new one is created that supersedes it, and the old one is deactivated. History stays readable, and "why could this person get in that day?" always has an answer.

Instants are stored in UTC; the timezone is separate data. Each zone carries its own. A passage is stored in UTC and displayed at the site's local time. Support looking for "the badge at 8am" sees the same hour as the person on the phone.

Hardware sits behind an interface. MockRelay locally, a GPIO relay in a cabinet; a simulated reader bus locally, OSDP on RS-485 in a cabinet. The decision engine does not know which one it is driving.

Everything is an environment variable with a documented default. No constant is hardcoded where an operator might need to change it. The complete list is in 12. Operating.

Local versus production¶

Local	Production	What changes
EMQX	AWS IoT Core	Nothing in the code: same protocol, same QoS, same shared subscriptions
MinIO	S3	Nothing: compatible API
PostgreSQL in a container	Managed PostgreSQL	Nothing
Keycloak in a container	Keycloak or any OIDC provider	Nothing
Development CA on a shared volume	Manufacturing PKI	A cabinet's private key never leaves its secure element
Plaintext MQTT on 1883	mTLS on 8883	A URL and three file paths
Simulated relay (logs a line)	Hardware relay (GPIO)	One adapter; the business code is identical
Simulated reader (text over TCP)	OSDP reader (RS-485)	`READER_BUS=osdp`; the business code is identical
SQLite key from the environment	Key from a secure element	⚠️ Not implemented. See 10. Security

You know the pieces. Now let us follow a badge, from the read to the door.

Next → 4. A decision