From docs/DEPLOY.md in the repository.
Deploying Situs for a real pilot
This build runs on one machine with make up. That is enough to show the control and the receipt. It is not enough to ask a bank to trust the receipt. This note says where each piece runs in a real pilot and what changes in the receipt when it does. It is a plan, not built here: none of it exists in Pilot 1.
What changes, in one table
| Concern | Pilot 1 (this repo) | Real pilot |
|---|---|---|
| Where authority, pep, and the broker run | Docker containers on a laptop | Confidential CPU VMs: Azure DCesv5 (Intel TDX) or GCP Confidential VM (AMD SEV-SNP or Intel TDX) |
| Attestation in the receipt | what.level_achieved: "L1-sim", what.quotes.cpu_tdx: null, what.quotes.gpu: null | what.quotes.cpu_tdx carries the attestation service token, and level_achieved: "L1" is set with the measured image digest |
| Signing keys | ed25519 PEM files in ./keys | Azure Managed HSM or Google Cloud KMS (HSM protection level). Private keys never leave the HSM. |
| Broker secret | PEP_BROKER_SECRET env var shared by pep and the mock bank | Released by the key-release policy only to an attested pep image, or replaced by the bank's own OAuth client credentials held in the HSM |
| Who operates what | One person runs everything | Administrative separation between the pep, authority, and release-policy roles |
| Transparency log | Local Merkle log, optional Rekor | Rekor (public Sigstore) or a bank-operated log, posted on every tree head |
| Bank | Mock bank on port 8003 | The bank's real payments API behind the pep's egress allowlist |
1. Confidential VMs for authority, pep, and the broker
Run the authority and the enforcement point, with its secret broker, each in its own confidential VM:
- Azure: DCesv5 or ECesv5 series (Intel TDX). Attest with Microsoft Azure Attestation (MAA) and use the MAA JSON Web Token as the quote.
- GCP: Confidential VM with AMD SEV-SNP or Intel TDX. Attest with Google Cloud Attestation, using the vTPM-backed attestation token.
Build each image reproducibly and record its measurement: the launch digest, or the PCR/RTMR values for the image. That measurement replaces the software-only what.gateway_build hash as the thing a verifier compares against.
The gateway can stay outside a confidential VM. It holds no credential, and the receipt does not rely on it for any authority decision. Its request and response hashes are evidence, not authority.
2. Swap the attestation slot in the receipt
In services/authority/receipts.py the receipt's what block is built with LEVEL_ACHIEVED = "L1-sim" and empty quotes. For a real pilot:
- At startup, each attested service fetches an attestation token from MAA or Google Cloud Attestation for its own VM, with a nonce bound to its public signing key.
- The authority records the token in
what.quotes.cpu_tdx, or in an SEV-SNP field if that is the platform. It also records the expected measurement inbundle.measurements. what.level_achievedbecomes"L1"only when the token verifies and its measurement matches the published image. Otherwise the receipt keepsL1-simand says why inunproven.- The verifier gains one step: validate the attestation token against the attestation service's signing keys, bundled with the receipt so verification stays offline. Then compare the measurement.
what.quotes.gpu stays null unless model inference itself runs in an attested GPU enclave. That is out of scope for a treasury pilot, and the receipt says so.
3. Keys into an HSM
Move each signing identity to Azure Managed HSM or Cloud KMS with HSM protection. Both support Ed25519 keys (Cloud KMS as EC_SIGN_ED25519; Managed HSM through its OKP key type where available, otherwise P-256, which is a change to situs_core/keys.py).
| Key id | Holder | Notes |
|---|---|---|
principal_jdoe | the treasurer | Their own HSM or a hardware key such as a YubiKey. Situs never holds it. |
approver_jdoe | the approver | Same. situs-approve signs with a hardware-backed key. |
authority | authority VM | Key-release policy: released only to the attested authority image. |
pep | pep VM | Key-release policy: released only to the attested pep image. |
situs_assembly | authority VM | Kept separate from authority so assembly and policy signing can be split later. |
bank | the bank | The bank signs its own acknowledgments with its own key. |
Publish the public keys at each service's /.well-known/keys.json, as today, and pin them in the verifier's key bundle.
4. Administrative separation
No single operator should be able to change policy, release the broker credential, and alter evidence. Split three roles across different people, and ideally different cloud subscriptions or projects:
- pep operator: deploys the pep image. Cannot change the key-release policy or the profile.
- authority operator: deploys the authority image and the Trust Profile. Cannot read the broker secret.
- release-policy owner: controls the HSM key-release policies that bind keys to measured images. Cannot deploy either service.
Changes to treasury.v1 (beneficiaries, limits, thresholds) go through the principal's signature, as the Intent Contract does today, not through an operator.
5. Network
- The pep is the only component with a route to the bank's payments API. Enforce it with the VPC's network security groups or firewall rules, not just the pep's egress allowlist.
- The gateway reaches only the authority and the pep.
- The console is internal. Approvals and revocations still require the signer's key, so a compromised console cannot approve anything.
6. Operational checklist before the first live transfer
- [ ] Images built reproducibly. Measurements published where the bank's auditors can see them.
- [ ] Attestation tokens verify, and receipts show
level_achieved: "L1". - [ ] All six signing keys are in the HSM, with key-release policies bound to measurements.
- [ ] The three roles are held by different people.
- [ ] The bank's egress route is reachable only from the pep subnet.
- [ ]
situs-verifypasses on a receipt from the pilot environment, on an offline machine. - [ ] A revocation drill: revoke a contract, confirm the next proposal is denied, and confirm the receipt shows it.
7. Runbook: the Pilot 1 GCP deployment (Task 36)
This part is built. It is one Confidential VM running the whole stack. It is the "where it runs" demo, not the separated deployment described above. deploy/gcp/ holds the Terraform, the startup script, the Caddyfile and gcp.sh. Every make gcp-* target runs gcp.sh and the /admin/cloud page runs the same targets.
What it creates. One n2d-standard-4 Confidential VM (AMD SEV, Debian 12, Shielded VM) with a reserved static IP. There are two firewall rules: 80 and 443 from anywhere, and 22 only from Google's IAP range so gcloud compute ssh --tunnel-through-iap works. The service account has one project role, roles/logging.logWriter, plus read access to one bucket. That bucket holds the source tarball. The startup script runs at every boot. It schedules the auto-stop first. It then installs Docker and uv once, downloads the tarball (download only, nothing is ever uploaded), and generates the broker secret, the JWT secret, the demo admin token and the console password on the VM. Next it runs make keys && make up and starts Caddy with Let's Encrypt TLS on <ip-with-dashes>.sslip.io. Keys and secrets never leave the VM's disk.
What Caddy exposes. Behind basic auth (user situs, password generated on the VM): the console and /bank/*. Without basic auth: /healthz, the authority's /approvals, /approvers and /.well-known under /api/authority/, and the gateway under /api/gateway/, which demands its own JWT. Nothing else is exposed: not the rest of the authority, not the pep, not the mailbox.
Cost (estimates). Compute Engine bills a running VM per second, and a stopped VM has no CPU or memory charge.
- Up: roughly $0.25 an hour, or about $6 for a full day.
- Stopped: roughly $10 a month for the 30 GB disk and the reserved IP.
- Destroyed: nothing.
From the admin page (the founder's path)
make admin-passwordprints the login (useradmin). The password is generated on first use intodata/admin_password(mode 0600, gitignored).- Open the page. There are two ways:
make adminserves the console on this machine athttp://localhost:8090/admin/cloud, using this machine'sgcloudandterraform.- The console in Docker also serves
/admin/cloud, but the container has nogcloud, so the page says the buttons cannot work there. - Bring up, with "auto-stop after" set to 1, 2, 4 or 8 hours (default 2).
- The first time it runs
make gcp-upwithSITUS_BUDGET_OK=1, because you clicked. That enablescompute.googleapis.com, applies the Terraform, pushes the source and waits for the VM. It then reads the console password back intoout/gcp/access.txt. - Later times it runs
make gcp-start: it writes the newauto-stop-hoursinto instance metadata, starts the stopped VM, and runsmake gcp-access. - The page shows the project (default
durable-timing-509623-j9), the zone, the VM state asgcloudreports it, how long the VM has been up, the auto-stop time left (as this page requested it), the console URL, and the deployed console's logins fromout/gcp/access.txt. - Extend sets
auto-stop-untilin instance metadata. A watcher on the VM reschedules itsshutdown -h. - Check runs
make gcp-check. It calls healthz through Caddy, runs one scripted storyline on the VM, copies the receipt back toout/gcp/receipt.jsonand verifies it here. - Take down runs
make gcp-stop. The IP, disk, keys and data survive, so only the disk and the IP are billed. - Destroy runs
make gcp-down(terraform destroy). You must typedestroyfirst. It deletes the VM, its disk with the keys and data, the bucket and the firewall rules, and releases the reserved IP, so the URL changes next time.
Only one action runs at a time, and its log streams on the page.
From the shell (the scriptable path)
export GCP_PROJECT=durable-timing-509623-j9 # the default when unset; GCP_REGION/GCP_ZONE too
gcloud auth login && gcloud auth application-default login
SITUS_BUDGET_OK=1 AUTO_STOP_HOURS=2 make gcp-up # refuses without SITUS_BUDGET_OK=1
make gcp-status # VM state, console URL
make gcp-check # healthz, one storyline on the VM, receipt verified locally
make gcp-logs # startup log and container logs over IAP ssh
make gcp-stop # stop paying for CPU; make gcp-start to resume (AUTO_STOP_HOURS=…)
AUTO_STOP_UNTIL=$(( $(date +%s) + 3*3600 )) make gcp-extend
make gcp-push # new source to the bucket; applied on the next boot
make gcp-down # destroy everything, including the reserved IPSet GCP_DRY_RUN=1 on any target to print what it would do and stop.
WHERE in the receipt
On the VM, docker-compose.yml passes SITUS_GCE_IDENTITY=1 (from the VM's .env) to the pep. At each enforcement decision the pep asks the metadata server for the instance identity token (audience=situs:<hash of this decision without its attestation>, format=full; Task 38). It places the token in the decision and the authority copies it into the receipt:
where.attestation:kind: gce-instance-identity, the token, and theconfidential_instance_typethe deployment wrote into instance metadatawhat.level_achieved: "L1-declared"
Anywhere else the pep checks the DMI product name, finds no "Google Compute Engine", and never contacts a metadata server. Receipts stay L1-sim, with no where.attestation key, and keep their hashes.
situs-verify checks the Google RS256 signature offline against situs_core/google_certs.json, along with the issuer, the audience (it must name the decision in this very receipt; a token copied from another receipt is INVALID) and the validity window, all at the receipt's own timestamp. It prints in yellow: "confidential VM declared by cloud identity; not a Confidential Space attestation". This is a label, not an eleventh check. situs-verify --refresh-google-certs merges Google's current certs into the snapshot and keeps the old key ids. The token is Google's statement of the project, zone and instance. Nothing in it measures the image. Confidential Space is the next step for a real L1: there, an attestation token binds the container image digest.
8. Runbook: the front door on Cloud Run (Task 38)
This part is built. https://situs.o11r.com is a small stateless service on Cloud Run (situs-front, us-central1, min instances 0). The VM keeps Caddy and its TLS and answers on its own name, https://situs-vm.o11r.com (the sslip.io name stays as a fallback). The front door is a front door, not a proxy (D38):
| Path on situs.o11r.com | What happens |
|---|---|
/, /docs/* | The overview's story and the Docs pages, rendered at image build time. No live status, no VM call. |
/status.json | vm (awake, asleep, starting, stopping or unknown) and vm_url, nothing else. |
/dashboard, /demo*, /policy*, /mailboxes*, /grants*, /contracts*, /receipts*, /verify, /passkeys*, and every handle_path prefix the VM's Caddy exposes (today /bank/*) | Awake (RUNNING and the VM's HEAD /healthz answers): 302 to https://situs-vm.o11r.com + the same path and query. Otherwise the data-free "Demo asleep — start it" page. |
/api/* | Never forwarded: JSON 503 demo_asleep, or JSON 421 not_forwarded naming the VM URL to call. The Approval API stays on the VM at https://situs-vm.o11r.com/api/authority/. |
/launcher | The founder-only launcher: Google sign-in, Start (auto-stop 1, 2, 4 or 8 h), Stop, Extend (never past 8 h from now). |
The only request the front door ever makes to the VM is HEAD /healthz. It holds no key, no evidence and no receipt, and its image holds only services/frontdoor/ and the static pages (services/frontdoor/image.py). Bring-up the first time, Destroy and Check stay on the local /admin/cloud (make admin), which is switched off on the VM's own console (SITUS_ADMIN_CLOUD=0) and blocked by Caddy.
The launcher's powers. Its service account situs-launcher holds a custom role with exactly compute.instances.get, compute.instances.start and compute.instances.stop, bound on the one instance; roles/storage.objectAdmin on the control bucket <project>-situs-control only; and roles/secretmanager.secretAccessor on the two launcher secrets only. Nothing at project level, nothing on the deploy bucket, no setMetadata. The auto-stop deadline it sets is the object gs://<project>-situs-control/autostop-until (epoch seconds). The VM's watcher reads it every 15 seconds, clamps it to 8 h from now and ignores anything malformed.
DNS (kept outside Terraform)
The coordinator or the founder keeps these records in Cloudflare; Terraform never touches DNS.
| Type | Name | Value | Cloudflare proxy |
|---|---|---|---|
| TXT | o11r.com | google-site-verification=... (exists; verifies the domain for the Cloud Run mapping) | n/a |
| CNAME | situs | ghs.googlehosted.com. | DNS only |
| A | situs-vm | the reserved IP (terraform -chdir=deploy/gcp output ip; 34.41.128.214 today) | DNS only |
| CAA | o11r.com | only if a CAA record exists: add 0 issue "pki.goog" and 0 issue "letsencrypt.org" | n/a |
Proxied must stay OFF on both. A Cloudflare proxy would terminate TLS at Cloudflare instead of on the VM (Caddy) and at Google (the mapping's managed certificate). The VM's passkeys and the console's Origin check are bound to situs-vm.o11r.com; they only mean something if the browser talks TLS to the VM itself.
make gcp-down releases the IP and then prints a reminder: remove the situs-vm A record, so the name cannot point at an IP someone else may be given.
The OAuth client (founder, once)
Google Cloud console, project durable-timing-509623-j9: APIs & Services → OAuth consent screen (External, Testing, the founder as test user; scopes openid and email), then Credentials → Create credentials → OAuth client ID → Web application:
- Authorized JavaScript origin:
https://situs.o11r.com - Authorized redirect URI:
https://situs.o11r.com/launcher/oauth/callback
Download the client JSON. It never enters the repo or Terraform.
The two secrets (out of band)
Terraform only references them (data "google_secret_manager_secret"), so no secret value is ever in its state. Create them once, before make gcp-up:
gcloud secrets create situs-launcher-oauth-client --replication-policy=automatic --labels=app=situs \
--data-file=client_secret_<id>.json --project durable-timing-509623-j9
openssl rand -base64 64 | tr -d '/+=\n' | head -c 64 | gcloud secrets create situs-launcher-session-key \
--replication-policy=automatic --labels=app=situs --data-file=- --project durable-timing-509623-j9Cloud Run mounts both (latest version) as SITUS_OAUTH_CLIENT_JSON and SITUS_FRONT_SESSION_KEY. The launcher takes client_id and client_secret from the JSON. Until both exist, every launcher route answers 503 and the rest of the front door works. make gcp-down leaves the secrets in place; remove them by hand with gcloud secrets delete situs-launcher-oauth-client situs-launcher-session-key.
Bring it up, push, check, take it down
SITUS_BUDGET_OK=1 make gcp-upnow also enables run, artifactregistry and secretmanager, creates the Artifact Registry repositorysitus, the control bucket, the launcher account and role,make gcp-front-pushes the image, and creates Cloud Runsitus-frontwith public invoke and thesitus.o11r.commapping (FRONT_DOMAIN_MAPPING=0skips the mapping). On an existing deployment the same command adds these in place and updates the VM's startup script; restart the VM once (Take down, then Bring up) so the new script writes.envand servessitus-vm.o11r.com.make gcp-front-pushrenders the pages (python -m services.console.front_site), builds the image locally for linux/amd64 from a context holding only the listed files (python -m services.frontdoor.image), pushes it, and rolls it onto the service. Nogcloud run deploy --source.make gcp-checkverifies the VM's receipt with--keysfetched fromhttps://situs-vm.o11r.com/api/authority/.well-known/keys.json(never the receipt's own copy), then checks thathttps://situs.o11r.com/answers, that/status.jsonsays awake, and that/dashboard?from=checkredirects to exactly the VM.FRONT_CHECK=0skips the front door part.make gcp-downdestroys the front door with everything else (not the secrets, not DNS).
Cost (estimates). Cloud Run with min instances 0 bills only while a request runs: cents a month at demo traffic. The image in Artifact Registry is well under 1 GB (about $0.10 a month). The VM's costs are unchanged.
On the VM (every boot). The startup script writes into .env: SITUS_RP_ID and SITUS_RP_IDS (situs-vm.o11r.com, plus the sslip name), SITUS_ORIGINS (https://situs-vm.o11r.com, https://<sslip>), SITUS_AUTHORITY_PUBLIC_URL (https://situs-vm.o11r.com/api/authority), SITUS_AUTHORITY_PATH_PREFIX (/api/authority), SITUS_BANK_BEHIND_PROXY=1 (the demo counterparty never pre-fills its admin token behind Caddy) and SITUS_ADMIN_CLOUD=0. Caddy answers on situs-vm.o11r.com and the sslip name and returns 404 for any /admin/reset and for /admin/cloud.
9. Runbook: the push relay (Task 39)
The front door's POST /push sends "Situs: an approval is waiting" to an approver's iPhone. It never carries an approval's content (docs/APPROVAL_API.md §6.5, docs/DECISIONS.md D39). It is built and deployed disabled: /push answers 503 push_not_configured until all of these exist.
Founder, once. In the Apple Developer account, create an APNs authentication key and hand over the .p8 path, the key id, the team id and the approver app's bundle id. The coordinator loads them without printing them:
gcloud secrets create situs-push-apns-key --replication-policy=automatic --labels=app=situs \
--data-file=AuthKey_<KEYID>.p8 --project durable-timing-509623-j9
printf '{"key_id":"<KEYID>","team_id":"<TEAMID>","topic":"<bundle id>"}' | gcloud secrets create \
situs-push-apns-meta --replication-policy=automatic --labels=app=situs --data-file=- \
--project durable-timing-509623-j9A new bundle id is a new version of situs-push-apns-meta; no code changes and the topic is never in the repo.
Turn it on. Once both secrets have a version: PUSH_ENABLED=1 SITUS_BUDGET_OK=1 make gcp-up. Terraform then gives the front door's account (and only it) secretAccessor on the two secrets, mounts them at latest as SITUS_PUSH_APNS_KEY and SITUS_PUSH_APNS_META, and changes the VM's startup script so its .env gets SITUS_PUSH_RELAY_URL=https://situs.o11r.com/push (restart the VM once). Without PUSH_ENABLED=1 none of that exists and the variable is empty (off).
The pinned key. make gcp-front-push reads https://situs-vm.o11r.com/api/authority/.well-known/keys.json while the VM is awake, keeps the authority entry as {kid, alg, pem} in out/gcp/push-authority-key.json, and sets it as SITUS_PUSH_AUTHORITY_KEY on the service; the next make gcp-up passes the same value to Terraform so an apply does not undo it. If the VM does not answer, the pin already on the service stays. Run it again whenever the VM's keys are regenerated (make gcp-down and a fresh gcp-up). The relay never fetches keys at request time.
Live check (pending a device). Enroll the iOS app on the VM, register its token (PUT /approvals/apps/{app_id}/push-token, sandbox for a development build), start a demo approval, and see "Situs: an approval is waiting" on the device; tapping it must open the item fetched from the VM, and the approval is made against the VM.