Deploy and Onboard

Use this guide to provision and onboard FDAI in Azure so it is ready to observe. This file owns the concrete deployment inventory, bootstrap sequence, and distribution/deployment responsibility split; the deployment lifecycle (CI/CD, progressive delivery, rollback, DR) remains in deployment.md. Azure focus: this document targets an Azure subscription. Non-Azure providers are TBD (see Implementation Focus). All identifiers are synthetic per generic-scope.instructions.md.

The day-zero service tiers and counts are decided in minimum Azure resource inventory. A deployment owner confirms the region, quota, retention, replica caps, and production tier overrides before deployment. The execution engine is decided: terraform apply against infra/ (Terraform HCL). The operator entry point is fdaictl provision azure, which coordinates exact plans locally and runs private data-plane apply on the managed host inside the target VNet. Tenant deployment does not use GitHub Actions. See Installable Deployment CLI and Deployment Artifacts. Interactive source installation writes append-only numbered stages, bounded details, and elapsed activity heartbeats. It doesn’t clear the terminal, estimate provider completion, or treat presentation progress as deployment readiness.

Separate Hub development validation

Use the vendor’s Release Hub root or the customer’s Lifecycle Hub root. Each owns its UI/API, private PostgreSQL, resources and state without peer inputs; explicit durable private local backends retain state, with reviewed remote backends optional. Login/catalog bindings aren’t needed for migration, health/readiness or static UI, even without Entra registrations. Management requires authentication; no anonymous APIs are enabled. The single-product procedure covers immutable images, state preservation, native Terraform/Azure CLI execution and bounded read-only database recovery comparison. The comparison rejects query-level connection-routing overrides, grants no authority and isn’t proof of full database equivalence; its private observer, retained evidence and independent security/cleanup readback requirements still apply.

For running Hubs, app_image retains the approved serving digest while image selects the migration candidate; keep apps and migration Jobs enabled. After manual migration and independent schema verification, activate with app_image=null in a new reviewed, sealed plan. Omission preserves single-image defaults. Optional resource_group_name overrides the default rg-<prefix>-<hub> boundary name, and optional custom_domain binds a public HTTPS subdomain and optional Azure DNS records while aligning the runtime origin and probe Host headers. A Hub accepts only its configured origin host, so the default Container App hostname answers 400 once a custom domain is set; custom_domain.redirect_default_hostname=true makes a Hub image that reads redirect_hosts answer GET and HEAD on that hostname with a 308 to the fixed origin and refuse everything else. Leave it unset for older images. Complete Job checks, globally zero-change dispatch plans, state ownership, authentication and approval remain required.

Remote development Agents use operator_auth.agent_principals to bind an installation, Entra object and client on Lifecycle Hub only. Release Hub rejects these bindings; omission preserves existing authentication. Terraform grants no Agent.Access, federated credentials or Kubernetes permissions. Prepare those separately within the approved scope and follow the Hub operator guide for namespace-only RBAC, durable state, enrollment and independent readback. This grants no operator or release-apply authority. The Agent state volume uses a zone-redundant disk class so a Pod rescheduled to another zone can attach it.

For a Hub-managed AKS handoff, enable_document_ingestion=true can provision the shared Azure prerequisites while enable_document_ingestion_workloads=false omits the ingestion API, worker and migration Container Apps. The default remains true for source/offline Container Apps deployments. Use the split only with an exact reviewed plan; it grants no Hub ownership or apply authority. For an existing installation, run that plan from infra/hub-stewardship-prerequisites with a separate remote state key. The root reads the shared platform resources and owns no application workload, so it cannot silently re-plan the retained source workloads state. When the operations VNet is already linked to an existing Blob private DNS zone, the document storage endpoint remains associated with the application zone and the root publishes a managed A record for the same private IP in the operations zone instead of attempting a second overlapping VNet link.

An optional Lifecycle Hub operator_auth.development_recovery input can recover one development installation identity whose private key is unavailable. Use it only with the exact full-authority-development installation, scope, Owner, old and replacement public keys, reason and authorization interval. Release Hub rejects this input. The Lifecycle Hub still requires replacement-key possession, no open Plan or existing configuration trust, append-only audit and first-binding-only single-Owner approval. Remove the input after verified recovery and key registration; it is not a production or general enrollment-rotation setting.

The Lifecycle Hub image now needs a digest-pinned OPA_IMAGE build argument because the adoption policy runs opa; the Release Hub image doesn’t contain it. The Lifecycle Hub catalog setting adds access, check_interval_seconds and an adoption object whose auto mode requires an attributed, expiring authorized_by and authorization_expires_at. Release Hub adds auth.public_catalog_export and timestamp_renewal_check_seconds. Each new field is optional and is omitted from the runtime JSON when unset, so older images keep parsing their configuration. See Automatic catalog adoption. Signed catalog roles add two more optional inputs: the Lifecycle Hub catalog.trust object (mode = "tuf", root_key_ids, root_threshold) replaces release_public_key, and the Release Hub catalog_roles object adds separate snapshot-key and timestamp-key secret versions. The offline-signed root is installed through the install-catalog-root operation. See Signed catalog roles. The Lifecycle Hub operation Job also accepts bind-catalog and rebind-catalog; they take an attributed actor and a reason instead of a request secret, and a rebinding keeps the revision floor.

The unchanged legacy paired fixture retains separate networks, identities/keys, migrations, Entra operator/service boundaries and catalog-pin approval for current state owners. Neither product requires the fixture; neither replaces an FDAI installer. Local/static checks don’t prove Azure network/RBAC isolation, Entra/MI, registry architecture, replica durability or deployment. Use an authorized live scope for those receipts; Agent apply and full FDAI installation remain excluded.

Prerequisites

Deployer Identity (Azure)

The production deployer permission boundary is owned by Production deployment hardening.

Azure Prerequisites

• Region with confirmed availability of every service in the inventory below.
• Confirmed quota headroom (AKS node-family and regional vCPUs, Event Hubs throughput units, PostgreSQL vCores, Key Vault operations).
• Diagnostic Settings destination (Log Analytics workspace) - new or existing; ownership TBD.
• Staged private networking. Basic deployment creates AKS Standard with API Server VNet Integration, dedicated workload and API-server subnets, and authenticated restricted public management access. It does not require VNet peering, private endpoints, private DNS or private-cluster mode unless subscription policy requires them from the first effect. After baseline Console health passes, /provisioning can request those changes through a separate exact plan. The protected executor establishes and verifies peering, routes, DNS, TLS, identity, private endpoints and registry access before disabling a public path. The browser never runs Terraform or receives deployment credentials. A policy-locked tenant instead uses an eligible internal host and the exact private plan during basic deployment; policy is never weakened.

What Terraform does not create

Standalone infra/bootstrap retains its existing-account and existing-application-group prerequisites. The separate genesis root below removes that creation cycle without running application data-plane work locally. For a fresh database, supply its administrator password through the protected input or explicitly opt into persistent Terraform generation; enabling generation for an existing server is a separately approved credential change. Reference plans stay compatible with Terraform >= 1.9, and root wiring is checked separately.

• Deployer identity: Role assignment needs User Access Administrator; Contributor alone is insufficient.
• State storage: Standalone bootstrap reads the existing account created by infra/bootstrap/create-state-account.sh. Its AzureRM lookup can read account keys, so protect that local state as secret-bearing.
• Application resource group: Standalone bootstrap expects the group to exist before assigning the runner’s roles.
• Runner inputs: Supply an SSH public key, quota headroom, and the Log Analytics destination. Mutation-enabled Genesis runs prepare-genesis-access-tools.sh before provider or policy mutation, pinning stable Bastion and Microsoft Entra SSH CLI extensions without creating Azure resources. Run it directly only to prewarm or repair the local CLI; inspection mode remains read-only. Connected bootstrap uses an exact Marketplace Ubuntu version. Offline bootstrap can supply a separately verified prebuilt image.

The genesis foundation root manages both resource groups, the private state account, and the tfstate and deployment-plans containers through ARM, including blob protection, and reuses bootstrap’s network, deployment identity, and runner without account-key lookup. The optional Standard Bastion subnet uses the complete Azure-required inbound and outbound Network Security Group rule set; a missing platform rule blocks tunnel creation. For a new platform state, foundation_resource_group_context_digest selects reference-only ownership and verifies the foundation tag and region. A verified failed public-development state can enter the empty Foundation-managed application backend only through the advanced no-delete adoption path, which removes the two resource-group owner addresses, retains the original state, writes a claim before one non-forced push, verifies readback, and blocks every delete or replacement plan before approval. fdaictl provision plan --stage foundation is a private dry run with a local backend; the exact state-migration archive activates the signed AzureRM backend example only for the attested host. Genesis authenticates separate Terraform archive/executable digests, selects a compatible host VM SKU and exact Marketplace Ubuntu version, and installs the checksum-pinned toolchain directly. Artifact-offline deployment can instead use a separately verified prebuilt image. An older retained migration claim with no sibling support files can restore only the exact reviewed-configuration bytes after a fresh foundation-state approval. A separate current-source verifier stays outside Terraform’s input tree, verifies its repair overlay, and resumes verify without repeating backend migration. New authority records bind exact backend blob bytes in addition to normalized Terraform output. After cleanup, current verified observer code uses the owner-only Bastion transfer and digest-readback path, reads the protected state blob directly through managed-identity Azure CLI authorization, and is removed after rechecking the authority-bound blob digest without reconstructing the deleted Terraform work tree or provider mirror. Completed legacy receipts retain their original zero-change evidence without inventing the newer digest. Foundation apply, Bastion enrollment, and verified state migration require exact approval; application deployment and readiness remain open in the Genesis ledger.

A tenant whose Azure Policy denies part of the inventory also needs either an exemption or the matching capability-mode toggle before the plan can converge (deployment-preflight.md). Upstream verifies the toggle contract with provider-free reference plans. The deployment or fork binds those outputs to its concrete resource modules and supplies any existing resource ids.

Ops/hub runner (private-everything tenants)

Some tenants force every data service private (Key Vault and storage), so even a terraform remote-state backend is laptop-unreachable. The infra/bootstrap layer stands up the durable hub that makes the deploy possible and survives app rebuilds:

The ops layer creates one outbound path by default for GitHub and the Azure management and identity planes. A closed network with an approved route sets enable_public_egress = false. If it also sets runner_bootstrap_mode = "offline", bootstrap requires one exact managed-image or numeric gallery-version ID and disables marketplace selection, cloud-init, and GitHub registration.

• an ops resource group + hub VNet (rg-fdai-ops-<region_short> / vnet-fdai-ops-...) separate from the app RG, with a runner subnet and a private-endpoint subnet;
• a terraform remote-state storage account locked to private, fronted by a blob private endpoint on privatelink.blob.core.windows.net linked to the ops VNet;
• a stable deploy user-assigned managed identity (UAMI) whose lifecycle is independent of the current and candidate runner VMs. Bootstrap owns its exact role manifest and outputs separate client and principal IDs;
• a self-hosted deploy runner VM (no public IP) with one to five slots on sustained Standard_D8ds_v5 compute (8 vCPU with a local SSD; Standard_D8ads_v5 or Standard_D8ds_v4 where the region offers no v5) and a Local ResourceDisk ephemeral OS. VM-side Bash expands slot paths; deallocation is blocked, and scheduled drift rejects managed OS disks or placement changes. Slots share the stable UAMI. Plans and read-only checks use service-specific locks, while apply and state migration share one environment writer lock. Platform and scenario Terraform pin deployer roles to that principal and stop when the authenticated principal differs. The UAMI holds Contributor + User Access Administrator on the app RG, Network Contributor on the ops RG, Storage Blob Data Contributor on state, and subscription Reader + EventGrid Contributor + Cognitive Services Contributor. Its conditional Role Based Access Control Administrator grant can assign only Reader, Monitoring Reader, and Cost Management Reader to service principals. During migration, the current VM keeps its system identity alongside the UAMI, but the promoted VM retains only the UAMI and workflows never select an identity implicitly. Each run clears the Azure CLI account cache, logs in with the configured UAMI client ID, and proves the exact repository-configured subscription, tenant, and ARM token oid before any storage, plan, or apply step. A reviewed blue/green cutover can preserve an existing candidate’s GitHub registrations by setting runner_vm_name before importing its VM and network interface into bootstrap state; bootstrap preserves the adopted image reference until an explicit reviewed replacement. Scheduled checks enforce all roots; a manual runner scope verifies only runner storage and bootstrap state, including protected-host SSH input recovery, actual disk inventory, UserAssigned-only identity, one configured UAMI, and empty structured drift actions. Full scope also requires exact bootstrap-plus-platform direct roles and absent or empty disposable scenario state. Before checkout, the runner removes only the legacy generated infra/None cache path so root-owned action residue cannot block the exact-commit clean step. That step creates the Azure CLI config under RUNNER_TEMP and exports it through GITHUB_ENV; because the job default is infra/, a fresh slot has no repository directory and never depends on earlier checkout residue. The app config peers its spoke VNet to the ops hub (both directions) and links its private DNS zones to the ops VNet via the extra_vnet_links seam, so the runner resolves the app’s Key Vault privately. The runner is the terraform apply principal, so the existing kv_officer_self grant makes it Key Vault Secrets Officer on the app vault - it writes the DSN secrets during apply. Tenant deploys run through the standalone local coordinator and the Bastion-reachable managed host. The historical deploy-dev workflow remains repository automation evidence and is not a supported tenant deployment entry point. The additional label selects the validated 8-vCPU local-SSD pool; conjunctive GitHub label matching queues jobs when that pool is unavailable instead of silently falling back to the slower managed-disk runner. Repository workflows allow only reviewed remote actions pinned to exact Node 24-compatible release refs; container supply-chain actions use immutable commit SHAs. The CI contract rejects unknown actions and mismatched refs. Terraform fixture tests use syntax accepted at the declared >= 1.9 floor and pin the mocked client-config object ID to the deploy principal. Plan-only retention admits an Operator API OpenAI User role replacement only when its scope and role are unchanged, principal_id is the sole role replacement path, and the paired Operator UAMI changes only its name while preserving location, resource group, and tags. The same review copy admits only exact role principal or scope migrations to the stable runner UAMI, delete-only retirement of the two prior indexed measurement Jobs when measurement capabilities are disabled, and the reviewed embedding family and SKU replacement. A measurement retirement must match the exact managed resource type, name, index, prior object, null result, and absent replacement path. Provider-computed field encoding does not grant authority, and retention never authorizes apply. The protected deploy workflow keeps repeated validation, target derivation, state preservation, and effect publication in focused helpers, removes rendered plan copies on every exit, and limits the exclusive development identity migration to state-owned reviewed deployer roles with stable-principal readback and zero superseded-principal assignments. A paired in-place storage prerequisite may only disable local users and retain the configured blob and container periods; absent role addresses are not targeted, and Console, Entra, database, health, and canary work stay outside the operation. Before recovery planning after an interrupted apply, storage and Foundry owner-resource state restores feature enablement even when the role address is absent; a surviving owner with an unavailable scope fails closed. The workflow imports only one exact existing stable assignment at each intended scope, rejects duplicates and assignment IDs already tracked elsewhere, and never reimports the redundant operational-history handoff address in identity mode. Privileged workflows first check out the shared source verifier from protected main; the verifier rejects a target commit that isn’t an ancestor or whose workflow controls differ before target commit code runs. A workflow that needs an additional deployment helper installs it only in runner-temporary storage, pins its exact release and SHA-256 digest, and verifies it before use. The exact CI version proves parsing and plan assertions. Upgrades verify action runtime metadata, and the self-hosted runner install always resolves the latest published GitHub Actions runner release rather than a pinned floor version. When private networking is enabled, PostgreSQL public access and the broad Azure-services firewall are disabled. Dev uses its approved private endpoint; delegated-subnet mode remains available for production. Protected requests checkout commit_sha explicitly and compare it with git rev-parse HEAD, so a release commit that advances main between dispatch and execution cannot change plan or apply code. The deploy job runs from infra/, so a step that invokes a repository-root script reaches it with ../scripts/ or overrides the working directory. A bare scripts/... path resolves under infra/ and exits 127 on the runner, before Terraform has produced anything to inspect. The protected runner invokes scripts/deployment/azure/run_live_preflight.py directly after Terraform planning. This standalone, read-only entrypoint checks Azure Policy, Compute quota, executor RBAC, and value-blind Key Vault secret metadata without depending on a runtime-service wheel or the separately distributed fdaictl package. Missing mappings, credentials, categories, or probe results fail closed before a plan artifact is stored. Dedicated repository modules build protected plan metadata and apply receipts so the YAML passes only sealed inputs and explicit output paths. The full plan projection and value-free summary use a current-UID mode-0700 temporary directory and are removed after the metadata builder returns. Exact apply validates the summary schema, digest, action and resource-type totals, non-destructive flag, and required post-apply observation list. Only plan-runtime-* and plan-observability-* may carry one terraform_data state replacement with zero create, update, or delete actions; every other destructive summary is rejected. Legacy v1 metadata without both fields remains valid until its bounded expiry, while partial summary evidence is rejected. Protected plans store the binary Terraform plan, bounded preflight evidence, and the Function source archive with separate SHA-256 digests. Exact apply verifies every artifact; peer receipts download each allowlisted isolated backend blob directly with the authenticated runner identity and a bounded timeout, avoiding repeated provider initialization without changing the state bytes. Service rollback removes only post-apply secret names absent from the immutable snapshot before restoring its exact Key Vault references. Independent-service Container App plans also seal a lowercase plan-time revision suffix into the saved Terraform plan, guaranteeing a fresh revision after an out-of-band verified image rollback left the desired Terraform image unchanged. The guard permits only that bounded suffix beside the exact image update, and health still requires a new revision running the attested image before recording an apply receipt. When configured, the history plan targets the dedicated decision-evidence account, private endpoint, and read-only identity grant; the separate workflow then verifies only protected main first-parent evidence, attests before publishing, and writes convergent records to immutable storage that runtimes can only read. Before storing a new plan, the runner selects only allowlisted plan, metadata, source, preflight, claim, and receipt blobs older than 24 hours. It scans fewer than 1001, deletes at most 1000 with eight workers, and fails the plan if selection is incomplete or any delete fails. When the development operations gateway is selected, Terraform targets that Function, core, Operator API, ingestion, the isolated Executor when selected, operational canary, inventory reconciliation Job, realtime inventory publishers, and their dependency graphs. This keeps the Job’s image and required shared runtime configuration converged while unrelated runtime-resource changes stay outside the plan. Terraform owns the provider-schema Job at the root address azurerm_container_app_job.provider_schema[0]. Deterministic resource IDs bind the existing Container Apps environment and inventory identity, a read-only identity lookup supplies its client ID, and a moved block preserves state from the former compute-module address. The provider-only target therefore does not inherit the compute module’s broad platform dependency graph. The legacy repository-automation provider-schema mode is not registered by the public deployment CLI. It retains plan-provider-* and apply-provider-* requests for the standard Core profile and plan-provider-cost-* and apply-provider-cost-* for the Cost Governance Core profile instead of the broad gateway target. The subtype must match the context-bound runtime image profile without another mutable input. Both forms admit only the provider-schema Job address, bind the exact attested Core image revision, reject mixed targets, and require a targeted zero-change plan after apply. Plan and apply require that image to be the active, healthy, provisioned Core revision with rollback retention enabled. Plan stores a sanitized immutable baseline; apply verifies it and observes the live baseline again before mutation. A bounded verifier starts the Job once and retains Core-baseline, source-revision, durable-generation, Heimdall-handoff, and matching Forseti-decision and Saga-audit evidence. The Job forces a fresh observation so periodic or failure-retry cadence cannot skip explicit verification. An accepted complete snapshot refreshes its freshness timestamp, and stale receipts are rejected. Verification-only resume never reapplies Terraform: it reuses exact immutable evidence or reruns only Job verification. An unchanged schema records review as not applicable. The retained compatibility status reader validates the sealed image profile and digest, fresh healthy Core baseline, durable source and generation, and complete review chain or explicit not-applicable result; apply status also requires the exact plan id and digest. The target set includes both source and destination addresses from active Terraform moved blocks, and the workflow contract test keeps those addresses synchronized so state migrations cannot invalidate a protected plan. This includes the unindexed destination addresses for the baseline-regression and pattern-growth jobs. A for_each key rename uses an explicit moved block so Terraform preserves the existing resource instead of planning a delete and replacement create. A targeted plan includes the collection resource address for that for_each move so Terraform can evaluate both keyed instances together; it also targets the AI account with its role collection so network and authorization settings converge in one apply. Terraform uses the reader managed identity for host and deployment storage; the workflow removes Flex-generated shared-key overrides before publishing. It grants Storage Blob Data Owner for the host and a separate idempotency role. Easy Auth admits only the core executor client before principal checks. Operator API deployment also requires non-secret FDAI maintainer and all non-autonomous agent operational ownership bindings from repository Variables; Container App preconditions reject an incomplete map. After exact apply converges, the official Flex One Deploy action remote-builds the verified source, retries bounded trigger sync, and requires both Function triggers before recording the apply receipt. If a later identity or health check fails after the immutable claim, verification resume validates that claim, skips Terraform apply, and reruns convergence and post-apply checks. Protected Console publication uses the non-secret Static Web App hostname and resource id synchronized to repository Variables by the exact apply. It falls back to the same exact Terraform state outputs only when those bindings are absent and never performs an arbitrary resource search. Protected Console publication updates only the combined Console and Manual Studio static artifact. The bot-owned catalog refresh is a separate protected operation that promotes the exact Core image, runs schema migrations, and verifies PostgreSQL projections without blocking static publication. Health acceptance always requires the core Container App’s latest revision to be Provisioned and Healthy before an apply receipt can be recorded. Selected Operator API and ingestion revisions must also be healthy, and their shared ingress /healthz responses must return the fixed success payload. Design-mocks-only applies are the sole exception because they do not plan the runtime. The protected-plan delete gate permits only bounded security retirements: closing the broad PostgreSQL Azure-services firewall path, or deleting one of the reviewed pre-split ingestion grants when every exact API or worker successor is pure-created in the same plan. It also permits the reviewed t1.embedding migration only when the plan JSON matches the exact address, account, family, source SKU/capacity, target SKU/capacity, and replacement action. A model-binding-only plan targets the Azure OpenAI capability deployment collection instead of the enclosing module. Existing account and role-assignment resources therefore remain outside the plan graph before the scope and destructive-change guards run. The workflow canonicalizes the sealed resolver artifact before hashing. A resource-no-op recovery plan is accepted only when that canonical digest differs from the attested active Core model digest; model-specific provider readback remains required after apply. A delete-only model change, a drifted replacement, a missing or non-create successor, and every other delete remain blocked. Full runbook: infra/bootstrap/README.md. Scheduled drivers remain Terraform-owned. SCHEDULER_TICK_CRON_EXPRESSION and ANALYZER_TICK_CRON_EXPRESSION configure the existing jobs; forecast_tick_cron_expression and forecast_targets_json opt into the forecast Job and inject FDAI_FORECAST_TARGETS_JSON. The forecast Job publishes only a raw tick, which Huginn normalizes for Heimdall to evaluate and close. automation_blueprint_cron_expression opts into the automation blueprint Job. It reuses the scheduler identity and state-store secret, and each run only proposes inert candidates or applies queued review proposals. Each target entry names target_kind and matches the repository-governed horizon, confidence level, sample floor, and fit floor. Core loads that policy even when forecast targets are disabled, so a missing or malformed policy blocks startup rather than leaving an ungoverned target path. The inventory reconciliation Job inherits the same required non-secret runtime config as core so recovery-delta forwarding can open its typed Event Bus publisher without a partial config. Scheduler and analyzer Jobs set FDAI_MI_CLIENT_ID to the client id of the user-assigned identity attached to that Job, so Azure Monitor and Event Hubs token acquisition never relies on implicit identity selection. The legacy generic OOB Job is a bounded, inert compatibility resource until a probe entry point owns it; the public contributor bootstrap omits it until a deployment-owned image exists, and implemented recurring work stays in the dedicated Jobs. The shared inventory enrichment builder gives both scheduled deployment and opted-in local refreshes the same subscription discovery path. The deployed inventory Job enables it by default under its dedicated Managed Identity’s subscription-scoped Reader, Azure Kubernetes Service Cluster User Role, and Azure Kubernetes Service RBAC Reader; each tick discovers current and future clusters, minimizes exec-format access profiles in memory, retains no kubeconfig, lets explicit fleet or legacy bindings narrow the scope, and never attaches these read-only roles to Core, Operator, or Thor. The same identity’s Reader role reads subscription role assignments and PIM schedule instances, but deciding guest and group-reached principals also needs the Microsoft Graph application permissions User.Read.All and GroupMember.Read.All with tenant admin consent. Terraform doesn’t grant them, matching the Operator API directory consent; until an administrator does, every subscription’s role_assignments property stays unobserved and the Rules that read it hold for review instead of passing. Managed identity role assignments additionally need Reader at the tenant root management group, because only a complete hierarchy read proves no grant is hidden; with subscription-scoped Reader alone they stay unobserved. On a public-network profile, Terraform also adopts the deterministic realtime-inventory Event Grid subscription when an operator restores it out of band, then converges its Event Hub destination, delivery identity, event filter, and retry policy on the next protected apply. Private-networking profiles do not create that unsupported Event Grid-to-private-Event-Hubs path. The VNet-integrated inventory Job instead forwards bounded Activity Log recovery deltas to the primary Event Bus after each reconciliation, using its topic-scoped Data Sender role and durable idempotency cursor. A disabled change accelerator neither reads its optional collection-policy entry nor contributes a cursor prefix or stale-cursor deadline. An empty cron disables its job. Existing scheduler or analyzer jobs are safely adopted before a plan, and later image or configuration changes converge through the same plan and apply path. The analyzer job defaults to a one-minute observation mode schedule and runs fdai.delivery.analyzer_tick_cli, which publishes one canonical Event per detected issue keyed by resource, signal, and tick window. It merges FDAI_ANALYZER_TARGETS with supported resources from the durable inventory projection when FDAI_INVENTORY_DSN is configured, deduplicates the merged set, and applies FDAI_ANALYZER_MAX_DISCOVERED_TARGETS before provider I/O. An unsupported resource type is omitted rather than guessed. An unreadable projection fails the tick so the Job retries instead of silently reducing coverage. Without an inventory DSN, the explicit-only path remains available; when both sources resolve no target, the tick is a clean no-op that exits 0. Set the analyzer cron to an explicit empty string to disable the job.

Inventory discovery with restricted egress

The preflight, source precedence, coverage, and stale-retention contract is owned by Restricted-network Azure inventory.

Onboarding automation

These customer-agnostic helpers keep both deployment routes repeatable:

• fdai-up.sh is the one-command source deployment after az login. Any clone deploys without a key: no integrity signing key selects the durable 30-day Trial, and a usable secrets/integrity-signing-key.pem selects a full installation entitlement. It uses bounded concurrency for independent preparation, read-only discovery, provider requests, and policy-probe siblings. Its Genesis checks admit only the clean checkout at the exact deployed commit and never query a CI result or require the GitHub CLI. The invocation approves each plan it shows, configures Foundation and tenant bindings, and applies through the manual managed host before requiring a zero-change plan. Approval, apply, cleanup, state, and handoff boundaries remain serial. GitHub repository configuration and workflow dispatch are not part of this path. Without a mode argument it deploys its own checkout from any working directory, selects the entitlement from secrets/integrity-signing-key.pem before any Azure call, starts az login when no Azure CLI session exists, and builds or signs no kit. It refuses the retired --signing-key option.
• check-signing-key.py identifies a candidate private key before a build or deployment consumes it. It reports the packaged roots’ fingerprints, which roles a candidate satisfies, and the owner-only custody the build requires. It prints no key material, so it is safe to run on any machine that may hold the key. --scan secrets shows whether a clone holds the upstream integrity key that selects full entitlement. The development profile pins one signer for both the complete-kit and bundle roles of the offline package; licensing uses the integrity key.
• genesis-up.sh retains the lower-level 15-stage Foundation route. Claims resume verification only and Foundation never implies readiness.
• verify-azure-context.sh binds Azure CLI and azd entry points to the approved subscription and tenant pair before mutation; Genesis verifies region availability through the exact subscription-bound ARM locations endpoint without changing the active CLI selection, and policy-probe cleanup parses multi-value TSV as ordered lines before proving absence.
• azd-up.sh remains a Core-only dev diagnostic. It is not an installation path and the public installer doesn’t adopt its state. It verifies provider locks, supplies the verified principal, and saves owner-only plans. Missing Linux x64 or ARM64 azd is installed from the SHA-256-pinned official 1.34.0 archive into ~/.local/bin without sudo or replacement. A failed apply stays fenced until refresh-only readback under a clean descendant records a separate closure. The old plan is never reapplied. An existing policy-private Key Vault routes the deployment to fdai-up.sh and its managed host. Each apply requires the exact SHA-256 in an interactive terminal, a plan no older than 20 minutes, unchanged source and actor bindings, and no unresolved prior attempt. It is not a private, shared, staging, or production deployment path.
• onboard.sh and set-gh-actions-config.sh are legacy repository-automation helpers. They are not invoked by public tenant deployment.
• register-runner.sh is legacy run-command recovery. Genesis instead sends registration material through SSH standard input over Bastion.
• check-runner-storage-posture.sh verifies the size and ephemeral placement; teardown-env.sh guards environment destroy. Both fail closed on unsafe runner storage or deallocation without changing the ops hub or state account.
• check-runner-terraform.sh runs inside the shared login-deploy-identity.sh helper before its first Azure call. It refuses a protected workflow on any runner whose Terraform is older than the validated 1.16.1. A Genesis-image runner now pins Terraform 1.16.1 and meets that floor; an enrolled runner built from an older image must re-run the runner image plan and apply to pick up the raised pin.

Production hardening knobs

Environment-specific ceilings are owned by Production deployment hardening.

Non-Azure Prerequisites

• A GitOps host (GitHub or Azure DevOps organization) with an installed GitHub App or service connection scoped to the catalog + fork repos.
• A Teams tenant with a group-connected team for human approval (the hil route). Teams is the default A1 primary. See channels-and-notifications.md.
• A Slack workspace with the FDAI Slack app installed and the mandatory userId ↔ Entra OID mapping store provisioned; required for the P1 Slack A1 channel (channel-specific Slack prerequisites).
• A container registry (ACR or an external registry) that supports signature + attestation storage.
• OpenTelemetry backend: Log Analytics with Application Insights bound to the workspace. A fork may replace the backend through the telemetry provider contracts, but the Azure day-zero inventory does not leave this choice open.

Deployment Artifacts

• IaC in infra/ (see project-structure.md) is the entry point. Every environment is provisioned identically from the same code with per-environment parameters and per-environment isolated state. Terraform exposes primary Event Hub names through event_bus_topics and auxiliary stage, approval, and inventory-ingress names through event_bus_auxiliary_topics so local runtime preparation binds only provisioned topics.
• Entry command: fdaictl provision azure coordinates Terraform against the infra/ HCL modules. Environment values stay outside source control. In source mode the standalone coordinator runs pin source snapshot -> inspect target -> Foundation plan and apply -> build images in the deployment registry -> application plan and apply -> post-provision checks; with --offline-kit it first verifies the signed package. Rerunning an existing offline installation’s work directory with a newer package continues its Foundation under the retained revision (offline kit upgrade). Terraform remains the execution engine and infrastructure source of truth.
• Tenant deployment transport is always manual. The active Azure user approves exact plans, and the managed host executes them with a distinct workload identity. No repository variable, repository secret, GitHub Environment, workflow dispatch, or GitHub runner participates.
• Same signed image is promoted dev → staging → prod; nothing is rebuilt per environment (deployment.md).

Resource Naming Convention

Resource naming and tagging are owned by Deployment Resource Conventions. The focused contract defines CAF prefixes, deterministic length handling, the fdai: tag namespace, deployment- supplied tags, and the fdai:managed=true ownership boundary. This heading remains as a stable target for existing links.

Azure Resource Inventory (minimum set)

The inventory is deliberately minimized for cost-efficiency first. Every choice below is driven by the Cost-Efficiency Principles at the end of this document. The inventory is rendered from the four CSP-neutral contracts (event bus, runtime, secret, workload identity) defined in csp-neutrality.md; Azure is today’s realization of each contract. Concrete tier values, exact names, region, and per-app replica caps are still deployment-specific and tuned per environment; the shape is stable.

#	Resource	Tier	Purpose	Notes
1	AKS Standard cluster	System and user node pools sized by the approved profile	default compute host for new installations	API Server VNet Integration, workload identity, Azure Policy, managed Key Vault CSI, and dedicated workload/API-server subnets are part of the basic profile.
2	Kubernetes workloads (five independent services)	Two replicas per long-running baseline service until measured scaling changes the profile	separates Core, Operator, ingestion, processing, and execution ownership	Typed Deployment, Service, ServiceAccount, HPA, PDB and NetworkPolicy resources use prebuilt signed image digests. Isolated Executor is the only effect-capable service.
3	Kubernetes CronJobs	Profile-bounded schedules and concurrency	scheduled probes, exact-scope WARA observation mode assessment, and out-of-band change detection	Each job uses a dedicated workload identity and bounded history, retry and active deadline.
4	Event Hubs namespace shards	2 x Standard (1 TU, auto-inflate off)	Kafka-wire event bus (endpoints on :9093)	Primary owns governed ingress, DLQs, human approval, and stages. Operational owns canary + DLQ, the dedicated synthetic startup round-trip, raw inventory, Executor command + DLQ, and Executor receipt entities. Core receives the operational bootstrap endpoint and startup topic through deployment configuration.
5	Event Grid inventory system topic + subscription + Diagnostic Settings	global subscription event delivery / Log Analytics	send resource writes/deletes to fdai.inventory.raw and platform diagnostics to the workspace	Terraform adopts one tracked topic with Azure’s canonical lowercase type, assigns the send-only inventory UAMI, and uses the dedicated system-topic subscription API; ambiguous discovery blocks the plan
6	PostgreSQL Flexible Server	Dev: Burstable B1ms, HA disabled, 7-day backup; prod: zone-redundant HA, 35-day geo backup	audit + KPI + pattern library + pgvector T1 embeddings, single store	Terraform allowlists vector and pg_trgm; production requires ZoneRedundant HA; local Compose uses the same Alembic-owned vector extension without a separate bind-mounted initializer
7	Key Vault	Standard	secret backend synchronized through the managed Key Vault CSI provider + workload identity - realizes the Secret contract	Premium (HSM) not required; app never calls a secret SDK
8	User-assigned Managed Identity	-	executor and separately scoped read identities; realizes the Workload Identity contract	The executor stays action-whitelisted; inventory and RCA use distinct read-only identities. Split-service hydration preserves the disabled default when the optional platform output is absent and otherwise accepts only the platform-exported RCA reader.
9	Log Analytics workspace + Application Insights	Pay-as-you-go, 30-day default retention	traces / metrics / logs / audit-forward	an appi-* resource binds to the workspace; retention is UI-configurable post-deploy
10	Container Registry (ACR)	Basic for the public baseline; Premium when private link or network isolation is selected	prebuilt signed images + provenance	tenant deployment mirrors or imports unchanged digests and never builds an image
11	Azure OpenAI accounts + Foundry account/project (opt-in, var.enable_llm)	Standard	T1 embedding + T2 mixed-model deployments, plus a dedicated GPT-4.1-nano web-search prompt agent at 100K TPM	Provisioning requires deployer permission and regional family capacity; otherwise the affected capability degrades to hil-only (see dev-and-deploy-parity.md § Deployer-Scoped LLM Provisioning). Terraform creates an Azure OpenAI account only when at least one OpenAI capability resolves; partner-only resolution keeps Foundry without requesting unavailable OpenAI account quota. Foundry project callers receive Azure AI Developer. The protected web-search stage keeps its exact domain allowlist and real-tool readiness probe. Private mode adds privatelink.services.ai.azure.com; tenant policy owns deny ACL details, which Terraform preserves.
12	ADLS Gen2 document account (opt-in, enable_document_ingestion)	StorageV2 Standard ZRS, HNS	private quarantine, immutable governed versions, derived envelopes	Shared Key and public access disabled in private mode; soft delete + lifecycle; blob and dfs private endpoints
13	Case-history Blob account (enable_case_history)	StorageV2 Standard ZRS	content-addressed prediction/incident case revisions for replay and governed Norns analysis	Shared Key disabled; private container, versioning, change feed, soft delete, bounded old-version lifecycle, Defender scanner private-link access, dedicated case-history UAMI data role, and blob private endpoint; the executor MI receives no Blob role
14	Document ingestion Deployments (opt-in)	Public API + internal worker with ClamAV on AKS	authenticated bounded upload relay plus independently scaled safety scan, extraction, pgvector indexing, and lifecycle events	API, worker, and migration identities are distinct; only the worker receives Event Hubs and OCR access; neither runtime identity receives executor permissions.
15	Operational-history archive + lifecycle CronJob (opt-in)	Private versioned Blob storage + scheduled AKS job	checkpoint, archive, verify, restore-sample, hold, pressure, and database-gated purge coordination	Scheduled execution is observation mode-only under the inventory identity; enforce and certify require external receipts, and only certify can reach the database purge gate.
16	Control-loop canary CronJob	Every 5 minutes	publishes one idempotent event to fdai.control.canary	Dedicated workload identity has only ACR pull and Event Hubs send; the Core records a no-op audit through a separate consumer path.
17	Development operations Function App (opt-in, enable_dev_operations_gateway)	Flex Consumption FC1	relays registered read, write, and execute operations from local development to private resources	dev and private-networking only, enforced by a lifecycle precondition and covered by infra/tests/dev_operations_gateway.tftest.hcl; terminates a public inbound endpoint behind Easy Auth - a developer has to reach it - so it stays off on a closed network; dedicated /27 subnet, private AAD-only deployment and idempotency storage, Easy Auth, separate reader/executor UAMIs, one-time server-issued mutation plan receipts, and no arbitrary URL, ARM path, command, or query surface
18	OHL scale-out evidence VM Scale Set + proposal Job (opt-in, enable_ohl_scale_out_evidence_target)	Uniform Standard_B1s, capacity 1; manual Consumption Job	bounded non-production target and normal-ingress observation mode proposal for governed ops.scale-out evidence	dev, private networking, and the operations gateway are required; the deployment supplies an exact region-available image version and rejects mutable latest; a dedicated /27 subnet has no public IP; the proposal UAMI has only ACR pull and primary Event Hub send; protected provider staging may increase capacity only to 2 before verified rollback
The protected history- request mode targets only the operational-history and decision-evidence storage,
their private endpoints, account-specific untagged A records in the runner’s existing Blob DNS zone, the read-only binding, and the independently declared root Job. The Job dependency includes private readback without public access or storage-key authentication. Exact apply is
re-dispatched by the bot-owned protected-operation workflow so the FDAI maintainer remains a distinct Environment
approver. Stable resource IDs and the versionless Key Vault secret URI exclude unrelated compute drift. The
deploy UAMI assignment is additive; if it exists outside Terraform state, the plan imports it only after its
storage scope, principal, and role match. Scoped post-apply skips unrelated inventory-image drift; replacements remain blocked. Set ENABLE_OPERATIONAL_HISTORY=true before the initial plan.
The local parity profile starts the same five service packages against loopback PostgreSQL and
Redpanda, with filesystem-backed document objects and ClamAV. It uses plaintext Kafka only on the
loopback broker; deployed modules continue to require Event Hubs Kafka with service-owned managed
identities and service-specific PostgreSQL roles.
The active local runtime uses pgvector/pgvector:pg16 on port 5432. A second local
pgvector/pgvector:pg16 cluster on port 5433, with its own volume, is reserved for destructive
migration validation because Alembic-managed roles are PostgreSQL cluster-global. Local runtime
preparation emits the dedicated validation DSN, and detached central validation maps only that DSN
to integration tests. The second cluster is a local validation dependency, not part of the Azure
resource inventory above.
Additional identity, channel, and console elements are deployment-owned or opt-in:

• App registrations × 3 - split audiences per user-rbac-and-identity.md#41-app-registrations: fdai-console-spa (SPA sign-in, PKCE), fdai-api (Web API audience for console + ChatOps backend), and fdai-approval-bot (Teams SSO). None hold the executor identity. Step-by-step az creation: ../runbooks/entra-app-registration.md. After a console apply, the deploy workflow safely retries an idempotent sync of the Terraform-emitted Static Web App origin into the target tenant’s SPA redirect URIs. A tenant mismatch or missing Graph permission blocks the deployment rather than shipping a console that cannot sign in.
• Entra security groups × 5 - aw-readers, aw-contributors, aw-approvers, aw-owners, aw-break-glass. Deployment-owned; objectIds injected via config and validated at startup (user-rbac-and-identity.md#42-security-groups-slots).
• Conditional Access policies - phishing-resistant MFA on aw-approvers/aw-owners, compliant-device on aw-owners, dedicated hardware token + sign-in alert on aw-break-glass. Available on Entra ID P1 (user-rbac-and-identity.md#43-conditional-access).
• Azure Bots - the approval bot remains deployment-owned. The independent System Knowledge Service root always creates its UAMI, private Blob claim container, and HTTPS endpoint through the protected workflow. Bot Framework adds an F0 Bot and Teams channel. Outgoing Webhook adds no Bot or Graph resource and binds its Teams-issued HMAC key from Key Vault only after endpoint bootstrap. The platform document_blob diagnostic setting continues to cover the reused claim storage account.
• Signed human approval webhook - production supplies the URL and a 32+ character HMAC secret through CI secrets. Terraform stores both in Key Vault; the core reads URL + secret and the Operator API receives only the callback secret. A group-connected approval team and channel are separate deployment slots shared by Core and Operator; RBAC group ids remain role assignments only.
• Topic-scoped Event Hubs roles - the executor receives Data Owner on each currently provisioned hub entity, not the namespace. Inventory and canary can send only to their own topics. The Operator API command identity sends proposals, decisions requiring human approval, and pantheon object messages, and receives the stage topic. Document ingestion API and worker identities can send only to fdai.pantheon.objects; the worker can also receive that topic for gated stage commands.
• Static Web Apps (Free tier, opt-in) - hosts the read-only console when enable_console=true.
• Design-mocks Static Web App (Free tier, opt-in) - hosts the isolated static design-review artifact when enable_design_mocks=true. The artifact builder copies only allowlisted browser assets from index.html, mocks/, examples/, and the shared agent icons. Static Web Apps authentication redirects anonymous requests to Microsoft Entra ID and admits only invited members of the reviewer role. The protected exact-apply workflow reads the deployment token from the Terraform-owned resource, masks it, and passes it only through the SWA_CLI_DEPLOYMENT_TOKEN environment variable to an exact-version Static Web Apps CLI. The workflow publishes the allowlisted artifact and verifies the authentication redirect. This path targets only module.design_mocks, rejects any planned resource change outside that module, and skips core canary and other runtime reconciliation. The token is never committed or stored as a repository secret.
• Workload identity federation - CI/CD short-lived OIDC tokens; not a resource, no cost.

Document ingestion deployment

Set enable_document_ingestion=true only with enable_llm=true, a resolved t1.embedding capability, the console API audience, all five Entra RBAC group ids, and explicit ingestion CORS origins. Terraform then provisions:

• distinct API, worker, and migration UAMIs plus role-scoped PostgreSQL DSNs. API can publish but not consume stages; worker alone receives stages and optional Document Intelligence OCR access;
• a StorageV2 account with HNS, the documents and derived filesystems, lifecycle controls, no Shared Key, and Terraform-owned Defender scanner private-link access;
• blob and dfs private endpoints. The app VNet links to the endpoint zones; the ops runner resolves Blob through an A record in its existing central Blob zone, while the DFS zone links to both VNets. This avoids linking one VNet to duplicate zones with the same namespace;
• a public ingestion API Container App and an internal worker app with replica-local ClamAV; initial cutover may snapshot exact empty legacy sidecar probes only for rollback, while the new revision still requires all three strict probes;
• a manual migration job that applies the document metadata and pgvector schema before traffic.

The deploy-dev workflow exposes deploy_document_ingestion. Plan remains the default. An apply runs the migration job; independent service apply masks the Key Vault admin DSN, advances its branch, and forces the declared role through PostgreSQL PGOPTIONS before traffic. It verifies both revisions and publishes ingestion_gateway_fqdn; build with VITE_INGESTION_API_BASE_URL=https://<fqdn>. Production gates require private networking and digest-pinned FDAI plus ClamAV images.

Document OCR uses a separate protected action. preserve retains the previously applied provider and Document Intelligence resource state. use_local_retain switches the worker to local kor+eng Tesseract without deleting Azure, use_azure_provision provisions or retains the private Document Intelligence account, and deprovision_use_local selects local OCR before removal. A Settings-originated plan-ocr-<proposal>-<digest> request is resolved from the authoritative PostgreSQL proposal, policy, and plan-request records before Terraform variables are set. Plan remains the default, and exact apply still requires separate protected-environment approval. The standalone module pins AzureRM ~> 4.14; its committed provider lock records the tested provider release for repeatable module tests. The workflow delegates OCR desired-state reduction to a focused script, keeping the protected job within its reviewed line and step budgets without changing approval or apply gates.

The public Static Web App never reaches ADLS directly. It streams through the authenticated gateway because the Storage account stays private. The gateway uses Managed Identity for ADLS, Event Hubs, and Azure OpenAI; no connection string or Storage account key is created.

Explicitly not provisioned on day zero (deferred to a later phase when a measured need justifies them):

• Service Bus namespace and Event Grid custom topics - the event bus is the Kafka endpoint on Event Hubs (csp-neutrality.md § Event Bus Contract); a subscription-scoped Event Grid subscription for inventory writes/deletes is enabled by default, but no separate custom topic is created.
• Dedicated vector database (pgvector inside PostgreSQL suffices at initial scale).
• Front Door, Application Gateway, API Management (no public inbound endpoint; console is read-only static hosting).
• Secondary-region resources for DR (Phase 4 - TBD; see Implementation Focus).

Compute Shape (current core and five-service target)

The current control-loop Core deploys as one signed image and one Python process. The five-service target adds an internal Isolated Executor and removes executor roles from Core at cutover; Operator, ingestion API, and ingestion worker remain separate. The prior topology is the rollback artifact, and the execution plan tracks every gate.

• Runtime: python -m fdai starts the Kafka consumer and composes routing, quality, risk, and audit stages. fdai-isolated-executor stays observation mode-only by default. The explicit cutover uses a stable Core receipt consumer group, the versioned command/receipt transport, the existing guarded direct-API executor, and a dedicated gateway caller identity. Deployment venue and RUNTIME_ENV remain independent.
• Health: internal /live and /ready probes open only after the authoritative control loop is assembled. The isolated Executor uses the same internal /live and /ready contract on FDAI_ISOLATED_EXECUTOR_HEALTH_PORT. The ingestion API uses /healthz; its internal worker uses /live and /ready.
• Core startup round trip: the independent Core waits 12 seconds for its unique operational Event Hubs consumer group to join before publishing the synthetic startup record. The per-probe deadline is 30 seconds and the phase deadline is 75 seconds, leaving bounded headroom for both default attempts. Deployments may tune these ordered values, but the probe remains not ready unless it consumes the exact record it published.
• Replica floor: the default is one replica. A zero floor without a verified Kafka scaler would never wake on Event Hubs data, so Terraform does not claim scale-to-zero.
• Graduation rule: the target is Core, Operator, Ingestion API, Processing Worker, and Isolated Executor; authority cutover follows every gate in Service Graduation and Data Ownership.
• Identity split: Operator API read/command and ingestion API/worker/migration principals stay distinct. The worker receives Saga/Muninn objects only from fdai.pantheon.objects and sends stage facts to fdai.pipeline.stages; ingestion_cohost_worker=true returns both scopes to the API identity.
• Cross-tenant SharePoint: sharepoint_connector_enabled=true activates the FDAI-native connector. The Azure ingestion UAMI supplies a federated assertion to the application registered in the Microsoft 365 tenant, which issues the Microsoft Graph token after administrator consent. Root composition forwards only the target tenant, application, site, drive, redirect allowlist, and server-owned document policy into the document ingestion module; it doesn’t construct connector state. The protected service workflow requires an explicit sharepoint_connector_transition and seals the exact enable or disable environment delta into the plan bundle. Power Platform isn’t deployed or invoked.
• Executor deployment and cutover: enable_isolated_executor=true provisions the internal app and a dedicated UAMI with ACR pull, command receive, receipt/DLQ send, and state-secret read only. The default is false; the private-runner workflow remains plan-only by default and installs its checksum-pinned GitHub CLI whenever model verification or isolated Executor image binding requires gh. Before Terraform initialization or any ACR command, it resolves the exact GHCR subject digest and verifies only the registry-hosted OCI bundle with --bundle-from-oci, the selected revision through --source-digest, the SLSA v1 predicate, and the repository’s container-supply-chain.yml signer. It has no GitHub API bundle fallback, keeps Docker authentication in owner-only temporary files, removes them on exit, and records no binding on failure. It syntax-checks embedded plan-metadata code, verifies the Core artifact under ghcr.io/<owner>/<repo>/fdai-core-control-plane, sends the host-only ghcr.io registry URI required by the ACR import API, binds the identical ACR digest, and includes the latest revision in health checks. Binding validates the ACR resource ID before import and emits content-free progress for registry lookup, import acceptance, and digest readback. Executable fake-boundary tests cover successful binding, an invalid ACR identity, and an import failure without exposing credentials. promote_runtime_image=true imports that verified digest into the ACR fdai repository without rebuilding, while exact apply rejects promotion, consumes only the protected plan, and restores the same runtime digest for convergence. enable_isolated_executor_authority_cutover=true additionally requires the development operations gateway, removes gateway and vertical effect access from Core, authorizes the isolated identity, and keeps Core transport/read access. verify_executor_effect=true runs one reversible NSG rule probe through an explicit pseudo-terminal on the non-interactive runner and preserves the remote exit status. Duplicate delivery shares one issued-at timestamp to retain immutable action and command identity; cleanup receives a fresh bounded deadline. The workflow checks the effect through Azure Resource Manager, rejects duplicate writes, records offsets and terminal receipts, cleans up, and fails after 900 seconds.
• OHL evidence target: enable_ohl_scale_out_evidence_target=true adds one dedicated Uniform VM Scale Set and one manual proposal Job only in dev. It requires private networking, the development operations gateway, an exact OHL_SCALE_OUT_EVIDENCE_IMAGE_VERSION, a retry-stable OHL_SCALE_OUT_EVIDENCE_CAMPAIGN_ID, the human OHL_SCALE_OUT_EVIDENCE_INITIATOR_PRINCIPAL_ID, and the non-secret OHL_SCALE_OUT_EVIDENCE_SSH_PUBLIC_KEY. The protected platform workflow owns the target, subnet, proposal UAMI, and Job, while service-deploy independently rolls the exact-revision Core and Executor images before the evidence run begins. Starting the Job publishes one observation mode proposal through the normal ingress; it holds no provider-effect authority.

Bootstrap Sequence

Provisioning is IaC-driven, but the logical bootstrap order to a first live event MUST be honored. Any earlier stage failing halts and unwinds; the deployment does not proceed to a later stage with a broken earlier one.

• Observation mode-only on first deploy: no rule / action starts in enforcement mode, ever. Promotion is a separate act (rule-governance.md).
• Migrations MUST run before the first control-loop tick. Workload containers do not migrate on start (to keep replicas identical + prevent races). CI runs root rollback checks before it applies service-owned migrations to an isolated service database, then runs service-dependent checks at that service head. Every tracked migration under alembic/versions/ defines downgrade(), but schema/data rollback can be destructive. Rehearse backup/restore and each migration-specific downgrade in staging before using it. The protected path uses 10-second connection, 5-minute database lock, 15-minute statement, and 20-minute migration-stage deadlines. The database cancels over-budget DDL and releases locks before the service plan is applied.
• Authoritative catalogs load after migrations from the exact verified Core image. The materialization Job applies a five-minute PostgreSQL statement deadline for large immutable projections. A management-plane prebind or prestart is accepted only when readback proves the digest-pinned image and latest successful execution; the VNet runner then compares every expected repository projection with PostgreSQL before reporting success. The protected model Settings workflow refreshes the model projection, creates a missing runtime Settings row without replacing existing runtime evidence, and verifies both environment-bound rows in a read-only transaction so a fresh Console can render setup controls. Runtime Settings materialization enumerates every shared RUNTIME_SETTING_SPECS entry, preserving each validated environment value and availability field rather than filtering the projection to one settings group. When operational evidence issuance is enabled, it also reads the verifier readiness endpoint once with a bounded timeout; an unreachable or malformed response leaves every operational evidence purpose unavailable.
• Cost Governance profile and post-deploy checks: plan-cost-* and apply-cost-* request identities bind one exact digest-pinned Cost Governance image to Core plus its two Jobs without changing package enablement or action mode. Post-deploy smoke tests and the synthetic canary remain defined in operating-and-verification.md.

Distribution and Deployment Responsibility Matrix

The upstream repo ships everything customer-agnostic. A downstream distribution may limit or extend capabilities through dependency injection without editing core/. A deployment supplies environment values, identity, secret references, and promotion state outside source control (generic-scope.instructions.md).

Concern	Upstream distribution	Downstream customization	Deployment configuration
IaC modules	parameterized modules	optional module overlays	environment tfvars, state, secret references
Provider adapters	Protocols and Azure implementations	optional replacement implementation through DI	endpoint and identity bindings
Rule catalog	generic seed and schemas	additional rules and policy overlays	assignments, exemptions, promotion state
human approval and RBAC	role and approval contracts	optional channel adapter	Entra groups, channel ids, approver bindings
Models	capability registry and resolver	optional provider adapter or preference overlay	resolved endpoints, quota, region, identity
Runtime values and capability license	validated key schema, signed-token contract, packaged public key, and Trial ceiling	no tenant values; optional composition-owned verifier or narrower catalog	environment variables and Key Vault references, including a versionless token reference plus image and deployment digests and no private key

Runtime Configuration Matrix

All values MUST come from env vars or Key Vault refs at runtime. No environment value is committed to this repo. The list below is the schema of keys the deployment expects; the full expanded catalog and defaults are authored during the inventory PR.

The Console projects a safe subset through Settings > Runtime policies. Readers can compare the environment, durable override, and effective value. Owners can change only the documented allowlist through revision and audit checks. IRP, analyzer, inventory freshness, and retention tick changes apply dynamically at their next event or Job boundary. Logging level and case retention or deletion day changes require a headless runtime restart. Deployment identity, transport, endpoint, secret, promotion, and test-only keys remain outside the editable surface. Incident automatic-open settings are also startup-bound. Materialization exposes the validated default minimum MEDIUM, while an environment value or audited durable override may select a different supported severity. The effective value takes effect only after Core restarts; repeat, correlation, evidence, deduplication, and downstream authority gates remain independent.

Key	Source	Owner	Notes
AZURE_TENANT_ID	env	deployment	non-secret
AZURE_SUBSCRIPTION_ID	env	deployment	non-secret
AZURE_RESOURCE_GROUP	env	deployment	target resource group
KAFKA_BOOTSTRAP_SERVERS	env	deployment	Event Hubs Kafka endpoint (<ns>.servicebus.windows.net:9093); realizes the Event Bus contract
FDAI_AUXILIARY_KAFKA_BOOTSTRAP_SERVERS	env	deployment	Operational Event Hubs Kafka endpoint used only by the core canary and raw inventory consumers; unset falls back to the primary endpoint for non-Azure adapters.
KAFKA_SECURITY_PROTOCOL	env	deployment	SASL_SSL on Azure; provider-specific value elsewhere
KAFKA_SASL_MECHANISM	env	deployment	OAUTHBEARER on Azure
FDAI_STATE_STORE_DSN; FDAI_LICENSE_TOKEN / FDAI_LICENSE_IMAGE_DIGEST / FDAI_LICENSE_DEPLOYMENT_BINDING / FDAI_LICENSE_TOKEN_REVISION	KV ref + env	upstream / deployment	The state-store value is the PostgreSQL connection URI for audit and KPI. The license token arrives only through a Container Apps native Key Vault secret reference; image and deployment values are lowercase SHA-256 digests checked against signed claims, and the token-revision digest forces a new Core revision on renewal without becoming an entitlement input. The shipped runtime leaves absent, invalid, expired, or misbound input read-only and denies all three Thor action paths. A local Git checkout may ignore the token only after its fixed owner-only secrets/integrity-signing-key.pem cryptographically matches the packaged upstream integrity public key; deployed runtime never opens that path.
FDAI_APPROVAL_PROFILE_JSON	protected GitHub secret or variable -> env	deployment / Mimir	Optional deploy-time bootstrap for one single-operator production approval-profile revision. materialize_tfvars.py validates the shared approval_profile_policy_digest, refuses the value with the full-authority development profile, and keeps it absent by default. After Mimir activates an approval policy revision, Core reads the active policy-administration pointer and treats this environment value only as a fallback; an active pointer wins and a mismatch is logged and audited.
FDAI_CASE_HISTORY_CONTAINER_URL / FDAI_CASE_HISTORY_MI_CLIENT_ID / FDAI_CASE_HISTORY_RETENTION_DAYS / FDAI_CASE_HISTORY_DELETION_DAYS / FDAI_CASE_HISTORY_RETENTION_TICK_SECONDS	env	upstream / deployment	Private Blob container URL, dedicated attached UAMI client id, active-retention/deletion-due offsets, and the bounded Muninn retention cadence for immutable case revisions. Azure defaults active retention, deletion due, superseded Blob versions, and change feed to 30 days. Terraform derives the storage and identity bindings, validates deletion is not earlier than retention, and startup fails if the dedicated identity id is missing or matches the executor identity; public/key-auth fallback is not used. The retention tick defaults to 86400; deployment overrides remain bound to the approved data schedule.
FDAI_OPERATOR_MEMORY_DSN	KV ref	upstream	Postgres DSN for human approval-approved operator memory. Same source as FDAI_STATE_STORE_DSN day-zero (single Flexible Server); a deployment MAY split it later without touching core code.
FDAI_T1_PATTERN_LIBRARY_DSN	KV ref	upstream	Postgres DSN for the pgvector-backed T1 pattern library. Same source day-zero; wired identically.
FDAI_CHANGE_MI_CLIENT_ID / FDAI_RESILIENCE_MI_CLIENT_ID / FDAI_FINOPS_MI_CLIENT_ID	env	deployment	Client ids of the three attached per-vertical user-assigned managed identities. They identify delivery principals only; execution authorization and fork-owned action whitelists still decide whether a selected identity may act.
FDAI_INVENTORY_DSN	KV ref	upstream	PostgreSQL DSN used only by the scheduled inventory collector to stage immutable candidates and atomically promote the active graph.
FDAI_INVENTORY_SCOPES / FDAI_INVENTORY_RESOURCE_TYPES	env	deployment	Comma-separated subscription scopes and optional CSP-neutral resource-type subset. Empty scope fails startup.
FDAI_INVENTORY_SOURCES	env	upstream	Ordered fallback list: arg,arm by default; declarative is accepted only with a fixture path and SHA-256.
FDAI_INVENTORY_MANAGEMENT_ENDPOINT / FDAI_INVENTORY_MANAGEMENT_AUDIENCE	env	deployment	Validated HTTPS ARM root and OIDC audience pair. Override both for an approved sovereign-cloud or validated Resource Management Private Link path.
FDAI_INVENTORY_FRESHNESS_SECONDS	env	upstream	Maximum active snapshot age before the projection becomes stale and graph-dependent autonomy degrades to human review. Default 86400.
FDAI_INVENTORY_RECONCILIATION_INTERVAL_SECONDS	env	upstream	Healthy full-scan interval and the observed-state freshness ceiling declared by the scheduled projector. The default is 21600.
FDAI_INVENTORY_CHANGE_MIN_INTERVAL_SECONDS	env	upstream	Floor between change-triggered reconciliations, preventing a change storm from becoming a scan storm. The default is 120.
FDAI_INVENTORY_PROGRESS_DEADLINE_SECONDS	env	upstream	Longest a source may run without a progress batch. Each batch re-arms the deadline. The default is 900.
FDAI_INVENTORY_ATTEMPT_DEADLINE_SECONDS	env	upstream	Absolute attempt ceiling. It must be at least the progress deadline and at most 1740, below the abandonment window. The default is 1500.
FDAI_INVENTORY_ARG_REQUESTS_PER_SECOND	env	upstream	Sustained Azure Resource Graph budget shared by every shard in one scan. The default is 3.
FDAI_INVENTORY_LOOP_SECONDS	env	dev-only	Delay between local --loop ticks. Deployed execution uses the cron. The default is 60.
FDAI_ANALYZER_TARGETS / FDAI_ANALYZER_WINDOW_SECONDS / FDAI_ANALYZER_MAX_DISCOVERED_TARGETS	env	deployment / upstream	Explicit analyzer targets, metric window, and the bounded inventory-discovery ceiling. Explicit and supported inventory targets merge deterministically. A malformed value or unreadable configured projection fails closed; only a fully resolved empty set is a clean no-op.
FDAI_TRACE_TOPOLOGIES_JSON	env	deployment	Optional bounded topology_ref, resource_ref, and ordered expected_hops declarations for workspace-based Application Insights continuity checks. The protected deployment workflow passes the TRACE_TOPOLOGIES_JSON repository variable through Terraform. Empty disables only this check; it does not disable the metric analyzers.
KAFKA_TOPIC_EVENTS	env	deployment	primary event ingest topic
KAFKA_TOPIC_DLQ_SUFFIX	env	deployment	dead-letter suffix (default .dlq)
FDAI_EXECUTOR_COMMAND_TOPIC / FDAI_EXECUTOR_RECEIPT_TOPIC	env	upstream / deployment	Isolated Executor command and versioned terminal receipt topics. Defaults are object.executor-command and object.executor-receipt; they must remain distinct.
FDAI_ISOLATED_EXECUTOR_MI_CLIENT_ID / FDAI_ISOLATED_EXECUTOR_AUTHORITY_CUTOVER	env	deployment	Dedicated isolated identity plus the exact default-off cutover marker. The identity has transport/state access in observation mode; cutover additionally makes it the sole development-gateway caller while Core keeps only transport/read access.
FDAI_ISOLATED_EXECUTOR_DEPLOYED	env	upstream / deployment	Exact opt-in marker for the independently deployed process. Only 1 starts this entry point; environment names do not imply deployment venue or authority.
FDAI_ISOLATED_EXECUTOR_HEALTH_PORT / FDAI_ISOLATED_EXECUTOR_INSTANCE_ID	env	upstream / deployment	Internal health port (default 8000) and bounded receipt-attribution instance id. Container Apps supplies HOSTNAME when the explicit instance id is unset.
LLM_MODE	env	deployment	local-fake for explicit services/core-control-plane/tests/mocks or azure for authoritative profiles. Environment does not select the binding; see dev-and-deploy-parity.md § Parity Contract.
LLM_RESOLVED_MODELS_PATH	KV ref	deployment	required when LLM_MODE=azure; points at the resolved-models.json written by the bootstrap resolver
T1_SIMILARITY_THRESHOLD / T1_MIN_SUCCESS_RATE	env	deployment	Validated [0,1] floors for similarity and historical success before learned-action reuse. Defaults are 0.8 and 0.9.
QUALITY_GATE_CONFIDENCE_THRESHOLD / QUALITY_GATE_QUORUM	env	deployment	Validated confidence floor and independent-model agreement quorum for T2. Defaults are 0.7 and 2; quorum cannot be lower than two.
RULE_CATALOG_REF	env	deployment	git ref of catalog snapshot
AUTONOMY_MODE_DEFAULT	env	deployment	MUST default to shadow
FDAI_LOG_LEVEL	env	upstream	Python logger level for the core app (DEBUG / INFO / WARNING / ERROR). Default INFO.
FDAI_OPERATOR_API_LOCAL_AZURE_CLI	env	local-only	Explicit CLI-principal debug alternative with a fixed role ceiling. Paired with VITE_LOCAL_AZURE_CLI_AUTH=1.
FDAI_OPERATOR_API_DEV_MODE	env	test-only	Authentication bypass for automated Operator API tests. The VS Code full-stack profile MUST NOT set it.
FDAI_OPERATOR_API_LOCAL_ENTRA	env	local-only	Canonical interactive profile. Browser Entra JWT and App Roles match deployment; the server Azure CLI session is confined to Azure adapters.
FDAI_START_PANTHEON	env	upstream / local	Disable-only control for the 15-agent runtime. Unset means enabled; 0, false, no, or off disables it. Event Hubs variables select transport and do not activate the Pantheon.
FDAI_LOCAL_SCENARIO_REPLAY	env	test-only	Generated scenario replay for automated tests and explicit mock applications. Interactive local startup rejects it.
FDAI_LOCAL_AZURE_DISCOVERY	env	local-only	Azure discovery is mandatory. Unset or 1 uses read-only AzureCliInventory; 0 is rejected and never selects a synthetic graph.
FDAI_LOCAL_AZURE_SUBSCRIPTION_ID	env	dev-only	Optional subscription passed to every local az group/resource list call. When omitted, discovery uses the active subscription in the selected Azure CLI profile. Never commit a populated value.
FDAI_LOCAL_AZURE_CONFIG_DIR	env	dev-only	Optional isolated Azure CLI profile. When omitted, the adapter removes an inherited AZURE_CONFIG_DIR and uses the default profile.
FDAI_POLICIES_ROOT	env	deployment	absolute path to the OPA / Rego bundle root consumed by T0 and the verifier. Defaults to the in-repo policies/ when unset.
FDAI_MI_CLIENT_ID	env	upstream	User-assigned MI client id for the current process. The core receives the executor id; the inventory job receives its distinct read-only discovery id.
FDAI_INVENTORY_RECONCILIATION_INTERVAL_SECONDS	env	upstream	Healthy full-scan interval for the Inventory Job. The default Job cron wakes every 10 minutes, but PostgreSQL attempt state skips scans until this interval is due and retries a newer failed or abandoned attempt on the next tick. An existing ontology projection whose generation is not the active snapshot, or whose release is not the running release, is pending and recollects after the minimum poll interval, so a deployment that changes the ontology release does not leave answers partial for this interval. A degraded pending projection retains its failure evidence but does not block the next authoritative full scan; an incomplete current projection still fails. A promoted full snapshot that skipped that degraded base is rebound to the actual manifest generation, while a missing manifest generation remains blocked. A fresh full snapshot can derive its prior state from retained complete topology history when its declared base is the exact skipped incomplete generation. Recovery can reuse an identical content-addressed state-transition coverage record across a recalculated batch without duplicating or weakening it. Global and active-scope projection checkpoints never advance past the journal high watermark observed by that snapshot append, even when later writes commit concurrently. Repository Variable ENABLE_RUNTIME_CALL_EVIDENCE=true independently enables the deployed runtime-call source after Operator state migration. The independent Operator API and channel edge receive distinct fixed semantic outbox namespaces so one runtime cannot claim the other’s pending request. The independent-service runtime_call_evidence_transition guard permits only those exact namespace additions or removals for the matching runtime. Within the configured freshness window, runtime-call completeness selects the newest observation per exact endpoint pair before ARM replica verification; a newer unverified observation fails closed, and a newer unpaired observation becomes incomplete after the trailing guard. Protected plan-runtime-* and apply-runtime-* requests target sibling flag, workspace, and exact-image transition resources whose verified updaters and rollback affect only the existing Job. The workspace transition exists separately so a previously created flag resource never hides a newly required binding. A changed attested digest permits only the image updater’s state-only replacement. Runtime-only convergence checks all three transition states, and independent readback verifies the flag, workspace digest, and selected image digest. Environment-integrated Container Apps log rows can carry an empty _ResourceId; the source then requires the platform container name and ID to match the exact ARM replica response instead of accepting an unbound name. The scope guard rejects every other address.
FDAI_INVENTORY_OPERATOR_REQUESTED	env	deployment / one-shot	Set to 1 only on an explicitly requested one-shot Job execution to make a healthy, idle adaptive scheduler collect immediately. The request does not bypass an active collection, provider backoff, throttling, an open circuit, or any inventory evidence gate. The persisted Job template leaves this value unset. The Core migration head admits the reviewed runtime_calls link type in snapshot and real-time inventory tables; downgrade removes only rebuildable runtime-call links before restoring the earlier allowlist.
FDAI_EMAIL_ENDPOINT / FDAI_EMAIL_SENDER_ADDRESS / FDAI_EMAIL_RECIPIENT_ADDRESSES_JSON / FDAI_NOTIFICATION_MI_CLIENT_ID	env	upstream / deployment	Enables the ACS Email A2/A4 channels. Terraform derives the endpoint and Azure-managed sender, attaches a dedicated notification MI, and injects the client id. Deployment configuration supplies recipients through NOTIFICATION_EMAIL_RECIPIENTS_JSON; no access key or connection string enters the app. Partial configuration fails startup.
FDAI_CONSOLE_BASE_URL	env	deployment	Public HTTPS origin used to build read-only evidence links in incident email. Terraform derives it from the Static Web App hostname when the Console is enabled. Without it, email delivery continues and the renderer omits the incident CTA.
FDAI_MEASUREMENT_MODE	env	upstream	Selects an opt-in Container Apps Job entry point in infra/modules/measurement-runners/: baseline runs frozen-scenario regression measurement, growth drains reviewed outcomes into pattern-growth intake, and operational-promotion evaluates immutable action-specific evidence without promoting it. Every job is disabled by default and uses the dedicated non-executor measurement identity. Action authority remains governed independently by promotion and risk gates.
FDAI_REVISION / FDAI_OPERATIONAL_PROMOTION_EVIDENCE_ROOT / FDAI_OPERATIONAL_PROMOTION_MANIFEST	env	deployment	Required by the opt-in operational-promotion Job. The revision is a full immutable source revision, the root is an absolute container path supplied by the pinned image or a protected read-only mount, and the manifest path is relative to that root. Evidence files contain bounded canonical facts and digests, not raw provider payloads.
FDAI_DIRECT_API_FAKE	env	test-only	1 swaps the executor direct-API path for the in-memory observation mode fake in automated tests. Interactive local preparation never sets it. Without exactly one scoped development operations gateway, managed-resource execution stays unavailable. Mutually exclusive with FDAI_DEV_OPERATIONS_GATEWAY_URL.
FDAI_TOOL_CALL_FAKE	env	test-only	1 swaps the executor tool-call path for RecordingToolExecutor in automated tests. Interactive local startup does not wire an executor.
FDAI_WORKFLOW_SHADOW	env	upstream	Event-triggered catalog Workflows run in non-mutating observation mode by default. Set 0, false, no, or off only for explicit maintenance disablement.
FDAI_WORKFLOW_ENFORCE_ALLOWLIST	env	deployment / local	Comma-separated Workflow names an Owner may start with mode=enforce. Requires Event Hubs command transport. Action steps re-enter the normal promotion/risk/human approval/executor path.
KAFKA_TOPIC_EVENTS / FDAI_STAGE_TOPIC	env	upstream / local	Event and stage topics shared by deployed runtime and Azure-backed interactive transport. When both Kafka bootstrap and event topic are absent, interactive local uses fdai.events plus the bounded local EventBus/SSE adapters.
FDAI_IRP_ENABLED / FDAI_IRP_BUDGET_SECONDS	env	upstream	Enables alert-shaped event handling through the budgeted investigation -> typed proposal path. The proposal re-enters the standard risk/human approval/executor loop.
FDAI_CHAOS_CONTEXT_JSON / FDAI_CHAOS_ENFORCE	env	deployment	Runtime context for promoted chaos injectors. Enforce stays disabled unless the explicit flag is 1, the scenario is promoted, and both injector and probe are registered. A failed protected scenario-lab plan emits only allowlisted diagnostic categories, Terraform addresses, and Azure error codes before cleanup shreds the raw provider log.
FDAI_JIRA_BASE_URL / FDAI_JIRA_ACCOUNT_EMAIL / FDAI_JIRA_API_TOKEN_SECRET / FDAI_JIRA_TOOL_MAP_JSON	env + KV ref	deployment	Configures the production JiraToolExecutor. TOOL_MAP_JSON maps tool.open-incident-ticket to a Jira project key. The token value is resolved from FDAI_SECRET_<API_TOKEN_SECRET> (KV-backed); never place the token in the mapping. Requires FDAI_STATE_STORE_DSN for the durable Jira ledger and distributed resource lock.
FDAI_JIRA_ENFORCE	env	deployment	Default unset/0 keeps Jira observation mode-only. 1 permits enforce requests only after the ActionType promotion gate and risk/decision requiring human approval also permit enforce. Observation mode receipts are never linked as real incident tickets.
FDAI_PROFILE_ID	env	deployment	selects one profile from rule-catalog/profiles/ (see rule-catalog-profiles.md). Bound at startup; blank or absent keeps the whole catalog.
FDAI_NARRATOR_PROVIDER / FDAI_NARRATOR_BASE_URL / FDAI_NARRATOR_MODEL / FDAI_NARRATOR_API_VERSION / FDAI_NARRATOR_API_KEY	env + KV ref	deployment	Operator-console narrator translator config (see operator-console.md); API_KEY MUST go through KV. Empty provider = deterministic fallback.
FDAI_CHATOPS_WEBHOOK_SECRET / FDAI_CHATOPS_TIMEOUT_SECONDS / FDAI_TEAMS_APPLICATION_ID / FDAI_TEAMS_TENANT_ID / FDAI_TEAMS_APPROVAL_TEAM_ID / FDAI_TEAMS_APPROVAL_CHANNEL_ID / FDAI_TEAMS_APPROVAL_ACTIVITY_URL / FDAI_TEAMS_BOT_MI_CLIENT_ID / FDAI_TEAMS_ALLOWED_SERVICE_URLS_JSON / FDAI_TEAMS_JWKS_URL / FDAI_TEAMS_PRINCIPAL_MAP_JSON / FDAI_SLACK_TEAM_ID / FDAI_SLACK_PRINCIPAL_MAP_JSON	env + KV ref	deployment	Human approval callbacks require PostgreSQL, Kafka, a bounded workflow timeout, and the shared HMAC secret for the internal Slack relay. Teams requires the complete Bot application, tenant, group-connected team/channel activity endpoint, dedicated Bot managed identity, service-token trust inputs, and principal map; Slack can operate independently when its workspace and principal map are complete. Missing or partial channel configuration remains unavailable.
FDAI_KAFKA_BOOTSTRAP_SERVERS / FDAI_HIL_DECISION_TOPIC	env	deployment / upstream	Event Hubs Kafka endpoint used by the Operator API durable decision outbox; topic defaults to fdai.hil.decisions. Operator persists first, publishes the exact decision fields, and marks delivery only after broker acceptance. Core consumes the same topic and owns resume/execution.
FDAI_NOTIFICATION_RECEIPT_SECRET / FDAI_NOTIFICATION_RECEIPT_TOPIC	env + KV ref	deployment	Authenticates the publication-receipt callback a publishing automation signs. The topic is the fixed logical name fdai.notifications.delivery-receipts, multiplexed over the existing primary physical topic; a different value blocks startup. The secret requires PostgreSQL, the semantic physical topic, and a configured Kafka receipt transport. Without the secret the ingress stays bound and fail-closed, so an unsigned callback is refused rather than silently dropped. Core alone applies the resulting delivered or retryable_failed transition.
FDAI_TEAMS_NOTIFICATION_ENDPOINT / FDAI_NOTIFICATION_BINDINGS_JSON	env + KV ref	deployment	Supplied together by Terraform only when enable_teams_notification_delivery and teams_notification_binding activate A2/A4 Teams delivery. Saving or testing an endpoint in the Console never sets them. The control plane holds read-only authority on the endpoint secret and refuses the seeded unconfigured placeholder at startup.
FDAI_TEAMS_NOTIFICATION_ACTIVATION	env	local	Local-only explicit activation. When set, the control plane reads the encrypted Operator-owned Teams binding record from the shared loopback database instead of a plaintext endpoint value. Unset means a saved binding delivers nothing.
FDAI_KAFKA_BOOTSTRAP_SERVERS / FDAI_SEMANTIC_TURN_REQUEST_TOPIC / FDAI_SEMANTIC_TURN_PROJECTION_TOPIC	env	deployment / upstream	Operator semantic transport configuration. All three values are configured together; partial configuration fails startup. The request and projection values name the provisioned operator-core-request and core-operator-projection entities. Optional FDAI_SEMANTIC_TURN_CONSUMER_GROUP_ID and FDAI_SEMANTIC_TURN_KAFKA_CLIENT_ID override the stable service defaults. FDAI_COMMAND_MI_CLIENT_ID selects the command identity for OAUTHBEARER; no connection string or shared key is supported. Local preparation copies these values only when the same topics already exist in Terraform output and disables the dev-only narrator for that run.
FDAI_GITOPS_API_BASE / FDAI_GITOPS_DEFAULT_BRANCH / FDAI_GITOPS_BRANCH_PREFIX / FDAI_GITOPS_TIMEOUT_SECONDS	env	deployment	gitops-pr adapter target repo config (GitHub App / Azure DevOps). Auth secrets flow through the platform’s App installation, not env vars.
FDAI_GITOPS_TOKEN / FDAI_GITOPS_OWNER / FDAI_GITOPS_REPO / FDAI_GITHUB_WORKFLOW_TOOLS_ENFORCE	KV ref + env	deployment	Binds the GitHub change feed and workflow tools for fix/release/security/incident/IRP artifacts. The platform exports the enabled operational ownership target and token secret id as stewardship_gitops_binding; the protected service workflow validates that object and injects only the Key Vault reference into the independent Core service. The enforce flag never bypasses the ActionType promotion and risk/human approval gates.
FDAI_RBAC_READERS_GROUP_ID / FDAI_RBAC_CONTRIBUTORS_GROUP_ID / FDAI_RBAC_APPROVERS_GROUP_ID / FDAI_RBAC_OWNERS_GROUP_ID / FDAI_RBAC_BREAK_GLASS_GROUP_ID	env	deployment	Entra ID group object ids for the five human roles (see user-rbac-and-identity.md). Unset group = role unassigned.
FDAI_STEWARDSHIP_REQUIRE_BINDINGS / FDAI_STEWARDSHIP_AUDIT_INTERVAL_SECONDS / FDAI_HANDOVER_KNOWLEDGE_INTERVAL_SECONDS	env	deployment / upstream	Set FDAI_STEWARDSHIP_REQUIRE_BINDINGS=1 so placeholder identities fail startup. ENABLE_STEWARDSHIP_GOVERNANCE runs in Console-governed mode by default without a GitHub App or webhook secret; optional GitOps adapters map GITOPS_OWNER, GITOPS_REPO, GITHUB_APP_CLIENT_ID, GITHUB_APP_INSTALLATION_ID, GITHUB_APP_PRIVATE_KEY, GITHUB_WEBHOOK_SECRET, or compatibility GITOPS_TOKEN with coordinated-rotation Key Vault exceptions. Terraform also supplies FDAI_MAINTAINERS, FDAI_STEWARD_<AGENT>, Key Vault references, and the readiness-gated Entra health and content-free ownership handover lifecycle cadences defined in Agent operational ownership lifecycle. The cadence defaults are 3600 and 60 seconds and grant no merge, IAM, approval, or execution authority.
FDAI_CATALOG_FRESHNESS_METADATA_URL / _ROOT_KEY_IDS / _ROOT_THRESHOLD / _REFRESH_SECONDS	env	deployment / upstream	Optional. The Core verifies the Release Hub’s public signed catalog metadata against the pinned ed25519 root key ids and returns operator-override promotions to observation mode while it is expired or unverified. Terraform sets them from the Core root’s catalog_freshness; the URL must be https. It only lowers authority. See Lifecycle Releases and Channels.
FDAI_ENTRA_TENANT_ID / FDAI_API_AUDIENCE	env	deployment	Required for the production Operator API Entra JWT verifier (EntraJwtVerifier): the deployment tenant id and the fdai-api application (client) ID, which v2 access tokens carry as aud. See user-rbac-and-identity.md#102-api-token-validation.
FDAI_ENTRA_ISSUER / FDAI_ENTRA_JWKS_URI	env	deployment	Optional verifier overrides; default to the tenant’s v2 issuer + public key set. Set ISSUER to https://sts.windows.net/<tenant>/ for a v1-token app; override JWKS_URI only for sovereign / air-gapped clouds.
FDAI_EXECUTOR_PRINCIPAL_ID / FDAI_EXECUTOR_EVENT_ROLE_DEFINITION_ID / FDAI_EXECUTOR_SECRET_ROLE_DEFINITION_ID	env	upstream	Operator API onboarding probe inputs. The probe uses ARG to verify the provisioned resource set and the executor’s Event Hubs / Key Vault roles.
FDAI_DR_DRILL_SOURCE_SERVER_ARM_ID / FDAI_DR_DRILL_TARGET_LOCATION / FDAI_DR_DRILL_TARGET_RESOURCE_GROUP / FDAI_DR_DRILL_TARGET_SERVER_PREFIX / FDAI_DR_DRILL_PITR_OFFSET_MINUTES / FDAI_DR_DRILL_INTEGRITY_TABLES / FDAI_DR_DRILL_DRY_RUN	env	deployment	Delivery-owned DB-DR drill config (see DB-DR drill runbook). The job uses a dedicated non-executor identity and a pre-created isolated target group. DRY_RUN=true validates complete configuration without making an Azure or database change.
FDAI_SECRET_KAFKA_TOKEN / other FDAI_SECRET_*	KV ref	deployment	generic escape hatch for a secret consumed by an adapter that does not yet have a dedicated env-var name; every FDAI_SECRET_* value MUST come from KV.

Rules that apply to every key:

The Onboarding console reports probe_mode=configured only when every Azure probe input is present. When the inputs are absent, probe_mode=not-configured means the displayed gaps are the required baseline, not observations from the signed-in tenant.

• Startup fails fast on missing or unparseable config (coding-conventions.instructions.md).
• Secrets go through Key Vault refs, never plain env; a secret in plain env fails the CI secret-scan gate.
• Per-environment values differ; the same image reads them from the injected environment.
• Terraform creates deployment-owned operator-request receipt signing seed secrets for Core and Operator in Key Vault and exports only their versionless secret ids and producer ids. The deploy workflow rejects a versioned secret reference before it hydrates those references into service tfvars, so a rotated seed resolves to its current version. Operator receives only its own seed. Core receives its own seed plus the Operator seed, derives trusted producer public keys at startup, and signs only as core-control-plane; Core remains the verifier inside the trusted computing base.

Event Source Subscription

Signals wired at bootstrap so the three initial verticals have something to observe. The concrete event types, subscription filters, and rate caps are TBD; the shape of the wiring is stable.

Vertical	Azure signal candidates	Delivery
Change	Activity Log (resource-write / delete), Change Analysis, Resource Health	Push into the canonical Event Hubs Kafka ingress; Huginn owns real-time discovery normalization and the Inventory sync job reconciles the full graph
DR / Chaos	Resource Health, backup vault events, PostgreSQL / SQL replication-lag metrics, restore-rehearsal outcomes	Diagnostic Settings + scheduled Container Apps Job probes → Kafka topic (fdai.dr.events)
FinOps	cost anomaly alerts, budget alerts, Advisor cost recommendations	Cost Management pull → Kafka topic (fdai.finops.events); anomaly alerts fan in through the same Diagnostic-Settings path
Operator safety control	Ordered poison-halt clear requests	Operator writes the logical fdai.operator.ordered-poison-halt.clear.v1 topic on the shared pantheon-object transport; Core validates the parked-record evidence before resuming a halted ordered consumer

Every event is stamped with an idempotency key at ingress so a replay is a no-op; DLQs MUST be reachable and covered by the alert-routing contract before enforce is enabled anywhere.

The Azure forwarding mechanism must preserve the no-shared-secret boundary. Do not enable Event Hubs local authentication only to satisfy a Diagnostic Settings export. When the selected Azure signal source cannot publish with managed identity, use the bounded Activity Log recovery reader until an approved push transport is available. The adaptive Inventory Job reads validated policy before each coordinator run, builds runtime-call, Resource Health, Static Web App, and Kubernetes enrichment through one shared ordered support boundary, and advances overdue reconciliation. The module split adds no service, identity, or state writer. Restricted-network inventory certification uses a task-owned resource set in the existing development application resource group with exact create and cleanup plans, without mutating or deleting the group. Its private Container Apps, PostgreSQL, DNS, and Blob path never peers with or changes the selected environment; private Blob I/O pins the endpoint address with hostname verification, migration preserves encoded credentials, the isolated server declares VECTOR,PG_TRGM, and distinct identities retain authority-free receipts.

Verification After Provisioning

Post-provision verification covers adapter reachability, canary round-trip, observation mode correctness, and readback of any enabled Inventory Job. The smoke test contract fails the apply when the inventory container is not successfully provisioned on the plan’s exact digest-pinned Core image, preventing stale collection; any failed verification aborts promotion and triggers rollback.

Cost-Efficiency Principles

The provisioning cost principles are owned by Cost Model, next to the illustrative monthly envelope they produce. This heading remains as a stable target for existing links.

Open Decisions

• Deployment interface - resolved: Terraform is the execution engine and the operator interface is fdaictl provision azure. The installable CLI runs read-only preflight and coordinates exact-plan work on the manual managed host without replacing Terraform. See Installable Deployment CLI.
• Concrete tier values within the minimum set (PostgreSQL storage size, Log Analytics daily cap, ACR retention window, Event Hubs throughput-unit ceiling).
• Region choice and the single-zone deployment posture (multi-zone deferred to Phase 4).
• Custom Azure role packaging for the deployer identity.
• Log Analytics daily-cap and query cost budget (retention default 30 days is configurable from the console UI; alert thresholds TBD).
• Kafka topic naming + Diagnostic-Settings forwarding filters, per-domain fan-in shape.
• Production networking baseline - resolved: VNet-integrated Container Apps, private Key Vault, and delegated-subnet private PostgreSQL. Development may retain the public PostgreSQL path; ACR/Event Hubs private endpoints remain tenant-policy-driven additions.
• Full runtime config key list (values matrix expanded).
• Day-zero semantic baseline - every schema-valid generic catalog entry in the signed exact-revision bundle, enumerated by the subscription genesis manifest.
• Core -> Isolated Executor target boundary - required for the five-service program; authority cutover waits for every binary gate and rollback receipt.

Hub workload state handoff

When an existing AKS workload state snapshot is unavailable, the Hub handoff workflow imports the exact approved Deployment, Service, HorizontalPodAutoscaler and CronJob inventory into a private state lineage. A separate forget-only release plan must prove zero live destruction. Rollback uses the generated import declarations and requires a final zero-change plan. Terraform 1.9 import plans may report an existing object in both before and after; the verifier accepts this only when both values are identical, the action is no-op, and the import id matches. Legacy completed transitions with a sub-millisecond state timestamp mismatch use the authenticated reconciliation command. It preserves the immutable transition receipt and rewrites only the Agent’s durable current state to the Hub-authoritative canonical timestamp and digest. Legacy receipt validation keeps the stored receipt digest unchanged while independently verifying the millisecond state digest and the full-precision Hub completion digest. Equivalent UTC +00:00 and Z forms are accepted only after the stored receipt digest verifies. An expired, unconsumed approval may be renewed only when all evidence and live state remain exact. The prerequisite root owns only the identities, federated credentials, ServiceAccounts, SecretProviderClasses, ConfigMaps and secret bindings that this inventory references. It does not create any managed Deployment, Service, HorizontalPodAutoscaler or CronJob. The Operator API identity receives Key Vault Secrets User only on its exact three generated secret resource ids; the root does not grant it vault-wide access. The separate Operator command identity receives Event Hubs sender and receiver roles only on the five exact change, semantic, human-approval, and pipeline topic bindings.

Related docs

To learn about	Read
Delivery status and remaining work	Implementation ledger