From Cloud Landing Zones to AI Landing Zones: What Enterprises Need to Add for Production AI

Most enterprises already have a cloud foundation: identity boundaries, network connectivity, policy, security monitoring and an operating model for workloads. Production AI does not make that foundation obsolete. It adds workload-specific controls around models, data retrieval, agent actions, quality evaluation and consumption.

The practical question is not whether to build a second cloud foundation called an “AI landing zone.” It is how to extend the existing platform so teams can deploy AI workloads within known guardrails, while adding controls that ordinary application patterns may not provide.

Keep the cloud foundation; extend the workload boundary

1. User identityEstablish who initiated the request and their access context.
2. ApplicationApply workload policy and decide which capabilities are available.
3. RetrievalReturn only data the requesting user is allowed to see.
4. ModelSend approved context through an authorized endpoint.
5. Tool actionAuthorize each operation; require approval when its impact warrants it.
6. EvidenceTrace outcome, quality, latency and cost with appropriate data protection.

A cloud landing zone establishes the shared conditions under which workloads can run: identity and access, subscription or account organization, network topology, governance, security and management. Those controls remain necessary for AI. Model endpoints, vector indexes, data pipelines, agent runtimes and supporting services still need owners, authorization, connectivity, monitoring and lifecycle management.

AI workloads introduce additional paths and trust relationships. A user request can retrieve content, send context to a model, invoke tools, write data or trigger an external action. Teams need to map and govern each of those paths. A workload landing zone can then consume shared platform services—such as identity, private connectivity, policy and logging—while its own architecture adds the AI-specific controls it needs.

Microsoft’s Azure landing-zone design areas describe foundational concerns such as identity, resource organization, network connectivity, security, management, governance and platform automation. Its AI workload landing-zone reference architecture shows one Azure implementation that builds on those foundations. Treat it as a starting pattern and adapt it to your platform, workload and risk model.

What production AI adds

1. Govern model access and lifecycle

Decide which models and endpoints a workload may use, who can deploy or change them, and how model versions move through development, test and production. Separate deployment administration from application inference access. Give runtime components only the identity and network path they need to call approved endpoints; avoid distributing long-lived keys through source code, prompts or developer workstations.

Record which model deployment, configuration and prompt version served a request. That context helps teams investigate a quality change, compare releases and roll back a deployment. The specific controls depend on the provider and hosting model, so keep the policy focused on outcomes—approved access, reviewable change and traceability—rather than assuming one vendor feature is universal.

2. Carry data permissions through retrieval

Retrieval-augmented generation (RAG) introduces a second authorization problem: the application must ensure that retrieved passages are permitted for the requesting user. Securing the chat interface or search service alone is not enough if the index returns content from documents that the user could not otherwise open.

Classify approved source data, preserve its ownership and permissions during ingestion, and enforce access at retrieval time. Also define how source changes, deletions, retention rules and index refreshes are handled. Microsoft describes approaches such as security filters and document-level access control in Azure AI Search; the right implementation depends on the source system, identity model and feature support in the selected services.

3. Bound agent and MCP permissions

An agent that can call tools has authority beyond generating text. A read-only search, a ticket update and a financial transaction have different consequences, so group them behind explicit tool and data boundaries. For MCP servers and other tool connectors, allowlist tools, constrain inputs and outputs, authorize each operation, and identify whether it acts as the user or as a service identity. Keep service identities narrowly scoped and make sensitive or irreversible actions require an appropriate human decision.

Treat retrieved content, user prompts and tool responses as untrusted input. Design for prompt injection and misuse instead of assuming a system prompt will enforce security. Log the tool selected, the authorization decision and the result, while limiting sensitive content in logs. Microsoft’s agent governance guidance similarly emphasizes organizational identity, access and data-governance controls. The Engineering Library’s AI Security, Identity, and Responsible AI guide goes deeper on these control areas.

4. Evaluate changes before release

Traditional application tests do not fully describe whether an AI workload is ready. Build a representative evaluation set and define what acceptable behavior means for the use case. Depending on the workload, checks might cover retrieval relevance, groundedness, citation quality, task completion, refusal behavior, safety, latency and cost. For agents, test whether tools are called correctly and whether permissions and approval boundaries hold.

Run evaluations when models, prompts, retrieval settings, data, tools or orchestration change. Put agreed thresholds into the release workflow, and keep a way to review failures rather than relying on one average score. Microsoft Foundry’s observability guidance covers lifecycle evaluation, tracing and production monitoring; teams can apply the same release discipline with their chosen stack.

5. Observe the whole request path

AI troubleshooting crosses more components than a typical API call. Trace the request from user and application through retrieval, model inference and tool execution. Capture enough context to investigate latency, failed retrieval, quality regressions, unsafe outputs and cost spikes: model and prompt versions, tool calls, token usage, errors and relevant evaluation signals.

Choose log retention and redaction rules deliberately. Prompts, retrieved passages and tool arguments can contain personal, confidential or security-sensitive data. Define who can access traces, how incidents are escalated, when a feature can be disabled, and what fallback or human review path is available.

6. Make AI consumption visible and governable

AI cost is more than model tokens. Retrieval, embeddings, indexing, storage, application hosting, observability, network transfer and human review all contribute. Establish an owner and a useful unit of measurement, such as cost per completed task or approved workflow, then relate usage and quality to that unit.

Set budgets, alerts, quotas and request-level limits where the platform supports them. Bound context size, output length, agent steps and tool fan-out when appropriate. Do not present a budget alert as a hard spending cap: verify the enforcement behavior of each service and use workload controls for limits that must be applied before a request runs. Microsoft’s Foundry cost guidance explains cost estimation, monitoring, budgets and the distinction between AI-service charges and total application cost.

Connect the work across three engineering layers

1. Cloud Foundation: establish identity, network, policy, governance, private connectivity and management boundaries.
2. Platform Engineering: make those controls repeatable through Infrastructure as Code, CI/CD, workload onboarding and operational tooling.
3. Enterprise AI: add governed model access, permission-aware retrieval, bounded agent tools, evaluation, observability and AI cost controls.

This is a connected delivery path, not three disconnected technology stacks. The landing zone provides shared boundaries; platform engineering makes them usable and repeatable; the AI workload applies those capabilities to its data, models and behavior.

For examples of how the pieces fit together, see Tongtech’s Secure AI Landing Zone, AI Production Accelerator and representative production AI architecture.

A practical readiness sequence

1. Define the workload, users, data classifications, business consequence and production owner.
2. Map identity, network, data, model and tool boundaries; identify the existing platform controls the workload can inherit.
3. Specify retrieval-time authorization, agent permissions, human decision points and audit requirements.
4. Agree evaluation criteria and release gates for quality, security, safety, reliability and cost.
5. Instrument the request path, cost measures, operational alerts, incident response and fallback behavior.
6. Implement the agreed scope as reviewable infrastructure and application changes, then validate the operating handover.

The depth of each control should match the data, actions and impact of the workload. A low-risk internal assistant and an agent that can change customer or financial records should not inherit identical permissions or approval paths.

Make retrieval authorization an explicit release check

A retrieval design must preserve the requesting user’s authorization through to the content returned to the model. Test users at different access levels against documents they can and cannot open, including group changes, revoked permissions and newly indexed content. A service identity that can read every document must not turn that broad access into answers for every user. Azure AI Search currently documents native identity-based ingestion and query enforcement as preview, while security filters remain an application-implemented pattern. Check the feature’s status and constraints before relying on preview capability for production authorization; keep and test an application-enforced fallback, and recheck support before major rollout. Include negative tests for cross-tenant and cross-group access, and test what happens when the identity provider or authorization lookup is unavailable. The safe behavior should fail closed for protected content, surface a useful operational signal, and avoid returning an incomplete answer as though it were fully grounded. Define a measurable revocation and deletion propagation window, then alert when indexing or synchronization falls outside it.

Continue reading

• Your Azure Landing Zone Is Finished. Now What? — Follows the cloud foundation into repeatable workload delivery and application platforms.
• Why Enterprise AI Is an Infrastructure Problem Before It Is an AI Problem — Looks at the cross-cutting system boundaries an AI workload must address before production.

References

• Azure landing zone design areas — Microsoft Cloud Adoption Framework.
• Baseline Microsoft Foundry chat architecture in an Azure landing zone — Microsoft Azure Architecture Center.
• Document-level access control — Azure AI Search.
• Observability in generative AI — Microsoft Foundry.
• Govern and secure AI agents across the organization — Microsoft Cloud Adoption Framework.
• Plan and manage costs — Microsoft Foundry.