Securing Enterprise Knowledge Retrieval with Azure AI

Executive Summary

Business context — Contoso: Contoso manages internal operations, service, and policy documents across approved Microsoft 365 and Azure repositories. Employees need current answers with source citations, and the assistant must enforce different departmental document permissions.

Contoso’s initial application can search a small set of documents and generate fluent answers, but it does not carry the source system’s authorization rules through retrieval. That makes a broad rollout unsafe: a user could receive a summary of a document they cannot open directly, recently changed content could remain in an index, and operators have limited evidence to investigate an incorrect answer.

The production design treats the assistant as a secured application and data pipeline. Microsoft Entra ID authenticates the employee. A backend resolves the user’s approved access context, queries Azure AI Search with server-enforced permission filters, and sends only the authorized passages to an approved model deployment. Content ingestion records source identifiers and access metadata, while evaluation, monitoring, and a named service owner support the release after launch.

Client reference: Contoso is used as a generic, anonymized business name. The design documents a production implementation pattern; project completion and measured results must be substantiated separately.

The Challenge

Contoso’s documents span policy, service operations, and team-specific procedures. The same employee may be allowed to read general policies but restricted from finance, human resources, or incident records. Several teams maintain content, so ownership, review dates, and withdrawal procedures are not consistent across repositories.

The existing application presents several production risks:

• Search permissions are broader than the employee’s access in the source repository.
• A document’s ACL changes or deletion may not be reflected in the index promptly.
• Content owners cannot easily identify who approves an ingestion source or resolves stale information.
• Retrieved text may contain prompt-injection instructions or conflicting procedures.
• Responses can sound confident when the search results are incomplete, outdated, or ambiguous.
• Users need a source link they can open under their own identity to verify a generated answer.
• Support teams need to trace failures without putting sensitive document text or employee prompts in general logs.

The objective is to make retrieval authorization, content lifecycle, and service operation explicit parts of the application rather than assumptions delegated to the model.

Objectives

The production design is intended to:

1. Authenticate employees and preserve user-level authorization through every retrieval request.
2. Index only approved sources and maintain the source identity, owner, classification, and permission metadata needed for filtering.
3. Remove or refresh indexed content when the source changes, access is revoked, or a document is deleted.
4. Ground responses in a small set of authorized passages and provide usable source references.
5. Decline or clarify when evidence is missing, conflicting, or outside the assistant’s supported scope.
6. Evaluate both answer quality and denied-access behavior before release and after material changes.
7. Operate the assistant with identity, networking, monitoring, cost ownership, and incident procedures suitable for production.

Authorized Retrieval Architecture

The employee signs in with Microsoft Entra ID through the approved application experience. A backend API validates the token, establishes the user and tenant context, and obtains the authorization claims required for the supported content sources. The browser never receives model credentials, search keys, or authority to construct its own security filter.

For this production example, Contoso uses Azure AI Search’s generally available security-filter pattern. Each indexed chunk carries a stable source identifier and the relevant user or group identifiers in a filterable field that is not returned with search results. The backend derives the caller’s trusted identifiers from the validated Entra identity and approved group-resolution process, then applies a mandatory search.in filter to every query before any passage can reach the model. A caller cannot supply or override that filter.

The filter is an application-enforced security-trimming pattern; an identity value in the search index is not, by itself, authentication or authorization. The backend validates the user, applies the filter on every search path, and uses narrowly scoped Microsoft Entra data-plane access to Azure AI Search. If identity resolution or filter construction fails, the request fails closed. The design is tested for search, suggestions, retries, and any future retrieval path so an alternate endpoint cannot bypass the constraint.

Permission metadata is synchronized with each source’s access model. Contoso makes the acceptable update delay explicit and prioritizes urgent revocations; when a permission update cannot be confirmed, the affected content is withheld or the source is paused until reconciliation completes. Native permission-enforcement capabilities that remain in preview are not part of this production baseline; they can be evaluated separately after their support status and source-specific behavior are confirmed.

The permission model is tested against source-system semantics before production. Group membership can be nested or changed, and repositories may represent permissions differently. Contoso resolves those details through approved APIs such as Microsoft Graph and maintains a documented mapping between source permissions and the indexed principal identifiers. Group changes and document ACL updates are included in synchronization and reconciliation tests. If the backend cannot establish the user’s access context, the request fails closed rather than falling back to an unfiltered query.

The response includes links or identifiers for the source documents. A user opening a link is authenticated and authorized by the source system again. A citation helps the user verify context; it is not treated as proof that the generated answer is correct.

Content Ingestion and Lifecycle

Each source has a named business owner, an approved purpose, a classification, and a documented ingestion path. The ingestion workflow uses an identity scoped to the selected repositories. It extracts supported content, preserves source identifiers and permission metadata, segments text into useful passages, and writes search documents with version and update information.

Updates are handled as a controlled replacement of the affected source content. The pipeline records successful and failed runs, avoids publishing partial updates as complete, and places malformed or unsupported files in a review path. Source deletions and permission changes trigger index updates or removal of affected chunks. A reconciliation process compares the source inventory with indexed records so that missed events do not leave content available indefinitely.

The indexing service uses its own managed identity and only the access it needs. Ingestion logs capture document identifiers and processing status without copying sensitive document bodies into general telemetry. Retention, deletion, and reprocessing follow the source owner’s policy.

Answer Behavior and Safety

The assistant is designed for read-only knowledge lookup. It does not approve access, modify source documents, or perform operational actions. Retrieved passages are treated as untrusted data; their contents cannot change server-side authorization or expand the assistant’s tool access.

The prompt asks the model to answer from the supplied evidence, distinguish direct support from uncertainty, and provide source references. Application logic checks for empty or weak retrieval and can ask the user to refine a question or explain that the approved corpus did not provide enough evidence. Conflicting procedures are surfaced for review rather than silently merged into a definitive instruction.

Content filtering and output checks are configured for the use case and tested with representative inputs. They complement authentication, authorization, retrieval filtering, and human judgment; they do not replace those controls. If Contoso later adds write actions or workflow tools, those are separately authorized, introduced behind explicit user confirmation, and reviewed as a new capability.

Evaluation and Acceptance

Before a production release, Contoso creates a reviewed evaluation set from approved content and common employee tasks. It includes questions with a clear answer, questions with no supported answer, conflicting or outdated source cases, and attempts to retrieve content using users with different access levels.

The evaluation checks retrieval relevance, answer grounding, citation correctness, appropriate abstention, permission boundaries, and response latency. Negative authorization tests verify that restricted documents do not appear in retrieved passages, citations, summaries, or application logs. A content owner reviews the expected answers and source links; security and service owners review the access model and operational evidence.

Acceptance criteria are agreed for the specific corpus and user groups before rollout. A passing automated score is one release signal, not a guarantee that all answers are correct or that access is safe for every future document. Material changes to source systems, permission mapping, chunking, search configuration, prompts, or model deployments trigger relevant re-evaluation.

Identity, Network, and Application Controls

The application backend uses managed identity for Azure AI Search and model access where supported. Permissions are scoped to the required resources and environments. Any remaining secrets are stored in Azure Key Vault and are not placed in source code, prompts, browser code, or pipeline logs.

Private endpoints and private DNS are used where required by Contoso’s network policy and supported by the selected services. The team verifies the complete path between the user-facing application, backend, search service, model endpoint, and ingestion components. Network restrictions are tested alongside service identity so that a private endpoint does not silently break a required operation.

Infrastructure and application configuration are versioned and deployed through reviewed CI/CD workflows. Development, test, and production use separate resources or clearly defined isolation boundaries. Evaluation data and production content are not copied into lower environments unless they are approved and appropriately minimized.

Production Operations

The service has an accountable application owner, a content operations owner, and an escalation path for source permission or data-quality issues. Dashboards track availability, failed requests, latency, throttling, search outcomes, model usage, and cost allocation. Access-denied and abstention signals are useful for operations, but telemetry avoids retaining full employee questions or sensitive retrieved passages by default.

Support procedures cover model or search service degradation, ingestion delays, permission-mapping failures, a source document that must be withdrawn, and suspected disclosure. Operators can disable a source, pause ingestion, or make the assistant unavailable while investigating. A rollback restores a known application and configuration version; it does not silently restore deleted or unauthorized content to the index.

Content owners periodically review source scope, document freshness, and permission mapping. Application owners review evaluation results, user feedback, and operational signals before expanding the supported corpus or adding new capabilities.

Implementation Sequence

1. Select a bounded set of read-only knowledge sources and appoint content, security, and service owners.
2. Map each source’s identity and permission model, including group changes, deletions, and restricted content.
3. Implement authenticated backend retrieval with server-enforced filters and a fail-closed path.
4. Build ingestion, update, deletion, and reconciliation workflows with source ownership and audit evidence.
5. Create a permission-aware evaluation set and test allowed, denied, stale, conflicting, and unsupported questions.
6. Deploy to a limited employee cohort, monitor service and authorization signals, and address issues before wider rollout.
7. Establish ongoing corpus review, re-evaluation triggers, incident response, and cost ownership.

Representative Deliverables

A typical engagement produces:

• Production architecture and end-to-end data-flow diagram
• Source inventory, owner register, and permission-mapping design
• Secure ingestion, update, deletion, and reconciliation workflows
• Backend retrieval implementation with user-scoped search filters
• Evaluation set and release criteria covering authorization and answer quality
• CI/CD configuration, infrastructure code, and environment controls
• Monitoring dashboards, incident procedures, and source-withdrawal runbook
• Documented limitations, support ownership, and rollout plan

Detailed Solution Topology and Trust Boundaries

Contoso's knowledge assistant uses an authenticated employee front end, a private backend service, a Search index with mandatory security trimming, a separately permissioned ingestion path, and an approved model deployment. The front end receives only the generated response and user-visible source links. It does not receive search keys, embedding credentials, raw ACL fields or model access tokens.

The backend executes all authorization checks before it sends retrieved passages to the model. Its service identity has read/query access to Search, while ingestion has only the write and source permissions it needs. Index administration is a separate privileged role. For sources whose effective ACLs cannot be faithfully mapped, the content is excluded until an approved permission model is available.

Production index schema and query behavior

For a validated caller, the server resolves allowed principal identifiers, constructs a mandatory OData filter (for example, an application-controlled search.in predicate against allowed_principals), and enforces the same restriction for keyword, vector/hybrid, semantic reranking, suggested results and any cached response. Effective source permission models can be more complex than group inclusion; inherited permissions, explicit denies, nested groups and sharing links require source-specific handling. The filter alone is not a substitute for a reliable ACL synchronization process.

The API rejects missing or inconsistent identity context. It never accepts an authorization filter from the browser and does not retry an authorization failure as an unrestricted search. Negative tests verify that restricted passage text, source titles, citations and autocomplete paths cannot escape the boundary. Cache keys include authorization context and index revision or caching is disabled for permission-sensitive responses.

Ingestion Workflow, Change Capture and Revocation

The ingestion pipeline registers a connector per approved repository, content owner, business purpose, access scope, retention class and extraction method. A source inventory records collection time, source document ID, owner, revision, ACL change marker, processing status, deletion marker and index build ID. New or updated content enters a staged index only after schema, classification, permission and extraction validation. The alias is promoted only after both functional and negative-authorization tests succeed.

Document deletion, permission tightening, revoked group membership or a connector authorization failure triggers a bounded response: mark affected source records unavailable, remove or rebuild affected chunks, reconcile the index against source inventory, and audit outstanding exceptions. Near-real-time event notifications can accelerate change capture, but periodic full reconciliation is still needed to detect missed events. Until access revocation is validated in the index, affected results are withheld; no universal revocation-latency guarantee is assumed.

Operational jobs separate source fetching, text normalization, chunking, embedding, index writing and reconciliation. Each stage has retry budgets, dead-letter review, idempotent identifiers, and a failure counter. Reprocessing does not accidentally reinstate withdrawn content. General-purpose logs record hashes, IDs and status without full sensitive text.

Application and Platform Implementation

The backend exposes bounded endpoints such as POST /api/ask, GET /api/sources/{id} (source permission revalidation), and a separate privileged administrative source-disable workflow. The request handler validates audience, issuer, tenant and token lifetime; resolves the user context; queries filtered Search; limits passage count and token usage; validates citations; and returns an answer, evidence list and non-sensitive correlation ID. Content that attempts to instruct the model to reveal secrets, change system directions or call tools is treated only as source data.

App Service or Container Apps runs the API with managed identity and an approved ingress model. VNet integration/private endpoints, DNS zones and routing are tested from actual running instances. App telemetry uses Application Insights with redaction and retention controls. Model and Search quotas are monitored independently. Terraform/Bicep separates dev, test and production state, deployment permissions and resource identifiers. Changes to chunking, search ranking, ACL mapping or prompt templates require evaluation and controlled promotion.

Example deployment repository

knowledge-assistant/
  infrastructure/{network,app,search,model,identity,monitoring}/
  services/{api,ingest,index-reconcile}/
  search/{schema,indexes,aliases,queries}/
  security/{acl-mapping,negative-tests}/
  evaluation/{answerable,abstention,adversarial,access-control}/
  deployments/{dev,test,prod}/
  runbooks/{source-revoke,index-restore,security-incident}/

Production Acceptance, Operations and Recovery

Acceptance covers: authorization tests with different source-system permissions; nested-group and revoked-access behavior; source deletion and index reconciliation; freshness of high-priority procedures; citation links resolving under user credentials; invalid prompt-injection directions being ignored; model/Search throttling; fail-closed query behavior; and a controlled data-withdrawal procedure. The application does not go live on the basis of answer fluency alone.

The content owner approves the corpus and stale-data limits; identity/security signs off on permission mapping; the application owner signs off on task quality and citation behavior; operations signs off on incident response, budgets and runbooks. Service monitoring tracks ingest backlog, last_sync_utc, authorization errors, empty-result/abstention rate, p95 latency, model throttling, source-disable events and evaluated answer quality. Alerts route to named owners rather than a generic team mailbox.

If Search or model inference fails, the application may display a service-unavailable or evidence-unavailable response; it does not silently generate an ungrounded authoritative answer. In a disclosure investigation, operators block affected sources or disable retrieval, preserve limited audit metadata, correct source/index permissions and validate negative tests before re-enabling. Index rollback cannot restore removed content or obsolete ACLs.

Before and After Implementation: Service Improvements

The table represents intended improvements and verifiable controls. It does not claim deployment statistics or actual customer outcomes without supporting records.

Outcome

Under this operating model, Contoso moves from an insufficiently governed application to a bounded internal service where retrieval is filtered using the signed-in employee’s approved access context, source changes have a managed path into and out of the index, and operators can investigate service behavior without routinely logging sensitive content.

Technologies

Microsoft Azure · Microsoft Foundry · Azure OpenAI in Microsoft Foundry Models · Azure AI Search · Microsoft Entra ID · Microsoft Graph · Azure App Service or Azure Container Apps · Managed Identity · Azure Key Vault · Private Link · Azure Monitor · Application Insights · Terraform / Bicep

Related Patterns

• Enterprise RAG and AI Search
• Zero-trust architecture for AI agents