AI-103 Part 5: Information Extraction Solutions

Part 5 of 5 — 10–15% of the AI-103 exam. This domain covers Azure AI Search (index design, vector search, hybrid search, semantic ranking, and RAG patterns) and Azure AI Document Intelligence (prebuilt models, Layout, custom extraction models, and composed models). These two services handle knowledge retrieval and structured document extraction in enterprise pipelines.

Information Extraction Architecture

Azure AI Search — Core Concepts

Azure AI Search is a cloud search service that hosts persistent indexes and provides full-text search, vector search, and hybrid search capabilities. The four main building blocks are the index (the schema and stored documents), the indexer (automated data ingestion pipeline), the skillset (AI enrichment steps applied during indexing), and the knowledge store (persistence of enriched content outside the index).

Service Tiers

Tier	Semantic Ranking	Max indexes	Notes
Free	Not available	3	Shared capacity, dev/test only
Basic	Yes (1,000 queries/month free)	15	Minimum tier for semantic ranking
Standard (S1–S3)	Yes	50–200	Dedicated capacity, scalable replicas/partitions
Storage Optimized (L1/L2)	Yes	10	Very large indexes, lower queries-per-second

Index Schema

An index is defined by its fields. Each field has a data type and a set of attributes (searchable, filterable, sortable, facetable, retrievable). For vector search, you add a Collection(Edm.Single) field and configure a vector search algorithm profile on it.

from azure.search.documents.indexes import SearchIndexClient
from azure.search.documents.indexes.models import (
    SearchIndex, SearchField, SearchFieldDataType,
    VectorSearch, HnswAlgorithmConfiguration, VectorSearchProfile,
    SemanticConfiguration, SemanticSearch, SemanticPrioritizedFields,
    SemanticField
)
from azure.core.credentials import AzureKeyCredential

index_client = SearchIndexClient(
    endpoint="https://my-search.search.windows.net",
    credential=AzureKeyCredential("YOUR_ADMIN_KEY")
)

fields = [
    SearchField(name="id", type=SearchFieldDataType.String, key=True, filterable=True),
    SearchField(name="title", type=SearchFieldDataType.String, searchable=True, retrievable=True),
    SearchField(name="content", type=SearchFieldDataType.String, searchable=True, retrievable=True),
    SearchField(name="source_url", type=SearchFieldDataType.String, retrievable=True),
    # Vector field: dimensions must match embedding model
    SearchField(
        name="content_vector",
        type=SearchFieldDataType.Collection(SearchFieldDataType.Single),
        searchable=True,
        vector_search_dimensions=1536,        # text-embedding-3-small = 1536
        vector_search_profile_name="hnsw-profile"
    ),
]

vector_search = VectorSearch(
    algorithms=[HnswAlgorithmConfiguration(name="hnsw-algo")],
    profiles=[VectorSearchProfile(name="hnsw-profile", algorithm_configuration_name="hnsw-algo")]
)

semantic_search = SemanticSearch(
    configurations=[
        SemanticConfiguration(
            name="my-semantic-config",
            prioritized_fields=SemanticPrioritizedFields(
                title_field=SemanticField(field_name="title"),
                content_fields=[SemanticField(field_name="content")]
            )
        )
    ]
)

index = SearchIndex(
    name="knowledge-base",
    fields=fields,
    vector_search=vector_search,
    semantic_search=semantic_search
)
index_client.create_or_update_index(index)
print("Index created.")

Vector Search

Vector search finds documents semantically similar to a query by comparing embedding vectors in a high-dimensional space. Instead of matching keywords, it measures cosine similarity between the query embedding and document embeddings stored in the index. This handles synonym variation, paraphrasing, and cross-language scenarios naturally.

HNSW vs Exhaustive KNN

HNSW (Hierarchical Navigable Small World) is the default and recommended algorithm for most scenarios. It is an approximate nearest-neighbor algorithm that builds a layered graph structure at index time, enabling fast approximate search in sub-linear time. Exhaustive KNN performs an exact nearest-neighbor search over all vectors — it guarantees finding the true top-k nearest neighbors but scales linearly with index size, so it is only practical for small indexes or when exact results are critical.

Integrated Vectorization

Integrated vectorization lets AI Search automatically call an embedding model (Azure OpenAI or Azure AI Vision) during indexing and at query time — you do not need to generate embeddings yourself. You configure a vectorizer on the index and an embedding skill in the skillset. At query time the service embeds the user's text automatically when you use a text-based vector query.

Python Example: Hybrid Query with Semantic Ranking

from azure.search.documents import SearchClient
from azure.search.documents.models import VectorizedQuery
from openai import AzureOpenAI

aoai_client = AzureOpenAI(
    azure_endpoint="https://MY-AOAI.openai.azure.com/",
    api_key="AOAI_KEY",
    api_version="2024-02-01"
)

search_client = SearchClient(
    endpoint="https://my-search.search.windows.net",
    index_name="knowledge-base",
    credential=AzureKeyCredential("YOUR_QUERY_KEY")
)

user_question = "What are the refund policies for digital products?"

# Step 1: embed the query
embedding_response = aoai_client.embeddings.create(
    model="text-embedding-3-small",
    input=user_question
)
query_vector = embedding_response.data[0].embedding

# Step 2: hybrid query — BM25 full-text + vector + semantic reranking
vector_query = VectorizedQuery(
    vector=query_vector,
    k_nearest_neighbors=50,
    fields="content_vector"
)

results = search_client.search(
    search_text=user_question,         # BM25 full-text component
    vector_queries=[vector_query],      # vector component
    query_type="semantic",
    semantic_configuration_name="my-semantic-config",
    query_answer="extractive",          # extract verbatim answer snippets
    query_caption="extractive",
    top=5,
    select=["id", "title", "content", "source_url"]
)

retrieved_chunks = []
for result in results:
    print(f"Score: {result.get('@search.reranker_score', 'N/A'):.3f} | {result['title']}")
    if result.get("@search.captions"):
        print(f"  Caption: {result['@search.captions'][0].text}")
    retrieved_chunks.append({
        "title": result["title"],
        "content": result["content"],
        "url": result["source_url"]
    })

Exam tip: Hybrid search (BM25 + vector) consistently outperforms either modality alone for most query types. Adding semantic reranking on top of hybrid results gives the best quality — semantic ranker re-scores the top 50 BM25/vector results using a cross-encoder model and returns a @search.rerankerScore between 0 and 4.

Semantic Ranking

Semantic ranking is a re-ranking step applied after BM25 retrieval. The semantic ranker processes the top-50 candidate results from BM25 using a cross-encoder model that reads both the query and each document together, producing a more nuanced relevance score than BM25's term-frequency statistics. The original BM25 score is preserved as @search.score; the semantic score is returned as @search.rerankerScore (scale 0–4).

Semantic Answers and Captions

When you specify query_answer="extractive", the semantic ranker attempts to extract a short verbatim passage from the top results that directly answers the question — returned in @search.answers. Semantic captions (query_caption="extractive") return highlighted excerpts from each result that are most relevant to the query — useful for showing context without displaying the full document.

Exam tip: Semantic ranking requires at least the Basic tier — it is not available on the Free tier. Semantic ranker only operates on the top 50 BM25 results; it does not consider documents that BM25 did not retrieve. This is why combining it with hybrid search (which surfaces semantically relevant documents that keyword search might miss) gives the best results.

Indexer, Skillset, and Integrated Vectorization

An indexer automates the process of pulling data from a data source, running it through an optional skillset, and writing the results to an index. It can be run on demand or on a schedule. Data sources include Azure Blob Storage, Azure SQL Database, Azure Cosmos DB, Azure Table Storage, and SharePoint Online.

A skillset is a reusable pipeline of cognitive skills. Built-in skills include OCR, language detection, entity recognition, key phrase extraction, sentiment analysis, and image analysis. You can also add a Web API custom skill to call any HTTP endpoint — for example, a custom classification model hosted as an Azure Function.

The text split skill is the key skill for RAG scenarios: it breaks long documents into smaller chunks (pages or fixed-length segments with overlap) so that each chunk fits in the LLM's context window. The maximumPageLength and pageOverlapLength parameters control chunk size and overlap.

from azure.search.documents.indexes.models import (
    SearchIndexer, SearchIndexerDataSourceConnection, SearchIndexerSkillset,
    SplitSkill, OcrSkill, SearchIndexerIndexProjection,
    SearchIndexerIndexProjectionSelector, SearchIndexerIndexProjectionsParameters
)

# Skillset with text split for RAG chunking
skillset = SearchIndexerSkillset(
    name="rag-skillset",
    skills=[
        SplitSkill(
            name="split-skill",
            description="Split documents into pages for RAG",
            text_split_mode="pages",
            maximum_page_length=512,
            page_overlap_length=64,
            inputs=[{"name": "text", "source": "/document/content"}],
            outputs=[{"name": "textItems", "target_name": "pages"}]
        )
    ]
)

# Indexer pulling from blob storage
indexer = SearchIndexer(
    name="blob-indexer",
    data_source_name="blob-datasource",
    target_index_name="knowledge-base",
    skillset_name="rag-skillset",
    schedule={"interval": "PT12H"}  # Run every 12 hours
)
index_client.create_or_update_indexer(indexer)

RAG Pattern with AI Search

Retrieval-Augmented Generation (RAG) combines AI Search with a generative model to answer questions over your own documents. The pattern has two phases: an offline indexing phase where documents are chunked, embedded, and stored in the index; and an online query phase where the user's question is embedded, similar chunks are retrieved, and those chunks are passed as context to the LLM.

The quality of grounding depends on including enough metadata in retrieved chunks for the LLM to cite the source. Always include at minimum the document title and a URL or identifier in each indexed chunk so the LLM can produce attributable answers.

# Complete RAG query-time implementation
import json

def rag_answer(user_question: str, top_k: int = 5) -> str:
    # 1. Embed the user question
    q_embedding = aoai_client.embeddings.create(
        model="text-embedding-3-small", input=user_question
    ).data[0].embedding

    # 2. Hybrid + semantic search in AI Search
    vq = VectorizedQuery(vector=q_embedding, k_nearest_neighbors=50, fields="content_vector")
    search_results = search_client.search(
        search_text=user_question,
        vector_queries=[vq],
        query_type="semantic",
        semantic_configuration_name="my-semantic-config",
        top=top_k,
        select=["title", "content", "source_url"]
    )

    # 3. Build grounded context for the LLM
    context_parts = []
    for r in search_results:
        context_parts.append(
            f"[Source: {r['title']} ({r['source_url']})]\n{r['content']}"
        )
    context = "\n\n---\n\n".join(context_parts)

    # 4. Generate answer with GPT-4o
    messages = [
        {
            "role": "system",
            "content": (
                "You are a helpful assistant. Answer the user's question using ONLY "
                "the provided source documents. Cite the source title and URL for each claim."
            )
        },
        {
            "role": "user",
            "content": f"Sources:\n{context}\n\nQuestion: {user_question}"
        }
    ]
    response = aoai_client.chat.completions.create(
        model="gpt-4o",
        messages=messages,
        max_tokens=800
    )
    return response.choices[0].message.content

answer = rag_answer("What are the cancellation terms in our service agreement?")
print(answer)

Azure AI Document Intelligence

Azure AI Document Intelligence (formerly Form Recognizer) extracts structured data from documents using trained models. It handles PDFs, images (JPEG, PNG, TIFF, BMP), and Office files. The SDK entry point is DocumentIntelligenceClient from the azure-ai-documentintelligence package (note: the older azure-ai-formrecognizer package used DocumentAnalysisClient — always use the newer package for new projects).

Prebuilt Models

Model ID	Input document	Key fields extracted
prebuilt-read	Any document	Text content with layout (lines, paragraphs, reading order). No key-value pairs or tables.
prebuilt-layout	Any document	Text + tables + key-value pairs + selection marks. Best general-purpose extraction model.
prebuilt-invoice	Invoice PDFs	VendorName, CustomerName, InvoiceId, InvoiceDate, DueDate, SubTotal, TotalTax, InvoiceTotal, line items
prebuilt-receipt	Store receipts	MerchantName, TransactionDate, Items (name, quantity, price), Subtotal, Tax, Total
prebuilt-idDocument	Passports, driver's licenses	FirstName, LastName, DocumentNumber, DateOfBirth, Expiration, CountryRegion
prebuilt-tax.us.w2	US W-2 forms	Employee name, SSN, wages, federal tax withheld, employer EIN
prebuilt-healthInsuranceCard.us	US health insurance cards	MemberName, MemberId, PlanName, GroupNumber, Payer
prebuilt-businessCard	Business cards	ContactNames, JobTitles, Emails, PhoneNumbers, Addresses, CompanyNames

Python Example: Invoice Analysis

from azure.ai.documentintelligence import DocumentIntelligenceClient
from azure.ai.documentintelligence.models import AnalyzeDocumentRequest
from azure.core.credentials import AzureKeyCredential
import base64

client = DocumentIntelligenceClient(
    endpoint="https://my-docint.cognitiveservices.azure.com/",
    credential=AzureKeyCredential("YOUR_KEY")
)

# Option 1: analyze from URL
poller = client.begin_analyze_document(
    model_id="prebuilt-invoice",
    analyze_request=AnalyzeDocumentRequest(
        url_source="https://example.com/invoice.pdf"
    )
)
result = poller.result()

for invoice in result.documents:
    fields = invoice.fields
    vendor = fields.get("VendorName")
    total = fields.get("InvoiceTotal")
    if vendor:
        print(f"Vendor: {vendor.content} (confidence {vendor.confidence:.2f})")
    if total:
        print(f"Total: {total.content}")

    # Line items
    items = fields.get("Items")
    if items and items.value_array:
        for item in items.value_array:
            desc = item.value_object.get("Description")
            amount = item.value_object.get("Amount")
            if desc and amount:
                print(f"  {desc.content}: {amount.content}")

# Option 2: analyze from local file (base64)
with open("contract.pdf", "rb") as f:
    b64_doc = base64.b64encode(f.read()).decode("utf-8")

poller2 = client.begin_analyze_document(
    model_id="prebuilt-layout",
    analyze_request=AnalyzeDocumentRequest(base64_source=b64_doc)
)
layout_result = poller2.result()

# Extract all tables
for table in layout_result.tables:
    print(f"Table: {table.row_count} rows x {table.column_count} cols")
    for cell in table.cells:
        print(f"  [{cell.row_index},{cell.column_index}]: {cell.content}")

Custom Extraction Models and Composed Models

Model type	Min training docs	When to use
Custom template	5 per form layout	Fixed-layout forms where field positions are consistent (tax forms, standardized surveys). Very fast and accurate for structured layouts.
Custom neural	5 per document type (10+ recommended)	Variable-layout documents where content shifts between instances (invoices from different vendors). Understands semantic meaning of fields, not just position.
Custom classification	5 per document class	Classifying incoming documents by type before routing to the right extraction model (e.g., invoice vs. purchase order vs. receipt).
Composed model	N/A (combines existing custom models)	Single endpoint for multiple document variants. Document Intelligence automatically routes each input to the best-matching constituent model.

When no prebuilt model covers your document type, you train a custom extraction model. The training workflow uses Document Intelligence Studio to label your sample documents (at least 5 per model, ideally 15–20 for better accuracy), train the model, and evaluate field-level accuracy. You then deploy the model and call it by its custom model ID.

A composed model combines multiple custom extraction models into a single model endpoint. When you call a composed model, Document Intelligence automatically classifies which constituent model best matches the input document and routes the extraction accordingly. This is useful when you have a varied document portfolio (invoices from different vendors with different layouts) and want a single API call that handles all variants.

A custom classification model classifies documents by type before extraction — for example, routing a document to the right extraction model in a multi-document scenario. Classification models can be trained independently and then used as a pre-processing step before extraction.

Azure AI Content Understanding for Document Extraction

Content Understanding is a Foundry-native extraction service that complements Azure AI Document Intelligence. While Document Intelligence excels at form and invoice extraction using prebuilt models, Content Understanding is better suited for arbitrary documents where you define a custom field schema and want markdown output alongside structured JSON — particularly useful for feeding documents into RAG pipelines.

Content Understanding vs. Document Intelligence

Aspect	Azure AI Document Intelligence	Azure AI Content Understanding
Where it lives	Standalone Azure resource	Foundry project (accessed via AIProjectClient)
Best for	Invoices, receipts, IDs, tax forms — high-volume structured forms	Arbitrary documents and images; custom field schema; RAG pre-processing
Output format	JSON key-value pairs and tables	Structured JSON matching your schema; optional markdown rendering of the full document
Training required	Yes, for custom extraction models (5–20 labelled samples)	No training — describe fields in the schema definition
Markdown output	Not supported	Supported — full document rendered as clean markdown for RAG chunking

Using Content Understanding in a RAG Pipeline

The markdown output mode from Content Understanding is particularly useful for document-heavy RAG pipelines. Instead of passing raw PDF bytes to a document splitter, you first run Content Understanding to get a clean markdown representation, then chunk that markdown and ingest it into Azure AI Search.

from azure.ai.projects import AIProjectClient
from azure.identity import DefaultAzureCredential

client = AIProjectClient(
    subscription_id="<sub-id>",
    resource_group_name="my-rg",
    project_name="my-project",
    credential=DefaultAzureCredential()
)
cu_client = client.inference.get_content_understanding_client()

# Create an analyzer — fields define the structured extraction schema
cu_client.analyzers.create_or_replace(
    analyzer_name="contract-analyzer",
    definition={
        "description": "Extract key contract fields and full markdown",
        "outputContentFormat": "markdown",   # full doc as markdown
        "fields": {
            "party_names": {"type": "array", "items": {"type": "string"}},
            "governing_law": {"type": "string"},
            "termination_clause": {"type": "string"}
        }
    }
)

# Analyse a PDF stored in Blob Storage
result = cu_client.analyze(
    analyzer_name="contract-analyzer",
    url="https://mystorageaccount.blob.core.windows.net/contracts/agreement.pdf"
)

# result.content contains the full markdown — chunk it for AI Search
markdown_text = result.content
fields = result.fields  # {"party_names": [...], "governing_law": "...", ...}

Exam note: The key differentiator for the exam is the outputContentFormat: "markdown" option. This turns Content Understanding into a document-to-markdown converter — ideal as a preprocessing step before AI Search ingestion when the source documents are PDFs with mixed text, tables, and images.

Exam Tips & Key Takeaways

• Semantic ranking requires Basic tier or higher. It is not available on the Free tier. If an exam scenario requires semantic ranking, the minimum service tier is Basic.
• Vector field dimensions must match the embedding model. text-embedding-3-small = 1536 dimensions; text-embedding-3-large = 3072 dimensions; text-embedding-ada-002 = 1536 dimensions. Mismatching dimensions causes indexing failures.
• prebuilt-read vs prebuilt-layout. The Read model extracts text and reading order but does not return tables or key-value pairs. The Layout model returns everything Read returns, plus tables, key-value pairs, and selection marks. Use Layout for any document where structured data extraction (not just raw text) is needed.
• RAG chunking with the text split skill. In a skillset, use the SplitSkill with text_split_mode="pages" and configure maximumPageLength (e.g., 512 tokens) and pageOverlapLength (e.g., 64 tokens) to create overlapping chunks that preserve context at chunk boundaries.
• Integrated vectorization. When you configure integrated vectorization on an AI Search index, the service automatically calls the Azure OpenAI embedding endpoint during indexing and at query time. You do not need to generate embeddings in your application code — you submit plain text and AI Search handles embedding transparently.
• Semantic ranker operates on top-50 BM25 results only. Semantic ranking does not scan all documents; it re-scores only the candidates returned by the initial BM25 retrieval. Using hybrid search (BM25 + vector) as the first-stage retrieval ensures the semantic ranker sees the most relevant candidates.

Further Learning – Microsoft Learn

• What is Azure AI Search?
• Vector search overview in Azure AI Search
• Semantic ranking in Azure AI Search
• Azure AI Document Intelligence overview
• RAG solution overview in Azure AI Search