Build GenAI Solutions with Microsoft Foundry

Part 2 terms — introduced before any code

How Foundry is organised for GenAI work

From Part 1 Topic 1 you know Foundry has three levels: Hub → Project → Services. Now let's go deeper into what each level means for building GenAI solutions.

What lives where in Foundry

Creating a Hub and Project — portal walkthrough

Choosing the right model for your use case

The Model Catalog in Foundry contains hundreds of models. The exam tests whether you can choose the right model for a given scenario. These are the key model families you need to know:

Building multi-step AI pipelines

A single API call to an LLM is often not enough for real-world AI applications. You typically need to: retrieve information, format a prompt, call the model, process the output, and maybe call the model again. Prompt Flow lets you build these multi-step pipelines visually and in code.

Types of nodes in a Prompt Flow

Flow types you need to know

Prompt templates — Jinja2 syntax

Foundry uses Jinja2 for prompt templates. You put variable names inside double curly braces: {{variable_name}}. At runtime Foundry replaces these with actual values from previous nodes.

# This is a Jinja2 prompt template (not Python code — it's a template string)
# Used inside a Foundry Prompt Template node

system:
You are a helpful customer service assistant for Contoso Bank.
Answer questions only based on the provided context.
If the answer is not in the context, say "I don't have that information."

user:
Context from our knowledge base:
# {{context}} is a Jinja2 variable — replaced at runtime with retrieved documents
{{context}}

Customer question:
# {{user_question}} is replaced with the actual question the user asked
{{user_question}}

# At runtime this becomes:
# system: You are a helpful customer service assistant...
# user: Context from our knowledge base:
#        [retrieved document text here]
#        Customer question: What is the overdraft fee?

Grounding your model in real data

RAG (Retrieval-Augmented Generation) is the most important pattern in enterprise AI. Instead of relying on the LLM's training knowledge (which may be outdated or simply wrong), you retrieve relevant documents from your own data and include them in the prompt. The model answers based on your data.

The 5-step RAG pipeline

Python — Building a simple RAG call

New Python concept introduced here: dictionary. A dictionary stores key-value pairs — like a lookup table. {"key": "value"}. Explained fully in the code comments.
Install: pip install openai azure-search-documents azure-identity

import os
from openai import AzureOpenAI          # Azure OpenAI client
from azure.search.documents import SearchClient   # Azure AI Search client
from azure.core.credentials import AzureKeyCredential


# ── Step 1: Set up Azure OpenAI client ────────────────────────────────
# AzureOpenAI connects to YOUR Azure OpenAI resource (not public OpenAI)
openai_client = AzureOpenAI(
    azure_endpoint = os.environ.get("AZURE_OPENAI_ENDPOINT"),  # Your Azure OpenAI endpoint
    api_key        = os.environ.get("AZURE_OPENAI_KEY"),      # Your Azure OpenAI key
    api_version    = "2024-02-01"                          # API version — always specify this
)

# ── Step 2: Set up Azure AI Search client ─────────────────────────────
search_client = SearchClient(
    endpoint    = os.environ.get("SEARCH_ENDPOINT"),  # Your search endpoint (search.windows.net)
    index_name  = "documents-index",              # The index where your chunks are stored
    credential  = AzureKeyCredential(os.environ.get("SEARCH_KEY"))
)


# ── Step 3: Embed the user's question ─────────────────────────────────
user_question = "What is the company's refund policy for digital products?"

# Call the embedding model to convert the question into a vector (list of numbers)
# embeddings.create() is a function on the client
# model= specifies the deployment name of your embedding model
# input= is the text to embed
embedding_response = openai_client.embeddings.create(
    model = "text-embedding-ada-002",   # Your embedding model deployment name
    input = user_question
)

# .data[0].embedding extracts the vector (list of numbers) from the response
# .data is a list of results (we sent one input, so data[0] is our result)
# .embedding is the actual list of ~1536 floating point numbers
question_vector = embedding_response.data[0].embedding


# ── Step 4: Search for the most relevant document chunks ──────────────
# New concept: dictionary — a collection of key-value pairs in curly braces {}
# Like a real dictionary: "word" (key) → "definition" (value)
# Here: "vector" is the key, question_vector is the value
vector_query = {
    "vector": question_vector,   # The embedding of the user's question
    "fields": "content_vector",  # Which field in the index stores the document vectors
    "k": 3                        # Return the top 3 most similar chunks
}

# search() queries the Azure AI Search index using the vector
results = search_client.search(
    search_text  = "",           # Empty string = vector-only search (no keyword search)
    vector_queries = [vector_query],   # Pass our vector query
    select       = ["content"]   # Only return the "content" field from each chunk
)

# Build a single string from all retrieved chunks
# For loop joins all chunk texts with two newlines between them
context_text = "\n\n".join([result["content"] for result in results])


# ── Step 5: Send the grounded prompt to the LLM ───────────────────────
# New concept: list of dictionaries — a list where each item is a dictionary
# The messages array tells the model the conversation so far
# Each message is a dictionary with "role" and "content" keys
messages = [
    {
        "role": "system",      # "system" sets the model's behaviour and persona
        "content": "You are a helpful assistant. Answer questions using ONLY the provided context. "
                   "If the answer is not in the context, say you don't know."
    },
    {
        "role": "user",        # "user" is the human's message
        "content": f"Context:\n{context_text}\n\nQuestion: {user_question}"
        # f-string (from Python) inserts variables: context_text and user_question
        # The model sees the retrieved documents AND the question in one message
    }
]

# chat.completions.create() sends the conversation to GPT-4o and returns a response
# model= is the deployment name of your GPT-4o model (not "gpt-4o" literally)
response = openai_client.chat.completions.create(
    model    = "my-gpt4o-deployment",  # Your deployment name from Foundry
    messages = messages               # The conversation history
)

# Extract the model's reply from the response
# .choices is a list of possible responses (usually 1)
# .choices[0] is the first (and usually only) response
# .message.content is the actual text the model generated
answer = response.choices[0].message.content
print(f"Answer: {answer}")

Measuring the quality of your GenAI solution

Before deploying a GenAI solution to users, you need to measure how well it actually performs. Microsoft Foundry includes built-in evaluation tools that automatically score your model's outputs against quality metrics.

The 5 key evaluation metrics

How to run an evaluation in Foundry

Calling your deployed flow from a Python application

Once your Prompt Flow is deployed as an endpoint in Foundry, you call it from your Python application just like any other REST endpoint. The Foundry SDK makes this straightforward.
Install: pip install azure-ai-projects azure-identity

import os
from azure.ai.projects import AIProjectClient    # Foundry project client
from azure.identity import DefaultAzureCredential  # Managed identity auth (from Part 1)


# ── Create the Foundry project client ────────────────────────────────
# Connection string format: region.api.azureml.ms;subscription-id;resource-group;workspace-name
# Found in: ai.azure.com → your project → Overview → Connection string
connection_string = os.environ.get("AZURE_AI_PROJECT_CONNECTION_STRING")

# DefaultAzureCredential automatically uses managed identity when running in Azure
# (no credentials to store — from Part 1 Topic 3)
credential = DefaultAzureCredential()

# AIProjectClient connects to your specific Foundry project
project_client = AIProjectClient.from_connection_string(
    conn_str   = connection_string,
    credential = credential
)


# ── Use a connection to get an OpenAI client ──────────────────────────
# Foundry manages the connection — you don't need to hardcode the endpoint or key
# get_azure_openai_client() returns a pre-configured OpenAI client using the
# connection you set up in your Foundry hub (Connection → Azure OpenAI)
openai_client = project_client.inference.get_azure_openai_client(
    api_version = "2024-02-01"
)

# ── Call your model deployment ────────────────────────────────────────
# Now use the client exactly as you would with direct Azure OpenAI
response = openai_client.chat.completions.create(
    model    = "my-gpt4o-deployment",   # Your deployment name
    messages = [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user",   "content": "What is Azure AI Foundry?"}
    ]
)

print(response.choices[0].message.content)

What this topic is specifically tested on

Test your understanding

5 questions — select an answer to see the explanation immediately.