Monitor & Manage Azure AI Services

Monitoring terms you must know

How Azure Monitor works for AI services

The monitoring data flow

The metrics you can track

Azure AI services automatically emit these metrics to Azure Monitor. No configuration needed — they appear as soon as your resource exists. You view them in the Azure portal → your resource → Metrics.

How to view metrics in the Azure portal

Monitoring what happens inside your application

Azure Monitor metrics show you what's happening at the service level — how many calls, how fast. Application Insights goes deeper — it shows you what's happening inside your Python application: which functions are slow, which exceptions are being thrown, how users are flowing through your app.

Adding Application Insights to your Python app

A new Python concept here: with statement (context manager). It is used by the Application Insights SDK to track operations. It is explained fully in the code comments.
Install with: pip install azure-monitor-opentelemetry

# Install first: pip install azure-monitor-opentelemetry

# This library instruments your Python app to send telemetry to Application Insights
from azure.monitor.opentelemetry import configure_azure_monitor

import os
from azure.ai.textanalytics import TextAnalyticsClient
from azure.core.credentials import AzureKeyCredential


# ── Step 1: Set up Application Insights ──────────────────────────────

# Your Application Insights connection string
# Found in: Azure portal → Application Insights resource → Overview → Connection String
# Format: InstrumentationKey=xxx;IngestionEndpoint=https://...;LiveEndpoint=https://...
connection_string = os.environ.get("APPLICATIONINSIGHTS_CONNECTION_STRING")

# configure_azure_monitor() sets up automatic telemetry collection
# After this line, Python exceptions, HTTP calls, and performance data
# are automatically sent to Application Insights — no extra code needed
configure_azure_monitor(connection_string=connection_string)


# ── Step 2: Use your AI service normally — telemetry is collected auto ─

key      = os.environ.get("LANGUAGE_KEY")
endpoint = os.environ.get("LANGUAGE_ENDPOINT")
client   = TextAnalyticsClient(endpoint=endpoint, credential=AzureKeyCredential(key))

# This call is now automatically tracked by Application Insights
# Duration, success/failure, and any exceptions are recorded
result = client.analyze_sentiment(documents=["Azure AI is excellent!"])

print(f"Sentiment: {result[0].sentiment}")
# Application Insights receives: duration of this call, success=True, HTTP status


# ── Step 3: Add a custom event (optional but powerful) ────────────────

# New concept: "from X import Y" — importing a specific class from a library
from opentelemetry import trace

# Get the tracer — this lets us create custom tracking spans
# A span is a named unit of work that gets recorded in Application Insights
tracer = trace.get_tracer(__name__)

# "with" statement — new Python concept explained:
# "with tracer.start_as_current_span(...) as span:" means:
#   "Start tracking a named operation called 'sentiment-analysis-batch'
#    and call it 'span' inside this block"
# Everything inside the indented block is timed and recorded as one unit
# When the block finishes (or if an error occurs), the span is automatically closed
with tracer.start_as_current_span("sentiment-analysis-batch") as span:

    # Add custom attributes to this span — extra information to record
    span.set_attribute("document.count", 3)   # How many documents we're analysing
    span.set_attribute("batch.type", "sentiment")  # What type of analysis

    # The actual AI call happens inside the "with" block
    # so Application Insights knows it's part of this named operation
    docs = ["Great service!", "Terrible experience.", "It was okay."]
    results = client.analyze_sentiment(documents=docs)

    # New concept: for loop — runs the indented block once for each item in a list
    # "for doc in results" means: take each item from results, call it "doc", run the block
    for doc in results:
        print(f"  Sentiment: {doc.sentiment} | Score: {doc.confidence_scores}")

# After the "with" block: Application Insights records the complete span
# including duration, custom attributes, and any exceptions that occurred

Routing logs to the right destination

By default, Azure AI service logs are not stored anywhere — they are generated and discarded. To keep them for analysis, you must configure diagnostic settings to route them to a destination.

Three log destinations

Log categories for Azure AI services

When setting up diagnostic settings, you choose which log categories to capture:

How to configure diagnostic settings — portal walkthrough

Writing KQL queries to analyse Azure AI service logs

KQL (Kusto Query Language) is how you search your logs in Log Analytics. It looks a bit like SQL but works differently. Let's learn the key building blocks before reading actual queries.

KQL building blocks

Exam-ready KQL queries for Azure AI services

These are the queries the exam most commonly tests. Read the comments — they explain what every line does.

Getting notified when something goes wrong

You shouldn't need to check metrics manually every hour. Alert rules watch your metrics and logs 24/7 and notify you automatically when a threshold is breached.

Anatomy of an alert rule

Alert rules commonly tested on the exam

Creating an alert rule — portal walkthrough

Keeping Azure AI costs under control

Azure AI services charge per API call (on S0 tier). Unexpected traffic spikes or misconfigured apps can run up large bills. The exam tests your knowledge of cost management tools and strategies.

F0 vs S0 — cost implications

Cost management tools

What this topic is tested on

Test your understanding

5 questions — select an answer to see the explanation immediately.