testing-course-samples
/testing-course-samples
Use when asked to validate, test, smoke-test, or run the course's notebook and code samples against a live Microsoft Foundry / Azure OpenAI configuration. Covers environment setup (.env, az login, packages), the scripts/validate-notebooks.ps1 runner, interpreting PASS/FAIL results, and which lessons need extra resources (Azure AI Search, GitHub MCP, Foundry Local, Playwright).
- Author
- microsoft ↗
- Repository
- ai-agents-for-beginners ↗
- Licence
- MIT
- Last repository change
- 24.10.2026
Assessment and security
- Files in this package
- 1
- Scripts
- 0
- HTML comments
- 0
- URLs in SKILL.md
- 2
No signals detected. Repository changed on 24.10.2026, without references to retired models or APIs.
MIT, open: use and modify while retaining author attribution.
What it does and when to use it
Use when asked to validate, test, smoke-test, or run the course's notebook and code samples against a live Microsoft Foundry / Azure OpenAI configuration. Covers environment setup (.env, az login, packages), the scripts/validate-notebooks.ps1 runner, interpreting PASS/FAIL results, and which lessons need extra resources (Azure AI Search, GitHub MCP, Foundry Local, Playwright).
Original author description
Use when asked to validate, test, smoke-test, or run the course's notebook and code samples against a live Microsoft Foundry / Azure OpenAI configuration. Covers environment setup (.env, az login, packages), the scripts/validate-notebooks.ps1 runner, interpreting PASS/FAIL results, and which lessons need extra resources (Azure AI Search, GitHub MCP, Foundry Local, Playwright).
How it works
- When to use
- Prerequisites (check first)
- Running the validation
- Interpreting results
- Lessons that need extra resources (expected to fail without them)
- Reporting back
SKILL.md preview
---
name: "testing-course-samples"
description: "Use when asked to validate, test, smoke-test, or run the course's notebook and code samples against a live Microsoft Foundry / Azure OpenAI configuration. Covers environment setup (.env, az login, packages), the scripts/validate-notebooks.ps1 runner, interpreting PASS/FAIL results, and which lessons need extra resources (Azure AI Search, GitHub MCP, Foundry Local, Playwright)."
---
# Testing the Course Samples
Validate that the lesson notebooks and code samples run against a live
Microsoft Foundry / Azure OpenAI setup. The repo ships a runner at
[`scripts/validate-notebooks.ps1`](../../../scripts/validate-notebooks.ps1) that
executes every Python notebook headlessly and prints a PASS/FAIL matrix.
## When to use
- "Validate all the notebooks / samples against my Azure subscription."
- "Smoke-test the course after upgrading packages or changing models."
- "Which lessons still pass / fail live?"
Do **not** use this for the AI Smoke Test GitHub Action (that validates *deployed*
hosted agents — see [`tests/README.md`](../../../tests/README.md)). This skill
runs the notebooks locally.
## Prerequisites (check first)
1. **Python 3.12+** with course deps: `python -m pip install -r requirements.txt`
plus the executor: `python -m pip install nbconvert ipykernel`.
2. **`.env` at the repo root** (copy from [`.env.example`](../../../.env.example)) with at least:
- `AZURE_AI_PROJECT_ENDPOINT` — Foundry project endpoint
(`https://<account>.services.ai.azure.com/api/projects/<project>`)
- `AZURE_AI_MODEL_DEPLOYMENT_NAME` — a non-deprecated deployment (e.g. `gpt-5-mini`)
- `AZURE_OPENAI_ENDPOINT` (`https://<account>.openai.azure.com`) and `AZURE_OPENAI_DEPLOYMENT`
for lessons that call Azure OpenAI directly (Lesson 06, 02-azure-openai, 14 handoff/human-loop).
3. **`az login`** completed — samples authenticate with `AzureCliCredential` (Entra ID, keyless).
4. Verify the model deployment exists:
`az cognitiveservices account deployment list -g <rg> -n <account> -o table`.
## Running the validation
```powershell
# All Python notebooks (skips .NET, .venv, site-packages, translations, skill assets)
pwsh scripts/validate-notebooks.ps1
# A single lesson, with a longer per-cell timeout
pwsh scripts/validate-notebooks.ps1 -Filter '08-*' -Timeout 600
# Just list what would run (no execution)
pwsh scripts/validate-notebooks.ps1 -List
# Explicit interpreter (if `python` is not on PATH, e.g. Windows Store alias)
pwsh scripts/validate-notebooks.ps1 -Python "C:/path/to/python.exe"
```
The script writes executed copies, per-notebook logs, and `results.json` to
`$env:TEMP\aiab-nbval` and exits with the number of failures.
Transient failures (shared-subscription HTTP 429 rate limits, an occasional
`AzureCliCredential` token hiccup, or a timeout) are retried automatically
(`-Retries`, default 2, with `-RetryDelaySeconds` backoff, default 20). If a
model deployment is regularly 429-ing, check the subscription's GlobalStandard
TPM quota (`az cognitiveservices usage list -l <region>`) — raising a single
deployment's capacity does not help when the *subscription* quota is exhausted.
## Interpreting results
- `PASS` — the notebook ran end-to-end with no cell error.
- `FAIL` — the first `*Error` / `*Exception` line is shown; open the matching
`log_*.txt` in the output dir for the full traceback.
- A single notebook's failure is bounded by `-Timeout` (per cell), so a hung
human-in-the-loop cell surfaces as `StdinNotImplementedError` rather than hanging.
## Lessons that need extra resources (expected to fail without them)
| Lesson | Extra requirement |
|--------|-------------------|
| 05 Agentic RAG | Azure AI Search (`AZURE_SEARCH_SERVICE_ENDPOINT`, key) — has an in-memory fallback path |
| 11 MCP / GitHub | GitHub MCP server + PAT |
| 13 memory (cognee) | `cognee` configured with a model provider |
| 15 browser-use | Playwright browsers installed (`playwright install`) + `AZURE_OPENAI_CHAT_DEPLOYMENT_NAME` |
| 17 local agent | Foundry Local runtime + a downloaded Qwen model (on-device, no cloud) |
| `*-dotnet-*` notebooks | .NET Interactive kernel (excluded by default; use `-IncludeDotnet`) |
## Reporting back
Summarise as a PASS/FAIL table grouped by lesson. Separate genuine regressions
(code/config bugs to fix) from environment gaps (missing Search/Foundry Local/PAT),
and cite the failing `log_*.txt` for each real failure.
Versions
| Repository | Score | Licence |
|---|---|---|
| This page · microsoft/ai-agents-for-beginners | 100 | MIT |
| microsoft/ai-agents-for-beginners | 100 | MIT |
| microsoft/ai-agents-for-beginners | 100 | MIT |
| microsoft/ai-agents-for-beginners | 100 | MIT |
| microsoft/ai-agents-for-beginners | 100 | MIT |
Identical content appears in 1 other GitHub folders. We show one copy with the highest score.
Database snapshot: 13.11.2026 · package version 2e161421d32b (SHA-256 digest of the entire package)
Downloads in the last 30 days: 0 · ranking
Report a problem with this skill: harmful content, incorrect descriptions or an author’s removal request.
Similar skills
- 85microsoft-foundry-local
Expert knowledge for Microsoft Foundry Local (aka Azure AI Foundry Local) development including best practices, decision making, configuration, and integrations & coding patterns. Use when compiling HF models with Olive, using Foundry Local CLI, embeddings/chat/audio APIs, or… - 85microsoft-foundry-local
Expert knowledge for Microsoft Foundry Local (aka Azure AI Foundry Local) development including best practices, decision making, configuration, and integrations & coding patterns. Use when compiling HF models with Olive, using Foundry Local CLI, embeddings/chat/audio APIs, or… - 95microsoft-foundry-classic
Expert knowledge for Microsoft Foundry Classic (aka Azure AI Foundry classic) development including troubleshooting, best practices, decision making, architecture & design patterns, limits & quotas, security, configuration, integrations & coding patterns, and deployment. Use… - 95microsoft-foundry
Expert knowledge for Microsoft Foundry (aka Azure AI Foundry) development including troubleshooting, best practices, decision making, architecture & design patterns, limits & quotas, security, configuration, integrations & coding patterns, and deployment. Use when building… - 95microsoft-foundry
Use this skill to work with Microsoft Foundry (Azure AI Foundry): deploy AI models from catalog, build RAG applications with knowledge indexes, create and evaluate AI agents. USE FOR: Microsoft Foundry, AI Foundry, deploy model, model catalog, RAG, knowledge index, create agent,… - 98foundry-hosted-agent-validation
Step-by-step process for validating a Python Foundry hosted agent sample (under python/samples/04-hosting/foundry-hosted-agents/) end to end — running it locally (native runtime and `azd ai agent run`) and after deploying it to an Azure AI Foundry project with `azd`. Use this…