Add mock AI providers for templates and testing - #7657
#7657Add mock AI providers for templates and testing#7657jeffhandley wants to merge 10 commits intodotnet:maindotnet/extensions:mainfrom jeffhandley:jeffhandley/mock-chat-client-planjeffhandley/extensions:jeffhandley/mock-chat-client-planCopy head branch name to clipboard
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces a new preview testing surface for Microsoft.Extensions.AI and updates the aichatweb / aiagent-webapi templates (and their snapshot-based integration tests) to support a deterministic local mock provider while removing GitHub Models as a default.
Changes:
- Add preview
Microsoft.Extensions.AI.Testingwith mock implementations (e.g.,MockChatClient,MockEmbeddingGenerator<TInput>), plus a new test project for it. - Update project templates to require an explicit
--providerand add a deterministicmockprovider path (including citations/retrieval/tooling flows). - Update template infrastructure + integration-test snapshots/execution matrices to reflect the new provider options and template outputs.
Reviewed changes
Copilot reviewed 141 out of 209 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/Services/SemanticSearch.cs | Snapshot update for semantic search service behavior in OpenAI variant. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/Services/Ingestion/PdfPigReader.cs | Snapshot update for PDF ingestion reader. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/Services/Ingestion/DocumentReader.cs | Snapshot update for document reader routing by media type. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/Services/Ingestion/DataIngestor.cs | Snapshot update for ingestion pipeline wiring. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/Services/IngestedChunk.cs | Snapshot update for vector record shape for ingestion/search. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/README.md | Snapshot update for provider configuration instructions. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/Properties/launchSettings.json | Snapshot update for launch profiles. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/Program.cs | Snapshot update switching from GitHub Models to OpenAI configuration. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/Components/Routes.razor | Snapshot update for routing component. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/Components/Pages/Error.razor | Snapshot update for error page. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/Components/Pages/Chat/ChatSuggestions.razor.css | Snapshot update for chat suggestions styling. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/Components/Pages/Chat/ChatMessageList.razor.js | Snapshot update for chat auto-scroll JS module. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/Components/Pages/Chat/ChatMessageList.razor.css | Snapshot update for message list styling. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/Components/Pages/Chat/ChatMessageList.razor | Snapshot update for message list component. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/Components/Pages/Chat/ChatInput.razor.js | Snapshot update for chat input JS behaviors. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/Components/Pages/Chat/ChatInput.razor.css | Snapshot update for chat input styling. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/Components/Pages/Chat/ChatInput.razor | Snapshot update for chat input component. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/Components/Pages/Chat/ChatHeader.razor.css | Snapshot update for header styling. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/Components/Pages/Chat/ChatHeader.razor | Snapshot update for header component. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/Components/Pages/Chat/ChatCitation.razor.css | Snapshot update for citation styling. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/Components/Pages/Chat/Chat.razor.css | Snapshot update for chat footer container styling. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/Components/Layout/MainLayout.razor.css | Snapshot update for layout error UI styling. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/Components/Layout/MainLayout.razor | Snapshot update for layout error UI markup. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/Components/Layout/LoadingSpinner.razor | Snapshot update for loading spinner. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/Components/App.razor | Snapshot update for app host page markup. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/Components/_Imports.razor | Snapshot update for component imports. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/appsettings.json | Snapshot update for runtime settings. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/appsettings.Development.json | Snapshot update for dev settings. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai.verified/aichatweb/aichatweb.csproj | Snapshot update for template project dependencies. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai_aais.verified/aichatweb/Components/Pages/Chat/ChatMessageItem.razor | Snapshot update for citation parsing/markup handling. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai_aais.verified/aichatweb/Components/Pages/Chat/Chat.razor | Snapshot update for XML-safe tool result formatting and survey removal. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai_aais.verified/aichatweb/Components/Layout/SurveyPrompt.razor.css | Snapshot removal of survey prompt styling. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.oai_aais.verified/aichatweb/Components/Layout/SurveyPrompt.razor | Snapshot removal of survey prompt component. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.o_q.verified/aichatweb/aichatweb.Web/Components/Pages/Chat/ChatMessageItem.razor | Snapshot update for citation parsing/markup handling (Aspire/Qdrant variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.o_q.verified/aichatweb/aichatweb.Web/Components/Pages/Chat/Chat.razor | Snapshot update for XML-safe tool result formatting and survey removal (Aspire/Qdrant variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.o_q.verified/aichatweb/aichatweb.Web/Components/Layout/SurveyPrompt.razor.css | Snapshot removal of survey prompt styling (Aspire/Qdrant variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.o_q.verified/aichatweb/aichatweb.Web/Components/Layout/SurveyPrompt.razor | Snapshot removal of survey prompt component (Aspire/Qdrant variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Services/SemanticSearch.cs | Snapshot update for semantic search service behavior in mock-provider variant. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Services/Ingestion/PdfPigReader.cs | Snapshot update for PDF ingestion reader (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Services/Ingestion/DocumentReader.cs | Snapshot update for document reader routing by media type (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Services/Ingestion/DataIngestor.cs | Snapshot update for ingestion pipeline wiring (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Services/IngestedChunk.cs | Snapshot update for vector record shape (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/README.md | Snapshot update documenting mock provider usage. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Properties/launchSettings.json | Snapshot update for launch profiles (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Program.cs | Snapshot update wiring mock chat + embedding providers. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Components/Routes.razor | Snapshot update for routing component (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Components/Pages/Error.razor | Snapshot update for error page (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Components/Pages/Chat/ChatSuggestions.razor.css | Snapshot update for chat suggestions styling (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Components/Pages/Chat/ChatMessageList.razor.js | Snapshot update for chat auto-scroll JS module (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Components/Pages/Chat/ChatMessageList.razor.css | Snapshot update for message list styling (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Components/Pages/Chat/ChatMessageList.razor | Snapshot update for message list component (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Components/Pages/Chat/ChatMessageItem.razor | Snapshot update for citation parsing/markup handling (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Components/Pages/Chat/ChatInput.razor.js | Snapshot update for chat input JS behaviors (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Components/Pages/Chat/ChatInput.razor.css | Snapshot update for chat input styling (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Components/Pages/Chat/ChatInput.razor | Snapshot update for chat input component (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Components/Pages/Chat/ChatHeader.razor.css | Snapshot update for header styling (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Components/Pages/Chat/ChatHeader.razor | Snapshot update for header component (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Components/Pages/Chat/ChatCitation.razor.css | Snapshot update for citation styling (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Components/Pages/Chat/ChatCitation.razor | Snapshot update adding citation viewer link generation (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Components/Pages/Chat/Chat.razor.css | Snapshot update for chat footer container styling (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Components/Pages/Chat/Chat.razor | Snapshot update for XML-safe tool result formatting and survey removal (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Components/Layout/MainLayout.razor.css | Snapshot update for layout error UI styling (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Components/Layout/MainLayout.razor | Snapshot update for layout error UI markup (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Components/Layout/LoadingSpinner.razor | Snapshot update for loading spinner (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Components/App.razor | Snapshot update for app host page markup (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/Components/_Imports.razor | Snapshot update for component imports (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/appsettings.json | Snapshot update for runtime settings (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/appsettings.Development.json | Snapshot update for dev settings (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.mock.verified/aichatweb/aichatweb.csproj | Snapshot update adding Microsoft.Extensions.AI.Testing dependency (mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A.verified/aichatweb/aichatweb.Web/Components/Layout/SurveyPrompt.razor.css | Snapshot removal of survey prompt styling (Aspire variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A.verified/aichatweb/aichatweb.Web/Components/Layout/SurveyPrompt.razor | Snapshot removal of survey prompt component (Aspire variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/README.md | Snapshot update documenting mock provider usage (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/Services/SemanticSearch.cs | Snapshot update for semantic search service (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/Services/Ingestion/DocumentReader.cs | Snapshot update for MarkItDown MCP PDF reading (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/Services/Ingestion/DataIngestor.cs | Snapshot update for ingestion pipeline wiring (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/Services/IngestedChunk.cs | Snapshot update for vector record key type (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/Properties/launchSettings.json | Snapshot update for launch profiles (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/Program.cs | Snapshot update wiring mock providers into Aspire-hosted web app. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/Components/Routes.razor | Snapshot update for routing component (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/Components/Pages/Error.razor | Snapshot update for error page (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/Components/Pages/Chat/ChatSuggestions.razor.css | Snapshot update for chat suggestions styling (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/Components/Pages/Chat/ChatMessageList.razor.js | Snapshot update for chat auto-scroll JS module (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/Components/Pages/Chat/ChatMessageList.razor.css | Snapshot update for message list styling (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/Components/Pages/Chat/ChatMessageList.razor | Snapshot update for message list component (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/Components/Pages/Chat/ChatMessageItem.razor | Snapshot update for citation parsing/markup handling (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/Components/Pages/Chat/ChatInput.razor.js | Snapshot update for chat input JS behaviors (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/Components/Pages/Chat/ChatInput.razor.css | Snapshot update for chat input styling (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/Components/Pages/Chat/ChatInput.razor | Snapshot update for chat input component (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/Components/Pages/Chat/ChatHeader.razor.css | Snapshot update for header styling (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/Components/Pages/Chat/ChatHeader.razor | Snapshot update for header component (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/Components/Pages/Chat/ChatCitation.razor.css | Snapshot update for citation styling (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/Components/Pages/Chat/Chat.razor.css | Snapshot update for chat footer container styling (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/Components/Pages/Chat/Chat.razor | Snapshot update for XML-safe tool result formatting and survey removal (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/Components/Layout/MainLayout.razor.css | Snapshot update for layout error UI styling (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/Components/Layout/MainLayout.razor | Snapshot update for layout error UI markup (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/Components/Layout/LoadingSpinner.razor | Snapshot update for loading spinner (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/Components/App.razor | Snapshot update for app host page markup (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/Components/_Imports.razor | Snapshot update for component imports (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/appsettings.json | Snapshot update for runtime settings (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/appsettings.Development.json | Snapshot update for dev settings (Aspire/mock variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.Web/aichatweb.Web.csproj | Snapshot update adding Microsoft.Extensions.AI.Testing dependency (Aspire/mock web). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.ServiceDefaults/aichatweb.ServiceDefaults.csproj | Snapshot addition/updates for Aspire service defaults project. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.AppHost/Properties/launchSettings.json | Snapshot update for Aspire AppHost launch settings. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.AppHost/appsettings.json | Snapshot update for AppHost runtime settings. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.AppHost/appsettings.Development.json | Snapshot update for AppHost dev settings. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.AppHost/AppHost.cs | Snapshot update removing OpenAI connection string dependency for mock provider. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_mock.verified/aichatweb/aichatweb.AppHost/aichatweb.AppHost.csproj | Snapshot update for AppHost project structure. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_aoai_aais.verified/aichatweb/aichatweb.Web/Components/Pages/Chat/ChatMessageItem.razor | Snapshot update for citation parsing/markup handling (AOAI/AAIS variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_aoai_aais.verified/aichatweb/aichatweb.Web/Components/Pages/Chat/Chat.razor | Snapshot update for XML-safe tool result formatting and survey removal (AOAI/AAIS variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_aoai_aais.verified/aichatweb/aichatweb.Web/Components/Layout/SurveyPrompt.razor.css | Snapshot removal of survey prompt styling (AOAI/AAIS variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb.A_aoai_aais.verified/aichatweb/aichatweb.Web/Components/Layout/SurveyPrompt.razor | Snapshot removal of survey prompt component (AOAI/AAIS variant). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb._defaults.verified/aichatweb/README.md | Snapshot removal of default GitHub Models README guidance. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb._defaults.verified/aichatweb/Components/Layout/SurveyPrompt.razor.css | Snapshot removal of survey prompt styling (defaults). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/Snapshots/aichatweb/aichatweb._defaults.verified/aichatweb/Components/Layout/SurveyPrompt.razor | Snapshot removal of survey prompt component (defaults). |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/AIChatWebSnapshotTests.cs | Update snapshot coverage to include explicit mock/openai provider args and Aspire+mock. |
| test/ProjectTemplates/Microsoft.Extensions.AI.Templates.IntegrationTests/AIChatWebExecutionTests.cs | Update execution matrix to include mock provider and remove GitHub Models. |
| test/ProjectTemplates/Microsoft.Agents.AI.ProjectTemplates.IntegrationTests/Snapshots/aiagent-webapi/aiagent-webapi.oai.verified/aiagent-webapi/README.md | Snapshot update removing GitHub Models from provider docs and adding mock provider. |
| test/ProjectTemplates/Microsoft.Agents.AI.ProjectTemplates.IntegrationTests/Snapshots/aiagent-webapi/aiagent-webapi.oai.verified/aiagent-webapi/Program.cs | Snapshot update adding DevUI registration. |
| test/ProjectTemplates/Microsoft.Agents.AI.ProjectTemplates.IntegrationTests/Snapshots/aiagent-webapi/aiagent-webapi.o.verified/aiagent-webapi/README.md | Snapshot update removing GitHub Models from provider docs and adding mock provider. |
| test/ProjectTemplates/Microsoft.Agents.AI.ProjectTemplates.IntegrationTests/Snapshots/aiagent-webapi/aiagent-webapi.o.verified/aiagent-webapi/Program.cs | Snapshot update adding DevUI registration. |
| test/ProjectTemplates/Microsoft.Agents.AI.ProjectTemplates.IntegrationTests/Snapshots/aiagent-webapi/aiagent-webapi.mock.verified/aiagent-webapi/Properties/launchSettings.json | Snapshot update for DevUI launch URL and profiles (mock). |
| test/ProjectTemplates/Microsoft.Agents.AI.ProjectTemplates.IntegrationTests/Snapshots/aiagent-webapi/aiagent-webapi.mock.verified/aiagent-webapi/Program.cs | Snapshot update wiring mock provider and adding DevUI registration. |
| test/ProjectTemplates/Microsoft.Agents.AI.ProjectTemplates.IntegrationTests/Snapshots/aiagent-webapi/aiagent-webapi.mock.verified/aiagent-webapi/appsettings.json | Snapshot update for runtime settings (mock). |
| test/ProjectTemplates/Microsoft.Agents.AI.ProjectTemplates.IntegrationTests/Snapshots/aiagent-webapi/aiagent-webapi.mock.verified/aiagent-webapi/aiagent-webapi.csproj | Snapshot update adding Microsoft.Extensions.AI.Testing dependency (mock). |
| test/ProjectTemplates/Microsoft.Agents.AI.ProjectTemplates.IntegrationTests/Snapshots/aiagent-webapi/aiagent-webapi.aoai_ID_T.verified/aiagent-webapi/README.md | Snapshot update removing GitHub Models from provider docs and adding mock provider. |
| test/ProjectTemplates/Microsoft.Agents.AI.ProjectTemplates.IntegrationTests/Snapshots/aiagent-webapi/aiagent-webapi.aoai_ID_T.verified/aiagent-webapi/Program.cs | Snapshot update adding DevUI registration. |
| test/ProjectTemplates/Microsoft.Agents.AI.ProjectTemplates.IntegrationTests/Snapshots/aiagent-webapi/aiagent-webapi.aoai_ID_F.verified/aiagent-webapi/README.md | Snapshot update removing GitHub Models from provider docs and adding mock provider. |
| test/ProjectTemplates/Microsoft.Agents.AI.ProjectTemplates.IntegrationTests/Snapshots/aiagent-webapi/aiagent-webapi.aoai_ID_F.verified/aiagent-webapi/Program.cs | Snapshot update adding DevUI registration. |
| test/ProjectTemplates/Microsoft.Agents.AI.ProjectTemplates.IntegrationTests/AIAgentWebAPISnapshotTests.cs | Update snapshot coverage to include explicit mock provider args. |
| test/ProjectTemplates/Microsoft.Agents.AI.ProjectTemplates.IntegrationTests/AIAgentWebAPIExecutionTests.cs | Update execution matrix to include mock provider and remove GitHub Models. |
| test/ProjectTemplates/Infrastructure/TemplateSnapshotTestBase.cs | Make template package resolution cross-platform by resolving to a concrete .nupkg path. |
| test/Libraries/Microsoft.Extensions.DataIngestion.Tests/Writers/SqliteVectorStoreWriterTests.cs | Switch test harness to MockEmbeddingGenerator from the new testing package. |
| test/Libraries/Microsoft.Extensions.DataIngestion.Tests/Writers/InMemoryVectorStoreWriterTests.cs | Switch test harness to MockEmbeddingGenerator from the new testing package. |
| test/Libraries/Microsoft.Extensions.DataIngestion.Tests/Utils/TestEmbeddingGenerator.cs | Remove local test embedding generator in favor of Microsoft.Extensions.AI.Testing. |
| test/Libraries/Microsoft.Extensions.DataIngestion.Tests/Microsoft.Extensions.DataIngestion.Tests.csproj | Add reference to Microsoft.Extensions.AI.Testing and remove embedded test generator compile include. |
| test/Libraries/Microsoft.Extensions.DataIngestion.Tests/Chunkers/SemanticSimilarityChunkerTests.cs | Update chunker tests to use MockEmbeddingGenerator and factor helper generator creation. |
| test/Libraries/Microsoft.Extensions.AI.Tests/MockEmbeddingGeneratorAlias.cs | Add alias to preserve TestEmbeddingGenerator naming expectations in tests. |
| test/Libraries/Microsoft.Extensions.AI.Tests/Microsoft.Extensions.AI.Tests.csproj | Add reference to Microsoft.Extensions.AI.Testing and remove old embedded embedding generator compile include. |
| test/Libraries/Microsoft.Extensions.AI.Testing.Tests/Microsoft.Extensions.AI.Testing.Tests.csproj | Add new test project for the new testing package. |
| test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/TestEmbeddingGenerator.cs | Remove old test embedding generator type in favor of Microsoft.Extensions.AI.Testing. |
| test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/Microsoft.Extensions.AI.Abstractions.Tests.csproj | Add reference to Microsoft.Extensions.AI.Testing. |
| test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/Embeddings/GeneratedEmbeddingsTests.cs | Update embedding tests to use MockEmbeddingGenerator. |
| test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/Embeddings/EmbeddingGeneratorExtensionsTests.cs | Update extension tests to use MockEmbeddingGenerator. |
| test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/Embeddings/DelegatingEmbeddingGeneratorTests.cs | Update delegating generator tests to use MockEmbeddingGenerator. |
| src/Shared/DiagnosticIds/DiagnosticIds.cs | Add new experimental diagnostic ID for AI Testing APIs. |
| src/ProjectTemplates/README.md | Update template docs to require --provider and list new mock option. |
| src/ProjectTemplates/Microsoft.Extensions.AI.Templates/templates/AIChatWeb-CSharp/README.Aspire.md | Add mock-provider instructions and gate provider-key note for non-mock configs. |
| src/ProjectTemplates/Microsoft.Extensions.AI.Templates/templates/AIChatWeb-CSharp/AIChatWeb-CSharp.Web/Services/IngestedChunk.cs | Add conditional key typing in the template’s vector record model. |
| src/ProjectTemplates/Microsoft.Extensions.AI.Templates/templates/AIChatWeb-CSharp/AIChatWeb-CSharp.Web/README.md | Add mock-provider instructions and gate provider-key note for non-mock configs. |
| src/ProjectTemplates/Microsoft.Extensions.AI.Templates/templates/AIChatWeb-CSharp/AIChatWeb-CSharp.Web/Program.cs | Remove GitHub Models path; add mock-provider wiring; keep OpenAI/AOAI/Ollama options. |
| src/ProjectTemplates/Microsoft.Extensions.AI.Templates/templates/AIChatWeb-CSharp/AIChatWeb-CSharp.Web/Program.Aspire.cs | Add mock-provider wiring for Aspire scenario. |
| src/ProjectTemplates/Microsoft.Extensions.AI.Templates/templates/AIChatWeb-CSharp/AIChatWeb-CSharp.Web/Components/Pages/Chat/ChatMessageItem.razor | Make citation parsing more robust by using XML parsing + broader regex match. |
| src/ProjectTemplates/Microsoft.Extensions.AI.Templates/templates/AIChatWeb-CSharp/AIChatWeb-CSharp.Web/Components/Pages/Chat/Chat.razor | Make tool result output XML-safe and remove survey prompt usage. |
| src/ProjectTemplates/Microsoft.Extensions.AI.Templates/templates/AIChatWeb-CSharp/AIChatWeb-CSharp.Web/Components/Layout/SurveyPrompt.razor.css | Remove survey prompt styling from template output. |
| src/ProjectTemplates/Microsoft.Extensions.AI.Templates/templates/AIChatWeb-CSharp/AIChatWeb-CSharp.Web/Components/Layout/SurveyPrompt.razor | Remove survey prompt component from template output. |
| src/ProjectTemplates/Microsoft.Extensions.AI.Templates/templates/AIChatWeb-CSharp/AIChatWeb-CSharp.Web/AIChatWeb-CSharp.Web.csproj-in | Add conditional dependency on Microsoft.Extensions.AI.Testing for mock provider. |
| src/ProjectTemplates/Microsoft.Extensions.AI.Templates/templates/AIChatWeb-CSharp/AIChatWeb-CSharp.AppHost/AppHost.cs | Remove GitHub Models wiring and add mock-aware conditional branches. |
| src/ProjectTemplates/Microsoft.Extensions.AI.Templates/templates/AIChatWeb-CSharp/.template.config/template.json | Make provider required; add mock choice; add mock-only file inclusion/exclusion rules. |
| src/ProjectTemplates/Microsoft.Extensions.AI.Templates/Microsoft.Extensions.AI.Templates.csproj | Add project reference to Microsoft.Extensions.AI.Testing for packaging templates. |
| src/ProjectTemplates/Microsoft.Agents.AI.ProjectTemplates/templates/AIAgentWebApi-CSharp/Program.cs | Remove GitHub Models wiring; add mock-provider wiring; add DevUI registration. |
| src/ProjectTemplates/Microsoft.Agents.AI.ProjectTemplates/templates/AIAgentWebApi-CSharp/AIAgentWebApi-CSharp.csproj-in | Add conditional dependency on Microsoft.Extensions.AI.Testing for mock provider. |
| src/ProjectTemplates/Microsoft.Agents.AI.ProjectTemplates/templates/AIAgentWebApi-CSharp/.template.config/template.json | Make provider required; add mock choice; add mock-only file inclusion/exclusion rules. |
| src/ProjectTemplates/Microsoft.Agents.AI.ProjectTemplates/Microsoft.Agents.AI.ProjectTemplates.csproj | Add project reference to Microsoft.Extensions.AI.Testing for packaging templates. |
| src/Libraries/Microsoft.Extensions.AI.Testing/Microsoft.Extensions.AI.Testing.csproj | Add new preview library project for deterministic AI testing mocks. |
| .github/agents/release-manager/write-release-notes/references/package-areas.md | Add Microsoft.Extensions.AI.Testing to package-area mapping. |
| .github/agents/release-manager/write-release-notes/references/collect-prs.md | Add servicing backport scope/attribution override guidance. |
| .github/agents/release-manager/write-release-notes/README.md | Add servicing backport release-notes guidance and tag preflight instructions. |
| .github/agents/release-manager/validate-release/references/stage-7-support-page.md | Update Stage 7 guidance to branch by monthly/servicing + new-package presence. |
| .github/agents/release-manager/validate-release/README.md | Align validate-release playbook with conditional Stage 7 handling. |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: dc263661-1b95-4f80-a7f4-3a4d3cd275ff
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: dc263661-1b95-4f80-a7f4-3a4d3cd275ff
…rary Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: dc263661-1b95-4f80-a7f4-3a4d3cd275ff
be9defc to
66a64dc
Compare
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: dc263661-1b95-4f80-a7f4-3a4d3cd275ff
Introduces the 'mock' provider option (--provider mock) to the aichatweb project template. The mock provider uses Microsoft.Extensions.AI.Testing's MockChatClient seeded with ~20 canned responses about the TrailMaster X4 GPS Watch, allowing the template to be used without any external AI provider credentials while still demonstrating the full chat experience with citations, suggestions, and vector-based semantic search. Also updates the Aspire test scenario to use --provider=mock instead of --provider=openai, and adds a test for the plain --provider=openai scenario. Updates the TemplateSnapshotTestBase to resolve template package paths using file timestamps rather than wildcard paths, improving cross-platform reliability. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: dc263661-1b95-4f80-a7f4-3a4d3cd275ff
Introduces the 'mock' provider option (--provider mock) to the aiagent-webapi project template. The mock provider uses Microsoft.Extensions.AI.Testing's MockChatClient seeded with 3 canned responses for the story writer/editor workflow (generate a story, edit for tone, format as HTML), allowing the template to be explored without any external AI provider credentials while still demonstrating the multi-agent pipeline with writer and editor stages. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: dc263661-1b95-4f80-a7f4-3a4d3cd275ff
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: dc263661-1b95-4f80-a7f4-3a4d3cd275ff
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: dc263661-1b95-4f80-a7f4-3a4d3cd275ff
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: dc263661-1b95-4f80-a7f4-3a4d3cd275ff
66a64dc to
f454867
Compare
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: dc263661-1b95-4f80-a7f4-3a4d3cd275ff
|
If MEAI.Testing is only for MEAI.Templates, it may be better to avoid adding a new package and instead emit all scaffolding in each template output. |
|
I considered that too, @jozkee. But I liked the idea of having mock providers we can use ourselves within our own tests, and also for customers to use in theirs. We also have documentation showing how to build Console-app chat clients and other walkthroughs that don't rely on the templates, where the mocks are valuable as well. The package also lets customers add the provider in after project creation. I hadn't previously realized we already have the precedent of the testing packages out of this repo, so I was comfortable following that direction. |
luisquintanilla
left a comment
There was a problem hiding this comment.
LGTM. Thanks @jeffhandley ![]()
Summary
Microsoft.Extensions.AI.Testingpackage with experimentalMockChatClientandMockEmbeddingGenerator<TInput>APIs.KeyValuePairresponse seeding toMockChatClientfor JSON-friendly plain-text fixtures with customizable request matching.mockproviders to theaichatwebandaiagent-webapitemplates, including citations, retrieval/tool flows, and the writing workflow.Validation
Microsoft.Extensions.AI.Testing.Testsacross supported target frameworks.aichatwebandaiagent-webapitemplate execution configurations.aichatweb project demo
aiagent-webapi project demo