AB-100: Agentic AI Business Solutions Architect › Practice Assessment for Exam AB-100: Agentic AI Business Solutions Architect › Question #26598
Single Choice Moderate

Q

A company has a customer service agent built in Microsoft Foundry that calls an Azure OpenAI model and a knowledge API. Telemetry and online evaluation scores are sent to Azure Monitor Application Insights.

Users report intermittent off-topic responses, even though latency and token usage remain normal.

You confirm that the issue occurs only when the knowledge API is used.

You need to detect quality regressions in production within five minutes of occurrence and automatically notify the on-call team.

What should you configure?

ID: #26598 Practice Assessment for Exam AB-100: Agentic AI Business Solutions Architect 10 views
Question Info
#26598Q ID
ModerateDifficulty
Practice Assessment for Exam AB-100: Agentic AI Business Solutions ArchitectTopic
Start Quiz on This Topic
 Your Answer

Choose the Best Option

Click any option to instantly check if you're correct.

  • A an Azure Monitor workbook that visualizes evaluation scores and knowledge API usage over time ✔ ✖
  • B an Azure Monitor log alert that detects dependency failures on the knowledge API based on HTTP status codes ✔ ✖
  • C an Azure Monitor metric alert in Application Insights based on the average evaluation scores across all requests ✔ ✖
  • D an Azure Monitor log alert in Application Insights that detects low evaluation scores when the knowledge API is used and triggers an action group ✔ ✖
Correct Answer

Explanation

Objective:

3.1 Analyze, monitor, and tune AI-powered business solutions

What This Item Tests:

Interpret telemetry data for performance and model tuning

Additional Reading:

Explore how to monitor with Azure

Monitor and analyze AI Agent usage and performance

Building and operationalizing copilot applications

Rationale:

A log-based alert in Application Insights can correlate low evaluation scores specifically with calls to the knowledge API and triggers an action group within minutes, enabling rapid detection of targeted quality regressions. Metric alerts based on overall averages, dependency alerts based only on HTTP status codes, and workbooks for visualization do not provide precise, automated detection and notification in this scenario.

 Continue Practice
Previous Question Next Question
 Share

Share This Question

Challenge a friend or share with your study group.

 More from This Topic

Related MCQ Questions

A company has a multi-agent solution built in Microsoft Foundry. The solution grounds responses in internal HR documents and uses agent tools to call internal and external systems. Security testing shows that ... A company uses an agent built in Microsoft Foundry that retrieves internal content and invokes tools to perform actions. Security testing shows that prompt injection attacks can trigger unauthorized tool calls... A company has a Microsoft Copilot Studio agent grounded in a Microsoft SharePoint Online knowledge source. After adding new knowledge documents and updating the agent instructions, responses to the same user q... A company has two production agents: One is built in Microsoft Copilot Studio, and the other is built in Microsoft Foundry. Both agents send telemetry to the same Azure Monitor Application Insights resource. U... A company has a Microsoft Copilot Studio agent that performs browser-based automation against a legacy web app by using the Computer Use tool. When multiple users trigger the automation simultaneously, some ru... A company uses Microsoft Dynamics 365 Sales with a custom model-driven app that sellers use instead of the standard Sales Hub app. The sellers cannot open the Copilot immersive experience from the custom app n... A company uses Microsoft Dynamics 365 Supply Chain Management with customizations developed in Microsoft Visual Studio and stored in Azure Repos. Deployable packages are published to the Lifecycle Services (LCS... A company plans to roll out an IT help desk agent that will be available in Microsoft Teams and Microsoft 365 Copilot. The agent must meet the following requirements: Answer questions grounded in Microsoft... A company has multiple Microsoft Copilot Studio agents that create and update service desk tickets by calling an internal ITSM API. The same ticketing actions must be shared across all agents, and the API sche... A company has a Microsoft Copilot Studio agent that generates a daily incident summary from a user-provided transcript and incident data stored in a Microsoft Dataverse table. The response must always be forma...