{"id":13192,"date":"2025-08-29T07:16:27","date_gmt":"2025-08-28T22:16:27","guid":{"rendered":"https:\/\/aireviewirush.com\/?p=13192"},"modified":"2025-08-29T07:16:27","modified_gmt":"2025-08-28T22:16:27","slug":"agent-manufacturing-unit-prime-5-agent-observability-greatest-practices-for-dependable-ai","status":"publish","type":"post","link":"https:\/\/aireviewirush.com\/?p=13192","title":{"rendered":"Agent Manufacturing unit: Prime 5 agent observability greatest practices for dependable AI"},"content":{"rendered":"<p> <br \/>\n<\/p>\n<div>\n<p>\n\t\t\tGuaranteeing the reliability, security, and efficiency of AI brokers is vital. That\u2019s the place agent observability is available in.\t\t<\/p>\n<p class=\"wp-block-paragraph\"><em>This weblog publish is the third out of a six-part weblog sequence known as\u00a0<\/em><a href=\"https:\/\/azure.microsoft.com\/en-us\/blog\/tag\/agent-factory\/\" target=\"_blank\" rel=\"noreferrer noopener\"><em>Agent Manufacturing unit<\/em><\/a><em>\u00a0which is able to share greatest practices, design patterns, and instruments to assist information you thru adopting and constructing agentic AI.<\/em><\/p>\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_53 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title \" >Table of Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\" role=\"button\"><label for=\"item-6a2a797e50574\" ><span class=\"\"><span style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input aria-label=\"Toggle\" aria-label=\"item-6a2a797e50574\"  type=\"checkbox\" id=\"item-6a2a797e50574\"><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/aireviewirush.com\/?p=13192\/#Seeing_is_understanding%E2%80%94the_facility_of_agent_observability\" title=\"Seeing is understanding\u2014the facility of agent observability\">Seeing is understanding\u2014the facility of agent observability<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/aireviewirush.com\/?p=13192\/#What%E2%80%99s_agent_observability\" title=\"What&#8217;s agent observability?\">What&#8217;s agent observability?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/aireviewirush.com\/?p=13192\/#Conventional_observability_vs_agent_observability\" title=\"Conventional observability vs agent observability\">Conventional observability vs agent observability<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/aireviewirush.com\/?p=13192\/#Azure_AI_Foundry_Observability_offers_end-to-end_agent_observability\" title=\"Azure AI Foundry Observability offers end-to-end agent observability\">Azure AI Foundry Observability offers end-to-end agent observability<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/aireviewirush.com\/?p=13192\/#5_greatest_practices_for_agent_observability\" title=\"5 greatest practices for agent observability\">5 greatest practices for agent observability<\/a><ul class='ez-toc-list-level-3'><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/aireviewirush.com\/?p=13192\/#1_Choose_the_suitable_mannequin_utilizing_benchmark_pushed_leaderboards\" title=\"1. Choose the suitable mannequin utilizing benchmark pushed leaderboards\">1. Choose the suitable mannequin utilizing benchmark pushed leaderboards<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/aireviewirush.com\/?p=13192\/#2_Consider_brokers_repeatedly_in_growth_and_manufacturing\" title=\"2. Consider brokers repeatedly in growth and manufacturing\">2. Consider brokers repeatedly in growth and manufacturing<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/aireviewirush.com\/?p=13192\/#3_Combine_evaluations_into_your_CICD_pipelines\" title=\"3. Combine evaluations into your CI\/CD pipelines\">3. Combine evaluations into your CI\/CD pipelines<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/aireviewirush.com\/?p=13192\/#4_Scan_for_vulnerabilities_with_AI_purple_teaming_earlier_than_manufacturing\" title=\"4. Scan for vulnerabilities with AI purple teaming earlier than manufacturing\">4. Scan for vulnerabilities with AI purple teaming earlier than manufacturing<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-10\" href=\"https:\/\/aireviewirush.com\/?p=13192\/#5_Monitor_brokers_in_manufacturing_with_tracing_evaluations_and_alerts\" title=\"5. Monitor brokers in manufacturing with tracing, evaluations, and alerts\">5. Monitor brokers in manufacturing with tracing, evaluations, and alerts<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-11\" href=\"https:\/\/aireviewirush.com\/?p=13192\/#Get_began_with_Azure_AI_Foundry_for_end-to-end_agent_observability\" title=\"Get began with Azure AI Foundry for end-to-end agent observability\">Get began with Azure AI Foundry for end-to-end agent observability<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-12\" href=\"https:\/\/aireviewirush.com\/?p=13192\/#What%E2%80%99s_subsequent\" title=\"What\u2019s subsequent\">What\u2019s subsequent<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-13\" href=\"https:\/\/aireviewirush.com\/?p=13192\/#Azure_AI_Foundry\" title=\"Azure AI Foundry\">Azure AI Foundry<\/a><\/li><\/ul><\/nav><\/div>\n<h2 class=\"wp-block-heading\" id=\"seeing-is-knowing-the-power-of-agent-observability\"><span class=\"ez-toc-section\" id=\"Seeing_is_understanding%E2%80%94the_facility_of_agent_observability\"><\/span>Seeing is understanding\u2014the facility of agent observability<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p class=\"wp-block-paragraph\">As agentic AI turns into extra central to enterprise workflows, guaranteeing reliability, security, and efficiency is vital. That\u2019s the place agent observability is available in. Agent observability empowers groups to:<\/p>\n<ul class=\"wp-block-list\">\n<li class=\"wp-block-list-item\">Detect and resolve points early in growth.<\/li>\n<li class=\"wp-block-list-item\">Confirm that brokers uphold requirements of high quality, security, and compliance.<\/li>\n<li class=\"wp-block-list-item\">Optimize efficiency and consumer expertise in manufacturing.<\/li>\n<li class=\"wp-block-list-item\">Keep belief and accountability in AI methods.<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">With the rise of complicated, multi-agent and multi-modal methods, observability is crucial for delivering AI that&#8217;s not solely efficient, but additionally clear, secure, and aligned with organizational values. Observability empowers groups to construct with confidence and scale responsibly by offering visibility into how brokers behave, make selections, and reply to real-world situations throughout their lifecycle.<\/p>\n<h2 class=\"wp-block-heading\" id=\"what-is-agent-observability\"><span class=\"ez-toc-section\" id=\"What%E2%80%99s_agent_observability\"><\/span>What&#8217;s agent observability?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p class=\"wp-block-paragraph\">Agent observability is the follow of reaching deep, actionable visibility into the interior workings, selections, and outcomes of AI brokers all through their lifecycle\u2014from growth and testing to deployment and ongoing operation. Key points of agent observability embody:<\/p>\n<ul class=\"wp-block-list\">\n<li class=\"wp-block-list-item\"><strong>Steady monitoring:<\/strong>\u00a0Monitoring agent actions, selections, and interactions in actual time to floor anomalies, surprising behaviors, or efficiency drift.<\/li>\n<li class=\"wp-block-list-item\"><strong>Tracing:<\/strong>\u00a0Capturing detailed execution flows, together with how brokers motive by way of duties, choose instruments, and collaborate with different brokers or providers. This helps reply not simply \u201cwhat occurred,\u201d however \u201cwhy and the way did it occur?\u201d<\/li>\n<li class=\"wp-block-list-item\"><strong>Logging:<\/strong> Data agent selections, device calls, and inner state modifications to help debugging and habits evaluation in agentic AI workflows.<\/li>\n<li class=\"wp-block-list-item\"><strong>Analysis:<\/strong>\u00a0Systematically assessing agent outputs for high quality, security, compliance, and alignment with consumer intent\u2014utilizing each automated and human-in-the-loop strategies.<\/li>\n<li class=\"wp-block-list-item\"><strong>Governance:<\/strong>\u00a0Imposing insurance policies and requirements to make sure brokers function ethically, safely, and in accordance with organizational and regulatory necessities.<\/li>\n<\/ul>\n<h2 class=\"wp-block-heading\" id=\"traditional-observability-vs-agent-observability\"><span class=\"ez-toc-section\" id=\"Conventional_observability_vs_agent_observability\"><\/span>Conventional observability vs agent observability<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p class=\"wp-block-paragraph\">Conventional observability depends on three foundational pillars:\u00a0metrics,\u00a0logs, and\u00a0traces. These present visibility into system efficiency, assist diagnose failures, and help root-cause evaluation. They&#8217;re well-suited for standard software program methods the place the main target is on infrastructure well being, latency, and throughput.<\/p>\n<p class=\"wp-block-paragraph\">Nonetheless, AI brokers are non-deterministic and introduce new dimensions\u2014autonomy, reasoning, and dynamic resolution making\u2014that require a extra superior observability framework.\u00a0Agent observability\u00a0builds on conventional strategies and provides two vital parts:\u00a0evaluations\u00a0and\u00a0governance. Evaluations assist groups assess how nicely brokers resolve consumer intent, adhere to duties, and use instruments successfully. Agent governance can guarantee brokers function safely, ethically, and in compliance with organizational requirements.<\/p>\n<p class=\"wp-block-paragraph\">This expanded strategy permits deeper visibility into agent habits\u2014not simply what brokers do, however why and the way they do it. It helps steady monitoring throughout the agent lifecycle, from growth to manufacturing, and is crucial for constructing reliable, high-performing AI methods at scale.<\/p>\n<h2 class=\"wp-block-heading\" id=\"azure-ai-foundry-observability-provides-end-to-end-agent-observability\"><span class=\"ez-toc-section\" id=\"Azure_AI_Foundry_Observability_offers_end-to-end_agent_observability\"><\/span>Azure AI Foundry Observability offers end-to-end agent observability<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/ai-foundry\/concepts\/observability\" target=\"_blank\" rel=\"noreferrer noopener\">Azure AI Foundry Observability<\/a> is a unified answer for\u00a0evaluating, monitoring, tracing, and governing\u00a0the standard, efficiency, and security of your AI methods finish to finish in <a href=\"https:\/\/azure.microsoft.com\/en-us\/products\/ai-foundry\" target=\"_blank\" rel=\"noopener\">Azure AI Foundry<\/a>\u2014all constructed into your AI growth\u00a0loop. From mannequin choice to real-time debugging, Foundry Observability capabilities empower groups to ship production-grade AI with confidence and velocity. It\u2019s observability, reimagined for the enterprise AI period.<\/p>\n<p class=\"wp-block-paragraph\">With built-in capabilities just like the Brokers Playground evaluations, Azure AI Pink Teaming Agent, and Azure Monitor integration, Foundry Observability brings analysis and security into each step of the agent lifecycle. Groups can hint every agent circulate with full execution context, simulate adversarial situations, and monitor reside site visitors with customizable dashboards. Seamless CI\/CD integration permits steady analysis on each commit and governance help with Microsoft Purview, Credo AI, and Saidot integration helps allow alignment with regulatory frameworks just like the EU AI Act\u2014making it simpler to construct accountable, production-grade AI at scale.<\/p>\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" alt=\"Azure AI Foundry Observability banner showing tabs for Leaderboards, Traces, Logs, Evaluations, Metrics, and Governance, with a lifecycle arrow indicating coverage across the agent and AI development lifecycle. \" class=\"wp-image-45844 webp-format\" srcset=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/Foundry-Observability-1024x485.webp 1024w, https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/Foundry-Observability-300x142.webp 300w, https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/Foundry-Observability-768x364.webp 768w, https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/Foundry-Observability-1536x728.webp 1536w, https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/Foundry-Observability-2048x971.webp 2048w\" src=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/Foundry-Observability-1024x485.webp\"\/><\/figure>\n<h2 class=\"wp-block-heading\" id=\"five-best-practices-for-agent-observability\"><span class=\"ez-toc-section\" id=\"5_greatest_practices_for_agent_observability\"><\/span>5 greatest practices for agent observability<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<h3 class=\"wp-block-heading\" id=\"1-pick-the-right-model-using-benchmark-driven-leaderboards\"><span class=\"ez-toc-section\" id=\"1_Choose_the_suitable_mannequin_utilizing_benchmark_pushed_leaderboards\"><\/span>1. Choose the suitable mannequin utilizing benchmark pushed leaderboards<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p class=\"wp-block-paragraph\">Each agent wants a mannequin and selecting the best mannequin is foundational for agent success. Whereas planning your AI agent, you want to resolve which mannequin could be the most effective to your use case when it comes to security, high quality, and value.<\/p>\n<p class=\"wp-block-paragraph\">You may choose the most effective mannequin by both evaluating the mannequin by yourself knowledge or use Azure AI Foundry\u2019s\u00a0<a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/ai-foundry\/how-to\/benchmark-model-in-catalog\" target=\"_blank\" rel=\"noreferrer noopener\">mannequin leaderboards<\/a>\u00a0to check basis fashions out-of-the-box by high quality, value, and efficiency\u2014backed by trade benchmarks. With Foundry mannequin leaderboards, yow will discover mannequin leaders in numerous choice standards and situations, visualize trade-offs among the many standards (e.g., high quality vs value or security), and dive into detailed metrics to make assured, data-driven selections.<\/p>\n<figure class=\"wp-block-image aligncenter size-large\"><img decoding=\"async\" alt=\"Screenshot of Azure AI Foundry model leaderboard dashboard, displaying comparative bar charts for model quality, safety, cost, and throughput, and detailed evaluation metrics for different AI models.\" class=\"wp-image-45845 webp-format\" srcset=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/model-leaderboards-1024x442.webp 1024w, https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/model-leaderboards-300x130.webp 300w, https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/model-leaderboards-768x332.webp 768w, https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/model-leaderboards-1536x664.webp 1536w, https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/model-leaderboards-2048x885.webp 2048w\" src=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/model-leaderboards-1024x442.webp\"\/><\/figure>\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"has-large-font-size wp-block-paragraph\"><em>Azure AI Foundry\u2019s mannequin leaderboards gave us the boldness to scale consumer options from experimentation to deployment. Evaluating fashions facet by facet helped prospects choose the most effective match\u2014balancing efficiency, security, and value with confidence.<\/em><\/p>\n<p><cite>\u2014Mark Luquire, EY World Microsoft Alliance Co-Innovation Chief, Managing Director, Ernst &amp; Younger, LLP*<\/cite><\/p><\/blockquote>\n<h3 class=\"wp-block-heading\" id=\"2-evaluate-agents-continuously-in-development-and-production\"><span class=\"ez-toc-section\" id=\"2_Consider_brokers_repeatedly_in_growth_and_manufacturing\"><\/span>2. Consider brokers repeatedly in growth and manufacturing<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p class=\"wp-block-paragraph\">Brokers are highly effective productiveness assistants. They&#8217;ll plan, make selections, and execute actions. Brokers sometimes first\u00a0<a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/ai-foundry\/concepts\/evaluation-evaluators\/agent-evaluators#intent-resolution\" target=\"_blank\" rel=\"noreferrer noopener\">motive by way of consumer intents in conversations<\/a>,\u00a0<a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/ai-foundry\/concepts\/evaluation-evaluators\/agent-evaluators#tool-call-accuracy\" target=\"_blank\" rel=\"noreferrer noopener\">choose the proper instruments<\/a>\u00a0to name and fulfill the consumer requests, and\u00a0<a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/ai-foundry\/concepts\/evaluation-evaluators\/agent-evaluators#task-adherence\" target=\"_blank\" rel=\"noreferrer noopener\">full numerous duties<\/a>\u00a0in line with their directions.\u00a0Earlier than deploying brokers, it\u2019s vital to guage their habits and efficiency.<\/p>\n<figure class=\"wp-block-image aligncenter size-large\"><img decoding=\"async\" alt=\"Alt-text: Diagram illustrating agent evaluation steps: intent resolution, tool calling, and response assembly, with example user query and evaluation criteria for each step. \" class=\"wp-image-45846 webp-format\" srcset=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/evaluation-agents-1024x565.webp 1024w, https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/evaluation-agents-300x165.webp 300w, https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/evaluation-agents-768x424.webp 768w, https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/evaluation-agents-1536x847.webp 1536w, https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/evaluation-agents.webp 1742w\" src=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/evaluation-agents-1024x565.webp\"\/><\/figure>\n<p class=\"wp-block-paragraph\">Azure AI Foundry makes agent analysis simpler with a number of agent evaluators supported out-of-the-box, together with Intent Decision (how precisely the agent identifies and addresses consumer intentions), Job Adherence (how nicely the agent follows by way of on recognized duties), Device Name Accuracy (how successfully the agent selects and makes use of instruments), and Response Completeness (whether or not the agent\u2019s response contains all obligatory info). Past agent evaluators, Azure AI Foundry additionally offers a complete suite of evaluators for broader assessments of AI high quality, threat, and security. These embody high quality dimensions equivalent to\u00a0<a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/ai-foundry\/concepts\/evaluation-evaluators\/rag-evaluators#relevance\" target=\"_blank\" rel=\"noreferrer noopener\">relevance<\/a>,\u00a0<a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/ai-foundry\/concepts\/evaluation-evaluators\/general-purpose-evaluators#coherence\" target=\"_blank\" rel=\"noreferrer noopener\">coherence<\/a>, and\u00a0<a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/ai-foundry\/concepts\/evaluation-evaluators\/general-purpose-evaluators#fluency\" target=\"_blank\" rel=\"noreferrer noopener\">fluency<\/a>,\u00a0together with complete\u00a0<a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/ai-foundry\/concepts\/evaluation-evaluators\/risk-safety-evaluators\" target=\"_blank\" rel=\"noreferrer noopener\">threat and\u00a0security checks<\/a>\u00a0that assess for code vulnerabilities, violence,\u00a0self-harm,\u00a0sexual\u00a0content material,\u00a0hate, unfairness,\u00a0oblique\u00a0assaults, and\u00a0the usage of\u00a0protected\u00a0supplies. The Azure AI Foundry Brokers Playground brings these analysis and tracing instruments collectively in a single place, letting you take a look at, debug, and enhance agentic AI effectively.<\/p>\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"has-large-font-size wp-block-paragraph\"><em>The sturdy analysis instruments in Azure AI Foundry assist our builders repeatedly assess the efficiency and accuracy of our AI fashions, together with assembly requirements for coherence, fluency, and groundedness.<\/em><\/p>\n<p><cite>\u2014<a href=\"https:\/\/www.microsoft.com\/en\/customers\/story\/24300-hughes-azure-ai-foundry?msockid=3e18774cbd40649e0ed362b3bc0c65eb\" target=\"_blank\" rel=\"noreferrer noopener\">Amarender Singh, Director, AI, Hughes Community Programs<\/a><\/cite><\/p><\/blockquote>\n<h3 class=\"wp-block-heading\" id=\"3-integrate-evaluations-into-your-ci-cd-pipelines\"><span class=\"ez-toc-section\" id=\"3_Combine_evaluations_into_your_CICD_pipelines\"><\/span>3. Combine evaluations into your CI\/CD pipelines<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p class=\"wp-block-paragraph\">Automated evaluations must be a part of your CI\/CD pipeline so each code change is examined for high quality and security earlier than launch. This strategy helps groups catch regressions early and might help guarantee brokers stay dependable as they evolve.<\/p>\n<p class=\"wp-block-paragraph\">Azure AI Foundry integrates along with your CI\/CD workflows utilizing <a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/ai-foundry\/how-to\/evaluation-github-action?tabs=foundry-project\" target=\"_blank\" rel=\"noreferrer noopener\">GitHub Actions<\/a> and <a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/ai-foundry\/how-to\/evaluation-azure-devops?tabs=foundry-project\" target=\"_blank\" rel=\"noreferrer noopener\">Azure DevOps extensions<\/a>, enabling you to auto-evaluate brokers on each commit, evaluate variations utilizing built-in high quality, efficiency, and security metrics, and leverage confidence intervals and significance checks to help selections\u2014serving to to make sure that every iteration of your agent is manufacturing prepared.<\/p>\n<figure class=\"wp-block-image aligncenter size-large\"><img decoding=\"async\" alt=\"Screenshot of Azure AI Evaluation dashboard comparing operational and AI quality metrics across different agent variants, including intent resolution, task adherence, and risk\/safety scores. \" class=\"wp-image-45847 webp-format\" srcset=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/github-action-multi-agent-result-733x1024.webp 733w, https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/github-action-multi-agent-result-215x300.webp 215w, https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/github-action-multi-agent-result-768x1073.webp 768w, https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/github-action-multi-agent-result-1099x1536.webp 1099w, https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/github-action-multi-agent-result.webp 1254w\" src=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/github-action-multi-agent-result-733x1024.webp\"\/><\/figure>\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"has-large-font-size wp-block-paragraph\"><em>We\u2019ve built-in Azure AI Foundry evaluations immediately into our GitHub Actions workflow, so each code change to our AI brokers is routinely examined earlier than deployment. This setup helps us rapidly catch regressions and keep prime quality as we iterate on our fashions and options.<\/em><\/p>\n<p><cite>\u2014Justin Layne Hofer, Senior Software program Engineer, Veeam<\/cite><\/p><\/blockquote>\n<h3 class=\"wp-block-heading\" id=\"4-scan-for-vulnerabilities-with-ai-red-teaming-before-production\"><span class=\"ez-toc-section\" id=\"4_Scan_for_vulnerabilities_with_AI_purple_teaming_earlier_than_manufacturing\"><\/span>4. Scan for vulnerabilities with AI purple teaming earlier than manufacturing<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p class=\"wp-block-paragraph\">Safety and security are non-negotiable. Earlier than deployment, proactively take a look at brokers for safety and security dangers by simulating adversarial assaults. Pink teaming helps uncover vulnerabilities that might be exploited in real-world situations, strengthening agent robustness.<\/p>\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/ai-foundry\/concepts\/ai-red-teaming-agent\" target=\"_blank\" rel=\"noreferrer noopener\">Azure AI Foundry\u2019s AI Pink Teaming Agent<\/a> automates adversarial testing, measuring threat and producing readiness experiences. It permits groups to simulate assaults and validate each particular person agent responses and sophisticated workflows for manufacturing readiness.<\/p>\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" alt=\"Metric dashboard showing attack risk categories and percentages for successful attacks, hate and unfairness, self-harm, sexual, and violence, used for AI red teaming evaluation. \" class=\"wp-image-45848 webp-format\" srcset=\"\" src=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/metric-dashboard-red-team-1-1024x187-1.webp\"\/><\/figure>\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" alt=\"Detailed metrics result table listing attack success, risk category, attack technique, complexity, and human feedback for various adversarial test cases in AI red teaming. \" class=\"wp-image-45849 webp-format\" srcset=\"\" src=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/detailed-metrics-results-1-1024x526-1.webp\"\/><\/figure>\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"has-large-font-size wp-block-paragraph\"><em>Accenture is already testing the\u00a0<a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/ai-foundry\/concepts\/ai-red-teaming-agent\" target=\"_blank\" rel=\"noreferrer noopener\">Microsoft AI Pink Teaming Agent<\/a>,\u00a0which simulates adversarial prompts and detects mannequin and software threat posture proactively. This device will assist validate not solely particular person agent responses, but additionally full multi-agent workflows through which cascading logic would possibly produce unintended habits from a single adversarial consumer. Pink teaming lets us simulate worst-case situations earlier than they ever hit manufacturing. That modifications the sport.<\/em><\/p>\n<p><cite>\u2014<a href=\"https:\/\/www.microsoft.com\/en\/customers\/story\/23953-accenture-azure-ai-foundry\" target=\"_blank\" rel=\"noreferrer noopener\">Nayanjyoti Paul, Affiliate Director and Chief Azure Architect for Gen AI, Accenture<\/a><\/cite><\/p><\/blockquote>\n<h3 class=\"wp-block-heading\" id=\"5-monitor-agents-in-production-with-tracing-evaluations-and-alerts\"><span class=\"ez-toc-section\" id=\"5_Monitor_brokers_in_manufacturing_with_tracing_evaluations_and_alerts\"><\/span>5. Monitor brokers in manufacturing with tracing, evaluations, and alerts<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p class=\"wp-block-paragraph\">Steady monitoring after deployment is crucial to catch points, efficiency drift, or regressions in actual time. Utilizing evaluations, tracing, and alerts helps keep agent reliability and compliance all through its lifecycle.<\/p>\n<p class=\"wp-block-paragraph\">Azure AI Foundry observability permits steady agentic AI monitoring by way of a unified dashboard powered by Azure Monitor Utility Insights and Azure Workbooks. This dashboard offers real-time visibility into efficiency, high quality, security, and useful resource utilization, permitting you to run steady evaluations on reside site visitors, set alerts to detect drift or regressions, and hint each analysis consequence for full-stack observability. With seamless navigation to Azure Monitor, you&#8217;ll be able to customise dashboards, arrange superior diagnostics, and reply swiftly to incidents\u2014serving to to make sure you keep forward of points with precision and velocity.<\/p>\n<figure class=\"wp-block-image aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"2537\" height=\"1731\" src=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/Tracing-Gif.gif\" alt=\"Screenshot of Azure AI Foundry tracing dashboard, showing a list of agent evaluation results with input, output, evaluation metrics, and timestamps for monitoring and debugging AI agent performance. \" class=\"wp-image-45850\"\/><\/figure>\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"has-large-font-size wp-block-paragraph\"><em>Safety is paramount for our massive enterprise prospects, and our collaboration with Microsoft allays any issues. With Azure AI Foundry, we have now the specified observability and management over our infrastructure and might ship a extremely safe surroundings to our prospects.<\/em><\/p>\n<p><cite>\u2014<a href=\"https:\/\/www.microsoft.com\/en\/customers\/story\/23681-spotfire-azure-ai-foundry\" target=\"_blank\" rel=\"noreferrer noopener\">Ahmad Fattahi, Sr. Director, Knowledge Science, Spotfire<\/a><\/cite><\/p><\/blockquote>\n<h2 class=\"wp-block-heading\" id=\"get-started-with-azure-ai-foundry-for-end-to-end-agent-observability\"><span class=\"ez-toc-section\" id=\"Get_began_with_Azure_AI_Foundry_for_end-to-end_agent_observability\"><\/span>Get began with Azure AI Foundry for end-to-end agent observability<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p class=\"wp-block-paragraph\">To summarize, conventional observability contains metrics, logs, and traces. Agent Observability wants metrics, traces, logs, evaluations, and governance for full visibility. Azure AI Foundry Observability is a unified answer for\u00a0agent governance, analysis, tracing, and monitoring\u2014all constructed into your AI growth\u00a0lifecycle. With instruments just like the Brokers Playground, clean CI\/CD, and governance integrations, Azure AI Foundry Observability empowers groups to make sure their AI brokers are dependable, secure, and manufacturing prepared. Be taught extra about <a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/ai-foundry\/concepts\/observability\" target=\"_blank\" rel=\"noreferrer noopener\">Azure AI Foundry Observability<\/a> and get full visibility into your brokers at the moment!<\/p>\n<h2 class=\"wp-block-heading\" id=\"what-s-next\"><span class=\"ez-toc-section\" id=\"What%E2%80%99s_subsequent\"><\/span>What\u2019s subsequent<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p class=\"wp-block-paragraph\">Partially 4 of the\u00a0<a href=\"https:\/\/azure.microsoft.com\/en-us\/blog\/tag\/agent-factory\/\" target=\"_blank\" rel=\"noreferrer noopener\"><em>Agent Manufacturing unit<\/em>\u00a0sequence<\/a>, we\u2019ll give attention to how one can go from prototype to manufacturing sooner with developer instruments and speedy agent growth.<\/p>\n<p class=\"wp-block-paragraph\">Did you miss these posts within the sequence?<\/p>\n<aside class=\"cta-block cta-block--align-left cta-block--has-image wp-block-msx-cta\" data-bi-an=\"CTA Block\">\n<div class=\"cta-block__content\">\n<div class=\"cta-block__image-container\">\n\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"768\" src=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/Azure-DevTools-Light-2-1024x768.jpg\" class=\"cta-block__image\" alt=\"A close up of a group of plastic objects\" srcset=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/Azure-DevTools-Light-2-1024x768.jpg 1024w, https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/Azure-DevTools-Light-2-300x225.jpg 300w, https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/Azure-DevTools-Light-2-768x576.jpg 768w, https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/Azure-DevTools-Light-2-1536x1152.jpg 1536w, https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2025\/08\/Azure-DevTools-Light-2-2048x1536.jpg 2048w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\"\/>\t\t\t<\/div>\n<div class=\"cta-block__body\">\n<h2 class=\"cta-block__headline\"><span class=\"ez-toc-section\" id=\"Azure_AI_Foundry\"><\/span>Azure AI Foundry<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p class=\"cta-block__text\">Construct adaptable AI brokers that automate duties and improve consumer experiences.<\/p>\n<\/p><\/div>\n<\/p><\/div>\n<\/aside>\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n<p class=\"wp-block-paragraph\"><em>*The views mirrored on this publication are the views of the speaker and don&#8217;t essentially mirror the views of the worldwide EY group or its member companies.<\/em><\/p>\n<\/p><\/div>\n<p><script>\n\t\tfunction facebookTracking() {\n\t\t\t!function(f,b,e,v,n,t,s){if(f.fbq)return;n=f.fbq=function(){n.callMethod?\n\t\t\t\tn.callMethod.apply(n,arguments):n.queue.push(arguments)};if(!f._fbq)f._fbq=n;\n\t\t\t\tn.push=n;n.loaded=!0;n.version='2.0';n.queue=[];t=b.createElement(e);t.async=!0;\n\t\t\t\tt.src=v;t.type=\"ms-delay-type\";t.setAttribute('data-ms-type','text\/javascript');\n\t\t\t\ts=b.getElementsByTagName(e)[0];s.parentNode.insertBefore(t,s)}(window,\n\t\t\t\tdocument,'script','https:\/\/connect.facebook.net\/en_US\/fbevents.js');\n\t\t\tfbq('init', '1770559986549030');\n\t\t\t\t\t\tfbq('track', 'PageView');\n\t\t\t\t\t}\n\t<\/script><br \/>\n<br \/><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Guaranteeing the reliability, security, and efficiency of AI brokers is vital. That\u2019s the place agent observability is available in. This weblog publish is the third out of a six-part weblog sequence known as\u00a0Agent Manufacturing unit\u00a0which is able to share greatest practices, design patterns, and instruments to assist information you thru adopting and constructing agentic AI. [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":13194,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[],"class_list":["post-13192","post","type-post","status-publish","format-standard","has-post-thumbnail","category-cloud-computing"],"_links":{"self":[{"href":"https:\/\/aireviewirush.com\/index.php?rest_route=\/wp\/v2\/posts\/13192","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/aireviewirush.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/aireviewirush.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/aireviewirush.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/aireviewirush.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=13192"}],"version-history":[{"count":1,"href":"https:\/\/aireviewirush.com\/index.php?rest_route=\/wp\/v2\/posts\/13192\/revisions"}],"predecessor-version":[{"id":13193,"href":"https:\/\/aireviewirush.com\/index.php?rest_route=\/wp\/v2\/posts\/13192\/revisions\/13193"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/aireviewirush.com\/index.php?rest_route=\/wp\/v2\/media\/13194"}],"wp:attachment":[{"href":"https:\/\/aireviewirush.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=13192"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/aireviewirush.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=13192"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/aireviewirush.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=13192"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}