Halilcan Soran

Tag: AI Agents

  • How MCP Actually Works Under the Hood When an Agent Calls a Tool

    How MCP Actually Works Under the Hood When an Agent Calls a Tool

    Diagram showing how MCP works under the hood between an AI agent and a tool server

    Most people treat MCP as the thing that lets their agent call tools, as if it were a fancier function-calling wrapper. That mental model gets you through demos and breaks the moment you try to run something real. If you want to understand how MCP works under the hood, you have to stop thinking about it as a plugin format and start thinking about it as a stateful client-server protocol built on JSON-RPC 2.0.

    Towards Data Science published a solid walkthrough of the protocol recently, MCP Explained: How Modern AI Agents Connect to the Real World, and it lines up with what I have been reading in the official Model Context Protocol spec. This post is my attempt to compress the mechanism into a mental model you can actually use when you design servers for Power Platform agents.

    The Surface Behaviour: What You See When an Agent Calls an MCP Tool

    From the outside, an MCP call looks trivial. You wire an MCP server into Copilot Studio or a Power Apps agent, the agent picks a tool, arguments get filled in, a result comes back, and the model uses it in the next turn. It feels indistinguishable from a REST connector.

    That is the illusion. Underneath, three things are happening that a REST connector never does: the client and server negotiated capabilities before any tool call happened, the server is exposing tools, resources, and prompts as separate primitives, and the connection is a live session, not a stateless request. The agent is not just calling an endpoint. It is holding a conversation with a server that has told it what it can do.

    Underneath: JSON-RPC, the Handshake, and Capability Negotiation

    MCP rides on JSON-RPC 2.0. Every message is a JSON object with a method, params, and either an id (for requests) or no id (for notifications). Nothing exotic. What matters is the sequence.

    When an MCP client connects to a server, the first thing it sends is an initialize request. This carries the client’s protocol version and its capabilities. The server responds with its own protocol version, its capabilities, and metadata like server name and version. The client then sends an initialized notification to confirm the handshake is done. Only after this does tool discovery happen, usually through a tools/list call.

    Capability negotiation is the piece people miss. The server advertises whether it supports tools, resources, prompts, sampling, logging, and so on. The client picks up only what it understands. This is why an MCP server built against a newer spec can still work with an older client. Backward compatibility is baked into the protocol.

    Then there are the three primitives:

    • Tools are model-invoked. The agent decides when to call them.
    • Resources are application-controlled. The host app decides what to expose.
    • Prompts are user-invoked. Templates the user can trigger.

    Most people conflate all three into tools. That is why their MCP servers feel bloated. If a chunk of data is really a reference document, it should be a resource, not a tool. Tools are for actions with side effects or computed answers.

    Edges and Limits: Transports, Session State, Schema Drift, and Auth

    MCP is transport-agnostic in principle but the two real options are stdio and HTTP with Server-Sent Events (or the newer streamable HTTP transport). Stdio is fine for local dev. For anything hosted, you are on HTTP, and that opens the usual questions about session identity, reconnection, and load balancing.

    Session state is the first sharp edge. An MCP session is stateful. If your server is behind a load balancer with no sticky sessions, you will see broken initialize sequences and mysterious method not found errors after a reconnect. This is not a bug. It is the protocol working as designed against infrastructure that assumed statelessness.

    Schema drift is the second. Tool schemas are exchanged at tools/list time. If your server changes a tool’s input schema mid-session without notifying the client via notifications/tools/list_changed, the agent will keep calling the old shape and fail silently or noisily depending on how strict your validation is. The new Dataverse MCP server tool shape is a good example of what a clean split between metadata inspection, querying, and search actually looks like in practice.

    Auth is the third. The core MCP spec deliberately left auth as a transport concern for a long time. The newer authorization spec pins it to OAuth 2.1. If you are wiring an MCP server into an enterprise agent, this is where you spend most of your time, not on the tool logic itself.

    What This Means When You Build an MCP Server for Power Platform Agents

    A few concrete implications for anyone pointing Copilot Studio or Power Apps agents at MCP.

    Design tool descriptions as retrieval hints, not documentation. I wrote about this in how Copilot Studio agent tool selection actually works. The orchestrator scores names, descriptions, and parameter names together. MCP does not change that. It just gives you a cleaner surface to expose them from.

    Split tools, resources, and prompts properly. If you dump everything as tools, you inflate the tool count, degrade selection quality past ten to fifteen entries, and force the model to reason about things that should have been passive context.

    Treat session lifetime as part of your design. Idle timeouts, reconnect logic, and schema change notifications are not edge cases. They are the load-bearing parts of a production MCP server. The same session and state management issues are a core reason AI agents fail in production when they worked fine in testing.

    MCP is not a plugin format. It is a protocol. Build servers like protocol servers and the sharp edges stop cutting. Skip that and you are building another silo with extra JSON on top. In my day-to-day, that shift in framing is what changes the design.

    Frequently Asked Questions

    How does MCP work under the hood when an agent calls a tool?

    MCP is a stateful client-server protocol built on JSON-RPC 2.0, not a simple function-calling wrapper. Before any tool call happens, the client and server go through a capability negotiation handshake, the server declares what it supports, and only then does tool discovery take place. The connection stays live as a session rather than being a stateless request like a REST call.

    What is the MCP handshake and why does it matter?

    When an MCP client connects to a server, it sends an initialize request containing its protocol version and capabilities, and the server responds with its own. The client then sends an initialized notification to confirm the exchange before any tools are listed or called. Skipping this step or misunderstanding it is a common reason real implementations break outside of demos.

    What is the difference between MCP and a standard REST connector?

    A REST connector is stateless, meaning each request stands alone with no shared context. MCP maintains a live session, negotiates capabilities upfront, and exposes tools, resources, and prompts as distinct primitives. This makes it better suited for agents that need to understand what a server can do before deciding how to use it.

    Why does MCP use JSON-RPC 2.0 instead of a simpler format?

    JSON-RPC 2.0 gives MCP a lightweight but structured message format that distinguishes requests from notifications and pairs responses to the correct request using an id field. This structure supports the sequenced, session-based communication that MCP requires, while remaining simple enough to implement across different clients and servers.

    This post was inspired by MCP Explained: How Modern AI Agents Connect to the Real World via Towards Data Science.

  • Anthropic Shipped Claude Opus 5 and the Long Horizon Agent Numbers Are What I Am Reading Twice

    Anthropic Shipped Claude Opus 5 and the Long Horizon Agent Numbers Are What I Am Reading Twice

    Claude Opus 5 release notes open on a laptop next to an agent workflow diagram

    Anthropic shipped Claude Opus 5 and the headline everyone is going to quote is the coding score. Fine. That is not what I am reading twice. The Claude Opus 5 release numbers I keep going back to are the long-horizon agentic task results. If the model can actually hold state and reason across a multi-step, tool-using workflow for meaningfully longer than Opus 4.8 could, that changes what I would trust an agent to do end to end.

    I have been running Opus 4.8 through Copilot Studio and a few MCP-connected setups for months. The tail failures I wrote about when 4.8 landed did not disappear. They just moved further out into the workflow. Opus 5 looks like it is aimed directly at that problem.

    What it actually does

    Opus 5 is the new frontier model in the Claude lineup. The release ships with three things I care about.

    First, a jump on long-horizon agentic benchmarks. Anthropic is publishing task-completion numbers on multi-turn, tool-using evals that run significantly longer than the ones they highlighted for 4.8. The distinction matters because average benchmark performance and the ability to keep a coherent plan across 30 or 40 tool calls are not the same skill.

    Second, better tool-use reliability. Structured output adherence, function argument shape, and schema compliance are all up. This is the boring plumbing that decides whether your agent works or spends half its runs retrying because it put a string where an integer belonged.

    Third, coding scores went up again. This is the number that will dominate every roundup post this week. It is real and it is useful, but for anyone building agents it is downstream of the first two.

    The pricing sits in the same neighborhood as Opus 4.8. Available on the direct Anthropic API, on Bedrock, and on Vertex.

    Why the long-horizon numbers matter more than the coding score

    Here is the thing I keep repeating to people at other organizations who ask about model swaps. A workflow that completes 70 percent of the time is a demo, not a workflow. The reason most internal agents get pulled after the pilot is that the human check-in halfway through is what actually makes them work, and once you add that check-in you have not saved anyone any time.

    Long-horizon task performance is the specific number that decides whether you can remove that check-in. If Opus 5 completes a 20-step research-and-write workflow at 85 percent instead of 60, that is not a marginal improvement. That is the difference between a pattern I would deploy and one I would kill. If you are thinking about how to measure whether that improvement actually shows up in practice, How to Measure Useful Work Per Dollar for Your Power Platform AI Agents is the framework I would reach for first.

    The coding score is downstream of this for a simple reason. Better tool-use reliability and better structured output are the same underlying capability that makes coding scores go up. A model that produces cleaner function calls also produces cleaner code. So if the coding number moved but the long-horizon number did not, I would be skeptical. Both moved. That is a good signal.

    What I am watching for over the next two weeks is whether the published numbers survive contact with real workflows. Anthropic ran a public rollback on Fable 5 earlier this year and the postmortem was clear about how launch evals can miss regressions that only show up in production. Opus 5 will get the same treatment from anyone paying attention.

    What I would do with it this week

    Three concrete things.

    One, pick the longest agent workflow I already have running on Opus 4.8 and rerun the last 50 production traces on Opus 5. Same prompts, same tools, same inputs. Diff the completion rates and the tool-call error rates. Not average latency. Completion rate on the tail. The failure modes worth hunting are exactly the ones I covered in Why Do AI Agents Fail in Production When They Worked Fine in Testing?

    Two, look at the system prompt. Every time a new model lands I see people paste the old prompt in and complain the results are worse. Opus 5 will have different steering. I would strip the prompt down to the shortest version that still enforces the policy and let the model do more of the work.

    Three, for the Copilot Studio side, wait. Model availability inside Copilot Studio lags the direct Claude API by weeks. There is no point speculating about which slot Opus 5 will fill until Microsoft actually plugs it in. Test on the direct API now, plan the migration path later. I learned the hard way that prototyping on one endpoint and moving to another at the last minute burns a sprint.

    If the long-horizon numbers hold up under real traces, this is the release that finally lets me push a handful of agents from human-in-the-loop to human-on-the-loop.

    This post was inspired by Claude Opus 5 via Anthropic.

  • Power Apps MCP Server Just Got Closed-Loop Learning and I Want to Point It at a Real Agent This Week

    Power Apps MCP Server Just Got Closed-Loop Learning and I Want to Point It at a Real Agent This Week

    Power Apps MCP server closed-loop learning diagram for enterprise agents

    Microsoft dropped the June 2026 Power Platform feature update on June 11, and the headline for me is Power Apps MCP server closed-loop learning for enterprise agents. Not a UX refresh. Not another connector. A real change in how agents get better after you ship them.

    The rest of the update is worth reading too, but this is the one I want to test against a real internal agent this week.

    What shipped on June 11

    The Power Apps MCP server now supports closed-loop learning. In practice that means the agent can adjust its grounding and skill selection based on how it is actually being used, without a developer manually rewriting prompts or retraining anything.

    Before this, teaching an agent your org meant one of two paths. Either you fed it documents and custom instructions and hoped the retrieval was good enough, or you stood up a data science workflow to run evaluation and optimization cycles yourself. Both paths break down at maintenance time. Docs go stale. Nobody wants to rerun eval jobs every quarter.

    Closed-loop learning changes the shape of that problem. Outcome signals from real usage flow back into the agent’s behavior at runtime.

    What Power Apps MCP server closed-loop learning actually does

    To be precise about what is and is not happening here: the underlying model is not being fine-tuned. What is being updated is the agent’s grounding layer and its skill selection policy, based on outcome data that lands in Dataverse.

    I wrote about this separation before in my post on the MCP server as the feedback layer. Skills are the policy layer. The MCP server signals are the feedback layer. Keep those two straight and the mental model holds up.

    The technical surface, as far as I can read from the docs and the Power Platform learn pages, looks like this. When a user interacts with the agent, outcomes get captured. Not just accept or reject clicks, but structured signals about what worked and what did not. Those signals feed a learning loop that updates how the agent grounds itself and which skills it reaches for in a given context. The Dataverse MCP server tool shape update that split metadata inspection, querying, and search into cleaner boundaries is part of what makes this grounding layer more precise.

    That is closer to how a good colleague gets better at their job than how a model gets fine-tuned. Which is the point.

    Why this changes the agent maintenance story

    Most internal agents I see, and most I hear about from people at other organisations, are frozen at deploy time. They make the same mistake in month four that they made in week one, until someone finally opens a ticket and a developer patches the prompt. That is not maintenance. That is triage. It is also exactly the kind of failure pattern I covered in why AI agents fail in production when they worked fine in testing.

    Closed-loop learning breaks that pattern if you set it up carefully. The agent adapts to how the process actually runs, not how the process was documented eighteen months ago when the SOP was last updated. Process drift stops being invisible.

    The trade-off is real though. Runtime learning means the audit story gets harder. Version history now has to cover learned behavior, not just authored skills. If a policy owner asks why the agent behaved differently in July than it did in May, someone has to be able to answer that from data, not from memory. Microsoft has not fully answered that governance question yet, and I want to see how the audit surface actually looks in a live environment.

    The other risk is signal quality. If your team treats every user click as positive feedback, you are teaching the agent that fatigue clicks and correct clicks mean the same thing. They do not. The learning loop is only as good as the signals you decide to trust.

    What I would do with it this week

    I have an internal agent in mind that has been quietly degrading for a few months. Not broken, just noticeably less useful than at launch. Perfect candidate.

    The plan looks like this. First, turn on closed-loop learning in a non-production copy and run it against a shadow traffic sample for a few days. Second, define the outcome signals I actually trust ahead of time, before I look at any data. Accept-without-edit is one signal. Accept-with-edit is another and it means something different. Explicit thumbs down is the cleanest. Fatigue clicks I want to filter out entirely.

    Third, set a review cadence. Not because I do not trust the loop, but because I want to see what it learns and whether the changes match the policy the skill owner would have made themselves. Having a clear way to measure whether the agent is actually improving is where a framework like measuring useful work per dollar for Power Platform AI agents becomes useful alongside the learning loop.

    If the learned behavior lines up with what a human policy owner would do, the loop is doing its job. If it drifts somewhere I did not expect, that is the interesting part.

    This is the first update in a while where I closed the browser tab and immediately opened a new one to start planning a test. I cannot wait to see how it holds up on a real workload.

    This post was inspired by What’s new in Power Platform: June 2026 feature update via Microsoft Power Platform Blog.

  • How to Measure Useful Work Per Dollar for Your Power Platform AI Agents

    How to Measure Useful Work Per Dollar for Your Power Platform AI Agents

    Power BI dashboard showing useful work per dollar for AI agents in Copilot Studio

    OpenAI published a piece on managing AI investments in the agentic era and the metric they anchor on is useful work per dollar. Not tokens per second. Not model benchmarks. Useful work per dollar for AI agents, measured against the actual business outcome the agent is supposed to produce. That framing maps cleanly onto Power Platform agents if you wire up the telemetry properly. This walkthrough shows how to instrument a Copilot Studio agent so you can see cost, task success rate, and useful work per dollar per workflow, then use that to decide what to scale and what to kill.

    The working result: a Dataverse table plus a Power BI page that gives you a per-agent, per-workflow view of cost efficiency in production. No guessing, no vibes.

    Step 1: Define what counts as useful work for the agent

    This is the step everyone skips and it is the one that decides whether the whole exercise is worth anything. Useful work is not “the agent responded.” It is the outcome the workflow is supposed to produce.

    For a support triage agent, useful work might be “ticket classified correctly and routed without human correction within 24 hours.” For an invoice extraction agent, it is “line items extracted with zero manual edits before posting.” For an internal knowledge agent, it is “user did not open a follow-up ticket on the same topic within 48 hours.”

    Write this down as a single boolean per invocation: isUsefulWork = true/false. If you cannot define it in one sentence, the agent probably does not have a clear job yet.

    Step 2: Log invocations and outcomes to Dataverse

    Create a Dataverse table called AgentInvocation with these columns at minimum: InvocationId, AgentName, WorkflowName, UserId, StartTime, EndTime, InputTokens, OutputTokens, ToolCallsCount, Outcome (choice: Success/Failure/Escalated), IsUsefulWork (bool), and Notes.

    In Copilot Studio, add a Power Automate flow at the end of the agent topic that writes a row to this table on every invocation. Pass the run context so you can correlate it later. For the IsUsefulWork field, you have two options: infer it automatically from a downstream signal (was the ticket reopened, did the invoice post, did the user thumbs-down) or capture it with a lightweight feedback prompt at the end of the conversation. I prefer downstream signals because self-reported feedback is noisy.

    Microsoft has a good primer on writing to Dataverse from flows if you have not done this pattern before.

    Step 3: Pull token and action costs into the same table

    Cost is not just tokens. For a Copilot Studio agent using Power Automate flows, tool calls, and premium connectors, the cost per invocation is a stack: model tokens plus per-message consumption plus any premium connector actions triggered inside tool flows.

    Build a simple cost model as a Dataverse calculated column or a Power BI measure:

    InvocationCost = (InputTokens * InputRate) + (OutputTokens * OutputRate) + (ToolCallsCount * FlowActionCost) + MessageMeterCost

    Get the current rates from your tenant’s Power Platform licensing page and your model provider. The exact rates will drift, so store them in a separate PricingRates table with an effective date rather than hardcoding them into the measure. Trust me on this one. I learned the hard way when connector pricing shifted and every dashboard I had was quietly wrong for a month.

    Step 4: Build the useful work per dollar view in Power BI

    Connect Power BI to the AgentInvocation table. Create three core measures:

    TotalCost = SUM(AgentInvocation[InvocationCost])

    UsefulWorkCount = CALCULATE(COUNTROWS(AgentInvocation), AgentInvocation[IsUsefulWork] = TRUE)

    UsefulWorkPerDollar = DIVIDE([UsefulWorkCount], [TotalCost])

    Build one page with a matrix visual: rows are WorkflowName, columns are the three measures plus task success rate and average invocation cost. Add a line chart showing useful work per dollar trending weekly per workflow. That trend line is the one you will actually look at.

    Slice by agent, by workflow, by user segment. The interesting patterns are usually in the segments, not the overall number. One workflow can be printing money while another quietly burns budget on retries.

    Step 5: Set a scale or kill threshold and review weekly

    Numbers without a decision rule are decoration. Set a threshold before you look at the data so you do not rationalize whatever the chart shows.

    My rule of thumb: if a workflow’s useful work per dollar is at least 3x the manual baseline (what the same work would cost in human time), it is a scale candidate. If it is below 1x for four weeks running, it gets killed or redesigned. Between 1x and 3x is the improvement zone: look at where the cost concentrates, usually retries, oversized system prompts, or unnecessary tool calls. If you want to understand what is driving those unnecessary tool calls, How Copilot Studio Agent Tool Selection Actually Works Under the Hood covers the orchestrator scoring pass and how to structure tools so the agent stops reaching for the wrong one.

    Review weekly. Monthly is too slow, daily is noise.

    Final state and one pitfall

    You end up with a Dataverse table capturing every invocation, a cost model that reflects real pricing, and a Power BI page that shows useful work per dollar per workflow with a scale-or-kill threshold. That is the whole loop.

    The common pitfall: measuring useful work by proxies that are easy to log rather than the ones that actually matter. “User did not thumbs-down” is not the same as “the work was correct.” Spend the extra effort to wire up a downstream signal, even if it means waiting 24 hours to mark an invocation useful. Noisy signals produce confident wrong decisions, and this is a place where being confidently wrong is expensive. If you want to see what happens when agents that looked fine in testing start failing on real signals, Why Do AI Agents Fail in Production When They Worked Fine in Testing walks through exactly that. For more on how I think about this stuff, see my LinkedIn.

    Frequently Asked Questions

    How do I measure useful work per dollar for AI agents in Power Platform?

    Start by defining a clear, binary outcome for each agent workflow, such as whether a ticket was routed correctly or an invoice posted without edits. Then log each invocation to Dataverse with cost and outcome data, and use Power BI to calculate the ratio of successful outcomes to total spend. This gives you a per-agent, per-workflow view of cost efficiency rather than relying on vague performance indicators.

    What is useful work in the context of a Copilot Studio agent?

    Useful work refers to the specific business outcome an agent is designed to produce, not simply the fact that it generated a response. For example, a support triage agent produces useful work only if the ticket is classified and routed correctly without human correction. If you cannot define that outcome in a single sentence, the agent likely lacks a clear purpose.

    How do I track AI agent invocation costs in Dataverse?

    Create a Dataverse table that records key details for every agent run, including token counts, tool calls, timestamps, and the workflow outcome. A Power Automate flow triggered at the end of each Copilot Studio topic can write this data automatically. Over time, this log gives you the raw numbers needed to calculate cost per successful outcome.

    When should I use downstream signals instead of user feedback to measure agent success?

    Downstream signals, such as whether a ticket was reopened or an invoice posted without edits, are generally more reliable than asking users to rate the interaction directly. Self-reported feedback tends to be inconsistent and easy to ignore, while automated signals reflect actual business outcomes. Use feedback prompts only when no suitable downstream signal exists for the workflow.

    This post was inspired by How to manage AI investments in the agentic era via OpenAI.

  • Dataverse July 2026 Update Extends the Agent Data Platform to More Coding Marketplaces and Certified MCPs

    Dataverse July 2026 Update Extends the Agent Data Platform to More Coding Marketplaces and Certified MCPs

    Dataverse July 2026 update expanding agent data platform with certified MCPs

    Microsoft shipped the July 2026 Dataverse update on July 6, and reading it back to back with the previous months makes the pattern obvious. The dataverse july 2026 update is not about new agent features. It is about reach and governance. The Dataverse plugin is expanding into more coding agent marketplaces, MCP tool coverage is growing, partner MCPs are getting certified, and internal MCPs are being pulled under enterprise governance.

    If you have been paying attention to the last three months of Dataverse announcements, this one fits the arc. Microsoft is turning Dataverse into the substrate every agent on the stack talks to, and now they are dealing with the mess that comes with that.

    What shipped in the July 2026 Dataverse update

    Four things landed together.

    First, the Dataverse plugin for coding agents is being pushed into more coding agent marketplaces. This is the same plugin that fixed the hallucination problem for coding agents generating Power Platform code. Wider distribution means more developers will hit it through their existing IDE and agent setups instead of having to hunt for it.

    Second, MCP tool coverage is growing. The Dataverse MCP server now exposes more of the platform surface, which continues the direction we saw when the tool shape shifted toward metadata inspection. Tighter tool boundaries, more of them, less guessing by the agent.

    Third, partner MCPs can now be certified. This is the one I think most teams will underestimate. Microsoft is publishing a certification path so third-party MCP servers can carry a trust signal. Think of it like connector certification, but for the agent tool layer.

    Fourth, internal MCPs get pulled under enterprise governance. DLP, environment scoping, audit. The plumbing you would expect from a Power Platform connector, applied to the MCP tool surface an agent can call.

    Why the certified MCP and governance angle matters

    Here is the part I keep thinking about. Every team I talk to that has been experimenting with agents in the last six months has quietly wired up MCP servers from wherever. Community repos. Vendor previews. Somebody’s GitHub. The tool surface an internal agent can reach is already sprawling, and nobody centralised the decision.

    Sound familiar? It is the same shape as the SharePoint site as a knowledge source problem I wrote about when knowledge sources and feedback loops shipped. Four makers, four wirings, no shared governance. MCP is worse because the surface is executable, not just readable.

    Certified partner MCPs are Microsoft’s answer to shadow adoption. Give platform teams a list of MCPs that carry a trust signal, register the ones that make sense, block the rest. The enterprise governance layer for internal MCPs is the other half. DLP policies that treat MCP tools like any other data movement. If you want to understand how the orchestrator evaluates tools before it ever calls them, how Copilot Studio agent tool selection actually works under the hood is worth reading alongside this update.

    The trade-off is real. Certification takes time. If your team needs a niche MCP that nobody has certified yet, you are either waiting or accepting risk. But the alternative is what I keep seeing on LinkedIn from people six months into agent projects. A tool surface nobody can inventory, and no clean way to answer the auditor when they ask what data the agent can touch. That sprawl is also one of the cleaner explanations for why AI agents fail in production when they worked fine in testing.

    What I would do with it this week

    I would not build a new agent this week. I would audit.

    Start with a simple inventory. Which MCP servers are already wired to any agent in your tenant. Where they came from. Who registered them. Whether they touch Dataverse, SharePoint, or anything with customer data. Most teams have never written this down.

    Then look at the certified partner MCP list as it lands and pick the two or three that actually match what your makers have been asking for. Register those under a governed environment. Publish the list internally. Say the quiet part out loud. If it is not on the list, it does not go into a production agent.

    Finally, get the Dataverse plugin for coding agents into the hands of the pro devs writing plugins and PCF controls. Not the citizen dev crowd. This is a pro dev tool and treating it otherwise misses the point. Scope it to a dev environment with a least-privilege app user, not System Administrator on production. I wrote about that setup in the original plugin post and it still applies.

    If you want to see how this ties back to the broader agent data platform direction, the Power Platform docs are catching up quickly and worth a scan. And if you want to compare notes on how other teams are handling MCP governance, I keep track of what people are saying on LinkedIn.

    The month the platform team gets ahead of MCP sprawl is worth more than the month you shipped one more agent.

    This post was inspired by Dataverse Is Your Agent Data Platform: Here’s What’s New in July 2026 via Microsoft Power Platform Blog.

  • Anthropic Shipped Reflect With Claude and the Journaling Angle Is More Interesting Than It Looks

    Anthropic Shipped Reflect With Claude and the Journaling Angle Is More Interesting Than It Looks

    Reflect with Claude guided journaling interface on a laptop screen

    Anthropic just shipped Reflect with Claude, a guided journaling and self-reflection experience built on top of the Claude models. On the surface this reads as a wellness feature. Look at it for two more minutes and Reflect with Claude is a positioning move that tells you where Anthropic thinks the interaction pattern is going.

    I want to talk about what it does, why I find it more interesting than the category suggests, and what I plan to actually do with it this week.

    What it actually does

    Reflect is a structured journaling surface inside Claude. You show up, Claude prompts you with reflection questions, and the session builds on prior sessions rather than starting from zero every time. It is opinionated about pacing. It asks follow-ups. It holds context across days.

    The mechanics that matter, if you read past the wellness framing:

    • The model prompts the user, not the other way around. That flips the default chat pattern.
    • Memory persists across sessions in a scoped, purposeful way.
    • The interaction is structured around a specific outcome, reflection, rather than open-ended tasks.
    • Privacy defaults are tighter than a general Claude conversation, which Anthropic calls out directly.

    You can try it on Claude.ai now. It is not an API surface, it is a product experience.

    Why it matters for people who build agents

    The wellness label is the wrapper. The interesting part is that Anthropic is publicly shipping a template for what I would call a companion pattern: model-initiated, memory-aware, purpose-bounded. That is not the same shape as a task assistant.

    Most enterprise agents I see people build are task assistants. User asks, agent does, session ends. That works for a lot of internal automation. It falls apart the moment you want an agent that follows up, checks in, or maintains a thread across days without being re-prompted from scratch every time.

    I have written before about how business logic and memory need real ownership in agent design. Reflect with Claude is a public reference point for what memory-aware, model-initiated interaction feels like when the vendor actually commits to it. The UX decisions in a product like this leak into what users start expecting from every agent they touch, including the ones I ship internally.

    The positioning piece is worth naming too. Anthropic is signaling that Claude is not just a coding model or a knowledge worker copilot. It is a companion product. That is a different market than OpenAI is currently chasing with agents-as-workers, and it is a different market than Microsoft is chasing with Copilot Studio. If you follow the model landscape at all, this is the kind of move that tells you where the roadmap is pointed. I keep tracking these positioning shifts because they change what stakeholders ask for six months later. Anthropic’s approach to red-teaming and safeguards methodology is part of what makes a product like this credible at the enterprise level, and worth studying regardless of whether you are building consumer or internal agents.

    My honest first take: this is more interesting than the launch post makes it sound. The category is wellness, the pattern is architecture.

    What I would do with it this week

    Three things, in order.

    First, I am going to actually use it for a week. Not skim it, use it. I want to feel how the prompting cadence works, how the memory shows up in session two versus session five, and where the friction lives. You cannot design a companion pattern for internal agents if you have never used a shipped one end to end.

    Second, I want to compare the interaction rhythm to what I have tried to build inside Copilot Studio agents. Copilot Studio is very good at request-response. It is not built for a model that opens the conversation. If I want an agent that pings a user with a reflection prompt at the end of the week, what does that actually look like on the Microsoft stack today? Probably a scheduled cloud flow that seeds a Copilot Studio conversation, which is a workaround, not the shape the platform was designed for. Worth stress testing. The reasons agents fail in production often come down to exactly this kind of mismatch between what you prototype and what the platform was actually designed to sustain.

    Third, I want to write down which parts of the Reflect UX I would steal for internal use cases. Not the wellness angle. The structural parts. Session continuity, model-initiated prompts, bounded memory, tight privacy defaults. These are the pieces that generalize to things like weekly project retrospectives, onboarding check-ins, or post-incident reviews with an agent.

    I cannot wait to try this properly. The launch might read as soft, but the shape underneath is the part I want to sit with.

    This post was inspired by Reflect With Claude via Anthropic.

  • Why Do AI Agents Fail in Production When They Worked Fine in Testing?

    Why Do AI Agents Fail in Production When They Worked Fine in Testing?

    Why AI agents fail in production despite passing tests

    Short answer: AI agents fail in production because test conversations are too clean, tool descriptions are too vague, and nobody logs the decisions the agent actually makes. The agent looked fine in testing because you asked it the questions you already knew it could answer. Real users do not do that.

    This is the question I keep getting from people at other organisations who just deployed their first Copilot Studio agent or LangGraph build and watched it fall over in week one. So here is the longer version.

    The longer answer on why AI agents fail in production

    Three things go wrong, and they usually go wrong together.

    1. The input distribution shifts the moment real users show up. In testing you type “what is the status of order 12345”. In production someone types “hey can you check where my thing from last tuesday is at, i think it was for the warehouse team”. The orchestrator now has to route ambiguous phrasing across multiple topics or tools, and it picks one confidently and gets it wrong. I wrote about this specific failure mode in how Copilot Studio agent tool selection actually works under the hood. The planner is scoring, not dispatching. Noisy input equals noisy scores.

    2. Tools return status instead of state. A tool that returns {"result": "done"} gives the agent nothing to reason about on the next turn. When something goes sideways the agent cannot recover because it does not know what actually happened. I have hit this the hard way. The fix is boring: tools return the state the agent needs to make the next decision, not a success flag.

    3. Nobody logged the decisions. Run logs tell you the flow executed. They do not tell you why the agent picked tool A over tool B, or why it summarised the ticket that way. Without a decision log you are debugging blind. You end up guessing at prompts.

    There is also a fourth thing that gets ignored. Silent failure. A Power Automate action inside a Copilot Studio topic can fail and the agent will still generate a confident-sounding response for work that was never done. The user gets an answer. Nothing throws. You find out three weeks later when someone asks why the record was never created.

    How to fix it

    Start with the tool contracts. Every tool your agent can call needs a precise name, a description written as a retrieval hint rather than documentation, and parameters with concrete names. customerAccountNumber not id. If you are on Copilot Studio, the Microsoft Learn docs cover the schema but not the hint-writing style. That part you learn by breaking things.

    Then cap the tool count. Past roughly 10 to 15 tools on a single agent, selection quality degrades because the relevance signal gets noisy. Split the agent before you add another tool.

    Log the decision, not just the execution. For every turn: what did the user say, what tools did the orchestrator consider, what did it pick, what did the tool return, what did the agent do with that. Store it. Query it. This is the only way to improve the policy over time. I go deeper on this in the post on decision ownership.

    Add adversarial test cases. Not “does it work”. Test misspellings, mixed languages, requests that sit between two topics, requests that reference something from two turns ago, requests where the user gives incomplete information and expects the agent to ask. This is where production breaks. Test it before production does. If you want a structured way to think about adversarial evaluation, the red team methodology in Anthropic’s jailbreak safeguards framework for Fable transfers directly to internal agent testing.

    Fail loudly. If a tool call fails, the agent should know and say so, not paper over it. Wire actual error handling into every Power Automate step the agent can invoke. Return the error state to the orchestrator so it can decide what to do next.

    Related gotchas

    Two more worth flagging.

    Context window drift. Long conversations quietly push earlier turns out of context. The agent forgets what the user told it in turn one by turn twelve. If your use case has long sessions, either summarise state into a persistent variable or split the interaction. I have written more about when a conversational interface is even the right choice in Copilot Studio is not always the answer.

    Model updates. The underlying model gets updated. Your prompts that worked last month behave differently this month. This is not theoretical. Anthropic and OpenAI both ship model updates that shift behaviour on edge cases. The Anthropic Fable 5 redeploy postmortem is a useful template for understanding how regressions surface and what a proper rollback and recovery process looks like. If you are running on Claude or any hosted model, version-pin where you can and regression-test when you cannot.

    None of this is glamorous. It is the boring work that separates an agent that survives in production from one that gets rolled back in week two. I talk about more of these lessons on LinkedIn if you want to compare notes.

    Frequently Asked Questions

    Why do AI agents fail in production after passing all their tests?

    AI agents often fail in production because test inputs are too clean and predictable compared to what real users actually type. Other common causes include tools that return vague responses the agent cannot reason about, and a lack of decision logging that makes it nearly impossible to diagnose what went wrong.

    How do I stop my AI agent from choosing the wrong tool for a user request?

    Write tool descriptions as retrieval hints rather than documentation, and use specific parameter names that clearly signal their purpose. Ambiguous phrasing from real users causes the agent to score tools incorrectly, so tighter tool contracts reduce misrouting significantly.

    What is silent failure in an AI agent and why does it matter?

    Silent failure happens when an action inside an agent workflow fails but the agent still generates a confident-sounding response, giving the user no indication anything went wrong. This is dangerous because the underlying task was never completed, and the problem may not surface until days or weeks later.

    When should I add decision logging to an AI agent?

    Decision logging should be built in before the agent goes live, not added after something breaks. Without it, you can see that the agent ran but not why it chose a particular tool or produced a specific response, which makes debugging little more than guesswork.

  • How Copilot Studio Agent Tool Selection Actually Works Under the Hood

    How Copilot Studio Agent Tool Selection Actually Works Under the Hood

    Diagram of copilot studio agent tool selection at runtime

    Most people building agents think copilot studio agent tool selection works like this: you attach a few tools, write a description for each, and the LLM reads the list and picks the right one. That is directionally correct and completely misses what is actually happening at runtime. The orchestrator runs a planning pass. It scores your tools against the current turn. Descriptions, input schemas, and even the order of your tools all feed that scoring.

    Once you see the mechanism, you stop writing tool descriptions like documentation and start writing them like retrieval hints. That changes how you name inputs, how many tools you attach to one agent, and when you split an agent instead of adding a fourteenth tool.

    What you see from the maker portal

    In the maker studio, you attach a tool to an agent. You give it a name, a description, and an input schema (either from a connector, a Power Automate flow, an MCP server, or a prompt). At runtime, you type a message, the agent thinks for a moment, and calls one of the tools. The trace view shows you which tool was picked and what inputs were passed.

    That surface makes it look like the model reads the list top to bottom and picks the best match. It is not that simple. The trace hides the planning pass, and the planning pass is where 80% of the reliability of your agent lives. Microsoft’s generative orchestration docs hint at this but do not spell it out in the way a builder needs.

    What the orchestrator is actually doing

    Between the user turn and the tool call, the orchestrator does something closer to retrieval than dispatch. It takes the current turn, the conversation history, and the agent instructions, and it scores each attached tool for relevance. The scoring uses the tool name, description, input parameter names, parameter descriptions, and enum values if present. Tools with vague names and thin descriptions score badly regardless of how logically correct they are.

    Then the planner picks a candidate tool, resolves inputs from the turn context (or asks the user for missing ones), and invokes. If the invocation fails or the result is empty, the planner may retry with a different tool. That retry loop is where token budget disappears and latency creeps up.

    This is the same pattern you see in the Copilot Studio release plans that describe how tools and knowledge sources get grounded per turn. It is a retrieval problem wearing a routing costume.

    Where the mechanism breaks down

    Three failure modes show up over and over. I wrote about the schema version of this in the Dataverse MCP server tool shape post, but it applies to every tool surface.

    Overlapping descriptions. Two tools both say something like “Get information about an order.” The planner cannot tell them apart at the description layer, so it falls back to parameter matching, which is noisier. You get silent misrouting where the agent confidently picks the wrong tool.

    Vague input schemas. A parameter called id of type string tells the planner nothing. A parameter called customerAccountNumber with a description like “6-digit customer account, not the order number” gives the planner something to bind against.

    Long tool lists. Once you attach more than roughly 10 to 15 tools, scoring quality degrades. The signal gets noisy. This mirrors what happens when you stuff too much into a system prompt, which I covered in the business skills post.

    How to build once you know this

    Write tool descriptions as retrieval hints, not documentation. State what the tool does, when to use it, and critically, when not to use it. “Use this to look up an order by its order number. Do not use this for customer profile lookups.” That negative clause is doing work.

    Name parameters like a human would search for them. orderNumber beats id. Add a description on every parameter, even the obvious ones. Enum values are gold because they narrow the planner’s search space to something concrete.

    Cap tool count per agent. If you find yourself attaching a fourteenth tool, split the agent by domain and use multi-agent orchestration to route between them. A focused agent with 6 well-described tools outperforms a monster agent with 20 tools every time. The same principle applies when deciding where your agent reads its data from — something I break down in SharePoint vs Dataverse as a Copilot Studio Knowledge Source.

    Test the routing, not just the tools. Write a set of representative user turns and check which tool the planner picks. If two tools tie or the wrong one wins, fix the descriptions before you touch the model or the instructions. That is the fastest debugging loop I have found, and it is one I keep coming back to in my own work.

    The mechanism is not magic. It is retrieval with extra steps. Once you build for that, your agents get more predictable, cheaper to run, and easier to explain to a stakeholder.

    Frequently Asked Questions

    How does Copilot Studio agent tool selection actually work at runtime?

    Rather than simply reading a list of tools and picking the best match, the orchestrator runs a planning pass that scores each tool based on relevance to the current turn. It factors in the tool name, description, input parameter names, and any enum values before selecting a candidate and resolving the required inputs. This scoring process is closer to retrieval than traditional routing.

    Why does my Copilot Studio agent keep picking the wrong tool?

    Vague tool names and thin descriptions are the most common cause, as the orchestrator scores tools for relevance and poorly described tools rank badly even if they are logically the right choice. Writing descriptions as retrieval hints rather than documentation, and being specific with parameter names, will improve selection accuracy significantly.

    How do I write better tool descriptions for a Copilot Studio agent?

    Instead of writing descriptions like reference documentation, treat them as signals that help the orchestrator match the tool to user intent. Be specific about what the tool does, what inputs it expects, and when it should be used rather than a similar tool. Precise parameter names and enum values also feed into the scoring process.

    When should I split one Copilot Studio agent into multiple agents instead of adding more tools?

    As the number of attached tools grows, the scoring pass has more candidates to evaluate and the risk of the wrong tool being selected increases. If you find yourself adding a large number of tools to a single agent, splitting responsibilities across multiple agents can improve reliability and reduce latency caused by retry loops.

  • Anthropic Redeployed Fable 5 After Rolling It Back and the Postmortem Is What I Am Reading Twice

    Anthropic Redeployed Fable 5 After Rolling It Back and the Postmortem Is What I Am Reading Twice

    Anthropic Fable 5 redeploy postmortem notes on a screen

    Anthropic pulled Fable 5 shortly after its initial release, then redeployed it with fixes and a public writeup. The anthropic fable 5 redeploy is not the story I care about. The postmortem is.

    Rolling back a flagship model publicly is rare. Doing it with a clear explanation of what broke and how they verified the fix is rarer. I have been reading this one twice because it maps almost directly onto how anyone deploying agents internally should think about regressions.

    What Anthropic actually shipped

    Fable 5 is the narrative-tuned variant in the Claude lineup, the one I wrote about when it first landed alongside Mythos 5. The redeploy brings the model back online after Anthropic identified regressions the initial release had introduced, including drift in persona stability and inconsistencies the launch evals had missed.

    The redeployed build ships with expanded regression tests, additional persona stability checks, and a documented verification pass before the rollout was reopened. In the writeup Anthropic names what broke, what they added to catch it next time, and what they changed in their pre-release process. That is the part I keep coming back to.

    You can read the full note on Anthropic’s site. It is short. It is worth your ten minutes.

    Why the rollback and redeploy matters

    Most frontier model releases treat launch as a one-way door. Ship, patch quietly, move on. Public rollback of a flagship variant tells you something about how the company thinks about the contract with the people building on top of it.

    Three things stand out.

    First, the transparency raises the bar. If you are building an agent on top of a model and the vendor tells you exactly what regressed and how they verified the fix, you can decide whether your workflow is affected. If the vendor patches silently, you find out through user complaints. I would rather have the writeup.

    Second, the operational discipline is the template. Naming the regression, adding tests that would have caught it, running a verification pass, and only then reopening the rollout is exactly the pattern I want internal agent deployments to follow. Most internal agent rollouts I hear about from people at other organisations skip at least two of those four steps.

    Third, this raises expectations for the whole space. When one vendor publishes a postmortem like this, the ones that keep patching silently start looking like they have something to hide. That is good pressure on the market.

    The uncomfortable read is what it says about launch evals in general. Fable 5 shipped, passed whatever gates it passed, and still needed a rollback. If that can happen at Anthropic, it is happening everywhere. The difference is whether you hear about it. That same assumption problem shows up in workforce planning too, and the OpenAI EU AI workforce report makes the case that the workflow change bucket is where most of the silent breakage actually lives.

    What I would do with this news this week

    Two concrete things.

    One. Write down what a rollback looks like for the agents you have in production. Not the theory. The actual steps. Who decides to pull it, how you flip the switch, what the fallback behavior is, and how you tell users. If you cannot answer those four in a paragraph, you do not have a rollback plan, you have a wish.

    Two. Look at your regression tests for agent behavior. If the only thing you check before a prompt change or a model swap is whether the happy path still works, you are shipping the same way Fable 5 shipped the first time. Add persona drift checks. Add tool-use reliability checks. Add a few adversarial prompts you know used to fail. This is exactly the kind of discipline I flagged when Opus 4.8 landed, because tail failures are what kill agentic workflows in production.

    If you are running Claude in a Power Automate flow or a Copilot Studio agent, the practical version is simple. Pin the model version in your connector config. Do not auto-upgrade. Keep a small eval set you can rerun on every model change. Before you reach for a desktop flow to automate that verification step, it is worth checking whether a cloud flow would do the job instead. The docs make version pinning straightforward, and it costs you nothing until the day it saves you.

    I have written more on how I think about model selection and agent reliability over on LinkedIn. The short version is this. Every model you depend on will regress at some point. The vendors that tell you about it are the ones worth building on.

    The next flagship release will land soon. The question is whether the postmortem discipline sticks.

    This post was inspired by Redeploying Fable 5 via Anthropic.

  • Dataverse MCP Server Got a New Tool Shape and the Metadata Inspection Story Is What Stands Out

    Dataverse MCP Server Got a New Tool Shape and the Metadata Inspection Story Is What Stands Out

    Dataverse MCP server tool shape diagram showing metadata inspection and query tools

    Microsoft published a new post on June 8, 2026 walking through the updated Dataverse MCP server tool shape. The headline is that agents can now inspect metadata, query records, and search across structured and unstructured data through cleaner, well-defined tool boundaries. You can read the original write-up on the Power Platform blog.

    This is not a feature dump. It is a redesign of how the agent talks to Dataverse. And that is more interesting than it sounds.

    What it actually does

    The old tool surface gave agents a handful of broad tools and expected them to figure out the rest from context. In practice, that meant the agent would call a query tool, get back something it could not interpret, call again with different parameters, fail, and burn tokens guessing at column logical names. I have watched this happen in traces. It is not pretty.

    The new shape breaks the surface into clearer categories. Metadata inspection is its own thing now. The agent can ask what tables exist, what columns a table has, what the relationships look like, before it tries to query anything. Record querying sits next to that with predictable inputs. Search across structured and unstructured data is its own tool boundary, so the agent does not have to invent a strategy for whether to hit the relational side or the knowledge side.

    The practical effect: fewer wasted tool calls, less prompt bloat from stuffing schema hints into the system prompt, and more predictable agent behavior when it hits a Dataverse environment cold.

    Why it matters

    This connects directly to a pattern I keep hitting. When you wire an agent to business data, the model is rarely the problem. The friction is the discovery loop. The agent does not know your schema. It does not know your naming conventions. It does not know that your account table has a custom column called cr1a3_segment that nobody documented.

    The old answer was to dump schema into the system prompt. That works until your prompt crosses a few hundred tokens and agent reliability starts degrading. I wrote about this in the context of business skills in Dataverse, and the same logic applies here. Pushing knowledge into a managed, queryable surface beats stuffing it into a prompt every time.

    The new tool shape gives the agent a path to discover the schema on demand instead of carrying it around. That is the architectural shift. It is the same lesson I keep seeing play out across the agent stack: tighter tool boundaries beat broader ones, because they shrink the search space the model has to reason about.

    It also pairs well with the low-latency Dataverse to Fabric sync story. If your agent can inspect metadata cleanly on the operational side and pull near real-time analytical data on the Fabric side, the data path behind the agent finally starts feeling coherent instead of stitched together.

    What I am skeptical about: tool shape changes are easy to undo if the team gets feedback that agents want more flexibility. I hope Microsoft holds the line on the boundaries. Broad, do-everything tools are how you end up back where you started.

    What I would do with it this week

    First, point an agent at a non-production Dataverse environment with the updated MCP server and watch the tool call traces. Not the chat output. The traces. That is where you see whether the new shape actually reduces wasted calls or just renames them.

    Second, strip schema hints out of system prompts on any existing Copilot Studio agent wired to Dataverse, and let the metadata inspection tool do the work. Compare reliability before and after. I expect the shorter prompt wins, based on what I have seen building internally and on what I keep reading from peers at other organisations. The Dataverse plugin for coding agents that Microsoft shipped at Build 2026 is worth reading alongside this, because it tackles the same hallucination-from-missing-schema problem from a different angle. You can also check the Power Platform docs for the current setup steps.

    Third, write down which tool should handle which question type for your agent and treat that as a contract. If the agent starts calling the search tool for things that should hit the query tool, that is a sign your boundary is wrong, not that the tool is wrong. I have been writing about these patterns on LinkedIn for a while now, and the discipline of writing the contract down before deployment saves you a month-six cleanup project.

    If the new tool shape holds up under real workloads, this becomes the default way agents talk to business data on the Microsoft stack.

    This post was inspired by Dataverse MCP Server: Understanding the New Tool Shape via Microsoft Power Platform Blog.