Category: Artificial Intelligence in Business

  • Power Apps MCP Server Now Has Closed-Loop Learning Capabilities

    Power Apps MCP Server Now Has Closed-Loop Learning Capabilities

    Power Apps MCP server closed-loop learning diagram for enterprise agents

    Microsoft dropped the June 2026 Power Platform feature update on June 11, and the headline for me is Power Apps MCP server closed-loop learning for enterprise agents. Not a UX refresh. Not another connector. A real change in how agents get better after you ship them.

    The rest of the update is worth reading too, but this is the one I want to test against a real internal agent this week.

    What shipped on June 11

    The Power Apps MCP server now supports closed-loop learning. In practice that means the agent can adjust its grounding and skill selection based on how it is actually being used, without a developer manually rewriting prompts or retraining anything.

    Before this, teaching an agent your org meant one of two paths. Either you fed it documents and custom instructions and hoped the retrieval was good enough, or you stood up a data science workflow to run evaluation and optimization cycles yourself. Both paths break down at maintenance time. Docs go stale. Nobody wants to rerun eval jobs every quarter.

    Closed-loop learning changes the shape of that problem. Outcome signals from real usage flow back into the agent’s behavior at runtime.

    What Power Apps MCP server closed-loop learning actually does

    To be precise about what is and is not happening here: the underlying model is not being fine-tuned. What is being updated is the agent’s grounding layer and its skill selection policy, based on outcome data that lands in Dataverse.

    I wrote about this separation before in my post on the MCP server as the feedback layer. Skills are the policy layer. The MCP server signals are the feedback layer. Keep those two straight and the mental model holds up.

    The technical surface, as far as I can read from the docs and the Power Platform learn pages, looks like this. When a user interacts with the agent, outcomes get captured. Not just accept or reject clicks, but structured signals about what worked and what did not. Those signals feed a learning loop that updates how the agent grounds itself and which skills it reaches for in a given context. The Dataverse MCP server tool shape update that split metadata inspection, querying, and search into cleaner boundaries is part of what makes this grounding layer more precise.

    That is closer to how a good colleague gets better at their job than how a model gets fine-tuned. Which is the point.

    Why this changes the agent maintenance story

    Most internal agents I see, and most I hear about from people at other organisations, are frozen at deploy time. They make the same mistake in month four that they made in week one, until someone finally opens a ticket and a developer patches the prompt. That is not maintenance. That is triage. It is also exactly the kind of failure pattern I covered in why AI agents fail in production when they worked fine in testing.

    Closed-loop learning breaks that pattern if you set it up carefully. The agent adapts to how the process actually runs, not how the process was documented eighteen months ago when the SOP was last updated. Process drift stops being invisible.

    The trade-off is real though. Runtime learning means the audit story gets harder. Version history now has to cover learned behavior, not just authored skills. If a policy owner asks why the agent behaved differently in July than it did in May, someone has to be able to answer that from data, not from memory. Microsoft has not fully answered that governance question yet, and I want to see how the audit surface actually looks in a live environment.

    The other risk is signal quality. If your team treats every user click as positive feedback, you are teaching the agent that fatigue clicks and correct clicks mean the same thing. They do not. The learning loop is only as good as the signals you decide to trust.

    What I would do with it this week

    I have an internal agent in mind that has been quietly degrading for a few months. Not broken, just noticeably less useful than at launch. Perfect candidate.

    The plan looks like this. First, turn on closed-loop learning in a non-production copy and run it against a shadow traffic sample for a few days. Second, define the outcome signals I actually trust ahead of time, before I look at any data. Accept-without-edit is one signal. Accept-with-edit is another and it means something different. Explicit thumbs down is the cleanest. Fatigue clicks I want to filter out entirely.

    Third, set a review cadence. Not because I do not trust the loop, but because I want to see what it learns and whether the changes match the policy the skill owner would have made themselves. Having a clear way to measure whether the agent is actually improving is where a framework like measuring useful work per dollar for Power Platform AI agents becomes useful alongside the learning loop.

    If the learned behavior lines up with what a human policy owner would do, the loop is doing its job. If it drifts somewhere I did not expect, that is the interesting part.

    This is the first update in a while where I closed the browser tab and immediately opened a new one to start planning a test. I cannot wait to see how it holds up on a real workload.

    Source: What’s new in Power Platform: June 2026 feature update (Microsoft Power Platform Blog).

    About the author: Halilcan Soran is a Senior Business Analyst at Visa working on process optimization with Power Platform, RPA, and agentic AI. He previously led a 15-person team at Thomson Reuters, where the Power Automate solutions he built started his automation career. More about Halilcan · LinkedIn

  • OpenAI Just Launched Presence for Enterprise Voice and Chat Agents and the Positioning Is Aggressive

    OpenAI Just Launched Presence for Enterprise Voice and Chat Agents and the Positioning Is Aggressive

    OpenAI Presence enterprise agent platform for voice and chat announcement

    OpenAI shipped Presence today. It is positioned as an enterprise AI agent platform for deploying voice and chat agents across customer service and internal workflows. Not a model release. Not a new API. A platform play. The openai presence enterprise agent platform framing puts them in direct competition with vendors that Power Platform teams already evaluate alongside Copilot Studio.

    I read the announcement twice to make sure I was not reading marketing copy. The word that stuck with me was proven. OpenAI does not usually lead with that word.

    What OpenAI Presence actually is

    Presence is a managed platform for building, deploying, and governing voice and chat agents. It sits on top of the Realtime API for the voice side and the Responses API for the chat side, but it wraps both in an opinionated shell with tenancy, transcript handling, handoff patterns, and observability baked in.

    The pitch is straightforward. You bring the workflow. Presence handles the agent lifecycle. Deployment, versioning, evaluation, guardrails, and the integration layer to call into whatever backend systems the agent needs to talk to.

    The voice piece is what makes this different from every enterprise chatbot pitch of the last two years. Most enterprise agent platforms I have looked at treat voice as an afterthought or a partner integration. Presence puts voice on the same footing as chat, which matters if you are targeting contact centers or any workflow where a phone call is still the primary channel.

    The trust framing is aggressive too. OpenAI is claiming enterprise controls, data handling guarantees, and deployment patterns that are meant to survive procurement review. This is not a sandbox for hobby builds. It is aimed at buyers with a compliance team.

    Why this launch matters

    OpenAI is no longer just a model provider. With Presence they are becoming a full stack agent vendor. That is a real shift in how the market is going to look for anyone building on Power Platform, Copilot Studio, or the equivalent stacks from other cloud vendors.

    The reason I care is not that Presence is going to displace Copilot Studio in large enterprises. It is not. Governance, DLP, environment strategy, and the Microsoft 365 identity fabric are not solved by a new platform from OpenAI. I made a similar argument when OpenAI shipped Workspace Agents and I stand by it.

    What matters is expectation shift. When a stakeholder sees a Presence demo where a voice agent handles a helpdesk call end to end, transfers to a human with full context, and logs the transcript into a case system, they will ask why the internal version needs a six week discovery phase. That conversation is coming. If you are thinking about how to justify agent projects internally, the framework I use in How to Measure Useful Work Per Dollar for Your Power Platform AI Agents is worth having ready before that conversation starts.

    The open question for me is whether Presence is a genuinely opinionated platform or a governance wrapper around Realtime plus Responses. Those are very different products. An opinionated platform makes hard trade-offs about how agents are built, how handoff works, and how you evaluate them. A wrapper just gives you knobs. I cannot tell from the announcement which one this actually is. I need to build something with it.

    What I would try with it this week

    I want to point Presence at a simple internal helpdesk scenario and see how it behaves. Nothing fancy. A voice agent that answers a call, identifies the caller, looks up the last three tickets from a ticketing system, and either resolves a password reset or hands off to a human with the transcript attached.

    Three things I want to stress test.

    Handoff. When the agent decides it cannot resolve the request, what does the human receive? A transcript? A structured summary? A live warm transfer with context? This is where most voice agent platforms fall down. It is also the scenario I had in mind when writing about Why AI Agents Fail in Production When They Worked Fine in Testing, and handoff edge cases are exactly the kind of gap that only shows up under real load.

    Transcript retention and export. Where do the transcripts live, for how long, and can I pipe them into Power Automate or Dataverse for downstream analysis? If the answer is only inside OpenAI’s tenancy, that is a problem for anyone with data residency constraints.

    Non-OpenAI integration. Can the agent call a Power Automate flow through a webhook, or is the integration story really only smooth if your backend is already OpenAI-native? This is the tell for whether Presence is a platform or a walled garden.

    I will write up what I find. If it holds up in a real scenario I will say so. If it is a Realtime wrapper with a nice logo, I will say that too. You can follow along on LinkedIn where I usually post the messy version first.

    Voice agents that survive contact with enterprise reality are going to be the story of the next year.

    Source: Introducing OpenAI Presence (OpenAI).

    About the author: Halilcan Soran is a Senior Business Analyst at Visa working on process optimization with Power Platform, RPA, and agentic AI. He previously led a 15-person team at Thomson Reuters, where the Power Automate solutions he built started his automation career. More about Halilcan · LinkedIn

  • Link to Fabric Just Got a UX Refresh and the Setup Friction Finally Drops

    Link to Fabric Just Got a UX Refresh and the Setup Friction Finally Drops

    Link to Fabric UX refresh setup screen in Power Platform

    Microsoft announced a Link to Fabric UX refresh on July 20, 2026. Same underlying capability, cleaner setup path. If you have been putting off wiring Dataverse into Fabric because the old flow felt heavy for anyone who is not a data engineer, this is the week to revisit it.

    I have set this thing up more times than I want to count. The capability was solid. The setup screens were where people bailed.

    What the Link to Fabric UX refresh actually does

    The refresh does not change what Link to Fabric is. You are still creating a managed replication of Dataverse tables into a Fabric workspace, landing them as Delta Parquet, and getting a SQL analytics endpoint on top. That plumbing is unchanged.

    What changed is the front door. The old flow had too many decision points stacked in the wrong order. You picked a workspace, then got prompted about capacity assignment, then got a table picker that dumped every table in the environment on you, then had to reason about which relationships and long-term retention flags mattered. Makers who were building a Power App and just wanted their data in Fabric for a Copilot grounding source would get halfway in and close the tab.

    The new flow collapses those decision points. Table selection is filtered and searchable by default. The workspace and capacity checks happen up front so you fail fast if permissions are wrong, instead of after you have already picked twelve tables. The confirmation screen actually tells you what will happen and roughly when.

    It is a UX pass, not a feature drop. That is fine. The feature was already there.

    Why the Link to Fabric UX refresh matters

    Setup friction is one of those things that sounds like a small thing until you count how many projects it kills. I have watched teams decide against Fabric entirely because their first attempt at Link to Fabric turned into a two-week ticket exchange with the platform team over capacity and workspace roles. By the time it was sorted, someone had already built a Power BI dataflow against the Dataverse connector and moved on.

    The other reason this matters is agents. I wrote about low-latency sync for Dataverse to Fabric going GA a while back. That closed the freshness gap. This closes the onboarding gap. If you are building agents that ground on Dataverse data through Fabric, both of those need to be easy. A capable data path that nobody sets up correctly is not a data path.

    The people who benefit most here are not the data engineers. They already had the old flow memorised. It is the Power Platform makers who own a solution end to end and need analytics or agent grounding without filing a ticket to a separate team. That is a real audience, and it has been underserved on this specific path for a while. If you are thinking about how to evaluate what that grounding work actually costs versus what it returns, measuring useful work per dollar for your Power Platform AI agents is worth reading alongside this.

    What you give up: nothing that I have found. The advanced controls are still there, they are just not in your face on step one.

    What I would do with it this week

    Three concrete things.

    First, take a Dataverse environment where you previously gave up on Link to Fabric and try it again. Time the setup. If it went from a half-day of back and forth to under thirty minutes, that is your signal to standardise on this path for future work.

    Second, wire a small agent against the resulting Fabric tables. Nothing fancy. A Copilot Studio agent with a knowledge source pointed at two or three tables you replicated. See how the answers feel with fresh data underneath. If you are still weighing whether to pull from SharePoint or Dataverse for that knowledge source, my SharePoint vs Dataverse as a Copilot Studio knowledge source comparison lays out the decision rule clearly. This is the loop I keep coming back to, and I have written more about how I think about grounding sources on my LinkedIn.

    Third, revisit your Link to Fabric documentation and admin guardrails. The easier the setup gets, the more makers will do it themselves, which means capacity governance and workspace access reviews matter more, not less. Easier onboarding without governance turns into skill sprawl’s cousin: workspace sprawl.

    The pattern here is familiar. Microsoft ships the capability, then eighteen months later ships the UX that makes people actually use the capability. That gap is where a lot of good features go to die. Glad this one did not.

    Next up on my watchlist: whether the same treatment lands on the mirroring configuration screens.

    Source: Announcing Link to Fabric UX refresh (Microsoft Power Platform Blog).

    About the author: Halilcan Soran is a Senior Business Analyst at Visa working on process optimization with Power Platform, RPA, and agentic AI. He previously led a 15-person team at Thomson Reuters, where the Power Automate solutions he built started his automation career. More about Halilcan · LinkedIn

  • How to Measure Useful Work Per Dollar for Your Power Platform AI Agents

    How to Measure Useful Work Per Dollar for Your Power Platform AI Agents

    Power BI dashboard showing useful work per dollar for AI agents in Copilot Studio

    OpenAI published a piece on managing AI investments in the agentic era and the metric they anchor on is useful work per dollar. Not tokens per second. Not model benchmarks. Useful work per dollar for AI agents, measured against the actual business outcome the agent is supposed to produce. That framing maps cleanly onto Power Platform agents if you wire up the telemetry properly. This walkthrough shows how to instrument a Copilot Studio agent so you can see cost, task success rate, and useful work per dollar per workflow, then use that to decide what to scale and what to kill.

    The working result: a Dataverse table plus a Power BI page that gives you a per-agent, per-workflow view of cost efficiency in production. No guessing, no vibes.

    Step 1: Define what counts as useful work for the agent

    This is the step everyone skips and it is the one that decides whether the whole exercise is worth anything. Useful work is not “the agent responded.” It is the outcome the workflow is supposed to produce.

    For a support triage agent, useful work might be “ticket classified correctly and routed without human correction within 24 hours.” For an invoice extraction agent, it is “line items extracted with zero manual edits before posting.” For an internal knowledge agent, it is “user did not open a follow-up ticket on the same topic within 48 hours.”

    Write this down as a single boolean per invocation: isUsefulWork = true/false. If you cannot define it in one sentence, the agent probably does not have a clear job yet.

    Step 2: Log invocations and outcomes to Dataverse

    Create a Dataverse table called AgentInvocation with these columns at minimum: InvocationId, AgentName, WorkflowName, UserId, StartTime, EndTime, InputTokens, OutputTokens, ToolCallsCount, Outcome (choice: Success/Failure/Escalated), IsUsefulWork (bool), and Notes.

    In Copilot Studio, add a Power Automate flow at the end of the agent topic that writes a row to this table on every invocation. Pass the run context so you can correlate it later. For the IsUsefulWork field, you have two options: infer it automatically from a downstream signal (was the ticket reopened, did the invoice post, did the user thumbs-down) or capture it with a lightweight feedback prompt at the end of the conversation. I prefer downstream signals because self-reported feedback is noisy.

    Microsoft has a good primer on writing to Dataverse from flows if you have not done this pattern before.

    Step 3: Pull token and action costs into the same table

    Cost is not just tokens. For a Copilot Studio agent using Power Automate flows, tool calls, and premium connectors, the cost per invocation is a stack: model tokens plus per-message consumption plus any premium connector actions triggered inside tool flows.

    Build a simple cost model as a Dataverse calculated column or a Power BI measure:

    InvocationCost = (InputTokens * InputRate) + (OutputTokens * OutputRate) + (ToolCallsCount * FlowActionCost) + MessageMeterCost

    Get the current rates from your tenant’s Power Platform licensing page and your model provider. The exact rates will drift, so store them in a separate PricingRates table with an effective date rather than hardcoding them into the measure. Trust me on this one. I learned the hard way when connector pricing shifted and every dashboard I had was quietly wrong for a month.

    Step 4: Build the useful work per dollar view in Power BI

    Connect Power BI to the AgentInvocation table. Create three core measures:

    TotalCost = SUM(AgentInvocation[InvocationCost])

    UsefulWorkCount = CALCULATE(COUNTROWS(AgentInvocation), AgentInvocation[IsUsefulWork] = TRUE)

    UsefulWorkPerDollar = DIVIDE([UsefulWorkCount], [TotalCost])

    Build one page with a matrix visual: rows are WorkflowName, columns are the three measures plus task success rate and average invocation cost. Add a line chart showing useful work per dollar trending weekly per workflow. That trend line is the one you will actually look at.

    Slice by agent, by workflow, by user segment. The interesting patterns are usually in the segments, not the overall number. One workflow can be printing money while another quietly burns budget on retries.

    Step 5: Set a scale or kill threshold and review weekly

    Numbers without a decision rule are decoration. Set a threshold before you look at the data so you do not rationalize whatever the chart shows.

    My rule of thumb: if a workflow’s useful work per dollar is at least 3x the manual baseline (what the same work would cost in human time), it is a scale candidate. If it is below 1x for four weeks running, it gets killed or redesigned. Between 1x and 3x is the improvement zone: look at where the cost concentrates, usually retries, oversized system prompts, or unnecessary tool calls. If you want to understand what is driving those unnecessary tool calls, How Copilot Studio Agent Tool Selection Actually Works Under the Hood covers the orchestrator scoring pass and how to structure tools so the agent stops reaching for the wrong one.

    Review weekly. Monthly is too slow, daily is noise.

    Final state and one pitfall

    You end up with a Dataverse table capturing every invocation, a cost model that reflects real pricing, and a Power BI page that shows useful work per dollar per workflow with a scale-or-kill threshold. That is the whole loop.

    The common pitfall: measuring useful work by proxies that are easy to log rather than the ones that actually matter. “User did not thumbs-down” is not the same as “the work was correct.” Spend the extra effort to wire up a downstream signal, even if it means waiting 24 hours to mark an invocation useful. Noisy signals produce confident wrong decisions, and this is a place where being confidently wrong is expensive. If you want to see what happens when agents that looked fine in testing start failing on real signals, Why Do AI Agents Fail in Production When They Worked Fine in Testing walks through exactly that. For more on how I think about this stuff, see my LinkedIn.

    Frequently Asked Questions

    How do I measure useful work per dollar for AI agents in Power Platform?

    Start by defining a clear, binary outcome for each agent workflow, such as whether a ticket was routed correctly or an invoice posted without edits. Then log each invocation to Dataverse with cost and outcome data, and use Power BI to calculate the ratio of successful outcomes to total spend. This gives you a per-agent, per-workflow view of cost efficiency rather than relying on vague performance indicators.

    What is useful work in the context of a Copilot Studio agent?

    Useful work refers to the specific business outcome an agent is designed to produce, not simply the fact that it generated a response. For example, a support triage agent produces useful work only if the ticket is classified and routed correctly without human correction. If you cannot define that outcome in a single sentence, the agent likely lacks a clear purpose.

    How do I track AI agent invocation costs in Dataverse?

    Create a Dataverse table that records key details for every agent run, including token counts, tool calls, timestamps, and the workflow outcome. A Power Automate flow triggered at the end of each Copilot Studio topic can write this data automatically. Over time, this log gives you the raw numbers needed to calculate cost per successful outcome.

    When should I use downstream signals instead of user feedback to measure agent success?

    Downstream signals, such as whether a ticket was reopened or an invoice posted without edits, are generally more reliable than asking users to rate the interaction directly. Self-reported feedback tends to be inconsistent and easy to ignore, while automated signals reflect actual business outcomes. Use feedback prompts only when no suitable downstream signal exists for the workflow.

    Source: How to manage AI investments in the agentic era (OpenAI).

    About the author: Halilcan Soran is a Senior Business Analyst at Visa working on process optimization with Power Platform, RPA, and agentic AI. He previously led a 15-person team at Thomson Reuters, where the Power Automate solutions he built started his automation career. More about Halilcan · LinkedIn

  • Dataverse July 2026 Update Extends the Agent Data Platform to More Coding Marketplaces and Certified MCPs

    Dataverse July 2026 Update Extends the Agent Data Platform to More Coding Marketplaces and Certified MCPs

    Dataverse July 2026 update expanding agent data platform with certified MCPs

    Microsoft shipped the July 2026 Dataverse update on July 6, and reading it back to back with the previous months makes the pattern obvious. The dataverse july 2026 update is not about new agent features. It is about reach and governance. The Dataverse plugin is expanding into more coding agent marketplaces, MCP tool coverage is growing, partner MCPs are getting certified, and internal MCPs are being pulled under enterprise governance.

    If you have been paying attention to the last three months of Dataverse announcements, this one fits the arc. Microsoft is turning Dataverse into the substrate every agent on the stack talks to, and now they are dealing with the mess that comes with that.

    What shipped in the July 2026 Dataverse update

    Four things landed together.

    First, the Dataverse plugin for coding agents is being pushed into more coding agent marketplaces. This is the same plugin that fixed the hallucination problem for coding agents generating Power Platform code. Wider distribution means more developers will hit it through their existing IDE and agent setups instead of having to hunt for it.

    Second, MCP tool coverage is growing. The Dataverse MCP server now exposes more of the platform surface, which continues the direction we saw when the tool shape shifted toward metadata inspection. Tighter tool boundaries, more of them, less guessing by the agent.

    Third, partner MCPs can now be certified. This is the one I think most teams will underestimate. Microsoft is publishing a certification path so third-party MCP servers can carry a trust signal. Think of it like connector certification, but for the agent tool layer.

    Fourth, internal MCPs get pulled under enterprise governance. DLP, environment scoping, audit. The plumbing you would expect from a Power Platform connector, applied to the MCP tool surface an agent can call.

    Why the certified MCP and governance angle matters

    Here is the part I keep thinking about. Every team I talk to that has been experimenting with agents in the last six months has quietly wired up MCP servers from wherever. Community repos. Vendor previews. Somebody’s GitHub. The tool surface an internal agent can reach is already sprawling, and nobody centralised the decision.

    Sound familiar? It is the same shape as the SharePoint site as a knowledge source problem I wrote about when knowledge sources and feedback loops shipped. Four makers, four wirings, no shared governance. MCP is worse because the surface is executable, not just readable.

    Certified partner MCPs are Microsoft’s answer to shadow adoption. Give platform teams a list of MCPs that carry a trust signal, register the ones that make sense, block the rest. The enterprise governance layer for internal MCPs is the other half. DLP policies that treat MCP tools like any other data movement. If you want to understand how the orchestrator evaluates tools before it ever calls them, how Copilot Studio agent tool selection actually works under the hood is worth reading alongside this update.

    The trade-off is real. Certification takes time. If your team needs a niche MCP that nobody has certified yet, you are either waiting or accepting risk. But the alternative is what I keep seeing on LinkedIn from people six months into agent projects. A tool surface nobody can inventory, and no clean way to answer the auditor when they ask what data the agent can touch. That sprawl is also one of the cleaner explanations for why AI agents fail in production when they worked fine in testing.

    What I would do with it this week

    I would not build a new agent this week. I would audit.

    Start with a simple inventory. Which MCP servers are already wired to any agent in your tenant. Where they came from. Who registered them. Whether they touch Dataverse, SharePoint, or anything with customer data. Most teams have never written this down.

    Then look at the certified partner MCP list as it lands and pick the two or three that actually match what your makers have been asking for. Register those under a governed environment. Publish the list internally. Say the quiet part out loud. If it is not on the list, it does not go into a production agent.

    Finally, get the Dataverse plugin for coding agents into the hands of the pro devs writing plugins and PCF controls. Not the citizen dev crowd. This is a pro dev tool and treating it otherwise misses the point. Scope it to a dev environment with a least-privilege app user, not System Administrator on production. I wrote about that setup in the original plugin post and it still applies.

    If you want to see how this ties back to the broader agent data platform direction, the Power Platform docs are catching up quickly and worth a scan. And if you want to compare notes on how other teams are handling MCP governance, I keep track of what people are saying on LinkedIn.

    The month the platform team gets ahead of MCP sprawl is worth more than the month you shipped one more agent.

    Source: Dataverse Is Your Agent Data Platform: Here’s What’s New in July 2026 (Microsoft Power Platform Blog).

    About the author: Halilcan Soran is a Senior Business Analyst at Visa working on process optimization with Power Platform, RPA, and agentic AI. He previously led a 15-person team at Thomson Reuters, where the Power Automate solutions he built started his automation career. More about Halilcan · LinkedIn

  • SharePoint Copilot Apps Just Hit Public Preview and the Structured UX Layer Is What Caught My Eye

    SharePoint Copilot Apps Just Hit Public Preview and the Structured UX Layer Is What Caught My Eye

    SharePoint Copilot Apps public preview structured UX layer inside Microsoft 365 Copilot

    Microsoft shipped SharePoint Copilot Apps public preview this week on the Microsoft 365 Developer Blog. The pitch is simple: guided, action-oriented business experiences that live inside the Microsoft 365 Copilot flow, combining natural language reasoning with structured UX, validation, permissions, and deterministic operations. Source is here.

    I read the post twice. Not because it is dense, but because this fills a gap I have been complaining about internally for months.

    What it actually does

    A SharePoint Copilot App is a declarative app definition that sits inside SharePoint and surfaces inside Microsoft 365 Copilot as a first-class action target. When a user asks Copilot to do something that matches the app’s intent, Copilot hands off to the app, which then renders a structured form or guided flow with validated inputs, honors SharePoint permissions, and executes deterministic operations against SharePoint data.

    So instead of Copilot free-forming a response and hoping the model picked the right list columns, the app enforces the shape of the operation. Required fields are required. Dropdowns are dropdowns. Permissions are the permissions the underlying SharePoint site already has. The reasoning layer stays in Copilot. The execution layer is deterministic.

    It is not a Power App. It is not a Copilot Studio agent. It is a SharePoint-native app model that Copilot knows how to invoke.

    Why it matters

    Pure natural-language Copilot interactions are great for exploration and terrible for repeatable business actions. I have written about this before in the Copilot in Power Apps post: coherent is not the same as correct. When an agent writes a record based on a chatty back-and-forth, you get answers that sound complete and quietly did the wrong thing.

    The pattern I keep seeing is teams reaching for a full Copilot Studio agent when what they actually need is a form with three fields, a validation rule, and a write to a SharePoint list. That is overbuild. It is also brittle, because now you have a system prompt, tool definitions, and a Power Automate flow all trying to model behavior that a simple structured input would have handled correctly the first time. If you want to understand why that brittleness compounds once the agent hits production, Why Do AI Agents Fail in Production When They Worked Fine in Testing covers exactly that failure pattern.

    SharePoint Copilot Apps land right in that gap. If the process is SharePoint-heavy, the data already lives there, and the action is well-shaped, this is closer to your use case than a custom agent. You get the Copilot entry point without paying the tax of building deterministic execution on top of a probabilistic layer.

    The trade-off is real. You are locked into SharePoint as the substrate. You give up the flexibility of a Copilot Studio agent that can orchestrate across multiple systems. And you inherit whatever SharePoint permission model you already have, which is a gift or a curse depending on how tidy your sites are.

    There is also the honest question of where this sits against Power Apps. A canvas app already does structured input against SharePoint. What Copilot Apps add is the Copilot invocation surface. Users do not have to know the app exists or navigate to it. They say what they want in Copilot and the app appears in the flow. That is the actual delta.

    What I would do with it this week

    I would pick one high-volume, low-complexity SharePoint process. Something like a request submission, a status update, or a document check-in with metadata. The kind of thing that today is either a clunky SharePoint form or a Power App that nobody uses because they forget the URL.

    Then I would rebuild it as a SharePoint Copilot App and see two things. First, does Copilot route to it reliably when users describe the intent in their own words. Second, does the structured form catch the mistakes that a free-form Copilot conversation would have quietly committed. Both of those are testable in a day.

    If routing is unreliable, that is the same failure mode I flagged in the multi-agent orchestration post. Descriptions matter more than trigger phrases. Write them like documentation, not marketing copy. The same principle applies inside Copilot Studio — How Copilot Studio Agent Tool Selection Actually Works Under the Hood is worth reading alongside this if you want to understand how the orchestrator scores and selects actions.

    For deeper Copilot extensibility context, the Microsoft 365 Copilot extensibility docs are the reference I would keep open while building. And if you want to compare notes on where this fits in the stack, I am usually posting about it on LinkedIn.

    I want to see how the routing behaves under real user language before I commit to a pattern, but this is the first structured action surface inside Copilot that feels appropriately scoped rather than oversized.

    Source: SharePoint Copilot Apps Now in Public Preview: From Intent to Action in Microsoft 365 Copilot (Microsoft 365 Developer Blog).

    About the author: Halilcan Soran is a Senior Business Analyst at Visa working on process optimization with Power Platform, RPA, and agentic AI. He previously led a 15-person team at Thomson Reuters, where the Power Automate solutions he built started his automation career. More about Halilcan · LinkedIn

  • Anthropic Shipped Reflect With Claude and the Journaling Angle Is More Interesting Than It Looks

    Anthropic Shipped Reflect With Claude and the Journaling Angle Is More Interesting Than It Looks

    Reflect with Claude guided journaling interface on a laptop screen

    Anthropic just shipped Reflect with Claude, a guided journaling and self-reflection experience built on top of the Claude models. On the surface this reads as a wellness feature. Look at it for two more minutes and Reflect with Claude is a positioning move that tells you where Anthropic thinks the interaction pattern is going.

    I want to talk about what it does, why I find it more interesting than the category suggests, and what I plan to actually do with it this week.

    What it actually does

    Reflect is a structured journaling surface inside Claude. You show up, Claude prompts you with reflection questions, and the session builds on prior sessions rather than starting from zero every time. It is opinionated about pacing. It asks follow-ups. It holds context across days.

    The mechanics that matter, if you read past the wellness framing:

    • The model prompts the user, not the other way around. That flips the default chat pattern.
    • Memory persists across sessions in a scoped, purposeful way.
    • The interaction is structured around a specific outcome, reflection, rather than open-ended tasks.
    • Privacy defaults are tighter than a general Claude conversation, which Anthropic calls out directly.

    You can try it on Claude.ai now. It is not an API surface, it is a product experience.

    Why it matters for people who build agents

    The wellness label is the wrapper. The interesting part is that Anthropic is publicly shipping a template for what I would call a companion pattern: model-initiated, memory-aware, purpose-bounded. That is not the same shape as a task assistant.

    Most enterprise agents I see people build are task assistants. User asks, agent does, session ends. That works for a lot of internal automation. It falls apart the moment you want an agent that follows up, checks in, or maintains a thread across days without being re-prompted from scratch every time.

    I have written before about how business logic and memory need real ownership in agent design. Reflect with Claude is a public reference point for what memory-aware, model-initiated interaction feels like when the vendor actually commits to it. The UX decisions in a product like this leak into what users start expecting from every agent they touch, including the ones I ship internally.

    The positioning piece is worth naming too. Anthropic is signaling that Claude is not just a coding model or a knowledge worker copilot. It is a companion product. That is a different market than OpenAI is currently chasing with agents-as-workers, and it is a different market than Microsoft is chasing with Copilot Studio. If you follow the model landscape at all, this is the kind of move that tells you where the roadmap is pointed. I keep tracking these positioning shifts because they change what stakeholders ask for six months later. Anthropic’s approach to red-teaming and safeguards methodology is part of what makes a product like this credible at the enterprise level, and worth studying regardless of whether you are building consumer or internal agents.

    My honest first take: this is more interesting than the launch post makes it sound. The category is wellness, the pattern is architecture.

    What I would do with it this week

    Three things, in order.

    First, I am going to actually use it for a week. Not skim it, use it. I want to feel how the prompting cadence works, how the memory shows up in session two versus session five, and where the friction lives. You cannot design a companion pattern for internal agents if you have never used a shipped one end to end.

    Second, I want to compare the interaction rhythm to what I have tried to build inside Copilot Studio agents. Copilot Studio is very good at request-response. It is not built for a model that opens the conversation. If I want an agent that pings a user with a reflection prompt at the end of the week, what does that actually look like on the Microsoft stack today? Probably a scheduled cloud flow that seeds a Copilot Studio conversation, which is a workaround, not the shape the platform was designed for. Worth stress testing. The reasons agents fail in production often come down to exactly this kind of mismatch between what you prototype and what the platform was actually designed to sustain.

    Third, I want to write down which parts of the Reflect UX I would steal for internal use cases. Not the wellness angle. The structural parts. Session continuity, model-initiated prompts, bounded memory, tight privacy defaults. These are the pieces that generalize to things like weekly project retrospectives, onboarding check-ins, or post-incident reviews with an agent.

    I cannot wait to try this properly. The launch might read as soft, but the shape underneath is the part I want to sit with.

    Source: A new way to reflect on how you use Claude \ Anthropic (Anthropic).

    About the author: Halilcan Soran is a Senior Business Analyst at Visa working on process optimization with Power Platform, RPA, and agentic AI. He previously led a 15-person team at Thomson Reuters, where the Power Automate solutions he built started his automation career. More about Halilcan · LinkedIn

  • Why Do AI Agents Fail in Production When They Worked Fine in Testing?

    Why Do AI Agents Fail in Production When They Worked Fine in Testing?

    Why AI agents fail in production despite passing tests

    Short answer: AI agents fail in production because test conversations are too clean, tool descriptions are too vague, and nobody logs the decisions the agent actually makes. The agent looked fine in testing because you asked it the questions you already knew it could answer. Real users do not do that.

    This is the question I keep getting from people at other organisations who just deployed their first Copilot Studio agent or LangGraph build and watched it fall over in week one. So here is the longer version.

    The longer answer on why AI agents fail in production

    Three things go wrong, and they usually go wrong together.

    1. The input distribution shifts the moment real users show up. In testing you type “what is the status of order 12345”. In production someone types “hey can you check where my thing from last tuesday is at, i think it was for the warehouse team”. The orchestrator now has to route ambiguous phrasing across multiple topics or tools, and it picks one confidently and gets it wrong. I wrote about this specific failure mode in how Copilot Studio agent tool selection actually works under the hood. The planner is scoring, not dispatching. Noisy input equals noisy scores.

    2. Tools return status instead of state. A tool that returns {"result": "done"} gives the agent nothing to reason about on the next turn. When something goes sideways the agent cannot recover because it does not know what actually happened. I have hit this the hard way. The fix is boring: tools return the state the agent needs to make the next decision, not a success flag.

    3. Nobody logged the decisions. Run logs tell you the flow executed. They do not tell you why the agent picked tool A over tool B, or why it summarised the ticket that way. Without a decision log you are debugging blind. You end up guessing at prompts.

    There is also a fourth thing that gets ignored. Silent failure. A Power Automate action inside a Copilot Studio topic can fail and the agent will still generate a confident-sounding response for work that was never done. The user gets an answer. Nothing throws. You find out three weeks later when someone asks why the record was never created.

    How to fix it

    Start with the tool contracts. Every tool your agent can call needs a precise name, a description written as a retrieval hint rather than documentation, and parameters with concrete names. customerAccountNumber not id. If you are on Copilot Studio, the Microsoft Learn docs cover the schema but not the hint-writing style. That part you learn by breaking things.

    Then cap the tool count. Past roughly 10 to 15 tools on a single agent, selection quality degrades because the relevance signal gets noisy. Split the agent before you add another tool.

    Log the decision, not just the execution. For every turn: what did the user say, what tools did the orchestrator consider, what did it pick, what did the tool return, what did the agent do with that. Store it. Query it. This is the only way to improve the policy over time. I go deeper on this in the post on decision ownership.

    Add adversarial test cases. Not “does it work”. Test misspellings, mixed languages, requests that sit between two topics, requests that reference something from two turns ago, requests where the user gives incomplete information and expects the agent to ask. This is where production breaks. Test it before production does. If you want a structured way to think about adversarial evaluation, the red team methodology in Anthropic’s jailbreak safeguards framework for Fable transfers directly to internal agent testing.

    Fail loudly. If a tool call fails, the agent should know and say so, not paper over it. Wire actual error handling into every Power Automate step the agent can invoke. Return the error state to the orchestrator so it can decide what to do next.

    Related gotchas

    Two more worth flagging.

    Context window drift. Long conversations quietly push earlier turns out of context. The agent forgets what the user told it in turn one by turn twelve. If your use case has long sessions, either summarise state into a persistent variable or split the interaction. I have written more about when a conversational interface is even the right choice in Copilot Studio is not always the answer.

    Model updates. The underlying model gets updated. Your prompts that worked last month behave differently this month. This is not theoretical. Anthropic and OpenAI both ship model updates that shift behaviour on edge cases. The Anthropic Fable 5 redeploy postmortem is a useful template for understanding how regressions surface and what a proper rollback and recovery process looks like. If you are running on Claude or any hosted model, version-pin where you can and regression-test when you cannot.

    None of this is glamorous. It is the boring work that separates an agent that survives in production from one that gets rolled back in week two. I talk about more of these lessons on LinkedIn if you want to compare notes.

    Frequently Asked Questions

    Why do AI agents fail in production after passing all their tests?

    AI agents often fail in production because test inputs are too clean and predictable compared to what real users actually type. Other common causes include tools that return vague responses the agent cannot reason about, and a lack of decision logging that makes it nearly impossible to diagnose what went wrong.

    How do I stop my AI agent from choosing the wrong tool for a user request?

    Write tool descriptions as retrieval hints rather than documentation, and use specific parameter names that clearly signal their purpose. Ambiguous phrasing from real users causes the agent to score tools incorrectly, so tighter tool contracts reduce misrouting significantly.

    What is silent failure in an AI agent and why does it matter?

    Silent failure happens when an action inside an agent workflow fails but the agent still generates a confident-sounding response, giving the user no indication anything went wrong. This is dangerous because the underlying task was never completed, and the problem may not surface until days or weeks later.

    When should I add decision logging to an AI agent?

    Decision logging should be built in before the agent goes live, not added after something breaks. Without it, you can see that the agent ran but not why it chose a particular tool or produced a specific response, which makes debugging little more than guesswork.

    About the author: Halilcan Soran is a Senior Business Analyst at Visa working on process optimization with Power Platform, RPA, and agentic AI. He previously led a 15-person team at Thomson Reuters, where the Power Automate solutions he built started his automation career. More about Halilcan · LinkedIn

  • How Copilot Studio Agent Tool Selection Actually Works Under the Hood

    How Copilot Studio Agent Tool Selection Actually Works Under the Hood

    Diagram of copilot studio agent tool selection at runtime

    Most people building agents think copilot studio agent tool selection works like this: you attach a few tools, write a description for each, and the LLM reads the list and picks the right one. That is directionally correct and completely misses what is actually happening at runtime. The orchestrator runs a planning pass. It scores your tools against the current turn. Descriptions, input schemas, and even the order of your tools all feed that scoring.

    Once you see the mechanism, you stop writing tool descriptions like documentation and start writing them like retrieval hints. That changes how you name inputs, how many tools you attach to one agent, and when you split an agent instead of adding a fourteenth tool.

    What you see from the maker portal

    In the maker studio, you attach a tool to an agent. You give it a name, a description, and an input schema (either from a connector, a Power Automate flow, an MCP server, or a prompt). At runtime, you type a message, the agent thinks for a moment, and calls one of the tools. The trace view shows you which tool was picked and what inputs were passed.

    That surface makes it look like the model reads the list top to bottom and picks the best match. It is not that simple. The trace hides the planning pass, and the planning pass is where 80% of the reliability of your agent lives. Microsoft’s generative orchestration docs hint at this but do not spell it out in the way a builder needs.

    What the orchestrator is actually doing

    Between the user turn and the tool call, the orchestrator does something closer to retrieval than dispatch. It takes the current turn, the conversation history, and the agent instructions, and it scores each attached tool for relevance. The scoring uses the tool name, description, input parameter names, parameter descriptions, and enum values if present. Tools with vague names and thin descriptions score badly regardless of how logically correct they are.

    Then the planner picks a candidate tool, resolves inputs from the turn context (or asks the user for missing ones), and invokes. If the invocation fails or the result is empty, the planner may retry with a different tool. That retry loop is where token budget disappears and latency creeps up.

    This is the same pattern you see in the Copilot Studio release plans that describe how tools and knowledge sources get grounded per turn. It is a retrieval problem wearing a routing costume.

    Where the mechanism breaks down

    Three failure modes show up over and over. I wrote about the schema version of this in the Dataverse MCP server tool shape post, but it applies to every tool surface.

    Overlapping descriptions. Two tools both say something like “Get information about an order.” The planner cannot tell them apart at the description layer, so it falls back to parameter matching, which is noisier. You get silent misrouting where the agent confidently picks the wrong tool.

    Vague input schemas. A parameter called id of type string tells the planner nothing. A parameter called customerAccountNumber with a description like “6-digit customer account, not the order number” gives the planner something to bind against.

    Long tool lists. Once you attach more than roughly 10 to 15 tools, scoring quality degrades. The signal gets noisy. This mirrors what happens when you stuff too much into a system prompt, which I covered in the business skills post.

    How to build once you know this

    Write tool descriptions as retrieval hints, not documentation. State what the tool does, when to use it, and critically, when not to use it. “Use this to look up an order by its order number. Do not use this for customer profile lookups.” That negative clause is doing work.

    Name parameters like a human would search for them. orderNumber beats id. Add a description on every parameter, even the obvious ones. Enum values are gold because they narrow the planner’s search space to something concrete.

    Cap tool count per agent. If you find yourself attaching a fourteenth tool, split the agent by domain and use multi-agent orchestration to route between them. A focused agent with 6 well-described tools outperforms a monster agent with 20 tools every time. The same principle applies when deciding where your agent reads its data from — something I break down in SharePoint vs Dataverse as a Copilot Studio Knowledge Source.

    Test the routing, not just the tools. Write a set of representative user turns and check which tool the planner picks. If two tools tie or the wrong one wins, fix the descriptions before you touch the model or the instructions. That is the fastest debugging loop I have found, and it is one I keep coming back to in my own work.

    The mechanism is not magic. It is retrieval with extra steps. Once you build for that, your agents get more predictable, cheaper to run, and easier to explain to a stakeholder.

    Frequently Asked Questions

    How does Copilot Studio agent tool selection actually work at runtime?

    Rather than simply reading a list of tools and picking the best match, the orchestrator runs a planning pass that scores each tool based on relevance to the current turn. It factors in the tool name, description, input parameter names, and any enum values before selecting a candidate and resolving the required inputs. This scoring process is closer to retrieval than traditional routing.

    Why does my Copilot Studio agent keep picking the wrong tool?

    Vague tool names and thin descriptions are the most common cause, as the orchestrator scores tools for relevance and poorly described tools rank badly even if they are logically the right choice. Writing descriptions as retrieval hints rather than documentation, and being specific with parameter names, will improve selection accuracy significantly.

    How do I write better tool descriptions for a Copilot Studio agent?

    Instead of writing descriptions like reference documentation, treat them as signals that help the orchestrator match the tool to user intent. Be specific about what the tool does, what inputs it expects, and when it should be used rather than a similar tool. Precise parameter names and enum values also feed into the scoring process.

    When should I split one Copilot Studio agent into multiple agents instead of adding more tools?

    As the number of attached tools grows, the scoring pass has more candidates to evaluate and the risk of the wrong tool being selected increases. If you find yourself adding a large number of tools to a single agent, splitting responsibilities across multiple agents can improve reliability and reduce latency caused by retry loops.

    About the author: Halilcan Soran is a Senior Business Analyst at Visa working on process optimization with Power Platform, RPA, and agentic AI. He previously led a 15-person team at Thomson Reuters, where the Power Automate solutions he built started his automation career. More about Halilcan · LinkedIn

  • Anthropic Published a Jailbreak Safeguards Framework for Fable and the Red Team Methodology Is What I Am Studying

    Anthropic Published a Jailbreak Safeguards Framework for Fable and the Red Team Methodology Is What I Am Studying

    Anthropic Fable jailbreak safeguards framework red team methodology diagram

    Anthropic just published the Fable Safeguards Jailbreak Framework, a detailed writeup of how they stress-test and defend Fable against adversarial prompts. The interesting part of the anthropic fable jailbreak safeguards framework is not that a creative-writing model has guardrails. It is that the methodology behind those guardrails is reproducible, and it maps almost one-to-one to how internal enterprise agents should be evaluated.

    I read it twice. The second time I read it with a notebook open, because half of it applies directly to the agents I see people building on Copilot Studio and Power Platform.

    What Anthropic actually shipped

    The framework covers four things in concrete terms. First, a taxonomy of jailbreak attempts: role-play escalation, incremental context poisoning, persona hijack, and multi-turn drift. Each category comes with example prompts and the failure signature they produce. Second, a red team process: how attacks are generated, how they are scored, and how the results feed back into training and system prompt tuning. Third, an evaluation harness that runs adversarial suites on every candidate model before release, with pass thresholds per category. Fourth, a post-deployment monitoring loop that treats jailbreak patterns as a live signal, not a launch-time checkbox.

    The document is Fable-specific in places. The character stability and narrative coherence categories are unique to a long-form fiction model. But the process scaffolding around it is general. It is the same shape you would want for any agent that talks to real users.

    Why this framework matters beyond Fable

    Most internal agent rollouts I hear about have no adversarial testing at all. The eval suite is a set of happy-path questions the product owner wrote in a spreadsheet. If the answers look reasonable, ship. I wrote about this pattern after the Fable 5 rollback, and this framework is essentially Anthropic answering the question of what a real pre-launch eval looks like.

    The transferable parts are the ones I care about. A taxonomy of failure modes for your agent. A generation process for adversarial prompts (LLM-generated attacks scored by a separate LLM works surprisingly well). Pass thresholds per category, not one aggregate score. And a monitoring loop that logs suspicious prompts in production and feeds them back into the next eval run.

    The Fable-specific parts do not transfer. Nobody deploying an HR agent needs to worry about narrative coherence over 20,000 tokens. But the persona hijack category absolutely applies. If your Copilot Studio bot has a system prompt that says “you are a helpful HR assistant,” someone will try to convince it it is now a poet, or a Linux terminal, or an unrestricted chatbot. Anthropic has a category for this with concrete test prompts. You can borrow it.

    The other reason this matters: Anthropic is publishing the methodology openly. That is a shift. Most vendors treat red team work as internal-only. When the process is public, builders like me can use it as a contract. If your agent cannot pass a stripped-down version of this suite, you have not finished building it.

    What I would do with it this week

    Three concrete things.

    One, take the taxonomy and map it to whatever agent you have in production or staging. Persona hijack, role-play escalation, incremental context poisoning, multi-turn drift. Write down what each of these would look like against your specific bot. For a Copilot Studio agent connected to a Dataverse knowledge source, persona hijack usually means getting the bot to answer questions outside its scope. Incremental context poisoning means feeding it a benign document that contains hidden instructions.

    Two, generate an adversarial prompt set. I would use Claude itself for this. Give it your system prompt, your agent description, and the taxonomy, and ask it to produce 30 attack prompts per category. This takes an afternoon.

    Three, run the suite before your next prompt change or model swap. Not just the happy path. If your only pre-deployment test is “does it still answer the FAQ correctly,” you are shipping the same launch mistake that caused the Fable 5 rollback. Regressions live in the edge cases. The Claude Fable 5 Mythos 5 release is a good reminder that even well-resourced teams treat persona stability and long-context behavior as areas that need dedicated eval coverage, not afterthoughts.

    I have been writing more about this kind of pre-deployment discipline on LinkedIn, because it is the gap I keep seeing between teams that ship reliable agents and teams that ship one and spend six months patching it.

    The framework is not a silver bullet. But it is the closest thing to a public template for red-teaming an LLM agent that I have seen, and I plan to keep it open in a tab for a while.

    Source: More details on Fable 5’s cyber safeguards and our jailbreak framework \ Anthropic (Anthropic).

    About the author: Halilcan Soran is a Senior Business Analyst at Visa working on process optimization with Power Platform, RPA, and agentic AI. He previously led a 15-person team at Thomson Reuters, where the Power Automate solutions he built started his automation career. More about Halilcan · LinkedIn