Every agent conversation I get pulled into starts with the same debate. Opus 5 or GPT. Copilot Studio’s default model or bring-your-own. Someone has a benchmark chart open before anyone has asked what the agent is supposed to do. After more than a year of building enterprise AI agents internally, I am convinced this is the least consequential decision in the entire project. Swapping the model has never taken me more than an afternoon. The data plumbing, the connector policy sprawl, the tool selection behavior, and figuring out whether the thing produces useful work per dollar have each eaten weeks. This post is my map of where the real cost centers sit, and it links out to everything I have written on each layer, so treat it as the index.
Every Enterprise AI Agent Kickoff Starts With the Wrong Debate
Model differences are real. I wrote about why Opus 5’s long-horizon task completion numbers matter more than coding benchmarks, and Anthropic shipping verticalized agents like Claude Finance tells you the frontier labs know where the value sits. But here is the asymmetry nobody prices in: the model is the one component you can replace without touching anything else. Copilot Studio lets you change it in a dropdown. A custom orchestration layer changes it in one config line. Meanwhile your Dataverse schema grounding, your DLP policies, your tool descriptions, and your measurement pipeline are all bespoke work that survives every model swap and cannot be bought off a pricing page.
So when a kickoff burns two hours on model selection, I now say the same thing every time. Budget this project like 10 percent model and 90 percent platform. Every agent I have seen die in month two died in the 90 percent.
Grounding Data Is Where the Real Engineering Hours Go
The hardest part of an internal agent is not intelligence. It is getting the agent to behave like someone who actually works at your organisation. Org policies, naming conventions, tribal knowledge, the exception the finance team makes every quarter-end. None of that is in any model’s training data.
The naive fix is stuffing everything into the system prompt. I learned the hard way that prompts past a few hundred tokens start degrading reliability, and a focused 400-token instruction set beats a 4000-token one every time. That is why I consider business skills in Dataverse an architectural shift rather than a feature: policy moves out of the prompt into a managed record with an owner and a version history. If you want persistent AI output that stays queryable alongside that structured data, Prompt Columns in Dataverse are worth understanding as a companion pattern — you define the prompt once and the result lives as a column rather than a one-off generation.
Same logic applies to schema. An agent that does not know your column logical names burns tokens guessing, which is why the Dataverse MCP server’s metadata inspection tools matter, and why grounding coding agents in live environment metadata is the only reliable fix for hallucinated column names. If you want to understand what is actually happening on the wire when an agent calls one of these tools, I broke down how MCP works under the hood. It is a stateful JSON-RPC protocol, not a REST connector, and treating it like one breaks in production.
Knowledge sources carry the same trap at a different layer. Wire the same SharePoint site into four agents four different ways and you get silent misrouting nobody can diagnose. I covered the governance side in my post on knowledge sources and agent feedback loops. None of this work goes away when you change models. All of it decides whether the agent is trustworthy.
Connector Policies and Tool Selection Decide What Your Agent Can Actually Do
The second cost center is everything between the agent and the systems it touches. I hit this building a GitHub integration: the connector sat in the Business data group in Dev and the Blocked group in Prod, so an agent that passed UAT would not even load its connection at go-live. The full story is in my post on enterprise environment sprawl, and the lesson generalises. The connector is never the problem. The environment strategy around it is.
Then there is tool selection, which most teams treat as magic. It is not. In Copilot Studio, the orchestrator scores tool names, descriptions, parameter names, and enum values against the current turn. It is a retrieval problem, not a dispatch problem, and I walked through the mechanics in how Copilot Studio agent tool selection actually works. A vague operation summary like GetData on a custom connector is not lazy metadata. It is a bad prompt that makes the agent pick the wrong action.
This is also why I push back on premature multi-agent architectures. A model that hits 95 percent tool selection accuracy with two connectors can drop to 70 percent with five, because descriptions start competing with each other. Multi-agent is a tax you pay when a single agent can no longer do the job, not an upgrade, and orchestration routing just moves the misrouting failure one layer deeper where it is harder to debug.
If You Cannot Measure Useful Work Per Dollar, You Built a Demo
The last cost center is the one almost everyone skips. If you cannot define what counts as useful work for your agent in one sentence, it does not have a clear enough job to be worth scaling. I documented my full approach in measuring useful work per dollar for AI agents: a single boolean per invocation, sourced from downstream behavioral signals like ticket reopens rather than thumbs-up ratings, divided into a cost stack of model tokens plus per-message consumption plus premium connector actions.
Measurement is also your early warning system. Agents that sailed through UAT fail in production because test conversations are too clean, and the input distribution shifts the moment real users show up with messy phrasing. A workflow that completes 70 percent of the time is not a workflow, it is a demo, because the human check-in that makes it reliable eliminates the time savings it was supposed to create. Without a useful-work metric you do not find that out until someone quietly stops using the agent.
The encouraging part: this layer is finally getting platform support. Feedback stored as queryable Dataverse rows, closed-loop learning through the Power Apps MCP server, and certified MCPs giving platform teams a trust-signal list instead of shadow adoption from random repos. The tooling is catching up to the problem.
So here is the payoff. Next time an agent project kicks off, do not open with the model comparison. Open with four questions. Where does the org knowledge live and who owns it. Which connectors and policies apply in the environment where this will actually run. How many tools before selection accuracy degrades. And what single boolean defines useful work. In my experience, the teams that answer those four before touching a model dropdown ship agents that survive month three. The teams that start with the benchmark chart ship demos.
Frequently Asked Questions
What is the most expensive part of building enterprise AI agents?
The biggest costs in enterprise AI agents are not the model itself but the surrounding layers: data grounding, governance policies, tool configuration, and measurement pipelines. These are bespoke to your organisation and cannot be swapped out the way a model can, which is why they consume the most engineering time and budget.
How do I choose the right AI model for an enterprise agent?
Model selection matters far less than most teams assume, since modern platforms let you swap models with minimal effort. A better approach is to spend the majority of your planning budget on data grounding, connector policies, and measurement before you finalise any model choice.
Why does my enterprise AI agent give unreliable answers despite a strong model?
Unreliable outputs are usually a grounding problem, not a model problem. If the agent lacks access to your organisation’s policies, naming conventions, and internal context, even a frontier model will produce generic or incorrect responses. Moving that knowledge out of long system prompts and into structured, versioned records tends to improve consistency.
When should I move AI policy instructions out of the system prompt?
Once your system prompt grows beyond a few hundred tokens, reliability starts to drop and maintenance becomes difficult. Migrating policy instructions into a managed data layer gives each rule a clear owner and version history, and it keeps the prompt focused on behaviour rather than organisational context.
Leave a Reply