Halilcan Soran

Tag: Agentic Workflows

  • Anthropic Shipped Claude Opus 4.8 and the Coding Numbers Are What Caught My Eye

    Anthropic Shipped Claude Opus 4.8 and the Coding Numbers Are What Caught My Eye

    Claude Opus 4.8 release notes shown on a developer screen with agent workflow diagrams

    Anthropic dropped the Claude Opus 4.8 release today. I read the post twice before writing anything because the coding and tool-use deltas are the part that actually matters for people building agents, not the headline number.

    Most model bumps are easy to ignore. This one I am paying attention to, and not for the reason most posts will tell you.

    What it actually does

    Opus 4.8 is an incremental release on top of the Opus 4 line. The model card focuses on three areas: coding quality, tool-use reliability, and long-horizon agentic task completion. Anthropic reports gains on SWE-bench style coding tasks and improvements on multi-step tool sequences where the model has to plan, call a tool, read the result, and decide what to do next without losing the thread.

    The pricing and context window stay in line with the Opus 4 family. It is available on the Anthropic API, on Claude.ai, on Amazon Bedrock, and on Google Vertex. Same deployment story as before.

    The thing that jumped out at me was the reduction in tool-call errors on longer agent traces. Not the average. The tail. That is a different problem to fix and a different problem to feel in production.

    Why it matters for agent builders

    If you have built anything resembling a real agentic workflow, you know the failure mode. The agent runs fine for 4 or 5 steps. Then on step 8 it calls a tool with malformed JSON, or hallucinates an argument, or forgets which tool it already called. The whole trace dies and you are looking at logs trying to figure out where the drift started.

    Coding quality and tool-use reliability are the same problem wearing two different hats. A model that writes better code is a model that produces better structured outputs, better function arguments, and better adherence to a schema. That is what makes a multi-step agent stop falling apart on turn 9.

    I have written before about latency being the quiet killer of agentic workflows. Reliability is the louder one. A workflow that completes 70 percent of the time is not a workflow. It is a demo. And most teams I talk to are sitting at 70 percent and calling it production.

    If Opus 4.8 actually pushes the tail of tool-call failures down, that is the difference between an agent you can schedule overnight and an agent that needs a human babysitting every run. That is the gap I care about.

    The other piece worth naming: this is Anthropic doubling down on coding and agents as the wedge. Not consumer chat. Not creative writing. They are picking a lane and shipping into it. Anthropic has been consistent about this for months and Opus 4.8 fits that pattern cleanly. That same strategic focus is visible in moves like Anthropic acquiring Stainless, where the SDK layer itself became a competitive asset.

    What I would do with it this week

    One test. Not a benchmark. A real one.

    Take the agent or automation in your stack that fails most often on long traces. The one where you keep adding retry logic and validation steps because the model keeps producing slightly wrong tool arguments. Run the same trace 20 times on whatever model you are using now. Log the failure points. Swap to Opus 4.8 with the exact same prompt and tools. Run it 20 more times.

    If the tail failures drop noticeably, you have your answer. If they do not, you keep what you have and you saved yourself a migration.

    I would also pay attention to cost per successful completion, not cost per token. A more expensive model that completes the workflow on the first try beats a cheaper model that needs three retries every time. That math is what people forget when they compare price sheets. It is the same logic that applies when you are deciding whether to build a multi-agent system instead of a single agent — the right architecture is the one that actually completes reliably, not the one that looks cleanest on a whiteboard.

    One more thing. If you are running Claude through Bedrock or Vertex, give the model version a week or two to land cleanly in your region before you build anything critical on it. I learned the hard way that prototyping on the direct API and then lifting to Bedrock at the last minute will burn a sprint on region availability and version naming. And if you are watching the broader competitive picture on AI coding agents in enterprise environments, the OpenAI and Dell Codex on-premise partnership is worth reading alongside this release — the two moves are aimed at the same buyer.

    Opus 4.8 is not a revolution. It is a sharpening. And for the kind of long-running agent work I keep seeing teams try to stand up, sharpening is exactly what is needed right now.

    This post was inspired by Claude Opus 4 8 via Anthropic.

  • Latency Is the Quiet Killer of Agentic Workflows and Almost Nobody Talks About It

    Latency Is the Quiet Killer of Agentic Workflows and Almost Nobody Talks About It

    Diagram showing agentic workflow latency across multiple model calls in a Copilot Studio and Power Automate loop

    Everyone obsesses over model quality, tool design, and prompt structure when building agents. The thing that actually kills adoption in production is something else entirely. Agentic workflow latency is the quiet killer, and most Power Platform and Copilot Studio builders are not thinking about it until users start abandoning the tool.

    I came across a post from OpenAI about using WebSockets and connection-scoped caching in the Responses API to speed up their Codex loop. It confirmed something I keep running into building multi-step agents internally. The math is brutal once you do it honestly.

    Why Agent Loops Feel Slow Even When Each Call Is Fast

    A single model call at 800ms feels fine. A tool call at 300ms feels fine. A Dataverse lookup at 500ms feels fine. Everyone looks at these numbers in isolation and says the platform is fast enough.

    Then you build an actual agent. It reasons, calls a tool, reads the result, reasons again, calls another tool, checks a condition, calls a third tool, summarises, responds. That is 8 to 15 round trips for one user request. Each round trip carries connection setup, authentication overhead, token streaming setup, and the model’s own time to first token.

    A 400ms overhead per call sounds small. Multiply by 12 calls. That is almost 5 seconds of pure overhead before any actual thinking or work happens. Users do not wait 15 seconds for a confident answer. They ask once, get nothing for a few seconds, and switch back to the old way of doing it.

    I have watched this kill internal tools that were technically correct. The agent did the right thing. Nobody used it.

    What OpenAI Just Shipped and Why It Matters Beyond Codex

    The short version of what they did: move from repeated HTTP requests to a persistent WebSocket connection, and keep cache state scoped to that connection so repeat context does not need to be re-processed on every turn.

    This is not a Codex-only trick. It is a general pattern. Connection-scoped caching means the expensive part of a call, the part that handles your system prompt and tool definitions and prior context, does not get redone from scratch every time your agent takes another step.

    For anyone building agents that loop, this is the shape of the next year of infrastructure work. The platforms that expose this properly will feel instant. The ones that do not will feel like they are thinking through molasses.

    What This Looks Like Inside Copilot Studio and Power Automate

    Here is where it gets uncomfortable. In Copilot Studio, you do not see the round trips. You see a topic, a few actions, a generative answer node. The platform hides every call behind its own orchestration.

    That hiding is the problem. A Copilot Studio agent doing generative orchestration with three tool calls backed by Power Automate flows is making far more round trips than most builders realise. Each tool call is a Power Automate HTTP trigger plus whatever that flow does internally, often including another connector call to SharePoint, Dataverse, or an external API. The agent then reads the response and decides what to do next, which is another model call. And if you are hitting Power Automate throttling limits under real load, every one of those round trips gets longer.

    I built one recently that felt snappy in testing with one user. In production with ten concurrent sessions, response times doubled. Nothing in the flow was slow on its own. The sum was slow, and throttling on shared connectors made it worse. This is the same class of problem I wrote about in Most Agentic Workflows Are Just Fancy If/Then Logic in a Trench Coat. The difference between a real agent loop and a glorified flow shows up in latency first.

    How I Would Budget Latency Before I Build the Agent

    I treat latency as a first-class design constraint now, not something I measure after the fact. Before I build, I do this:

    • Estimate the number of model calls per user request. Not best case. Typical case.
    • Estimate the number of tool calls and what each one hits. A SharePoint list call in the same tenant is not a Graph API call with auth handshake.
    • Set a budget. I aim for under 4 seconds total for anything conversational, under 10 seconds for anything that is clearly doing work.
    • Cut calls aggressively. Can two tools be one? Can I pre-fetch context in a single call instead of three? Can the agent skip a reasoning step when the intent is obvious?
    • Parallelise where I can. Power Automate lets you run actions in parallel branches. Most builders do not use them.

    The other thing I stopped doing: chaining LLM calls for steps that do not need reasoning. If a step is deterministic, I call the tool directly, not through the model. Every model call I can remove from the loop gives me back 500 to 1500ms.

    Latency is also where the question of who owns the decision in an agentic workflow becomes a performance problem, not just a governance one. Every checkpoint that routes back to a human approver adds another wait state to the loop. The more of those you have, the more your total response time is dominated by human latency, not model latency.

    I have written more about my approach to this kind of trade-off on my LinkedIn, because I keep having the same conversation with people at other organisations who hit the wall when their demo hits real users.

    The agents that win in production are not the smartest ones. They are the ones that answer before the user gives up.

    Frequently Asked Questions

    Why does agentic workflow latency get so bad in multi-step agents?

    Each individual call in an agent loop may seem fast, but the overhead adds up across 8 to 15 round trips per user request. Connection setup, authentication, and token streaming costs stack on every single step, turning individually acceptable delays into a frustrating overall wait time.

    What is connection-scoped caching and how does it help agent performance?

    Connection-scoped caching keeps expensive context like system prompts, tool definitions, and prior conversation state ready across multiple calls instead of reprocessing it each time. This avoids redundant work on every step of an agent loop and significantly reduces the overhead that accumulates across a multi-turn interaction.

    How do I reduce latency in Copilot Studio and Power Automate agents?

    Start by auditing how many round trips your agent actually makes for a single user request, since this is where most hidden latency lives. Look for opportunities to batch tool calls, reduce unnecessary steps in your loop, and watch for platform-level improvements like persistent connections that reduce per-call overhead.

    Why do users abandon AI agents even when the agent gives correct answers?

    If the response takes too long, users lose confidence and revert to familiar alternatives before the agent finishes. Technical correctness does not matter if the experience feels slow enough to suggest something has gone wrong.

    This post was inspired by Speeding up agentic workflows with WebSockets in the Responses API via OpenAI.

  • Most Agentic Workflows Are Just Fancy If/Then Logic in a Trench Coat

    Most Agentic Workflows Are Just Fancy If/Then Logic in a Trench Coat

    People keep asking in the community what makes an agentic workflow actually useful. The honest answer is that most things being called agentic workflows right now are not. They are linear automations with a language model bolted on for the response step. That distinction matters more than most teams realise when they start building.

    What a Useful Agentic Workflow Actually Does

    A useful agentic workflow does something a standard Power Automate flow cannot: it makes decisions mid-execution based on context it discovered during the run, not based on conditions you hard-coded before it started.

    That sounds obvious. It is not, in practice.

    A flow that checks a field value and routes left or right is not an agent. An agent is something that can retrieve information it did not start with, reason about what that information means for the current task, and take a different action than you would have anticipated when you designed it. The key word is discovered. The agent had to go and find out something, then act on it.

    If you can fully diagram the execution path before the workflow runs, it is probably not agentic. It is a well-structured flow. There is nothing wrong with a well-structured flow. But you should not be paying the overhead of agent infrastructure to build one.

    Where Teams Go Wrong Building Agentic Workflows

    The most common mistake I see is treating the language model as the agent. The LLM is not the agent. The LLM is the reasoning layer. The agent is the system that decides when to call what tool, handles what comes back, and determines whether the result is good enough to proceed or whether it needs to try something else.

    When that orchestration layer is weak or missing, you get a workflow that calls one tool, takes the output at face value, and moves on. That is not reasoning under uncertainty. That is a glorified lookup with a friendly response message.

    I wrote about silent action failures in the context of Copilot Studio earlier (the production testing post covers this in detail). The same failure mode appears in agentic workflows, but it is worse because the agent has more steps where it can silently accept a bad result and keep going. A flow fails at a specific action. An agent can propagate a bad intermediate result through three more steps before anything looks wrong.

    The Two Things That Make or Break an Agentic Workflow

    Based on what I have built internally and what I hear from people at other organisations, it comes down to two things.

    First: tool design. The actions available to your agent need to return enough context for the agent to evaluate them, not just a success or failure signal. If your Power Automate flow returns {"status": "done"}, the agent has no way to assess whether done means what the user needed. It will treat it as success. Your tools need to return structured, interpretable output. This is not a language model problem. It is an API design problem.

    Second: failure handling that is explicit, not optimistic. A useful agent knows when it is stuck and does something about it. That might mean escalating to a human, asking the user for clarification, or stopping cleanly with an honest message. What it does not do is generate a confident-sounding response for a task that did not complete. That is the failure mode that destroys trust in agents faster than anything else, because the user finds out later, not immediately.

    I covered how this plays out in Copilot Studio specifically in the post on when Copilot Studio is the wrong choice. But the principle applies regardless of the tooling. An agent that cannot fail gracefully is not useful in production. It is a liability.

    What Agentic Workflows Are Actually Good For

    The use cases where agentic workflows justify their complexity share a few characteristics. The task has multiple possible paths and you cannot enumerate them all upfront. The inputs are unstructured or variable enough that rule-based routing breaks down. The system needs to recover from partial failures without a human in the loop for every edge case.

    Document processing that involves extracting, validating, cross-referencing, and then acting on extracted data is a reasonable fit. Multi-step research tasks where what you search for next depends on what you found are a reasonable fit. Anything where the decision logic changes frequently and hard-coding it into a flow becomes a maintenance problem is worth evaluating. Before committing to that architecture, though, it is worth asking whether the underlying process is actually sound — automating a bad process just makes it fail faster, and agentic workflows are no exception.

    A status check is not a fit. A single-action task triggered by a button is not a fit. Anything you can build cleanly as a Power Automate flow with proper error handling is probably not worth the overhead of an agentic architecture. The orchestration cost is real and the debugging surface is larger.

    The Test I Use

    Before committing to an agentic workflow architecture, I ask one question: does this task require the system to discover something during execution that changes what it does next, and would that discovery be different for different runs? As Halilcan Soran on LinkedIn, I have found this single question filters out more false positives than any other test I use.

    If yes, agents are worth the investment. If no, you are adding complexity to solve a problem that a well-built flow could handle, and you will spend more time debugging Claude or other AI agent behaviour than you saved on logic design.

    The technology is not the constraint. Knowing what you are actually building is.

    Frequently Asked Questions

    What