L3Let the Model Choose the Next Step, but Let People See Every Step

A new hire in operations asks: “How did air fryers do in last month’s big sale?”

L2 files this under “sales query” and asks three questions from a fixed script. L3 first looks at what the person has already said, asks only for the one thing that’s missing, and then decides for itself whether to pull the numbers from the order system or from BI. The difference between them isn’t whether they ask follow-up questions. It’s who decides the next step.

The last post ended at the point where L1 and L2’s fixed flows can’t keep up: when a question has to be broken into steps and the next step depends on the result of the last. This post is about L3, which hands the next step to the model, and about what you then have to leave in place so people can see each step and stop it.

From this level on, the model decides the next step

The technology behind L3 is called function calling; Anthropic’s documentation calls it tool use. It works like this:

1. The developer gives the model a list of tools: what each one is called, what it does, and which parameters it needs.
2. The user makes a request, and the model judges whether to use a tool, and which one.
3. Instead of writing the answer directly, the model outputs a structured call request (the tool’s name and its parameters).
4. The application carries out that request for real.
5. The result goes back to the model, which writes the final answer.

The dividing line is step 2: the model decides which tool to use and when, and code no longer fixes that in advance. Anthropic’s documentation puts it this way: “Claude determines when to call a tool based on the user’s request and the tool’s description.”¹ L2 can recognize intent too, but the path it takes after that has been written in advance.

Strictly speaking, an L3 that calls a tool only once is closer to what Anthropic calls an “augmented LLM”; calling tools in a loop, based on what each call returns, is what they define as an Agent. The series introduction made the same point.

A minimal loop

I use “minimal execution loop” to describe one complete run of L3. It’s a descriptive framework I’ve drawn from engineering practice, not an industry-standard term. The closest thing in the research is ReAct, proposed in 2022 by researchers at Princeton University and Google Research: the model alternates between reasoning and acting.²

flowchart LR
  A[Understand the intent] --> B[Find the information gaps]
  B --> C["[Ask to fill the gaps]"]
  C --> D[Choose a tool]
  D --> E[Fill in parameters]
  E --> F[Run the call]
  F --> G[Check the result]
  G --> H[Produce the deliverable]
  G -- "If the result looks wrong, go back" --> B

In this loop, the two steps the model decides on its own are choosing the tool and filling in the parameters: whether to query the order system or pull a report from BI, and which line of the conversation the time range should come from. Asking follow-up questions is only one node, used when information is missing. L2 can ask too, from a preset script, so asking isn’t what defines L3.

L3 can call a tool just once (look up one record and answer), or call several in a row (pull the data, calculate the metrics, write the report). Calling several in a row brings errors that pile up on each other; more on that later.

A system that asks follow-up questions

Asking follow-up questions doesn’t define L3, but L3 asks differently from L2. L2’s questions are a fixed checklist: once it detects a “sales query,” it asks about the period, the channel and the category, in that order. L3’s questions are inferred. Back to the new hire in operations; L3 might ask:

“I need to check two things. Should I look only at first-party sales, what the platform sells itself, or include third-party stores too? And do you want a comparison with the same sale last year?”

The person already gave the time period, so it doesn’t ask again. If they mention an unusual way of slicing the analysis, it checks that once more, instead of forcing it into a standard template.

There are two ways to build this. One is to write the operating guidelines into the system prompt, spelling out what to ask in which situation. The other is to make “clarify the scope of the analysis” a tool of its own, which the model calls when it doesn’t have enough information. The second is more flexible, and it also keeps a record of the questions that can be checked later.

For a new hire, these questions have another use: each one tells them what a complete analysis has to consider. Experience that used to live only in senior colleagues’ heads becomes guidance built into the system.

Letting the model call tools: where to put the boundaries

First, a common misunderstanding: using function calling doesn’t mean forbidding the model from writing its own queries. Text-to-SQL (having the model write SQL and then run it) is a common pattern within function calling. What needs care is the environment it runs in, and whether anyone reviews it:

How the tools themselves are designed matters a great deal too. Anthropic’s engineering team has written up some lessons,³ and two of them, I think, matter most when you put this to work in a business.

First, how well a tool is described decides how accurately the model picks it. The model reads a tool’s description to decide whether to use it. A description like “get_data: gets data” leaves the model unsure, and more so as the number of tools grows. A good description says what the tool does, when to use it, and when not to. Another Anthropic article reports that after they added usage examples to tool definitions, accuracy on complex parameters rose from 72% to 90% in their internal testing.⁴

Second, design a small number of tools around the workflow, and don’t wrap every interface in a tool of its own. Where it helps, fold several steps into one tool, then check the effect with evaluation data. Anthropic’s example: rather than offering list_users, list_events and create_event, offer a single schedule_event that finds a free slot and then books the event itself.³

The most underrated use: getting AI to use the systems you already have

I think the most underrated direction for L3 in companies is teaching AI to use the systems already in place, without replacing them.

Take an e-commerce platform that has been running for many years. Orders, logistics, customer service tickets, membership, BI: each system has been in use for years and is tied to real business processes, so replacing any of them costs a great deal. In a setting like that, “bring in a brand-new AI system” meets far more resistance than “teach AI to use the systems already there.”

L3 suits this well. Wrap the existing systems’ interfaces as tools, and the Agent calls those tools and works the way a skilled operator would. The systems don’t need to change, the mature systems still guarantee that the data is accurate, and the AI only handles the scheduling. For the IT department, this amounts to the existing systems gaining a smarter user, and that makes the project much easier to push through.

The infrastructure for this is the Model Context Protocol (MCP), which Anthropic released in November 2024.⁵ It’s an open standard for how AI applications connect to outside data and tools. Before it, every new system needed its own custom connection code; with N systems and M AI applications, the number of connectors to maintain is the product of the two. MCP solves this with one shared protocol, and the official documentation compares it to “a USB-C port for AI applications.”⁶ By December 2025, major AI products including ChatGPT, Gemini and Microsoft Copilot supported MCP, and that month Anthropic donated it to the Agentic AI Foundation, newly set up under the Linux Foundation.⁷

On the e-commerce platform, that means the order, logistics, customer service ticket and membership systems can all connect to the same L3 Agent through one protocol, with no separate integration layer to build for each.

Let users see it, and let them stop it

With L1 and L2 the interaction is simple: type a question, see an answer. L3 adds a layer. Behind the scenes the model takes a series of actions, choosing tools, filling in parameters, calling interfaces, checking results, and the user sees none of it. For business users who don’t code, that causes two problems: distrust (“What is it doing? Where did that number come from?”), and no way to correct it (“The result is wrong, but I can’t tell which step went wrong”).

So the job of an L3 interface is to put the tool calls happening in the background in front of the user, in business language. I do this in four layers.

The first layer is showing the process. Instead of displaying interfaces and parameters, the system says something like “Pulling air fryer sales for last month’s sale from the order system…” Users don’t need to understand interfaces; they only need to know the system is working on their request properly.

The second layer is confirming parameters. Once the model has pulled the parameters out of the conversation, it shows them to the user before anything runs.

Users can catch a wrong parameter before any computing power is spent, which is far easier than finding it later in a wrong report.

The third layer is reviewing drafts, and it’s only for write operations: sending email, pushing data to downstream systems, changing membership records, producing formal reports. Before any of these runs, a person sees a preview.

This echoes the advice in Anthropic’s Building Effective Agents: an Agent can pause at checkpoints and wait for human feedback.⁸ In the interface, that means always leaving a person the final confirmation before any action that can’t be undone.

The fourth layer is tracing results: next to each important number, show where it came from and when, for example “Air fryer sales during the sale were up 18% year on year (source: order system, as of September 15).” That reassures people, and it leaves a trail to follow when the data turns out to be wrong.

The same result can also change format for its reader. A store’s operations staff get a brief that fits on one phone screen; the category lead gets a report with comparison charts. You do this by adding a “who is this for” parameter to the report tool, which the model fills in based on who the user is.

Three common failures

Give an Agent the power to act, and its errors stop being wrong words and become real operational mistakes.

• Parameter hallucination: the model gets a parameter wrong when extracting it, for example matching “Q4 last year” to the wrong year, or a region name to the wrong code. The most effective fix is the second layer above: have a person confirm the parameters before anything runs.
• Cascading errors: when several tools are called in a row and the first step pulls the wrong date range, every growth rate after it rests on the wrong baseline. The conclusion in the final report can end up far from the facts, even though each step looks “fine” on its own. The fix is to pause at the steps that matter, show a person the intermediate result, and continue only once it’s confirmed. A single tool call doesn’t have this problem.
• Overreaching permissions: if a tool’s permissions are too broad, or its description too vague, the Agent can be led (by accidental ambiguity, or by a deliberate prompt injection) to touch data it shouldn’t. The fix is least privilege: each tool gets only the permissions it needs, read-only tools are kept apart from tools that write, and every write has to pass an independent review.

On top of these fixes, my rule is simple. Reading data and doing analysis, the Agent can do on its own, and it can be checked afterwards. Sending messages, changing records and pushing data must start as a draft, and run only after a person confirms.

What L3 can’t do

Limit 1: every intermediate result shares one context

When a request needs several independent systems at once, a single L3 Agent can issue several tool calls in the same turn and let the application run them at the same time. As of October 2026, the Claude and OpenAI APIs both allow this by default.⁹ So “one call at a time” isn’t L3’s bottleneck.

L3’s real bottleneck is that every intermediate result is crammed into the same context. Take “handling an unusual refund case”: checking the financial transactions, confirming the shipping status, going through the customer’s history. If each of these needs several steps of exploration and a judgment of its own, the context swells fast, and the separate threads start to interfere with each other.

This is where L4’s approach fits better: sub-agents work in their own separate contexts, and an orchestration layer brings the results together. When Anthropic built its own multi-agent research system, one of its reasons was to let sub-agents work in parallel, each with its own context window.¹⁰

Limit 2: tasks whose path can’t be listed in advance

L3’s tools are all prepared ahead of time. Faced with a completely new situation that has no matching tool in the toolbox, the Agent can’t invent a new path on the spot. It’s good at combining things flexibly inside known boundaries, and not good at exploring the unknown on its own.

Limit 3: when goals conflict, no one checks and no one decides

When a task holds several conflicting goals (on a sale-event product page, say, marketing wants the selling points to hit harder, while legal wants every absolute claim removed), a single Agent has to speak for both sides inside one prompt. It absorbs the conflict quietly, and you can’t see which side gave way, or why.

Limit 2 is a topic for L5. Limits 1 and 3 point to L4: several Agents, each working in its own context and each speaking for one goal, with an orchestration layer that brings their work together, and rules or a person making the final call. That’s the next post.

Closing

Back to the new hire in operations. L3 asked him the one question that was missing, pulled the numbers from the order system and BI by itself, and wrote the brief. But before sending it, it stopped and showed it to him.

Whether L3 works well usually comes down to three things: whether the tools have clear boundaries, whether the process is visible, and whether a person confirms before anything is written. Anthropic’s advice in Building Effective Agents points the same way: start with the simplest solution that solves the problem, and add complexity only when it’s truly needed.⁸

When a task holds several conflicting goals that need different views to check each other, and someone has to make the final call, you’ve reached the edge of L3, and that is where L4 begins. If all you need is parallel calls, or work across several systems, L3 can still carry it.