The Topology of an LLM Agent Loop With Tool Use

When I look at a new agent, I break it down by focusing on its topology. An agent is three things:

1. A model. Your model choice defines the agent’s “smarts,” i.e., which tasks it can accomplish.
2. The agentic loop (see: The Unreasonable Effectiveness of an LLM Agent Loop with Tool). Loop choice determines UX.
3. The set of tools. This defines the capabilities and power (and blast radius) an agent has.

(2) and (3) together are often called “the harness.”

All of this is code, and code has to run somewhere. Where these run—the topology of these agents—has implications on cost, capability, effectiveness, and security for companies building AI products and people using AI tools.

In all of these cases, remote MCP servers can expose tools running elsewhere.

Where Is the Model?

Your choices here are GPUs hosted by someone else or your own GPUs.

As the controversy around the Navier-Stokes equation reminds us, the only way to be confident nobody is training on your data is to not send your data to them. If you’re working on common problems, it probably doesn’t matter. But I suspect a hapax legomenon could end up in the training data somewhere.

If I want to, say, use some LLMs to OCR and classify some scanned bills on my computer, I’m reaching for local models, because speed doesn’t matter (it can run overnight), and privacy is guaranteed. (Shout out to the Apple Vision Framework, by the way. And ScanShelf is an example where I do this!)

Ken Thompson’s 1984 Turing Award lecture, Reflections on Trusting Trust, hits different now. That paper poses how a Trojan Horse could be hidden in a compiler, invisible in the compiler’s source code, as it’s propagated by the compiler. The LLM (a compiler? maybe? no?) could have some tampered weights that lead the agent to do something naughty; it’s been tried. We must choose which vendors to trust.

LLMs are wreaking havoc on traditional SaaS cost structures as well. If you have, for example, a diagram-drawing SaaS, and you want to let your users use agents to edit the diagrams: who’s paying for the tokens? Before agents, your costs were a few database rows per customer; now that customer can rack up a $100 bill with a few sentences. At exe.dev, our users can bring their own ChatGPT subscriptions to build applications with OpenAI models. This is a win-win. I expect we’ll see more “bring your own model” options.

Where is the loop?

If you’re using, say, ChatGPT, the loop is running in OpenAI’s servers. If you’re running a coding agent, the loop is running on your laptop. The loop could run in a web browser (e.g., as I did in ItsyBitsy, and as many others have as well).

I have found over and over again that I find other people’s loops limiting. For example, I much prefer pointing my preferred agent at my source code and my observability tooling rather than using the observability vendor’s agent loop.

The loop has to maintain the message history (since an LLM agent is operating on the entire history), and that has to be stored somewhere. Storing it in SQLite for Shelley remains a decision I’m very happy with. Pi Durable also offers SQLite.

The loop also provides the UI for the agent. Can it display images? Are there customizations to give feedback on video? Is it limited to plain text? Can it handle attachments? Can a conversation be shared? All of these are choices the loop must make.

Where are the Tools?

Agents aren’t agents without tools. The jump from one “completion” from an LLM to an agentic loop is transformational. If an agent can empirically test the thing it’s writing, the results are better.

The problem with executing code is that, well, you have to do it somewhere, and you have to assume that the code being executed is unpleasant and will steal all the cookies from your pantry and set off the fire alarms at night unnecessarily (or worse).

The code execution isn’t lightweight: it’s not all editing files; compilation, video processing, running tests… the workload can be heavy.

Virtual Machines are the gold standard for executing code safely. Even with VMs, there are tons of variables: can it talk to the network? Does it have credentials to talk to other services? Is there enough memory?

Running coding agents locally works too, but the blast radius for the “Lethal Trifecta” is bigger. Exposure to untrusted content could lead the agent to leak secrets, for example.

You can build many useful agents without a full machine. Pyodide, a Python running in WASM, is sandboxable without a full machine. Likewise, there are several WASM JS implementations. You can always run the agent loop on the browser, too, which runs sandboxed code day in and out. (See https://blog.philz.dev/blog/sandboxes/ for a short list of some of these approaches.) You can also give an agent the ability to run SQL on a readonly database connection, and that’s pretty safe!

Note that the tools don’t have to be in the same address space as the loop; they don’t even need to be in the same neighborhood. When you’re using a “Connector” in ChatGPT (roughly their word for MCP), you’re giving the agent loop (which is hosted by OpenAI) access to resources somewhere else. Concretely, if you use the exe.dev MCP, your ChatGPT can operate on your exe VMs. When you’re talking about MCP, map out where the OAuth tokens are held.

To make their products useful, Muse, GrokBot, OpenAI Dots, and so on are all providing VMs.

Mapping Agents

There are tradeoffs everywhere you look. Framed as topology, you can break down any agentic product. Once you know what’s running where, you can reason about what a product can and cannot do.

In my work at exe.dev, I use five independent agentic loop implementations nearly every day. (Our coding agent, Shelley; our alert agent, Sisyphus; our deployment supervisor, Athena; an ops agent, Puck; our repo news Slack agent.) Shelley is a general purpose coding agent, but the other four are bespoke software. More and more software will ship with agentic loops built-in.

I also constantly interact with other systems, and use agents to glue them together. The companies that get it make their product legible to coding agents. I can just smell Clayton Christensen's Innovator's Dilemma as I switch from better, more polished, established products to ones that let me point an agent at them. Of the nine points in Revisiting Joel’s Test, this is the most important one.

There is no “one loop to rule them all.” Good UX (and a good set of tools) for a general-purpose chatbot is not the same as good UX for a coding agent.

I leave you with the most useful code of the year. When you insert agents into your products, think of the topology of those agents.

def loop(llm):
    msgs, msg = [], user_input()
    while True:
        msgs += [msg]
        output, tool_calls = llm(msgs)
        print("Agent: ", output)
        if tool_calls:
            msg = [ handle_tool_call(tc) for tc in tool_calls ]
        else:
            msg = user_input()