Chapter 23. The Agent Era

Improper architecture is locally cheap and globally expensive: fusing the factors into one program is always less work today, and the costs land on caches, crawlers, integrators, and the future. For thirty years the deferral had no consequences. Now it does. Software agents are trying to read the web, and they find what Part IV measured: rendered pixels and private APIs. The response is a compensating industry: scraping harnesses, headless browsers, and a per-application protocol server bolted onto every system whose owners want it read by machines. Read that list against Chapter 11’s scores: it is the S4 cost at industry scale, one adapter per application, exactly as the model predicts. The machine-readable web is being retrofitted from outside because the platform abandoned it — Chapter 9’s maintenance failure, its cost still compounding.

On the left, four agents each wired to three applications through a dense tangle of one-off adapters, labelled N times M; on the right, the same agents and applications each connecting once to a single shared substrate, labelled N plus M

The arithmetic of that compensating industry is the integration industry’s arithmetic (Chapter 18) at a new scale. N agents meeting M applications through bespoke adapters need on the order of N × M integrations. The moment state shares one model and one query semantics, the count collapses to N + M — each side implements the common substrate once. Engineers have re-learned this sum in every generation of middleware, and they are re-learning it now, in the agent era, with N growing by the month. The per-application protocol server (MCP, the Model Context Protocol, the emerging convention as of this writing) is the N × M answer. The protocol is shared, but the model and query semantics are not: each server exposes its own vocabulary, so every agent still learns every application one at a time. The derived stack is the N + M answer, shipped since 1999.

The arithmetic also has a way out. Instead of one adapter per consumer, write one adapter per application, translating it into the common model: written once, it serves every consumer. Such an adapter is temporary by design. Once the application serves its own state natively, its answers are the ones the adapter was already giving, so nothing downstream changes and the adapter can be switched off. Until then it can answer in either of two ways: translate from the silo at query time and store nothing, or import the translated facts into the consumer’s own dataspace, where they survive after the silo account that supplied them is closed. And the adapters need not be built from scratch: the industry already maintains scrapers and protocol servers around every silo, and aiming them at the common model turns them into the bridge.

The vision, dated 2001. This chapter derives a scenario that was written as fiction twenty-five years ago. The May 2001 Scientific American article “The Semantic Web” is by Tim Berners-Lee, James Hendler, and Ora Lassila. It opens with Lucy’s agent negotiating a course of medical appointments over machine-readable data on her behalf, an agent reading the web, not scraping its pixels. It read as science fiction because the agents did not exist. They exist now. Chapter 8 said the substrate was built for machine consumption and the machines were twenty years out; those twenty years have now elapsed.

Underneath runs the grounding problem. Statistical models interpolate, and interpolation hallucinates. What agents need beneath them is a substrate whose answers are computed rather than guessed. Fact-sets with a formal query semantics are that substrate, and Part III named the deployed one. The two divide the work, statistical model above, fact-set substrate below. Where the substrate covers a question, the answer is computed from it; the model interprets the rest, the rare and one-off questions it does not yet cover. What proves valuable there is promoted into the substrate, and promotion is simply a write: the fact is asserted into some party’s dataspace and attributed to that party, like any other fact. The substrate is governed as any origin is, by whoever holds the right to write it. And integration accumulates instead of repeating: every promoted fact composes by union, and stays.

“Machine-consumable” requires nothing the book has not already derived: state with a universal model (R1), a coordination-free merge (R2), global reference (R3), and addressable intermediates (S4). An agent is a user agent. The requirement was in Definition 1.1’s type signature from the start.

And reading is only half of Definition 1.1. The write side serves agents even better. An agent’s change arrives as Chapter 7’s delta: two fact-sets, (D⁻, D⁺). The delta is a reviewable object: a human can inspect it before it applies, an audit log can store it verbatim, an operator can invert it by swapping the sets. The alternative is an opaque API call: its effect is whatever the endpoint’s code does, and nothing can reverse it. Agent autonomy is a governance problem only where agent actions are opaque, and the delta normal form makes the action a document. And what holds for one change holds for a whole plan of them. An agent’s entire intended course can be stated as one document and read before any of it runs: what it will read, what it will change, what it will do if an answer comes back empty.

The chapter ends with a question, put to any agent directly: is it more efficient for you to write a custom system for every domain, or to reuse one generic system and define the domain as data? Reusing one generic system is plainly the more efficient of the two. What stands between agents and that option is the set of human mental models Part IV audited: pre-web paradigms, defended now by habit rather than argument. The web that agents need is the web this book derived.