Technology, Systems Thinking
Agents Get the Glamour. Foundries Give Them the Firepower.
Agents get the demos; foundries quietly give them the firepower. What a foundry should decide for everyone, what it should leave to the builder, and why its capabilities set the limit of what can be built.
Insight
7 min
Since the age of metals, a foundry has been where tools are made. It is also where the cannon were cast.
The battles got the glory. The foundry got none of it, though it gave the army its firepower.
The agentic world looks much the same today. The agents are the glamorous part: they plan, they act, they get the demos. Behind them, quietly, foundries give them the firepower to do it.
What they cast today is not metal. It is systems of decisions, made faster every day.
So the question is no longer what a foundry can build. It is what it should decide, and what it should leave to you.
It is the question an anthill taught me long before I met enterprise architecture or AI: what are we really trying to achieve, and what must be preserved?
What a foundry is
An AI foundry means different things to different players. This is the definition I work with.
It is a centralised ecosystem that bundles foundation models, orchestration, data grounding, security and governance in one place. It lets builders create and scale agents, agentic workflows and systems without thinking too much about the underlying infrastructure.
That definition has held up. It has been about three years for me, from visualising a foundry to building production-grade agentic systems, and it began before vibe coding. New practices such as AI-DLC, the AI-driven development lifecycle, have changed how every workflow, agent and agentic system gets built.
What has not changed is that all of them need the right model, the right inference engine and the right reasoning capability. Underneath sits a world of compute and hardware choices. Many builders are unaware of it, or have never specialised in it.
That is where a foundry earns its keep.
This essay stays with enterprise software agents. Physical AI and edge AI is a different essay.
The week I lost
I was building a RAG pipeline. I chose Rust, because I come from a C background and have always loved its memory management and clean structure. I started with an open-weight Llama 3.1 8B running locally on Ollama. Guilty.
Then I hit a wall. There was no way to run the RAG workload on a local setup, so I moved to the cloud: Groq for generation, stepping up from the 8B to a 70B model, with Cohere for embeddings and reranking. The index was a flat file, with cosine similarity written in Rust.
As soon as I wrote the knowledge management layer, I saw errors in the workflow. The embeddings were not performing well, and the move to the cloud did not fix that. Every round of tuning the pipeline took days. About a week later I had a pipeline that worked, and worked efficiently.
A few weeks after that, our foundry was ready.
I could drag and drop workflows and assign a model to each step, with limits. I no longer had to think about where each model was hosted, which hardware to run it on, which services to wire together, or where the workload would execute.
The embedding problem did not disappear. What disappeared was the cost of trying again: simulating and experimenting became cheap and fast.
In my experience, a RAG pipeline built without a foundry takes about four times as long and costs three times as much, the extra going into tuning the outcomes and choosing the right embedder and reranker.
Who decides what
Looking back, I had made a long list of decisions alone. Hosting. Hardware. Performance targets. Business flow.
Some of them were rightly mine.
I set the performance targets for each RAG activity: reading, chunking, embedding and retrieving.
I decided the business flow: how the pipeline judged OCR confidence and language.
Some should never have been mine. Where the model is hosted, which GPUs it runs on, where the workload executes. Those are specialist decisions, and the foundry makes them once, for everyone.
And one sits above both. The intent behind any threshold comes from the business: the person accountable for the outcome of the process the system is part of. An HR head. A CFO.
That is the system of decisions a foundry makes: it decides what must be the same for everyone, and it preserves for the builder what only the builder can judge.
Intent comes from the business. Thresholds come from the builder, the forward deployed engineer. The foundry enforces the limits.
Limits, and the drift they guard against
The foundry lets you set the boundaries of an operation as limits. The list of limits has grown over the years: context size, speed thresholds and drift checks.
Take context size. The context budget is a hard limit the foundry applies to every builder, not a suggestion. In my experience, as models have evolved, that budget has only ever been revised down, never up. Models do not need more information, they need less. A tighter window keeps the model in the range where it operates well, and keeps its answers steadier.
The research points the same way: performance degrades as input grows, often well short of a model's stated maximum. See Lost in the Middle (Liu et al., 2024), Same Task, More Tokens (Levy et al., 2024) and Chroma's Context Rot study.
Drift is change over time, measured against a baseline. In my experience it is a recurring pattern in agentic workflows. In a RAG pipeline I think of it as three kinds of integrity.
Source integrity: are the sources and their chunks current, authoritative and traceable?
Retrieval integrity: as information is added or changed, does the index stay aligned with its sources, and does retrieval still surface the right evidence?
Response integrity: are the answers still grounded, relevant and correct? A grounded answer can still repeat an outdated source.
Each needs controls before an answer is released and monitoring after. Before: freshness and provenance checks, testing during the build through statistical tests, simulation runs and trace analysis, and validation at runtime. Better observability over the years has made deterioration easier to see. After: monitoring catches what testing misses and triggers correction. Retries and rephrased questions are useful signals, but weak evidence on their own.
One rule applies to all three: provenance. Chunks generated by the model are always held at a lower confidence, as an explicit policy and not something the model is trusted to do. If generated content re-enters the knowledge base unchecked, it can crowd out real source material, because the model can favour its own words over the evidence.
The research is mixed. A 2026 preprint, RAG Collapse, reports collapse in simulations when a system retrieves its own writing. An earlier study, LLMs are Biased Evaluators But Not Biased for Retrieval Augmented Generation, found no significant self-preference in its tests. The risk is real enough to set the rule.
What centralising costs
None of this is free.
Buying a foundry off the shelf is the easy decision. What you lose is the ability to enforce hard limits, such as a context budget, across every builder at once. In my experience, organisations that built their own have scaled meaningfully, because the team learns the foundry through the implementation and becomes an advocate for the use cases to be built.
But centralising costs adoption and speed. It also invites layers: when every threshold needs its own business owner team to set and approve it, governance turns into red tape and the system slows. And some parts of the organisation will not want to centralise at all. Sales and marketing do not run the same workflows as manufacturing, or at the same speed of delivery.
AI coding AI
I believe the foundry is what has made the new AI software factory real, where AI codes AI.
It is already visible. Anthropic has published that Claude authored over 80% of the code merged into its production codebases as of May 2026.
When I set out, the human building the agents was at the centre of the design. That principle held. What is interesting is that as model and harness capabilities evolved, the builder may not be a human at all, and the definition still holds.
When the builder is an agent, the thresholds belong to the business owner, the only one who can monitor and tune them directly. And that raises a question for another essay: how do we prepare them for it?
No matter how fast systems are designed, by humans, by agents or by a combination of both, the capabilities of the foundry define their limit.
CONTINUE EXPLORING
More writing on technology, AI economics, systems thinking, leadership, and mentoring.
CONNECT