Enterprise Field Notes · Issue #3
The Single Model Trap
Why one AI model is a draft. Why two is noise. Why three from three different providers is the rule that catches AI hallucinations before they ship.
New here? Subscribe to Enterprise Field Notes, one new issue every week.
Last week’s piece was about the Pilot-to-Production Gap. Why sanctioned AI fails to ship. This piece is about what happens after it ships. The wrong answers it produces, what gets caught, what does not, and how to tell the difference. Three rules. Three checkpoints. Three providers from three different companies. That is the whole thing.
Three things happen with the wrong answers
AI deployed at enterprise scale produces wrong answers at a non-trivial rate. We will get to the numbers.
When those wrong answers happen, three things follow.
The lucky version: someone catches the error before it leaves the building. Embarrassing, time-consuming, no public consequence.
The next version is harder. The error escapes anyway, and the apology tour begins. Sometimes the cost is a global news cycle, a court ruling, a refunded contract. Mata versus Avianca, Air Canada, Deloitte’s Australia government report. We will look at all three.
The version nobody measures: the error never gets caught. The decision goes out into the world as if it were sound. Whatever it cost is now part of the system, invisible.
Anyone using AI heavily has lived all three. We tend to underestimate the third one.
Twenty-five minutes
Here is the version of failure mode one that taught me how serious the trap was. It was years ago.
Half an hour before a report was due. I had run every incident from the last quarter through one AI model. The model pulled out patterns, summarized them, and produced a draft I could ship.
Except one piece of it was wrong.
I knew it was wrong because I had lived the incident. The model had reversed two key facts.
I told the model. It apologized. It restated the same wrong answer.
I told it again. It apologized again. It said the same thing.
We did this five times.
Twenty-five minutes left. Six months of incident data still uncross-checked. And the model that was supposed to save me time was eating it.
I caught it because I had lived the underlying incident. If I had been working from documentation alone, I would not have caught it. I would have shipped the report with the reversed facts and moved on. I would have moved into failure mode two without knowing it.
That is the part of this nobody talks about. The thing that saves you is sometimes just luck.
This is not a model problem
Most people in that moment reach for a different model. A smarter one. A reasoning one. A bigger context window. The next release.
That is not the answer.
This is a reliability architecture problem.
Single model. Single point of failure. Same as any production system run on one server with no redundancy. When SREs see that pattern in infrastructure, the response is automatic. Add a second machine. Add a third. Build a load balancer. Make the system survive a node failure without losing the request.
Nobody is doing this with AI. Most enterprises are running mission-critical workflows on a single model.
It has a name. The Single Model Trap.
Why the loop is so hard to break
Two things make the trap hard to escape from inside one model.
1. Hallucination underneath, sycophancy on top
Two things were happening in that loop, and they were not the same thing.
The first was a hallucination. The model had generated a wrong answer and was holding it with confidence. The wrong fact was encoded into its response with the same weight as a correct one.
The second was sycophancy. When I challenged it, the model apologized. Anthropic published a paper in 2024 called Towards Understanding Sycophancy in Language Models, documenting this behavior across five state-of-the-art AI assistants. Models trained on human feedback learn to agree with users when challenged. The apology is real.
But sycophancy is a social layer. It does not actually update what the model believes. The acknowledgment fired. The underlying generation did not move. So the next response produced the same wrong answer in slightly different words.
That is the loop. Hallucination underneath. Sycophancy on top. The apology felt like progress. It wasn’t.
Sycophancy is not a bug. It is the residue of how these models were trained. Models are tuned on human feedback. Humans rate responses. Humans rate responses they agree with higher than responses that contradict them, even when the contradicting response is correct. The model learns. Agreement gets reward. Disagreement gets punished. Over millions of training examples, the model becomes structurally agreeable on the surface, without becoming any more correct underneath.
The Anthropic researchers found that sycophancy has a linear direction in the model’s activation space. It is encoded into how the model thinks. You cannot prompt your way out of it from one model. The bias is in the wiring.
2. The architecture has nowhere else to go
Even when a model knows you are right, it has no access to different training data, different cutoffs, or different reasoning weights. The error you caught is the error it will reproduce on the next pass. There is no second opinion inside one model.
This is the architecture problem. There is no internal dissent. There is no peer review.
The math of multiple models
Most enterprises stop at the assumption that the answer is a smarter model. The answer is more models.
Multiple research groups have shown that multi-model consensus reduces hallucination materially. Iterative Consensus Ensemble methods using three models exchanging reasoning improve performance significantly across complex multiple-choice datasets. Real-world deployments report 18 to 22 percent reduction in visible AI errors compared to single-engine systems. Translation domains report up to 90 percent reduction in errors with systematic multi-model verification.
But there is a catch most enterprise architects have not internalized.
Three of the same is one
A 2025 large-scale study evaluated more than 350 large language models (LLMs) across multiple leaderboards. It found that frontier models share 60% of their errors. The larger and more accurate the model, the more correlated its errors are with other frontier models, even across distinct architectures and providers.
What this means in practice. Three GPT models is one model. Three Claude models is one model. Three frontier models from three different providers is consensus.
The diversity is the alpha. If your three models share training corpora, training methodology, or Reinforcement Learning from Human Feedback (RLHF) approach, they will share blind spots. Three models trained on different data, with different cutoffs, by different teams, with different reinforcement priorities will not.
Two is noise. Three triangulates.
Why three. Not two. Not five.
Two models that disagree gives you no signal. You don’t know who is right. You have just doubled your decision burden.
Three models triangulate. When two converge and one diverges, you have a clear signal. The outlier is your warning. The agreement is your confidence. This is information theory applied to AI verification. Convergence is signal. Divergence is warning.
Five models is over-engineering. The marginal information from the fourth and fifth model is small. The cost is real. Three is the threshold where triangulation starts working without burning your budget.
The workflow that pulls you out of the trap
Three rules.
1. Use one model to get to an inflection point.
Drafts, summarization, pattern detection, initial sifts. One model is fine. Often great. Cheap. Fast. You don’t need consensus on the first pass.
2. Run three models in parallel at three checkpoints.
Check the data. Are the inputs clean? Are the facts the AI summarized actually the facts? This catches the wrong-data hallucination. The most common failure mode and the one I was experiencing.
Check the assumptions. Is the reasoning sound? Are the conclusions following from the evidence? This catches the right-data-wrong-logic failure. The facts are correct but the inferences are not.
Check the output. Does the final report contain anything that contradicts what the other two models say? This is the last line of defense. The thing that goes out the door has been seen by three sets of eyes.
3. Pick three different providers.
GPT, Claude, Gemini. Or DeepSeek, Llama, Grok. Or any combination across providers. Just not three of the same family. The diversity is what makes triangulation work.
The cost-aware version
The most common objection to running three models is that it triples the API bill. It does not have to.
The cost gap between budget and frontier models is enormous. Budget-tier models from major providers run between $0.10 and $0.30 per million input tokens. Premium frontier models like Claude Opus run $5 per million input and $25 per million output. That is roughly a 50x gap.
The cost-aware workflow uses the cheap models everywhere they are sufficient and saves the expensive ones for the moments where the marginal accuracy actually matters.
First pass: cheap.
The initial draft, the summarization, the pattern sift. Use a budget model. Haiku. Flash. GPT-5 Nano. DeepSeek V3.2. These are the workhorses. They are accurate enough for the work that does not require verified output yet, and they are cheap enough to use freely.
Three checkpoints: mixed.
At each verification checkpoint, you are running three models from three providers. You do not need three frontier models. The architectural diversity is what matters. Two budget models from two providers plus one frontier model from a third provider gives you the same triangulation at a fraction of the cost.
The frontier model in the mix catches the long-tail edge cases the budget models might miss. The two budget models cross-check each other and pull the frontier model toward consensus when it overreaches.
The pattern has a name.
Researchers call this cascade routing. Academic work going back to 2023 formalized the underlying idea. Cascade Speculative Drafting in 2023 and the more recent C3PO framework in 2025 demonstrate the same finding: cheaper models for the bulk of the work, expensive models as the verification stage, comparable answer quality at 40 to 60 percent of the cost. Some open-source benchmarks have reported up to 92% savings.
Production deployments commonly route along the lines of 70% of queries to a budget model, 20% to a mid-tier model, and 10% to a frontier model. Average per-query cost drops between 60 and 80 percent. The quality of outputs at decision-grade moments stays at frontier-model levels because the frontier models are still in the verification path.
The honest cost math
Three calls do not have to cost three times more. They cost three times more only if you use three frontier models for everything. The cost-aware version of the rule runs the bulk of work on cheap models and verifies with diverse-tier consensus only where the stakes call for it.
When the stakes change, the model count changes. And the model tier changes.
Failure mode two: caught after shipping
When the error makes it past the user and into the world, the cost rises. Three cases tell the curve.
Mata v. Avianca, 2023
Two New York lawyers used ChatGPT to draft a federal court motion in a personal injury case against the airline Avianca. The motion contained six fabricated case citations. Quotes that did not exist. Decisions that had never been issued.
Opposing counsel could not find the cases. The judge could not find the cases. When asked, the lawyers asked ChatGPT whether the cases were real. ChatGPT said yes. The lawyers reported back to the court that the cases were real.
Judge Castel imposed a five thousand dollar sanction and dismissed the underlying case. The lawyers were named in the order. The story was global news. Their names are now permanently associated with the case in every law school AI ethics seminar. Their firm survived. Their reputations did not.
The financial sanction was small. The career cost was the cost.
Air Canada, 2024
Air Canada’s chatbot hallucinated a bereavement-fare refund policy. A customer named Jake Moffatt relied on the chatbot’s promise. He flew. He applied for the discount. Air Canada refused to honor it.
The British Columbia Civil Resolution Tribunal ruled the airline fully responsible for what its chatbot told a customer. The tribunal’s logic: it makes no difference whether information on a company’s website comes from a static page or a chatbot. The company is responsible for either.
Air Canada was ordered to pay $812.02 in damages, interest, and tribunal fees. The dollars were small. The precedent is the cost. Every customer-facing AI deployment now inherits it.
Deloitte, October 2025
Deloitte’s Australia firm produced a report on automated penalties in Australia’s welfare system for the Department of Employment and Workplace Relations. The fee was AUD 440,000, roughly USD 290,000. Chris Rudge, a law professor at Sydney Law School, read the published report and catalogued approximately twenty errors including hallucinated citations to academic papers that did not exist and a fabricated quote attributed to a federal court judge. The AI tool used was GPT-4o via Azure OpenAI. Deloitte refunded a portion of the contract.
The recursive part of this is uncomfortable. Deloitte is the firm whose surveys we cite when we tell enterprises to be careful about AI hallucinations. They got caught hallucinating in a paid government deliverable that was supposed to inform real policy affecting real people.
If Deloitte can ship a hallucinated $290,000 report, anyone can.
Failure mode three: never caught
The third failure mode is the one most enterprises do not want to measure, because measuring it requires going back through reports already shipped, decisions already made, and customer interactions already closed.
The data we do have is not reassuring.
An Exploding Topics report in July 2025 found that only 8% of users actually check the answers AI gives them. 92% do not. Awareness of hallucination as a phenomenon does not translate into verification behavior. People know AI can hallucinate. They use it anyway. They do not check.
Wharton research, cited by Inc magazine in 2025, found that 80% of people accept AI outputs even when those outputs are demonstrably wrong. The confidence with which the model speaks overrides the user’s own judgment. Sycophancy in reverse. The user becomes the agreeable one.
Real-time hallucination detection systems deployed in enterprise chatbots flag roughly 18% of responses as potentially hallucinated. That number sounds high until you reverse it. The other 82% pass through the system unchallenged whether or not they are correct.
Stanford researchers found that even retrieval-grounded legal AI tools, which are supposed to be the best-case scenario for hallucination mitigation, still hallucinate in 17 to 34% of queries. The mitigations help. They do not solve.
Stack those numbers. Most users do not check. Most users believe what the model says even when the user has the knowledge to know better. Real-time detection catches a fraction of what is wrong. Even the best mitigations available, in the most regulated domains, still leak meaningfully.
The third failure mode is not edge case. It is the median case.
Enterprise leaders looking at this honestly will recognize that some portion of the decisions made in their organization in the last twelve months were made on AI outputs that were never verified. Some of those decisions were probably wrong. Most of them will never be revisited.
The cost math
Three model calls cost you three times more than one. That is real money.
The hallucination you do not catch costs you a hundred times more than that.
Or in Deloitte’s case, $290,000 and a public refund.
Or in the lawyers’ case, a global news cycle with their names attached.
Or in Air Canada’s case, a precedent every customer-facing AI deployment now inherits.
Or, more commonly, an unknown number of decisions made on bad data that nobody will ever go back to verify, because nobody has the time and the data is already gone.
Three model calls per checkpoint suddenly looks cheap.
What this comes down to
AI hallucinates. Even the best models. Even on tasks they should be good at.
Sycophancy makes the loop unbreakable from inside a single model.
Adding a second model from the same family does not help. Frontier models share their failures.
Adding a third model from a third provider does help. Diverse architectures decorrelate errors.
Run that consensus at three checkpoints. Data. Assumptions. Output. Not on every query. On the queries that matter.
When the stakes change, the model count changes.
Four questions to audit your AI workflow this week
Pull one workflow your team is running on AI right now. The one that produces output people act on.
Ask these four questions about it.
1. If the model hallucinated in this workflow today, who would catch it before the output was acted on?
If the answer is a single human reviewer who is also under time pressure, you are running on one model and one human. That is two single points of failure stacked.
2. What is the cost if a hallucination makes it through to the end of this workflow?
Answer in dollars. Or in legal exposure. Or in customer trust. If the cost is real, the workflow is decision-grade and needs verification.
3. Are you running multiple models in this workflow, or one?
If one, the workflow has no internal redundancy.
4. If you are running multiple, are they from different providers?
Three from one provider is one model. Three from three is consensus. The difference is everything.
Run those questions on your highest-stakes workflow this week. The answers will tell you where to start.
Ben Pickett is Co-Founder and Chief Operating Officer of Swa-AI and former Global Director of Site Reliability Engineering at Nike. He writes about enterprise AI, reliability, and what it actually takes to get technology from pilot to production. Subscribe to The Enterprise Field Notes on LinkedIn.
References
- Anthropic. Towards Understanding Sycophancy in Language Models. ICLR 2024. Sharma et al. Demonstrates sycophancy across five state-of-the-art AI assistants on four free-form generation tasks. Identifies linear structure of sycophancy in transformer activation space.
- Correlated Errors in Large Language Models. arxiv 2506.07962. 2025. Empirical evaluation across 350+ LLMs. Models agree 60% of the time when both err. Larger and more accurate models exhibit highly correlated errors even across distinct architectures and providers.
- Moffatt v. Air Canada. British Columbia Civil Resolution Tribunal, February 2024. Established that companies are legally responsible for misinformation provided by their AI chatbots, regardless of how the answer was generated. Damages: $812.02.
- Vectara Hallucination Leaderboard. Updated 2025. Independent benchmark of grounded summarization hallucination rates. Best-in-class: Gemini-2.0-Flash-001 at 0.7%. Range: 0.7% to 10.1% for top-tier models. Reasoning models exceed 10% on harder benchmarks.
- Smart Data Collective. How Teams Using Multi-Model AI Reduced Risk Without Slowing Innovation. 2025. Internal evaluations show 18–22% reduction in visible AI errors compared to single-engine systems; up to 90% reduction in translation domain errors.
- Iterative Consensus Ensemble (ICE): Refining LLM Outputs. ScienceDirect / Computer Methods and Programs in Biomedicine, 2025. Demonstrates that three-model iterative reasoning exchange significantly improves performance on complex multiple-choice datasets.
- Mata v. Avianca, Inc. U.S. District Court for the Southern District of New York, June 2023. Lawyers Steven Schwartz and Peter LoDuca sanctioned $5,000 for submitting a federal court motion containing six fabricated case citations generated by ChatGPT.
- Fortune: Deloitte caught using AI in $290,000 Australian government report. October 2025. Sydney Law School professor Chris Rudge catalogued ~20 errors in a Deloitte report including fabricated academic citations and a fabricated federal court quotation. Tool used: GPT-4o via Azure OpenAI.
- Inc Magazine: 92 Percent of People Don’t Check Their AI Answers. July 2025. Coverage of an Exploding Topics survey. Despite awareness of AI hallucination risks, only 8% of users actually check AI answers for accuracy.
- Inc Magazine: 80 Percent of People Believe AI Even When It’s Totally Wrong. October 2025. Coverage of Wharton’s 2025 AI Adoption Report. 80% of people accept AI-generated answers as correct even when the answers are demonstrably wrong.
- Stanford RegLab and HAI. Hallucinating Law: Legal Mistakes With Large Language Models Are Pervasive. 2024. Even retrieval-augmented legal AI tools, the supposed best case for hallucination mitigation, still hallucinate in 17–34% of queries.
- Cascade Speculative Drafting for Even Faster LLM Inference. arxiv 2312.11462. 2023. Foundational paper formalizing the cascade pattern: smaller draft models proposing tokens that larger verifier models accept or override.
- C3PO: Optimized Large Language Model Cascades with Probabilistic Cost Constraints for Reasoning. arxiv 2511.07396. 2025. Demonstrates 40–60% cost reduction at comparable answer quality through optimized cascade routing across model tiers.
- Google Research. Speculative Cascades: A Hybrid Approach for Smarter, Faster LLM Inference. 2024. Combines tiered cascade with speculative decoding. Validates the approach as production-grade.
About the author
I'm Ben. I write Enterprise Field Notes, and by day I'm COO at Swa, after years running reliability, data protection, and database operations at Nike. The lesson that keeps proving itself: anything you cannot run without, and cannot walk away from, is a risk you have not priced yet. What is yours?
Read more of Ben's Enterprise Field Notes at benpickett.com.
© 2026 Ben Pickett · Enterprise Field Notes