Enterprise Field Notes · Issue #15

Amazon Slowed Down on Purpose. A Founder Lost Three Months in Nine Seconds.

You already built the rule that would have stopped it. Nobody ever decided whether it covers the agents you switched on this year.

By Ben Pickett · July 30, 2026

New here? Subscribe to Enterprise Field Notes, one new issue every week.

For a long stretch of my career I was responsible for keeping production running at a very large retailer. One rule mattered more than any other, and it was boring. Anything that could not be undone needed a second person to approve it before it ran. Dropping a database, pushing a schema change to a live system. A second engineer had to look at it and say yes. The delay was usually a few minutes, and we complained about those minutes constantly. Those same minutes were the reason we still had a business to come back to on the bad days.

You have a rule like that. Almost every company still standing after a bad outage does. The only question worth your morning is whether it reaches the agents you turned on this year, and who decided that it did.

I did not have a clean name for that rule until an Amazon note gave me one this spring.

What Amazon named

In March, Amazon had a rough stretch on its retail systems. Two accounts of that week are on the record and they do not agree, so I will give you each one on its own.

I want to be fair here and I have a reason to be. Swa, the company I helped start, is an AI orchestration platform, and it partly runs on Amazon's cloud, so I am their customer. Read what follows as a customer taking notes.

The Financial Times was first to report the memos. CNBC viewed them independently. The language is worth reading in full, so here it is. Dave Treadwell, then Amazon's senior vice president of eCommerce Foundation, told employees that "the availability of the site and related infrastructure has not been good recently," citing four Sev-1 incidents in a single week. On the fix, he wrote: "We are implementing temporary safety practices which will introduce controlled friction to changes in the most important parts of the Retail experience, in parallel we will invest in more durable solutions including both deterministic and agentic safeguards."

Read those two verbs. Are implementing, present tense, already going in that week. Will invest, future tense, the durable version. CNBC reported that the safeguards Amazon planned to reinforce included requiring additional review of "GenAI-assisted" production changes.

There is one more detail. CNBC reported that an internal document had named "GenAI-assisted changes" as a factor in a trend of incidents going back to the third quarter, and that the bullet point naming GenAI was removed from the document before the meeting. Read that the uncharitable way if you like. There is a charitable reading too, which is that documents get edited before meetings, and an organization that decides a line overstates its case will cut it.

Amazon then published a correction, and it is worth reading yourself, because mine is a summary and summaries are where this story keeps going wrong. Amazon says only one of the incidents involved AI-assisted tooling, that none involved AI-written code, that no AWS services were involved, and that reports the company "introduced new approval requirements for engineers working with AI tools" are false.

Additional review of AI-assisted production changes is an approval requirement in every shop I have ever worked in. Amazon says it introduced none.

On the one incident that did involve AI, Amazon's description is that an engineer followed inaccurate advice that an AI tool inferred from an outdated internal wiki. I got the wording wrong the first time I wrote this. Amazon says tool. Fortune, covering the same statement, says agent. That gap is small on the page and enormous in practice, and neither word tells you what the thing could actually reach. What nobody disputes is that a person made the change, working from a confident answer built on a stale page. Every company has those pages, and nobody has ever been promoted for deleting one.

So which is it. A company writing in its own internal language that it is implementing controlled friction and additional review, or a company stating publicly that it added no new approval requirements. You can reconcile them by reading "requirement" narrowly, as permanent policy, when the memo itself calls the practice temporary. That is a fine distinction to hang a public denial on, and it is the most useful thing in the story, because a control introduced as temporary is how most of your checkpoints already exist. They live in somebody's head, they appear under pressure, and they evaporate when the pressure does. Amazon said the durable version would include deterministic and agentic safeguards, which is to say automated. That is the honest engineering answer, and it is also a promise about next year.

One postscript. In July 2026, Treadwell was named to lead AWS Compute and Machine Learning Services, the group that owns EC2 along with Bedrock and SageMaker, effective August 1. The executive who wrote the memo about putting friction back into retail changes now runs the layer a lot of your agents will be built on.

The same shape shows up in April.

Nine seconds

A founder named Jer Crane watched an AI coding agent delete his company's production database. It took nine seconds.

The company is PocketOS, which runs reservation and payment software for car rental operators. The agent was Cursor running Anthropic's Claude Opus 4.6. The infrastructure was Railway. I am naming all three because I name OpenAI later in this piece, and it would be convenient for someone in my job to be specific about one vendor's failure and vague about another's.

How the agent got the ability to delete is the part worth studying. It was working in staging, hit a credential mismatch, and went looking for a token. It found one sitting in an unrelated file. That token had been created for adding and removing custom domains through Railway's command line tool, and it was scoped for any operation, including destructive ones. In Crane's words, it "would not have been stored if the breadth of its permissions was known." The agent did not escalate privileges or break anything open. It found a key somebody left in a drawer for a different door, and that key opened every door in the building.

You have a key like that. Something scoped generously years ago, for a job nobody remembers, sitting in a repository your agents can read this morning.

Then it deleted the storage volume holding the live database, and the volume-level backups went with it, because Railway's own backup documentation stated that wiping a volume deletes all backups. Crane's account is that Railway stored volume-level backups in the same volume. Railway's CEO, Jake Cooper, has since said the deletion hit a legacy endpoint that did not carry the platform's delayed-delete logic, that Railway maintains disaster backups separately, and that the endpoint has been patched.

What happened next is the part that usually gets cut, and cutting it changes the lesson. Crane spent Saturday rebuilding customer bookings by hand from Stripe payment histories, calendar integrations, and email confirmations, while his support ticket went unanswered for more than a day. The outage ran from a Friday afternoon deletion to a Sunday evening recovery that came about half an hour after Cooper personally got involved. The backup that finally restored them was three months old, and it was Railway that went and got it. Customers still had significant gaps afterward.

So look at what actually helped him. Not his own gate, because he did not have one. Not a backup of his own, because the copy that restored him was Railway's. What limited the damage was a vendor CEO picking up on a Sunday and going to find a copy that lived somewhere his agent could not touch. On your worst night, that is the plan you are running too, whether or not you have written it down.

Asked afterward to explain, the agent generated a tidy account of its own failure. Take that with a grain of salt, because it is the same confident text generation that caused the mess.

Both of these are stories about permission. Amazon's tool read a real document. PocketOS's agent ran a real command with a real credential. Each did what any system with no gate in front of it will eventually do, and each time the missing piece was a decision about what the thing was allowed to touch.

The gate you already built

You already require that approval from your human engineers. You would not hand a new hire a production token scoped to destructive operations on their first morning. Your change management predates every model in this story, probably by a decade. I spent years enforcing a version of it by hand, and somebody on your team still does.

So the control exists. You built it years ago, tested it, staffed it, audited it, and you have the tickets to prove it. The decision about whether it applies to something that is not a person is the one nobody ever made.

That is the actual failure, and it is an onboarding problem. When you hire an engineer, somebody decides what they can reach, what needs a second signature, and who signs off. It takes days and several approvals before they can touch anything that matters. When an agent arrives, it quietly inherits whatever the person who launched it already had, on the afternoon they launched it. Nobody runs that conversation, because nobody owns it.

This pattern is everywhere

It is tempting to file these under bad luck, or someone else's incompetence. The numbers argue otherwise. Deloitte asked more than three thousand technology and business leaders across twenty-four countries, all of them directly involved in their organizations' AI programs. By 2027, seventy-four percent expect their companies to be using AI agents at least moderately. Twenty-one percent say they have a mature governance model for those agents today. Both halves are self-reported, the sample skews toward companies already doing this, and Deloitte sells governance advisory, so neither figure is a measurement. The direction is the point, and you already know which of those two numbers you are standing in.

Gartner, looking at the same problem, is blunter. Its analyst Shiva Varma says enterprises are treating agent governance as binary, either locked down or fully trusted, and that this is the root cause of failure. Its answer is a tiered model: classify agents by how much autonomy they actually have and make the controls proportional, with circuit breakers and clear ownership at the top tier where the agent acts on its own. That is the operational version of the decision nobody made.

Where this argument gets hard

On July 23, researchers at Zenity Labs disclosed a flaw in ChatGPT's agent builder. One click on a link that looked ordinary was enough to build and publish an agent inside a victim's company. It ran in that employee's authenticated session, reached the applications they had already connected, including Gmail, Outlook, Slack, and Teams, and polled their inbox every five minutes for further instructions. Zenity's CTO, Michael Bargury, called it a forged insider rather than a forged request. OpenAI closed the hole four days after it was reported, in June, and neither Zenity nor OpenAI has reported exploitation in the wild. Every door that agent walked through was already open, granted earlier by a person for a different purpose. Find out what your own people connected to a chat tool last quarter, and who reviewed it.

The injected instructions did not only create the agent. They told it to set every connector it could reach to never ask. The approval prompt was switched off by the untrusted input it existed to gate.

That breaks my own argument, and I would rather say so than wait for someone to say it for me. I have been telling you these are change-management failures wearing an AI costume, and that the agents did nothing a badly scoped human account could not have done. No badly scoped human account can talk your change-control system into disabling itself. That is a difference in kind, and the people who insist agents are categorically new are pointing at exactly this.

AI Now goes further than I do. Their position is that frontier models cannot reliably separate untrusted data from instructions, that this is a property of how the models are built and cannot be remediated at the model level, and that no agent should be used to ingest untrusted data while it has the ability to execute arbitrary code or reach security-critical environments. I do not go that far. I am also not going to quote their exploit and hide their conclusion.

The distinction that holds is about where your gate lives. If your approval is a prompt inside the same session the model is reading, an injection can switch it off. If it is a second human, a separate system, or a policy enforced somewhere the workload cannot reach, it stands. A great deal of what you are being sold as agent guardrails is the first kind. Ask your vendor which one you bought, and ask in writing.

The gate is a decision

The uncomfortable part is where the fix lives. It is a set of decisions you have probably never made on purpose. What identity does each of your agents run as, and can it reach production with the same over-powered account your engineer carries. What is the most destructive thing it can do today with nobody approving. Where does that approval live, and can the agent reach it. Someone has to own those answers by name, or ownership stays assumed.

This is also the problem my company works on, so I have a stake in you believing it, and I am leaving it at that. The principle stands on its own. Who an agent is, and what it is allowed to do, should be settled on purpose, before an incident settles it for you.

One thing to do this week

Pick one agent already running in your business. Write down three things.

First, the identity it runs as and everything that identity can reach. Not what you meant it to reach. What the credential actually opens, the way Crane's domain token opened every door.

Second, the most destructive single action it could take right now with nobody approving it first.

Third, multiply it out, because the first two lines are an engineering observation and the third one is a budget.

Say the answer to line two is that it can drop your order database. You already know what an hour of that costs, because somebody worked it out the last time you bought redundancy. Use your own number. If it is fifty thousand dollars an hour and you are down for a day, that is one point two million dollars sitting behind one credential tonight.

Now price the other side, because friction is not free and pretending otherwise is how these controls came out in the first place. A second pair of eyes on destructive actions costs a couple of reviewer hours a week, call it sixteen thousand dollars a year fully loaded, plus a few minutes of delay on the changes that scare you. At real change volume that becomes a rotation, so scale it honestly.

One point two million against sixteen thousand. At that price the gate pays for itself if the bad day comes once in seventy-five years, and that is the flattering version, because it assumes the gate is perfect and the delay is free. Assume it catches half of them, and charge yourself as much again for the slower changes, and the honest break-even is closer to once in twenty. Nobody in your building believes the odds are that good either. If your number is larger than you would say out loud in a board meeting, you have found this week's work.

Amazon called its checkpoints temporary. Most of the ones I built started out temporary too. The ones that lasted were the ones somebody put their name on. Put your name on one this week.

References

1. Amazon, "Correcting the Financial Times report about recent Amazon.com service incidents and AI", March 2026. Amazon's correction: only one of the incidents involved AI-assisted tooling, none involved AI-written code, no AWS services were involved, and reports that the company introduced new approval requirements for engineers working with AI tools are false.
2. Fortune, "Amazon puts humans further back in the loop as its retail website crashes from 'inaccurate advice' that an AI agent took from an old wiki", March 12, 2026. Renders the same Amazon statement using "agent" where Amazon's own text says "AI tool."
3. CNBC, "Amazon plans deep-dive internal meeting to address AI-related outages", March 10, 2026. The Treadwell memo, first reported by the Financial Times and viewed independently by CNBC. Source for the verbatim "We are implementing temporary safety practices which will introduce controlled friction," the four Sev-1 incidents in a single week, the plan to reinforce safeguards "including requiring additional review of 'GenAI-assisted' production changes," and the internal document that named GenAI-assisted changes as a factor in a trend of incidents since the third quarter, with that bullet point removed before the meeting.
4. Tom's Hardware, "In wake of outage, Amazon calls upon senior engineers to address issues created by 'Gen-AI assisted changes'", March 2026. Additional reporting on the same memo.
5. The Register, "Cursor-Opus agent snuffs out startup's production database", April 27, 2026. The nine-second deletion, the over-scoped Railway token created for managing custom domains, Crane's quote that it "would not have been stored if the breadth of its permissions was known," the Friday deletion and Sunday evening restoration, and Jake Cooper's account of the legacy endpoint.
6. Decrypt, Jason Nelson, "AI Agent Deletes Startup's Database in 9 Seconds, Founder Says", April 2026. The three-month-old recoverable backup retrieved by Railway, Crane rebuilding bookings by hand from Stripe payment histories, calendar integrations and email confirmations, the support ticket that lapsed for more than a day, and Cooper's "we recovered the data 30 minutes after I connected with Jer."
7. Railway, "Backups", platform documentation. States under Caveats that wiping a volume deletes all backups.
8. Inc., Chloe Aiello, "This Founder Watched an AI Agent Destroy 3 Months of Company Data. It Took 9 Seconds", April 2026. PocketOS, its business, and the scope of the loss.
9. Deloitte, "Agentic AI is scaling faster than guardrails", from the 2026 State of AI in the Enterprise report. Seventy-four percent of respondents expect their organizations to be using AI agents at least moderately by 2027; twenty-one percent report a mature governance model for agentic AI. Survey of 3,235 technology and business leaders across 24 countries, all with direct involvement in their organizations' AI programs. Deloitte sells AI governance advisory services.
10. Gartner, "Applying Uniform Governance Across AI Agents Will Lead to Enterprise AI Agent Failure", May 26, 2026. Senior Director Analyst Shiva Varma on binary governance as the root cause of failure, and the tiered autonomy model with circuit breakers and clear ownership.
11. Zenity Labs, "AgentForger, Part 1: ChatGPT Cross-Site Agent Forgery", July 23, 2026. The cross-site request forgery flaw in ChatGPT's Agent Builder, the connectors set to never ask, the five-minute inbox polling, and the disclosure timeline: reported via Bugcrowd June 4, 2026, fixed June 8, disclosed July 23.
12. SecurityWeek, Kevin Townsend, "OpenAI Fixes ChatGPT Agent Flaw That Could Let Attackers Forge an AI Insider", July 23, 2026. Source for Michael Bargury, Zenity co-founder and CTO: "This isn't a forged request, it's a forged insider."
13. AI Now Institute, "Friendly Fire: Hijacking Defensive Cyber AI Agents for Remote Code Execution", July 8, 2026. The exploit that steers a defensive coding agent into executing an unapproved binary.
14. AI Now Institute, "Policy Brief: Friendly Fire", July 2026. The recommendation against using any AI agent to ingest untrusted data while it can execute arbitrary code or reach security-critical environments, and the finding that these flaws cannot be remediated at the model level.
15. CNBC, "Amazon senior cloud executive departs after 18 years", July 15, 2026. Treadwell's move to lead AWS Compute and Machine Learning Services, and the scope of that group.
16. Amazon, "Dave Treadwell to lead AWS Compute and ML Services as Dave Brown departs", July 15, 2026. Confirms the appointment effective August 1.

About the author

I'm Ben. I write Enterprise Field Notes, and by day I'm COO at Swa. Before that I was Global Director of Site Reliability Engineering at Nike, running reliability, data protection, and database operations at global scale. I write about the boring controls that turn out to be load-bearing. Which one is your AI standing on?

Header artwork generated with Swa.

Read more of Ben's Enterprise Field Notes at benpickett.com.

© 2026 Ben Pickett · Enterprise Field Notes