Monday morning. Boot up five Claudes. By lunch I’ve diagnosed and merged PRs for two production issues, shipped updates across three projects, and started scoping a new feature.
Same Monday morning. Mrinank Sharma, the guy who led Anthropic’s Safeguards Research Team, posts his resignation letter. 8.8 million views. “The world is in peril.”
I kept shipping.
The Pattern
Sharma’s departure isn’t isolated. It’s the latest in a cycle that now spans every major AI lab:
- Anthropic: Safeguards Research lead resigns, warns of “interconnected crises” and internal pressure to “set aside what matters most”
- OpenAI: 11+ scientists and executives departed in 2025. Superalignment team dissolved after its leader quit. AGI Readiness team dissolved after its leader quit. Research on negative findings actively suppressed.
- xAI: Half of the 12 co-founders gone. Two left this week alone.
- Google DeepMind: 11 AI executives left in 2025, mostly to rivals
The cycle repeats: safety team built, leader quits publicly, team dissolved. At OpenAI, it happened twice.
— Jan Leike, former OpenAI safety leadSafety culture and processes have taken a backseat to shiny products.
The Trust You Don’t Think About
Let’s be clear: the code side is fine. We’re still reviewing PRs, still testing, still running multiple models and cross-checking output. Nobody’s blindly shipping Claude’s first draft to production. The developer workflow hasn’t gotten less rigorous. If anything, multi-model validation has made it more so.
The trust gap isn’t in your codebase. It’s upstream.
Sharma’s team wasn’t reviewing your pull requests. They were building defenses against AI-assisted bioterrorism. Studying sycophancy: models telling you what you want to hear instead of what’s true. Writing safety cases for capabilities that haven’t been released yet. The work that sits between the model and the stuff no amount of PR review catches.
That’s the part developers never think about. We think about capabilities. Context windows. Latency. Cost per token. The safety layer is invisible to us by design. And the people who built it are leaving.
The Dissonance
Here’s the part that’s hard to write honestly.
I’m more productive than I’ve ever been. Claude Code is genuinely transformational. I ship features in hours that used to take days. I’ve built products, tooling, entire systems faster than I thought possible.
And the people who were supposed to make sure this all stays safe are walking out, one by one, saying it can’t be done from inside.
Sharma wrote that employees “constantly face pressures to set aside what matters most.” Internal surveys showed Anthropic engineers worry they’re “coming to work every day to put myself out of a job.”
That last line hits different when you’re a developer using their output. Aren’t we doing the same thing? Accelerating our own displacement while enjoying the productivity boost? The difference is safety researchers were at least conscious of the tension. Most of us just ship.
— Mrinank Sharma, resignation letterWe appear to be approaching a threshold where our wisdom must grow in equal measure to our capacity to affect the world, lest we face the consequences.
What We Can’t See
One X user asked Sharma directly: “How long does the confidentiality agreement last? Is it for the rest of your life, like OpenAI?”
No answer. The letter was, as Futurism put it, “painfully devoid of specifics.”
This should concern developers most. These researchers have seen internal safety evaluations, red team results, capability thresholds that haven’t been made public. They can’t tell us what they know. They can only leave, and hint that things aren’t right.
When OpenAI’s Tom Cunningham left, he cited pressure between “conducting rigorous analysis and functioning as a de facto advocacy arm.” When Miles Brundage departed, OpenAI dissolved his entire team. The message to remaining safety staff: your work is expendable. Your concerns are inconvenient.
We’re building on platforms where the people with the most information about risk are systematically silenced or pushed out.
At OpenAI, the Superalignment team was dissolved after Sutskever and Leike left. The AGI Readiness team was dissolved after Brundage left. The message isn’t subtle: raise concerns, leave, and your team disappears behind you.
The Harder Problem Nobody Wants to Say Out Loud
Here’s the hot take: maybe the safety teams aren’t just losing a political battle. Maybe they’re losing a mathematical one.
I’ve written before about how LLMs are fundamentally stochastic, fallible, and unintelligible. I’ve argued for deterministic enforcement because prompts can be convinced otherwise. But the safety problem goes deeper than corporate politics or bad incentives.
Non-deterministic systems can’t be made provably safe. This isn’t opinion. Recent research has formalized it: the set of safe policies occupies zero volume in the space of all possible policies. Even at temperature zero, outputs show 5-12% instability across seeds. The alignment problem itself is formally undecidable via Rice’s theorem. And capabilities emerge unpredictably: training a model on one narrow task can cause broad misalignment in completely unrelated areas.
Traditional safety engineering works because bridges and aircraft are deterministic. You can model the load, test the stress points, prove the margins. LLMs don’t work like that. The same input produces different outputs. Capabilities appear without warning. And the training data you’d need for safety (abundant examples of rare catastrophic failures) is a logical contradiction.
This is what makes the safety team exodus different from, say, engineers leaving Boeing over MCAS. Boeing’s problem was knowable and solvable. The engineers who left could point to specific design failures with specific fixes. AI safety researchers are leaving because the problem might not have a solution within the current paradigm.
We’ve been living this at the code level for years. That’s why we don’t trust a single model’s output. We cross-check, we test, we use deterministic verification (CI, type systems, linters) to constrain non-deterministic generation. The safety teams are fighting the same battle upstream, except the stakes are bioweapons instead of bugs, and there’s no test suite for “did the model just help someone do something catastrophic.”
Is This Rational?
For most developers, this doesn’t change anything today.
The safety risks are abstract. Bioweapon synthesis. Existential AI risk. Societal disruption. These feel distant. The productivity gains are concrete and immediate. You can see them in your git log.
So we keep shipping. That’s probably rational in the narrow, local sense. The same way it was rational for individual developers to “move fast and break things” at Facebook. Rational for engineers to ship Boeing’s MCAS without adequate testing. Rational for bankers to package subprime mortgages.
The benefits are concentrated and visible. The risks are diffuse and invisible. Until they’re not.
Not all departures are principled. Sutskever left OpenAI and raised $1B for his own safety startup. Murati founded Thinking Machines Lab. Sharma is pursuing a poetry degree. Not everyone who quits in protest is a whistleblower. But when the pattern spans every major lab and the teams keep getting dissolved after each departure, the trend matters more than any individual exit.
What Developers Can Actually Do
I don’t have a clean answer. I’m not going to stop using Claude Code. You’re probably not either. But pretending this doesn’t matter is willful ignorance.
- Stop treating safety as someone else’s problem. Read the safety reports. Follow the departures. Understand what guardrails exist on the tools you depend on.
- Verify more than you trust. I’ve written about this before: trust the tool, verify the output. That applies to safety, not just code quality.
- Support transparency efforts. Current and former employees across OpenAI, DeepMind, and Anthropic signed a “Right to Warn” letter calling for whistleblower protections. California passed SB 53 establishing mandatory safety incident reporting. These matter.
- Be honest about the trade. You’re trading safety uncertainty for productivity gains. That’s a valid choice. Make it consciously, not by default.
The safety team left. We’re still shipping.
I don’t know if that makes us pragmatic or complicit. Probably both. The least we can do is stop pretending we haven’t noticed.
Related Posts
