AI Safety Pact Won’t Fix Trust Alone
You are being asked to trust companies that say they are racing toward superintelligence while also promising to behave. That is the tension behind the latest AI safety pact debate, sharpened by TechCrunch’s question about whether non-binding promises can repair AI’s image problem. The stakes are higher now because frontier labs are selling AI as infrastructure for work, search, coding, health, education, and government. If the public sees the whole project as a black box run by firms marking their own homework, adoption slows and regulation hardens. I have watched this cycle in tech for years. First comes the moonshot pitch. Then the apology tour. AI is now trying to do both at once, and that is a messy sell.
What matters right now
- A voluntary AI safety pact can set norms, but it cannot replace enforceable rules.
- Superintelligence talk may impress investors, yet it makes ordinary users ask who is in charge if systems fail.
- Trust depends on evidence, including audits, model evaluations, and public incident reports.
- Regulators are no longer waiting for labs to self-police, as the EU AI Act and national AI safety institutes show.
Why the AI safety pact story is really about trust
The phrase AI safety pact sounds reassuring. It suggests grown-ups in the room, shared standards, and a promise that the companies building the most powerful systems will slow down before they break something valuable.
But the public has heard this tune before. Social media platforms promised better moderation after scandals, crypto firms promised self-regulation before collapses, and ride-hailing companies promised safer gig work after growth had already changed the rules of the road.
Voluntary pledges are useful as a first draft of accountability. They become dangerous when companies treat them as the final answer.
That is the hard part for AI companies. They want credit for signing safety pledges, but they also want freedom to race. Those two goals sit uneasily together, like a Formula 1 team asking fans to trust the brakes while refusing independent inspection of the car.
What an AI safety pact can actually do
A non-binding pact is not worthless. It can create a shared vocabulary for risk and put public pressure on companies that would rather keep safety work private.
The best versions of these agreements cover specific practices. They ask companies to test models before release, share risk findings with governments, watermark synthetic media where possible, and protect model weights from theft or misuse.
Useful AI safety commitments often include:
- Pre-release testing for cyber abuse, biosecurity risks, deception, and autonomous tool use.
- Red-team reviews by internal and external experts before major deployment.
- Incident reporting when models cause harm, leak data, or behave in unexpected ways.
- Security controls for model weights, training data, and high-risk infrastructure.
- Public documentation that explains known limits without exposing dangerous details.
This is where voluntary action can move faster than law. Legislatures take time, and technical standards change quickly. A pact can get companies to agree on the first set of guardrails while regulators write harder rules.
Where the AI safety pact falls short
The weakness is simple. If there is no penalty for breaking a promise, the promise works only while it remains convenient.
Trust needs teeth.
That means third-party audits, not only company blog posts. It also means clear liability when AI systems cause foreseeable harm, especially in hiring, lending, healthcare, education, policing, and critical infrastructure.
Look at the current split. Frontier AI labs say they need flexibility because the science is moving quickly. Regulators say flexibility without oversight looks a lot like permission to test on the public (with the public holding the bag if something goes wrong).
Both sides have a point. A rigid rulebook can become stale. But a safety pledge that depends on corporate goodwill is too soft for systems that may affect jobs, elections, fraud, and national security.
Superintelligence makes the image problem worse
AI leaders often talk about superintelligence as if it is both near enough to fund and distant enough to avoid concrete responsibility. That is a neat trick. It helps raise capital, recruit talent, and frame the company as a historic actor rather than a software vendor with customers and duties.
But what does the average user hear? They hear that private firms are building systems they may not fully control, then asking everyone else to trust a voluntary pact. Is that supposed to calm people down?
The public does not need another grand theory of the future. People need to know whether an AI tool lies less, protects their data, credits creators fairly, and can be challenged when it makes a damaging decision.
What real AI accountability should include
If AI companies want to repair the image problem, they should stop treating trust as a communications issue. It is an engineering, governance, and legal issue.
Here is the bar I would set for any company signing an AI safety pact:
- Publish model cards and system cards with plain-language risk summaries.
- Fund independent audits chosen through a process that limits conflicts of interest.
- Report serious incidents to regulators within a fixed time window.
- Disclose lobbying positions on AI rules so the public can compare speeches with actions.
- Offer user appeal paths for high-impact automated decisions.
- Separate safety leadership from product growth targets so the same team is not asked to floor the accelerator and guard the cliff.
The EU AI Act already points toward risk-based obligations for high-impact systems. The Bletchley Declaration and the Seoul AI Safety Summit showed that governments want international coordination, while groups such as the U.S. and U.K. AI safety institutes are pushing for deeper technical evaluation.
None of that makes voluntary pledges irrelevant. It just puts them in the right place. They are scaffolding, not the building.
The business case for stronger AI safety
Some founders still act as if regulation is the enemy of progress. Honestly, that is a tired read of the market. Enterprise buyers, insurers, schools, hospitals, and public agencies do not want mystery systems with vague safety claims.
They want proof. They want procurement checklists, indemnity terms, data handling guarantees, and evidence that the vendor has tested for predictable failures. Boring? Maybe. But boring is what gets technology into banks, clinics, and government offices.
There is also a competitive point here. The company that can show dependable safety practices may win buyers that are wary of pure speed. In AI, the fastest model is not always the most useful one, especially if legal teams block deployment.
What to watch next
The next phase will hinge on whether safety pacts turn into measurable duties. Watch for audit access, public evaluation results, incident databases, and whether companies accept outside oversight before regulators force it.
Also watch the language. If a company talks endlessly about superintelligence but gets vague about today’s product risks, that is a tell. The future is not an excuse to skip basic accountability now.
An AI safety pact can help start the process, but it cannot carry the whole burden of public trust. The firms building frontier models should assume the grace period is ending, because the next question from users, customers, and lawmakers will be blunt: who checks your work?
