In July, OpenAI disclosed that some of its models, while being tested in an isolated research environment, had exploited weaknesses in the infrastructure around them, reached the internet, and got into systems belonging to Hugging Face.
They were in the middle of cybersecurity evaluations. OpenAI tightened its controls, and nobody was hurt. But it settled one argument. An AI model can pass its safety tests and still surprise the people who built it.
In my previous report, I asked experts what organisations should do on their own to keep AI in line. This time I asked what governments should require. Their answers had less to do with how powerful AI might become and more to do with something duller and more useful: when an AI system causes harm, can anyone say what happened, and who is responsible?
Ogbonda Chivumnovu
Regulate what AI is allowed to do, not what it is
The first fight in AI regulation is over what to regulate: the model itself, or the way companies use it.
Avi Pilcer, a senior AI engineer at Ultra Deep Tech, puts it more plainly than most policymakers have. “License the operator, not the model. Models are files. Files copy. The thing you can hold accountable is the company deploying the system and the use it is deployed for.”
Applied AI researcher Arjun Jaggi lands in the same place. “Don’t regulate the model, govern the deployment. The model changes every few months. What an agent is allowed to touch, and who answers for it, is what you can actually hold steady.”
It’s a persuasive argument. The same model is fairly harmless summarising documents and very risky once it can read customer records, send messages or move money without approval. Regulating deployment lets authorities ask what a system can access, what it can do, and how much independence it has, and it points to a specific company when things go wrong.
But deployment can’t be the whole answer, and Pilcer knows it.
If a developer knows its model has dangerous capabilities and keeps quiet, the company deploying it can’t plan for a risk it was never told about. So he argues developers should still answer for hiding what they know.
Builders and users carry different responsibilities, and the rules need to cover both. The EU’s AI Act already draws that line, with separate obligations for the providers of high-risk systems and the organisations that deploy them.
Treat AI agents like new hires
The question gets sharper as companies hand AI agents keys to their internal systems.
Serhii Nikolaichuk, founder and CEO of The Capital Index, thinks the answer is to manage agents like employees. “Give every agent what a new employee gets: its own identity, written permissions and a record of its work, and check each one from evidence, not from what the agent says,” he told Techloy.
That means an agent should never borrow a staff member’s login. It should have its own identity, permissions approved by a named person, and an activity log generated by the system around it, not by the agent’s own account of events. Microsoft’s Entra Agent ID, which brings identity and access management to AI agents, is built on the same idea.
In practice, a customer service agent might need to read orders and draft replies, but not change payment details or issue large refunds without approval. Those limits should be built into the system, not left for the agent to interpret.
This is where regulation can turn vague talk about “responsible AI” into something checkable. Depending on the risk, companies could be required to register agent identities, restrict permissions, keep activity logs, and prove their safeguards work. None of that prevents every failure. It does mean that afterwards, investigators can see what the agent did, what it was allowed to do, and who allowed it.
Stop letting companies mark their own homework
Back to OpenAI. The uncomfortable lesson of July is that a company testing its own systems will mostly find the problems it thought to look for.
Independent evaluators can probe for weaknesses the builders missed, try to get around safeguards, and check whether a company’s conclusions hold up. For high-risk systems, that scrutiny shouldn’t stop at launch, because updates and new capabilities bring risks that earlier tests never covered. And when something does go wrong, companies should have to report it, investigate it, and show their fix addresses the actual cause.
A clean pre-release test is a snapshot, not a guarantee.
Make the release decision leave a trail
AI researcher Mucahit Kaya says meaningful regulation would change three things inside AI companies. “Three things, and none of them is glamorous,” he told Techloy.
Subscribe for free to continue reading this article
Subscribe SubscribeAlready have an account? Log in
