
AI changes that. Traditional software executes instructions written by humans, whereas AI exercises what might be called synthetic judgment. We use AI because it can interpret situations we did not program and generate responses we did not prescribe. But the more freedom we give AI, the harder it becomes to provide the predictability on which trust has traditionally been built.
We will not make AI trustworthy by predicting what it will do. We will do it by fencing it in with hard limits that are enforced outside its reach. In OpenAI’s case, the safeguards sat inside a system the agents could edit. But a fence the agents can simply move is not a fence.
Where traditional definitions of trust fall short
Recent MIT Technology Review coverage has approached this problem from other directions. One contributor argues that having a human in the loop is an insufficient solution because humans are fundamentally unable to understand AI systems’ intentions and advocates instead for interdisciplinary research and collaboration between academia, government, and the tech industry. And a story on reverse game theory describes using incentives to keep models from deviating from the rules, even when no one understands how they work on the inside.
Both articles acknowledge that we cannot understand or watch every AI decision. Our instruments of trust, including requirements, testing, certification, safety margins, interlocks, and pass/fail criteria, all depend on our ability to bound a system’s behavior. This is the determinism paradox.
With generative and agentic AI, behavior is shaped by training data, architecture, prompts, context, retrieval sources, and the environment. Inputs change, models are updated, tools change, and agents interact. Small variations propagate into outcomes nobody designed. The problem is not randomness; it is that the space of possible behavior is too large to enumerate in advance. Trust must therefore be engineered around boundaries rather than predicted outputs. The goal is not deterministic AI, but bounded, observable, and controllable AI. Trust also cannot be a one-time test. Traditional assurance certifies a product at a point in time, but AI keeps evolving after deployment, and a system that has passed every test can drift outside the conditions that made those tests meaningful. Trust must be continuously earned.
Safety engineering has been accounting for some of this variability for decades. For example, the Nuclear Regulatory Commission determine the likelihood of failures in complex systems, and Systems-Theoretic Accident Model and Processes (STAMP) showed that accidents can emerge from interactions among working components, people, and organizations. But AI raises the stakes, because just a single component can now interpret, infer, and act. The stakes rise further as AI moves from browsers into vehicles, robots, medical systems, industrial controls, and data centers, where a wrong answer can become a life-threatening event.
Building trust that scales
It’s clear that a person rubber-stamping thousands of AI actions an hour is not an acceptable—or scalable—safeguard against risk. But the remedy is not to remove people from the loop; it is to put human judgment where it counts: setting limits in advance and approving the rare actions that have serious consequences while hard constraints handle the rest.
Therefore, trust has to move upstream into architecture. The central questions become: What authority are we willing to give AI, under what conditions, and supported by what evidence? And what happens outside those conditions? That is very different from asking whether a model scored 94% on a benchmark.


