From white box to black box
How AI is like every other complex technology
AI systems are increasingly opaque. But opacity is not unusual in mature technology. Cars, phones, doors, computers, and infrastructure all become useful by hiding their internal mechanisms behind interaction affordances. Cybernetics asks what kind of oversight architecture can make AI black boxes governable.
Image generated by GPT Image 2, 2026.
In a previous post, I argued that contemporary AI systems are best treated the way Ashby treated any sufficiently complex system: block boxes to be regulated from the outside rather than explained from the inside. That argument depends on a premise: that it is normal, even expected, for a mature technology to be opaque to its users. This post addresses that premise.
AI’s move from white box to black box
The history of AI is usually told as a move from symbolic systems to machine learning. Work in the mid-20th century predominantly focused on symbolic, rule-based systems connected to knowledge bases. Systems tended to leverage decision trees or other logic-based algorithms that were relatively straightforward and therefore easy to inspect. This made them, and the decisions they supported, inherently transparent. Of course it also made them useful only for very narrow problems.
The shift to statistical machine learning in the late-20th century, characterized by regression models, Bayesian networks, SVMs, small neural networks, etc., made AI systems less inspectable. Still, model results could be made interpretable by, for example, exposing a relatively small set of feature weights, coefficients, or conditional probabilities using methods that were relatively easy for end users to understand and, in some cases, tweak themselves (Carter et al., 2011). The rise of deep learning led to the explainable AI techniques discussed in a previous post. However, SHAP, LIME, and their variants barely kept pace with the models they were meant to explain. A parallel effort focused on interpretable-by-design models (Rudin, 2019), but those approaches have remained uncompetitive with frontier systems.
All of these efforts were made moot in November 2022 with OpenAI’s release of ChatGPT. Transformer-based AI architectures were introduced in 2017 (Vaswani et al., 2017), but until this moment their long-term utility was unclear. Now they exceed all previous expectations for the intelligence of AI systems, making past gold-standard tests like the Turing test seem quaint. The cost of this great leap forward was any hope of conveying to end users the complete mechanics behind a particular result. Even experts struggle to understand how the systems work. Indeed, most research is advanced by trying a wide range of ideas and determining which is best purely through experimentation (Kamath et al., 2024).
This would sound familiar to Ashby, Walter, and other cyberneticians. But it would also sound familiar to UX professionals.
Abstraction as design
In The Design of Everyday Things, Don Norman makes the point that usable products often hide their functional mechanisms. A door handle is a black box not because its designer failed but because a user who had to think about the latch mechanism could never simply walk through the door (because, to borrow from Heidegger, it would make the latch present-at-hand),. Abstraction is not a luxury of mature design; it is a requirement for functional systems. The designer’s job is to choose which level of abstraction the user will operate at, and to make that level legible through affordances and feedback. What lies beneath is, out of necessity, sealed off.
The cybernetic black box is a description of what the system looks like from the outside; the UX black box explains why a good designer puts it there. The user experience of the black box is a deliberate, and necessary, abstraction.
All sufficiently complex products follow a similar trajectory. Early-20th-century cars were operated through levers, rods, and cables directly connected to the subsystems they controlled. Driving was, in a literal sense, mechanical: the user’s hands and feet were coupled to the machinery, and the machinery’s behavior was revealed through its use. Over the following century, layer after layer of mediation accumulated, from hydraulics to electronics, embedded software, sensor fusion, driver-assist, and now full autonomy. The modern driver operates a car almost entirely through abstractions, expressing intent and supervisory control with essentially no real knowledge of how the underlying subsystems work. The car has gone from white to black, passing through every shade of gray along the way. Phones, computers, and consumer electronics generally have done the same.
AI has gone through a particularly fast instance of the same pattern. In thirty years it has moved from inspectable rule systems to functionally unreachable transformer architectures. While the speed with which this has happened is surprising, the fact that it has happened is not surprising at all.
Accountability after opacity
This raises a question: if AI is following the same trajectory as every other complex technology, why does it provoke special anxiety? In one sense, it doesn’t: technological progression regularly produces panic. But the specific concern with AI is that it influences decisions that put human lives at risk. Even that, though, can’t be the whole story; everyday life is saturated with consequential risk. Injuries and violence account for roughly 4.4 million deaths worldwide each year, and the technologies that produce them are also black boxes to their users. In fact, we built regulatory regimes around those technologies because they are black-boxed: crash standards, recall mechanisms, mandatory recorders, licensing, liability frameworks. Opacity did not exempt these technologies from accountability.
The pattern is already repeating. New York’s RAISE Act (signed in 2025) built on California’s SB 53 and helped establish an early U.S. state reporting and safety regime for frontier model developers. The US administration’s EO 14365 (signed in December 2025) is designed to preempt a patchwork of state laws under a federal framework. As of this writing, the administration is negotiating a further executive order establishing a voluntary 90-day federal security review of frontier models before public release, a kind of pre-market evaluation process familiar to the pharmaceutical and aviation industries.
Here cybernetics and UX converge: if AI systems cannot be made transparent, then they must be wrapped in the controls appropriate to opacity: monitoring, intervention, redundancy, and feedback. This could be, as Melanie Mitchell has argued, a job for experimental tools that evaluate model behavior rather than dashboards that attempt to summarize it. It is also, as the regulatory record makes clear, a job that institutions tend to figure out the hard way, over decades, after the technology has already gone black.
We should expect AI to continue down this regulatory pathway. The gray box is not a stable category, but rather a phase every technology passes through on its way to becoming infrastructure. The goal shouldn’t be to try to force the black box back open, but rather to build the requisite regulatory and operational architecture around it.