Research applications as patterns

Research applications are designed with exploration in mind. In HCAI, we use applications to probe the world to help us understand how different well-understood behavioral mechanisms play out in situ.

Storybook is one such application. Storybook uses generative AI to transform user stories into illustrated narratives with consistent character representation. People describe themselves and their situation in their own words, design a personalized avatar, and produce a visual story that advice givers then read and respond to. The system can be tuned for different deployments via a human-legible configuration file. Researchers can control dozens of aspects of the system, including how avatars are created, what AI models are used, how stories are structured, the types of responses that are allowed, etc. To create a new deployment, researchers modified an extensive configuration file and enabled a new route on an existing installation of Storybook, or cloned the code and launched their own installation from scratch. This configuration method was well established, but limited the tunable components to those that the developers had enabled.

Like so much that has changed since the fall of 2025, our approach to launching Storybook deployments completely shifted. Now, rather than a relatively stable code base with parameterized deployments, the codebase and deployment configuration files serve as ingredients to an agentic coding system. A researcher baking a new Storybook deployment mixes the old coding patterns with their new requirements as well as guidelines to incorporate the latest third-party models and techniques. The result is an outcome designed specifically for their use case.

Experience Sampling becomes a feature

Two EV charging research systems built by HCAI researchers can also help illustrate the impact of agentic coding.

ChargeMinder is a full research application for changing plug-in behavior among electrified vehicle owners. It gives drivers charging reminders, goals, streaks, feedback about their charging and driving behavior, and educational content. It also uses Smartcar, an API for connected vehicles, to observe driving and charging-related vehicle state. Because the intervention lives inside the app, researchers can control the surrounding experience: what the driver sees, when the prompt appears, what feedback follows, and how the behavior is represented over time.

Another project focused on vehicle-to-grid (V2G) participation. V2G systems depend on drivers making their cars available to the grid at useful times. In this project, researchers investigated whether people would choose to participate in V2G, and if so for how long and at what price.

The system used to run the V2G study leveraged Samply, an open-source experience-sampling platform with a researcher-facing server and a participant-facing mobile app. Samply let the team send prompts to participants in the field without building a full V2G application. Those prompts were combined with Smartcar, which exposed vehicle state and charging control, Qualtrics surveys, and backend services that linked participants, vehicles, prompts, and payments.

Rather than build a polished V2G app, the team in this case assembled a field system from other tools: Samply for experience-sampling, Smartcar for vehicle state and charging control, and Qualtrics for surveys. Backend services linked participants, vehicles, prompts, and payments.

These systems were two sides of a tradeoff between building a complete functional system versus using pre-existing tooling that required little building. ChargeMinder is a coherent system, but it requires the researchers to do everything that comes with building and deploying a bespoke app: design and development, QA testing, code reviews, configuring deployment infrastructures, etc. The V2G setup leveraged third-party tools and was therefore faster–most of the build and deployment concerns were already complete, but it had a less tailored participant experience. One treated the intervention as part of a product. The other treated the intervention as a field probe.

Agentic coding offers a way to reject this tradeoff by converting field assessment methodologies into a feature. Once bespoke research apps can be generated and revised quickly, ESM does not need to remain outside the system. Diary prompts, event-contingent questions, check-ins, follow-ups, and post-hoc surveys can be built directly into the deployment.

The ability to tailor applications to meet not only the needs of users but also researchers on a per deployment basis has the potential to revolutionize field research. Small research teams can now build systems specifically tailored to the study at hand: polished enough to make the intervention real, flexible enough to keep learning from the field.

One caveat is that agents are strongest where training data is plentiful, such as web frontends and Python services, versus where it is relatively sparse, such as iOS Swift. The agentic advantage is lower for projects with complex requirements, such as protecting participant data or running unattended for months. Generating a variant is cheap; verifying one is still human work.

Maintenance

The obvious cost of variance is maintenance. If every deployment becomes its own codebase, researchers inherit a family of systems rather than one system with many settings.

One answer is to rely on increasingly capable agents to handle maintenance work directly: updating dependencies, running migrations, and keeping related deployments in sync.

But another answer is that the road metaphor may not fit. A road is built for permanence. Many research systems are not. They may need to exist only long enough to answer a question, reveal a failure case, support a field study, or suggest a path to tech transfer. Others may branch into a family of related systems, each adapted to a different site, population, or intervention.

Agentic coding is a paradigm shift for knowledge work. Perhaps it is not surprising that it should also challenge our intuitions about research methodologies, systems, and contributions.

The title is a nod to an earlier HCAI paper.

From perspective-taking to agent design

There are many places we could go from here. This chain-of-perspectives reasoning of course derives from Theory of Mind. This cognitive capacity is usually associated with mechanisms that act to prevent egocentrism and promote empathy and better communication (though it can have less prosocial impacts as well, which Liu Cixin famously explores with the concept of “chains-of-suspicion” in his book The Dark Forest).

But what struck me most about this problem and the ensuing logic of its solution was the unreality of both the mise en scene, the relentless perspective hopping, and the ruthless adherence to logic over common sense. Humans would likely not follow a many-hopped logic but instead dismiss concerns as soon as it became common sense to do so (e.g., once it was clear that everyone knows that there are blue eyed people, folks would shrug and move on with their lives). They would also (unless they are literally imprisoned) be unlikely to act exclusively as lone agents: they would share information (even if told not to do so, they would find a black market method).

This led me to worry that agents in AI system design may be as similarly "alien" in their thought processes as the islanders. And while we probably do want agents to use this kind of ruthless inhuman logic in some cases, we also want them to be properly empathetic with humans and understand that they are likely not to do so. We may also want them to share information rather than hoard it.

When he derived the Viable System Model (VSM), Stafford Beer recognized that recursive coordination requires recursive viability: each subsystem must itself be a viable system, with its own regulators and feedback loops. He was explicit that the VSM is not a command hierarchy but a recursive containment structure with bidirectional flows, control distributed across levels rather than concentrated at the top. This model stands in contrast to the way that many agentic systems are designed now: a rather rigid hierarchy in which an orchestrator manages sub-agents. We might instead envision architectures that give agents more agency at each level of control, and include shared spaces to push information up, down, and across the agent graph. This mechanism would also allow agents to regularly update their shared understanding of the peculiarities of the people or communities they are modeling or interacting with.

This is the space stigmergic blackboard architectures are exploring (the term stigmergy comes from biology: the study of how individual agents [e.g., ants] modify their environment with traces to coordinate with other agents). Salemi et al. (2025) take aim at the master-slave paradigm in current agent frameworks in which subordinate agents are forced to execute instructions from a coordinator even when they hold better information, and a central controller required to maintain accurate models of every agent's capabilities, which is rarely realistic when agents have only partial or shifting knowledge. Their alternative is to have the coordinator post a request on a shared blackboard, with agents self-selecting based on what they can contribute. They report improvements of 13–57% on their data discovery benchmarks over orchestrator baselines.

Han and Zhang (2025) generalize the pattern, building a control unit, a blackboard split into public and private spaces, and a roster of agents with roles like planner, decider, critic, cleaner, and conflict-resolver, all communicating through the shared workspace. Gao et al.’s AutoScientists agents are related but more decentralized, interpreting a shared experimental state, self-organizing into teams around promising hypotheses, critiquing proposals, and running experiments while sharing both successes and failures so the system as a whole avoids redundant exploration. Similarly, the open-source Stigmergic Blackboard Protocol (SBP) is a coordination mechanism for AI agents that work together with two layers: ephemeral "pheromones" that decay over time, and durable traces that persist as institutional memory. A related line of work on reflective modeling, in which agents watch other agents (see Wang et al. 2024 and Storf et al. 2026), fits alongside the shared blackboard pattern since a shared workspace gives these monitoring agents something to watch.

Takeaway

Stigmergic blackboard architectures show how we can leverage some of the ways humans coordinate to design systems that have more regulatory feedback, share more knowledge to deepen understanding, and ultimately help agents act in a way that is overall more humane.

That said, I do think the main takeaway of this article is: never read logic puzzles when you are trying to go to sleep.

AI Reporter

If Requisite Variety only hosted written content, we could’ve used a third-party platform like Substack. But we want this site to be a “conversation with materials”, where the material here is web-based interactive technology. As such, we firmly believe that variety means variety of media. After all, if the medium is the message, and our goal is variety, then it stands to reason that we need a variety of media!

We deliver this variety through a set of agents that run regularly to update the site. One such agent is AI Reporter, which every week gathers candidate research material from configured sources, chooses a human-AI topic, writes a short article or Jupyter Notebook-style post, creates supporting media when needed, and formats the result as a Quarto page.

Importantly, the AI Reporter makes its own decisions about what to write or build. It also cycles through different models to use to do the writing or code building. Variety at every turn!

We are not convinced that the pieces AI Reporter produces will always be good (in fact, we suspect the hit rate to be pretty low), but its posts will give us insight into how well different LLMs can choose topics, build arguments, and communicate a point-of-view.

AI Artist

“I think of art, at its most significant, as a DEW line, a Distant Early Warning system that can always be relied on to tell the old culture what is beginning to happen to it.”

— M. McLuhan

“Art is how a culture feels the future before it can explain it.”

— Codex 5.5 extra high

Ted Chiang says that AI isn’t going to make art. AI Artist puts that thesis to the test.

AI Artist is an agent that generates a new piece of dynamic web art on a daily cadence. It reads a creative guideline, asks an LLM to produce a self-contained HTML/JS artwork, archives the result by date, and updates the current published artwork. The agent can select from a variety of tools to generate art, including different LLMs and selected media creation models available through platforms such as fal.ai. Its harness prompts it to consider different styles of art. It also explicitly tries to combine conventions and styles or generate entirely novel styles on its own. (It creates pieces in a portrait orientation so that they can be shown on a display in the TRI lab space with a similar orientation.)

The agent also keeps track of what it has made in the past so that it doesn’t get repetitive. Human curators regularly push particularly impressive creations to a Hall of Fame list; each Wednesday the AI Artist serves up a Hall of Fame entry, advancing one slot through the list each week, rather than creating a new piece.

Artworks generated by AI Artist and selected by human curators for the Hall of Fame.

In his piece, Chiang writes that, “An artist—whether working digitally or with paint—implicitly makes far more decisions during the process of making a painting than would fit into a text prompt of a few hundred words.” We agree, which is why the “text prompt” is only a small aspect of AI Artist: the agent also reasons through stylistic choices and experiments with dynamic DOM elements, video, image editing, and other media.

Another common critique of AI-based creative tools is that they replace work that humans would have done, de-skilling people and reducing their agency. But that critique is zero sum and not relevant: The fact that AI Reporter creates a new post does not prevent humans from writing their own posts (like this one!). AI Artist is on even stronger ground here: no one in our group (indeed, very few people ever) has been doing this kind of experimentation with dynamic, web-based art.

AI Critics

What gets measured gets better! Separate AI critics evaluate both AI Reporter and AI Artist and write feedback to files that both agents read when creating new content. Look for clickable scales of justice (⚖️) icons to see the critic’s feedback. Of course, measurement has its problems, so the agents take the critics’ feedback with a grain of salt.

Benchmarking Tool

Model benchmarks are vital information for any user-facing system builder. They also guide model development.

Our AI benchmarking tool is another independent agent. It gathers publicly available results from regularly updated benchmark leaderboards that researchers on our team have found relevant to human-centered concerns. The tool normalizes the top results into a shared format, merges new results with the previous data, and writes the updated benchmark data file back into the Requisite Variety repo.

So far, all benchmarks are third-party, but we are hard at work building our own benchmarks to better understand how best to integrate AI into our workaday tasks.

The first of these will be a Diplomacy benchmark. Diplomacy is a game of strategy and negotiation. It is often used in studies of human behavior to study bounded rationality, cognitive limitations under time pressure, and trust dynamics. Diplomacy rewards negotiation and alliance-building as much as tactics, which makes it a way to observe agent behavior. We plan to score several capabilities, including strategy, social inference, negotiation, coordination, resilience, and execution, from replayed critical game states, and to publish the formula, uncertainty, and source evidence behind every number. Look for this and other benchmarks in the coming months!

AI’s move from white box to black box

The history of AI is usually told as a move from symbolic systems to machine learning. Work in the mid-20th century predominantly focused on symbolic, rule-based systems connected to knowledge bases. Systems tended to leverage decision trees or other logic-based algorithms that were relatively straightforward and therefore easy to inspect. This made them, and the decisions they supported, inherently transparent. Of course it also made them useful only for very narrow problems.

The shift to statistical machine learning in the late-20th century, characterized by regression models, Bayesian networks, SVMs, small neural networks, etc., made AI systems less inspectable. Still, model results could be made interpretable by, for example, exposing a relatively small set of feature weights, coefficients, or conditional probabilities using methods that were relatively easy for end users to understand and, in some cases, tweak themselves (Carter et al., 2011). The rise of deep learning led to the explainable AI techniques discussed in a previous post. However, SHAP, LIME, and their variants barely kept pace with the models they were meant to explain. A parallel effort focused on interpretable-by-design models (Rudin, 2019), but those approaches have remained uncompetitive with frontier systems.

All of these efforts were made moot in November 2022 with OpenAI’s release of ChatGPT. Transformer-based AI architectures were introduced in 2017 (Vaswani et al., 2017), but until this moment their long-term utility was unclear. Now they exceed all previous expectations for the intelligence of AI systems, making past gold-standard tests like the Turing test seem quaint. The cost of this great leap forward was any hope of conveying to end users the complete mechanics behind a particular result. Even experts struggle to understand how the systems work. Indeed, most research is advanced by trying a wide range of ideas and determining which is best purely through experimentation (Kamath et al., 2024).

This would sound familiar to Ashby, Walter, and other cyberneticians. But it would also sound familiar to UX professionals.

Abstraction as design

In The Design of Everyday Things, Don Norman makes the point that usable products often hide their functional mechanisms. A door handle is a black box not because its designer failed but because a user who had to think about the latch mechanism could never simply walk through the door (because, to borrow from Heidegger, it would make the latch present-at-hand),. Abstraction is not a luxury of mature design; it is a requirement for functional systems. The designer’s job is to choose which level of abstraction the user will operate at, and to make that level legible through affordances and feedback. What lies beneath is, out of necessity, sealed off.

The cybernetic black box is a description of what the system looks like from the outside; the UX black box explains why a good designer puts it there. The user experience of the black box is a deliberate, and necessary, abstraction.

All sufficiently complex products follow a similar trajectory. Early-20th-century cars were operated through levers, rods, and cables directly connected to the subsystems they controlled. Driving was, in a literal sense, mechanical: the user’s hands and feet were coupled to the machinery, and the machinery’s behavior was revealed through its use. Over the following century, layer after layer of mediation accumulated, from hydraulics to electronics, embedded software, sensor fusion, driver-assist, and now full autonomy. The modern driver operates a car almost entirely through abstractions, expressing intent and supervisory control with essentially no real knowledge of how the underlying subsystems work. The car has gone from white to black, passing through every shade of gray along the way. Phones, computers, and consumer electronics generally have done the same.

AI has gone through a particularly fast instance of the same pattern. In thirty years it has moved from inspectable rule systems to functionally unreachable transformer architectures. While the speed with which this has happened is surprising, the fact that it has happened is not surprising at all.

Accountability after opacity

This raises a question: if AI is following the same trajectory as every other complex technology, why does it provoke special anxiety? In one sense, it doesn’t: technological progression regularly produces panic. But the specific concern with AI is that it influences decisions that put human lives at risk. Even that, though, can’t be the whole story; everyday life is saturated with consequential risk. Injuries and violence account for roughly 4.4 million deaths worldwide each year, and the technologies that produce them are also black boxes to their users. In fact, we built regulatory regimes around those technologies because they are black-boxed: crash standards, recall mechanisms, mandatory recorders, licensing, liability frameworks. Opacity did not exempt these technologies from accountability.

The pattern is already repeating. New York’s RAISE Act (signed in 2025) built on California’s SB 53 and helped establish an early U.S. state reporting and safety regime for frontier model developers. The US administration’s EO 14365 (signed in December 2025) is designed to preempt a patchwork of state laws under a federal framework. As of this writing, the administration is negotiating a further executive order establishing a voluntary 90-day federal security review of frontier models before public release, a kind of pre-market evaluation process familiar to the pharmaceutical and aviation industries.

Here cybernetics and UX converge: if AI systems cannot be made transparent, then they must be wrapped in the controls appropriate to opacity: monitoring, intervention, redundancy, and feedback. This could be, as Melanie Mitchell has argued, a job for experimental tools that evaluate model behavior rather than dashboards that attempt to summarize it. It is also, as the regulatory record makes clear, a job that institutions tend to figure out the hard way, over decades, after the technology has already gone black.

We should expect AI to continue down this regulatory pathway. The gray box is not a stable category, but rather a phase every technology passes through on its way to becoming infrastructure. The goal shouldn’t be to try to force the black box back open, but rather to build the requisite regulatory and operational architecture around it.

Explainability and its limits

In contemporary AI discourse, a “black box” refers to deep models with many hidden layers where inputs and outputs are known, but the internal decision-making is hidden, making the models hard to audit and hard to trust. Explainable AI (known as XAI, not to be confused with the AI company “xAI”) offers techniques, such as LIME (Ribeiro et al., 2016) and SHAP (Lundberg and Lee, 2017), to peek inside these boxes, revealing why a decision was made. These techniques are perceived as crucial for fairness, debugging, and compliance, especially in high-stakes areas like medicine or finance, even though explainability sometimes comes at the cost of peak performance. For example, a SHAP analysis might show how much each input feature (e.g., salary, credit score) contributes to a prediction (e.g., likelihood of repayment of a loan); and a LIME analysis might approximate aspects of a more complex model with simpler, more interpretable one (e.g., a linear regression model).

But this focus on explainability has begun to hit structural limits. Modern models keep growing in complexity, exceeding the reach of any post hoc rationale, while models with built-in interpretability (Rudin, 2019) are increasingly uncompetitive with state-of-the-art models. Meanwhile, most modern AI innovation is concentrated in commercial labs that are increasingly unforthcoming about the inner workings of their models (Stanford HAI Index). On the other hand, many chat systems that interact with foundation models now include “reasoning traces” feedback meant to help the user have some understanding of the tools, lines of thought, web pages, etc., that the system uses in deriving a response to a user prompt. However, this feedback typically amounts to short, rapidly cycling phrases that provide only the most cursory information about the true set of executions under-the-hood.

A new kind of black box

From a cybernetics perspective, the effort to fully explain these systems was always going to hit a ceiling. Arguably around 2023-24, frontier AI models crossed a threshold: they became classic black boxes, not just to their users, but in important respects to their builders. This is a stronger claim than Ashby’s. The door handle is opaque to the child but transparent to the engineer who made it; a frontier model can be opaque to everyone at once. After all, who can explain precisely how their phone, car, laptop, monitor, watch, etc., works, much less the far more complex and interacting systems that govern everyday infrastructure that supports the banal cadence of everyday life (e.g., power and water procurement and distribution)? Concerns focused on the inner workings of those systems are important, but only to the extent to which they impact relevant external (and monitorable) systems.

At first blush, this stance could appear worrisomely indifferent. What if, for example, these systems, executing some unknown algorithm, act in ways that are inappropriate when given some new input? In fact the cybernetic stance demands more caution, not less. Precisely because these systems are never going to be fully knowable from the inside, they must always be embedded in architectures of control from the outside. There must always be a regulatory system in place that mediates between a system’s outputs and the others it interfaces with.

This is, in fact, the direction production AI is already moving. Serious deployments rarely expose a single model directly to users. Instead, they nest models inside one another: a narrowly-scoped classifier polices the output of a more capable generator; a separate model checks for policy violations before anything reaches the interface; logs and rate-limits sit outside the whole stack. None of these guardrails require understanding why the underlying model produced what it produced. They only require monitoring what it produced and intervening when needed.

A complement to XAI

This is the cybernetic complement to the XAI program. The goal of XAI is to make the box transparent enough to trust. Cybernetics instead focuses on a regulator around the box so that its opacity doesn’t matter.

The irony may be that designers who give up on explanation will feel the need to build systems with more regulatory controls. When you cannot inspect the engine, you add instrumentation and sensors, design in redundancies, improve logging and feedback, etc. You define acceptable outputs and enforce them.

In the end, presuming less control may force better overall accountability.