If you were to take somebody from the early 1990s and put them in front of modern AI products, they would have a strangely familiar experience. Information would return to them word by word, reminiscent of watching text arrive through an Internet Gopher client over a slow connection.

It is as if much of the progress in user interfaces for navigating information, beginning with Mosaic, developed by Marc Andreessen and Eric Bina, and later Netscape, co-founded by Andreessen and Jim Clark, had never happened.

What happened to the graphical Web?

That revolution, together with the earlier development of graphical operating systems, from Xerox PARC and the Xerox Star to Apple's original Macintosh, helped computing break out of niche applications and become accessible to ordinary users. It changed how humans interacted with computers, not simply what computers could do.

Yet much of the modern GenAI experience focuses on displaying and controlling information in a way that reflects the mechanics of the computation itself. The user types a request, and information is returned token by token as it is generated. We already know the limitations of this approach. Humans have a limited ability to process complex information as a stream of words, whether in text or audio.

That is why visual information display is so critical. It allows people not only to receive information, but also to review it, make sense of it, compare alternatives, and act on it. Without this review, human oversight and engagement with AI will be limited. Large volumes of generated text almost mandate skimming the content and ignoring portions of it rather than carefully reviewing it.

Streaming tokens are a technical delivery mechanism, not a human-interface design principle.

Yet much of the development has focused on chatbots that provide a text interface to existing applications, or agents that react to human input and scheduled events, effectively becoming sophisticated replacements for scripts and cron jobs.

It's as if command-line interfaces and prompting have become fashionable again. My premise is that this is not intentional, but rather convenient.

What would AI look like if we started with a focus on humans?

Humans like to operate, navigate, and manipulate familiar objects in familiar places. We create mental models of the workspace, where to find items, and how interacting with them will change the outcomes.

Text-based interaction often fails at this goal, and so does voice communication when the task involves complex information. Neither naturally provides a consistent spatial model of the interface.

With each new request, information comes back as another sequence of text, moving previous information out of view and providing no consistent way of navigating and interacting with the result.

Interacting with that information is also unnecessarily difficult. In most chatbot interfaces, users can't directly manipulate objects and instead have to describe every intended change in words.

Even when changes have already been made elsewhere in the interface, the user may have to explain them again to the agent.

The objective of the user interface should be to make the capabilities of AI accessible through interfaces designed around human perception. I think this offers a significant opportunity for leadership in user interface design.

Perhaps it will come from Apple, which is often characterized as a fast follower, when its historical strength has been something quite different: distilling complex technologies into products that reflect how humans perceive, understand, and use them.

Or perhaps it will come from other groups and leaders who see an opportunity to differentiate through human-centered design.

This is one of the principles guiding my work at Soundness. AI should be measured not only by what models can do, but also by what they enable users to understand, control, and accomplish.


Full archive: All my articles, including those published outside this newsletter, are available at yakovshkolnikov.com.