The frontier

Three communities, seven threads

Generative UI sits where three research communities meet: HCI (interaction paradigms, mixed initiative, malleable software), ML and NLP (code generation, multimodal models, benchmarks) and agents on the web (GUI agents and interfaces built for them). An essay-and-industry canon often ran ahead of all three. Seven threads organise the work.

1. The interface as the model's output. From small, ephemeral widgets (DynaVis, BISCUIT, 2024) to whole generated pages. Google's Generative UI paper reports generated pages preferred over markdown answers 82.8% of the time, and at least comparable to expert-made sites in about half of comparisons. Stanford's Generative Interfaces for Language Models adds a structured intermediate representation and a refinement loop. UCSD's malleable-UI line (Jelly and successors) argues for generating a data model first and rendering from it.
2. Evaluating generated UI. Evaluation has climbed a ladder: pixels (Design2Code) → interactions (Interaction2Code) → functionality executed by an agent in a browser (WebGen-Bench, best setup 27.8% pass rate) → checklist-guided judging (ArtifactsBench) → whether the tool's own claims are true (Design Theater: over 25% of stated design rationales aren't implemented).
3. Machines reading and operating UIs. Screen understanding (ScreenAI, Ferret-UI) → grounding as the bottleneck (SeeAct) → realistic environments graded on outcomes (WebArena, OSWorld) → native end-to-end agents (UI-TARS-2).
4. Safety and consequences of actions. WebGuard labels state-changing actions SAFE/LOW/HIGH and finds frontier models catch under 60% of high-risk ones. ST-WebAgentBench scores task completion under policy, including user consent. On the human side: Horvitz's 1999 account of when to act, ask or confirm, and evidence that confirmations people click through don't protect them.
5. Interfaces for two audiences. From the 1997 Shneiderman–Maes debate (direct manipulation vs agents) to SWE-agent's finding that agent-computer interface design changes agent performance, to WebMCP's tool hints — which the draft itself admits can't be verified.
6. Beyond chat. Systems like Graphologue, Sensecape and DirectGPT (tasks done about 50% faster with 72% shorter prompts) and essays from Victor, Nielsen, Wattenberger, Appleton and Litt established that structure, persistent objects and direct manipulation beat linear chat for complex work. Counterweights: over-reliance, and the irony that automation turns users into evaluators.
7. Protocols and products. Production has settled on three architectures: free code in a sandbox, a declarative component catalog, and a data model rendered by rules.

The central tension

Across every thread runs one trade-off: expressiveness versus verifiability. Free-form generation wins preference studies. Catalogs and data models win on safety, conformance and testability. The 2026 shift in the literature is from "can models generate UI?" to "can we control what they generate?" — design-system conformance (GUIDE), intent semantics, just-in-time objectives, designer feedback loops.

Notice the rung missing from the evaluation ladder: consequences. Does the generated UI confirm destructive effects? Can they be undone? Does the button's label honestly describe what happens? Across this survey, no benchmark measures that end to end for generated interfaces. A lab that can make free-form generation verifiable after the fact, with execution evidence, sits exactly on the field's central tension. That is the bet behind RUI.

The twelve must-reads

#	Work	Read it for
1	Leviathan et al., Generative UI: LLMs are Effective UI Generators (Google, 2026)	the architecture (tools, a long system prompt, post-processors) and the preference-evaluation protocol — then list what it can't see
2	Chen et al., Generative Interfaces for Language Models (Stanford, 2025)	when generated interfaces beat chat, and which queries don't benefit
3	Cao, Jiang, Xia, Jelly (CHI 2025)	model-first generation: what a data model lets you verify that HTML can't
4	Imteyaz et al., Design Theater (2026)	evidence that stated ≠ implemented
5	Zheng et al., WebGuard (2025)	risk tiers for state-changing actions, and where prediction fails
6	Lu et al., WebGen-Bench (2025)	agent-executed functional tests you can adapt
7	WebMCP draft (W3C community group, 2025–26)	the dual-audience API and its security section
8	Horvitz, Principles of Mixed-Initiative User Interfaces (CHI 1999)	when to act, ask or confirm — the vocabulary for consent policy
9	Amershi et al., Guidelines for Human-AI Interaction (CHI 2019)	18 guidelines, several checkable automatically on generated UI
10	Lee et al., GUIDE (2026)	design-system conformance as a measurable score
11	Karpathy, Software Is Changing (Again) (2025)	the autonomy slider and why GUIs make verification fast
12	Bernstein, The Interface of Theseus (UIST Visions 2026)	interfaces as disposable; generators and AI-facing specs as the durable artifacts

All of them, plus about seventy more, are annotated with links in the Library.

The essays that keep the field honest

Papers prove things; essays name them. A few you should be able to quote:

• Jakob Nielsen, "AI: First New UI Paradigm in 60 Years" — intent-based outcome specification: you say what you want, not how to do it. Generative UI is that paradigm getting its own screens.
• Kate Moran and Sarah Gibbons (NN/g), "Generative UI and Outcome-Oriented Design" — the design profession's first serious framing of interfaces generated per user.
• Amelia Wattenberger, "Why Chatbots Are Not the Future of Interfaces" — the argument for affordances over a blank text box.
• Geoffrey Litt and Ink & Switch on malleable software — people should be able to reshape their tools; LLMs make that cheap enough to matter.
• Bret Victor, Magic Ink — most software is information software, and most of it should be designed, not navigated.
• Andrej Karpathy, "Software Is Changing (Again)" — partial autonomy, the autonomy slider, keeping the AI on a leash, and GUIs as the fastest way for a person to verify.

Open problems you could work on

Each of these is a real gap with a 6–10 week project shape. The first four share one piece of infrastructure — an instrumented browser that records DOM, network and state changes and judges them against declared intent — so building it once makes each project a new dataset plus a new family of assertions.

1. Consequence-aware evaluation. A benchmark of ~100 generation prompts that require effects (delete, pay, send, publish). Run each generated UI against a mocked backend; assert confirmation before high-risk effects, an undo path, and effects that match labels.
2. Label–effect faithfulness ("action theater"). Detect controls whose label disagrees with their effect — a "Preview" that POSTs, a "Cancel" that deletes.
3. Reversibility grounded in execution. Annotate ~1,000 actions with post-execution evidence and show how much traces improve risk classification over screenshot-only prediction.
4. Verifying declared agent affordances. Call WebMCP and MCP Apps tools in a sandbox and measure how often their read-only and consequential hints match what they actually do.
5. Dual-audience usability. Do agent-friendly components help or hurt people? Same tasks, people and agents, agent-ready vs plain components.
6. Accessibility of streamed UI. Focus jumps, live-region spam and layout shift during generation — nobody has measured them across products.
7. The conformance–expressiveness frontier. Same prompts and design system across free code, catalog and data-model architectures; plot conformance against task success.
8. When LLM judges certify broken UIs. Pair judge verdicts with execution ground truth; characterise the false passes.
9. Over-reliance on polish. Does visual polish raise trust independent of correctness — and does an evidence badge restore calibration?
10. Regeneration drift. Regenerating "the same" UI moves controls, breaking people's spatial memory and agents' selectors. Define and measure drift.

Natural venues: CHI late-breaking work, UIST posters and demos, NeurIPS Datasets & Benchmarks, IUI, ASSETS.

Where this is going

Three directions are visible from here.

• Interfaces become disposable; their contracts don't. If screens are generated per moment, the durable artifacts are the catalogs, the design-system specs written for AI, the tool contracts, and the policies about consequences. Teams will version those the way they version APIs today.
• Every interface gets two audiences. People and agents will use the same surfaces. The components that win will expose one implementation to both, with declared consequences.
• Trust moves from review to evidence. Nobody can review every generated screen. The question shifts from "did a person approve this design?" to "can we prove what this screen does, and that a person consented before anything irreversible happened?"

Check yourself

1. State the central tension of the field in one sentence, and name one architecture on each side of it.
2. Google's paper reports an 82.8% preference for generated pages. Name three things that number can't tell you.
3. Why is consequence the missing rung on the evaluation ladder, and what would measuring it require?
4. Pick one open problem above and write its research question, dataset and metric in three lines.

Go deeper

• The Library: 81 annotated papers, benchmarks and essays with verified links
• The six-week reading program is in research/03-papers-and-thinkers.md in this repo
• Essay: The model only proposes
• Start again from the top, faster: 00 · Orientation