Skip to case study

Mix.dialog · Nuance · 2019–2022

Making complex conversation logic visible and controllable

I led interaction design for behavior authoring, condition building, and QA workflows so enterprise conversation designers could build, understand, and verify deeply nested IVR logic.

Role
Senior UX Designer and interaction design lead
Focus
Behavior authoring, condition building, and QA workflows
Partners
Conversation designers, product, engineering, and research
PRODUCT WORK · CONDITION STACK Mix.dialog QA node with the condition stack open: nested if branches on a morning variable, message and return actions inside each level, and Else If and Else controls on every block.
The condition stack in the product, authoring nested logic on a QA node. Open full size to zoom.

Enterprise IVR teams needed behavior they could inspect and control

Conversation designers specified deterministic voice experiences: explicit rules decided what the system said or did after a recognized intent, a no-match or no-input event, a retry, or an escalation.

Unlike simpler chatbot flows, that behavior could vary by channel and modality and sit inside deeply nested business rules. Authors needed to predict and verify each path because a wrong branch could misroute a call or interrupt a transaction.

The workaround was a long conditional spreadsheet passed between conversation designers, customers, and engineers. It was difficult to read and change, and it separated design intent from implementation.

Mix.dialog turned intent recognition into authored behavior

Within the connected Mix platform, Mix.dialog was the product conversation designers used to author deterministic prompts, conditions, actions, and recovery behavior across voice and digital channels.

NLU

Mix.nlu

Defined intents and entities the experience could recognize.

Dialog

Mix.dialog

Authored prompts, conditions, actions, and recovery behavior.

DASHBOARD

Dashboard

Managed and deployed dialog and NLU configurations.

Mix.dialog authoring a travel booking project: a component tree on the left, a flow canvas of connected greeting, intent, and menu nodes in the centre, and node properties for the selected message on the right.
The authoring surface conversation designers worked in, with the flow canvas and node properties side by side. Open full size to zoom.

I led interaction design for the behavior-authoring model

I worked with conversation designers, product leadership, and engineering directors to translate authoring needs into feasible data and interface structures. I led the condition-building model and core node, modality, and channel patterns; prototyped interaction and failure states; and defined reusable patterns for the wider tool.

The core authoring surfaces I shaped

I designed the authoring primitives and verification surfaces that conversation designers built on—not the conversations themselves.

ANCHOR PATTERN

Question + QA nodes

Shaped question behavior and the QA verification surface, using their complex properties to inform the wider authoring experience.

BEHAVIOR BUILDING

Messages + conditions

Designed how authors created prompts and nested conditional logic without translating spreadsheets or writing generated code.

CHANNEL SEMANTICS

Modalities + actions

Defined rich text, audio, DTMF, and per-channel output patterns that connected recognized intents to downstream behavior.

The same QA node with the channel dock expanded, showing omni-channel, mobile, web, and TV virtual assistant targets above the nested condition stack.
The channel dock expanded over the condition stack: one authored behavior, targeted per channel. Open full size to zoom.

Making conditional logic visible and direct

For the condition stack, I tested interaction models with conversation designers and engineering, then used what we learned to shape both the interface and its underlying logic structure.

01 · DIVERGE

Test competing interaction models

Code-forward concepts exposed power but raised the learning burden. Form-heavy designs hid context, while flatter structures could not express the depth IVR teams needed.

02 · LISTEN

Find the real comprehension problem

Feedback showed that syntax was only part of the problem. Authors needed clear starting points, visible hierarchy, readable content, and confidence about what an edit would affect.

03 · CONVERGE

Make valid logic directly manipulable

The selected direction treated conditional groups as objects. Authors could read hierarchy, add branches, move content, and understand the output without translating a spreadsheet or writing code.

04 · REFINE

Define predictable editing rules

Adding, deleting, reordering, and changing an if branch into an else-if branch each needed explicit rules so deep logic felt safe to edit.

Nestable condition blocks made the logic directly manipulable

Every condition block kept its if, optional else-if branches, and else together as one movable structure. Authors could nest whole blocks, move messages, variable assignments, and events between levels, and reorder else-if branches without detaching them from their parent logic.

Enable JavaScript to explore the interactive condition-stack reconstruction.

The actions inside a condition were a shared system, not a list

A condition is only useful if something can happen inside it, and that set of actions did not exist yet. I ran design interviews with the VUI designers who would be authoring with them to establish what was actually needed inside a branch, then worked the set out with product management and my UX design team.

That defined the actions—messages, variable assignments, events, notes, and returns—which I then designed as one system rather than as features of the condition builder. Each had to hold up in more than one place: nested several levels deep inside a condition, sitting flat on a node with no conditions at all, and reused by the other node types. That meant settling their structure, their properties, their editing and reordering rules, and how they read when stacked together.

Because the actions were shared, they set the boundaries other designers worked inside. New behavior had to compose from the same pieces and follow the same structure, which kept the authoring language consistent as the product grew.

Condition stack design exploration, slide 1 of 14.
Condition stack design exploration, slide 2 of 14.
Condition stack design exploration, slide 3 of 14.
Condition stack design exploration, slide 4 of 14.
Condition stack design exploration, slide 5 of 14.
Condition stack design exploration, slide 6 of 14.
Condition stack design exploration, slide 7 of 14.
Condition stack design exploration, slide 8 of 14.
Condition stack design exploration, slide 9 of 14.
Condition stack design exploration, slide 10 of 14.
Condition stack design exploration, slide 11 of 14.
Condition stack design exploration, slide 12 of 14.
Condition stack design exploration, slide 13 of 14.
Condition stack design exploration, slide 14 of 14.

Research, design, validate, implement

This was not a screen-design handoff. The condition model evolved in a continuous loop with conversation designers, product, and engineering. Interface discoveries frequently became architecture conversations.

01

Research real authoring work

Study spreadsheets, tools, handoffs, and nested IVR scenarios.

02

Prototype competing mental models

Tables, forms, graphs, code-like syntax, and direct manipulation.

03

Validate comprehension and control

Test context, nesting, add points, movement, and recovery.

04

Shape implementation with engineering

Align interaction rules with valid generated logic and data.

A more usable language for complex behavior

The work improved deeply nested condition authoring while establishing interaction patterns that influenced the wider Mix platform.

17 IVR + digital-VA designers
5.0 → 7.0 CSAT · +40%
52.67 → 61.50 SUS · +8.83 points

Three-session longitudinal study run by our embedded PhD researcher, tracking the condition stack as it was built. Both old-table-to-Session-3 gains were statistically significant at p < .05; the source deck does not report per-session completion counts.

A validated direction

The new stack significantly improved satisfaction and perceived usability while revealing findability and hierarchy issues that still needed iteration.

Proven in enterprise contexts

The workflows supported customers including Sony Interactive Entertainment and Rakuten, where reliability and complex branching mattered.

A platform-level template

The QA node and condition work established properties and interaction patterns that informed other nodes and shared product behavior.

“The work he has done for the Dialog tool and the condition stack is simply put amazing. The Mix platform would not be where it is today without Erik and his dedication to excellence and being a true team player.”

Senior Design Manager

“You are constantly pushing for the potential of Engineering and UX collaboration on complex design implementations.”

Senior Software Engineering Manager

“You have both UX and frontend development knowledge and are in this unique position to provide guidance to both designers and developers.”

Principal Software Developer

Colleague feedback from internal reviews, quoted by role rather than by name.

Condition stack results, slide 1 of 5.
Condition stack results, slide 2 of 5.
Condition stack results, slide 3 of 5.
Condition stack results, slide 4 of 5.
Condition stack results, slide 5 of 5.

Customer context. Sony Interactive Entertainment used Mix.dialog and its condition system in work supporting voice functionality for PlayStation 5. This case study focuses on the authoring system—not Sony’s implementation details.

The product patterns became a shared system

Mix.dialog exposed a broader need for consistency, accessibility, and shared language between design and engineering. I made the case for a platform design system over several years, secured leadership buy-in, and led the architecture of what became Verse. Built with React, Radix, and vanilla-extract, the system was implemented across Mix.dialog and the surrounding platform by multiple engineering teams.

As the design–engineering liaison, I aligned teams around shared components, interaction patterns, tokens, and implementation practices. After the Microsoft acquisition, we migrated Verse to Fluent 2 to align with Microsoft’s design language. The representative slides below are from my 2022 Nuance i3 talk, “Design like a developer… Develop like a designer!”

Read the full Verse Design System case study →

Let’s make complex systems understandable.

Senior UX Designer · UX Engineer · Design systems and AI tools