AI · Conversations · Children's Experience
Building a Conversation Engine with a Narrative for Kids
Role
Project Lead · Sole Designer · AI Architect
Timeline
Apr-Sep 2024
Team
5 core + outsourced art
Release
A/B tested · Shipped
The Problem
How do you give children the freedom to talk about anything while still guiding them through a meaningful narrative experience?
Conversations are disappearing from childhood, replaced by passive screen time rather than genuine back-and-forth exchange. Research consistently links meaningful dialogue to language development and emotional intelligence in children aged 6-12. We wanted to bring those conversations back, through an AI companion children would actually talk to.
The moment we tried to build it, a fundamental tension appeared. A fully scripted character feels dead. It can't respond to what a child actually says. A fully open LLM has no sense of where the story needs to go. It wanders, forgets, and breaks character. We needed a system that lived in the space between: open enough to feel real, controlled enough to go somewhere.
That tension (scripted narrative vs. generative conversation) is not unique to this project. It's the central unsolved problem in AI-driven interactive experiences. What was unique was having to solve it for children aged 6-12, using voice as the only input, with STT models that weren't built for young voices, in real-time, with a character whose integrity could not break. No existing architecture gave us what we needed.
Problem: Fully scripted
“Do you have any pets at home?”
“I have a robot! His name is Zorp.”
“Ok now I need to tell you about the Dreamer's Lodge.”
Can't respond to what the child actually said. Dead end.
Problem: Fully open LLM
“The ancient map is hidden somewhere in the Lodge!”
“I love dinosaurs. Do you like T-Rexes?”
“Oh yes! T-Rexes had tiny arms but were incredible hunters...”
Story gone. No concept of where the narrative needs to go.
We needed a system that lived in the space between: open enough to feel real, controlled enough to go somewhere.
The Dreamer's Lodge: children weren't chatting in a UI, they were talking inside a world
The Architecture
The Handoff System
The approach we landed on is what practitioners call an FSM-style dialogue manager with bounded generative exchanges: a finite-state machine that owns the narrative, with the LLM operating inside it rather than on top of it. The state machine decides when the story moves forward. The LLM handles what happens in between.
We called each bounded exchange a conversation pocket: a defined space with a hardcoded entry line, a small number of open LLM turns, a context injection of lore and state, and a hardcoded exit that returned control to the narrative. The child could say anything inside the pocket. The narrative always moved forward when the pocket closed.
Hardcoded
Entry line Narrative beat Context injected
LLM Open
2-4 turns Anything goes Child leads
LLM Primed
Next beat revealed Wind-down cued LLM closes naturally
Hardcoded
Narrative resumes Seamless transition Story moves forward
LLM Open
Next pocket New context New topic
LLM Primed
Next beat revealed Wind-down cued LLM closes naturally
Hardcoded: FSM owns this
LLM Primed: told what's next, winds down
LLM Open: child leads freely
The soft exit
The LLM was never cut off. It was asked to finish.
The most fragile moment in the handoff loop isn't the entry. It's the exit. If the LLM is mid-sentence and the state machine fires the hardcoded transition, the child hears a non-sequitur. The illusion breaks immediately.
The solution was a priming step. Before the exit trigger fired, the system injected a context update into the LLM: what narrative beat was coming next, what Dux needed to do to get there, and an instruction to bring the current exchange to a natural close. The model wasn't told “stop now”. It was told “here's where the story is going, guide the child there.” The LLM would write its own wind-down line, often something that felt like Dux naturally shifting attention: noticing a sound, remembering something, asking a closing question. The hardcoded narrative line landed on the back of a conversation that had already started moving in that direction.
The child experienced it as a continuous conversation. The system had handed control back to the narrative state machine. Neither of those things felt like the other.
Why we couldn't just prompt our way out of this
Fully scripted
Couldn't respond to what a child actually said. Every unexpected answer caused a dead end. Children aged 6-12 say unexpected things constantly.
Fully open LLM
No concept of narrative time or story structure. Dux would go so deep into a side topic that the story ceased to exist. Also: no memory, lore drift, character breaks.
Better prompting
Prompted instructions erode across a conversation. The longer the exchange, the less reliably the model followed them. A 6-year-old will always outlast a system prompt.
Existing tools (2024)
Dialogue managers like Rasa or Botpress were built for task completion, not narrative. No children's voice product had shipped a hybrid FSM + LLM architecture at this point.
How we planned it out
The full architecture was documented and iterated on in detail before a single line was written. Every handoff point, every state transition, every fallback was mapped, because with children, edge cases are the product.
The full dialogue architecture for Prologue C: each column is a state, blue blocks are interruption points where the mic stays open during scripted lines, branching handles every STT failure mode
Design Decisions
Four places the seam between scripted and open conversation needed design work.
A hybrid architecture doesn't eliminate problems. It moves them. The hard part isn't the state machine or the LLM individually. It's what happens at the boundary between them. These are the four places where that boundary required the most deliberate design.
Decision 1
The entry line had to do two jobs
Each pocket opened with one hardcoded line that closed the previous beat for the child and primed the model's context: telling it where in the story it was, what it knew about the child, and what was in-bounds.
Decision 2
The LLM had to not know it was bounded
We never told the model it had a turn limit. Dux's personality (curious, one question at a time, 40 words max) made short replies feel like character, not constraint, while the exit fired quietly on turn count.
Decision 3
STT failure breaks the handoff
Children's voices hit word error rates 2-5× worse than adults'. Every pocket handled five input states (clear, unknown, wrong, silent, noisy) each with an in-character Dux recovery that never broke the seam.
Decision 4
The exit had to feel like Dux, not the system
The hardcoded exit is the most fragile moment. Every one was written as something Dux would genuinely say (a sound elsewhere in the Lodge, a callback to something the child mentioned) so the rails never showed.
Lore context injection: structured world facts prepended before each GPT exchange
The Dux system prompt: every constraint exists for a UX reason, not a technical one
The Character Layer
The handoff only works if the character never breaks.
A hybrid architecture is invisible to the user if (and only if) the character stays consistent across the seam. Every transition between scripted and open had to feel like Dux, not like a system state change. That required every element of Dux's design to reinforce the same coherent personality.
Three states. One character.
Every voice interaction had to signal what Dux was doing. Without a clear visual state, children didn't know when to talk. Getting the latency moment right (turning processing time into visible character behaviour) was one of the smallest but most impactful design decisions in the whole project.
Solving the blank page: before the pocket opens
Open pockets only work if the child has something to say. In testing, children went quiet not because they were shy, but because “you can talk about anything” is a harder prompt than it sounds. Topic Cards gave the child a visible starting point: something to choose, not something to be told. The pocket still opened. The conversation still flowed. We just removed the blank page.
Topic Cards: giving children a visible starting point so the open pocket doesn't begin with silence
Testing
The proof that the seam was invisible.
“Children started talking to Dux before we taught them how.”
No instruction. No onboarding prompt. They simply started talking, because Dux felt like someone, not a feature. That signal tells you the handoff architecture was working: if children had noticed the transition between scripted and open conversation, the illusion would have broken. They didn't notice. The seam was invisible.
A second observation confirmed it further: when the Dreamer's Lodge opened up and children had an entire world to explore, they kept coming back to Dux. The character held attention even competing against the environment. The architecture kept the story coherent. The character made children want to stay in it.
What This Taught Me
Designing AI products is not about designing the model. It's about designing the systems around the model.
The LLM was one component. The state machine, the context injectors, the hardcoded entry and exit lines, the character poses that mapped to pipeline states, the in-character STT fallbacks, the topic cards that solved the blank page. None of that is in the model. All of it is design. And all of it had to work together for the handoff to be invisible.
The hybrid scripted/LLM architecture is now a known pattern across EdTech, game AI, and voice products. What isn't common is designing that architecture for children: where STT fails constantly, where narrative coherence has to survive a six-year-old, and where the character's integrity is the product. That specificity is what made this genuinely hard. And genuinely worth solving.