AI Digital Human
A photoreal MetaHuman that holds a spoken conversation in real time, streamed to any browser. It role plays difficult customers so staff can practise on someone who never gets tired and never takes it personally. Built, delivered, and approved.
Client: A European retail training platform
Speech to lipsync, end to end
Hops inside that budget
Installs. It runs in a tab.
The difficulty in a digital human is not the face. It is the loop.
A person speaks into their browser. The audio is transcribed, sent to a language model, returned as text, spoken by a neural voice, and driven onto a photoreal character as real time lipsync, which streams back to them as video. Five hops, several of them network round trips to different providers, and each one costs time.
Latency is the design constraint. Past a couple of seconds a person stops talking to a character and starts waiting for a computer, so the whole pipeline is built around holding the round trip inside two seconds.
The character plays a set of difficult customer types, which lets staff rehearse the conversations they find hardest, at any hour, without a colleague standing in and without a real customer at the other end.
It handles dialect: a trainee speaks in the regional form they use at work, and the character answers in the standard written form of the language.
The same pipeline powers Mirnak, the avatar on this site.
What we built
The systems inside it
Real-time conversational loop
Speech recognition, a language model, a neural voice, and facial animation, pipelined so the whole round trip from a person finishing a sentence to the character starting to answer stays inside two seconds.
MetaHuman with live lipsync
A photoreal character in Unreal Engine 5, with facial animation driven from the generated audio as it arrives rather than pre baked.
Pixel streamed delivery
Rendered on a cloud GPU and streamed to the browser, so a trainee on an ageing office laptop gets the same character as someone at a workstation, and installs nothing.
Customer personas
The character plays a set of difficult customer types, so a trainee practises the conversations they actually dread rather than the easy ones.
Dialect handling
A trainee speaks the way they speak at work, in the regional form, and the character answers in the standard written form of the language.
More work
Bring us the hard part
Performance walls, network replication that will not scale, streaming pipelines, real-time systems that have to hold their frame budget. That is the work we want.