Making an AI character talk in AR took a day. Making it look alive took a month and a half - and six failed attempts.

We wrote an honest technical breakdown of how we build AR AI assistants inside AR Spatially - with code, real numbers, and every dead end we hit along the way.

Inside:

▫️ Why the character would sometimes smile and say nothing. Turns out, in the Realtime API, a function call and speech physically can't happen in the same turn - and one invisible mandatory tool call was silently breaking every conversation

▫️ How we rebuilt lip sync six times. From randomly jittering the mouth to bone-based animation we tore out completely in a single day - before finally landing on something that worked

▫️ Why the character's body still moves independently from what it's actually saying

Also - how one line in the character's prompt turned into a real production bug. A prompt isn't just personality - it's mechanics too.

Full article here 👉 https://arspatially.com/blog/ai-character-gestures-lipsync-realtime-api

For anyone building voice AI agents themselves - there's a step-by-step breakdown and exact timing values worth taking for your own project.