🦿Avatars Grow Legs

I'm thrilled to share with you my latest research paper (CVPR 2023)! This was a joint effort with my intern at Meta Reality Labs before our team transitioned to GenAI.

Our innovative method, dubbed Avatars Grow Legs (AGRoL), aims to control a 3D avatar's entire body in VR without the need for extra sensors. Typically in VR, your interaction is limited to a headset and two handheld joysticks, leaving out any direct input from your legs. This limitation persists despite the Quest's downward-facing cameras, as they rarely capture the unocludded view of the legs.

To tackle this, we've introduced a novel solution based on a diffusion model. Our model synthesizes the 3D movement of the whole body conditioned solely on the tracking data from the hands and the head, circumventing the need for direct observation of the legs.

Moreover, we've designed AGRoL with an efficient architecture
enabling 30 FPS synthsis on a V100 GPU.

❱❱ Code and weights
❱❱ Paper
❱❱ Project Page

@gradientdude