We present VRGaussianAvatar, an integrated system that enables real-time full-body 3DGS avatars in virtual reality using only HMD tracking signals.
We introduce VRGaussianAvatar, an integrated VR system for controllable full-body 3DGS avatar representation. (A) A single input image is used to reconstruct a full-body 3D Gaussian Splatting avatar. (B) The reconstructed avatar supports real-time rendering and full controllability in VR using only the internal sensors of commercial head-mounted displays (HMDs).
We present VRGaussianAvatar, an integrated system that enables real-time full-body 3D Gaussian Splatting (3DGS) avatars in virtual reality using only head-mounted display (HMD) tracking signals. The system adopts a parallel pipeline with a VR Frontend and a GA Backend. The VR Frontend uses inverse kinematics to estimate full-body pose and streams the resulting pose along with stereo camera parameters to the backend. The GA Backend stereoscopically renders a 3DGS avatar reconstructed from a single image. To improve stereo rendering efficiency, we introduce Binocular Batching, which jointly processes left and right eye views in a single batched pass to reduce redundant computation and support high-resolution VR displays. We evaluate VRGaussianAvatar with quantitative performance tests and a within-subject user study against image- and video-based mesh avatar baselines. Results show that VRGaussianAvatar sustains interactive VR performance and yields higher perceived appearance similarity, embodiment, and plausibility.
VRGaussianAvatar is structured as a parallel framework with a VR Frontend and a GA Backend. The VR Frontend estimates full-body pose from HMD sensors and provides per-frame camera parameters. The GA Backend animates and renders the pre-reconstructed 3DGS avatar, incorporating a Binocular Batching strategy for efficient stereoscopic rendering. In our reference setup, the VR Frontend and GA Backend communicate over localhost to minimize network overhead.
@article{song2026vrgaussianavatar,
title={VRGaussianAvatar: Integrating 3D Gaussian Avatars into VR},
author={Song, Hail and Yoon, Boram and Yang, Seokhwan and Kang, Seoyoung and Kim, Hyunjeong and Metzmacher, Henning and Woo, Woontack},
journal={IEEE Transactions on Visualization and Computer Graphics},
year={2026},
publisher={IEEE}
}
