Jose Luis Ponton

We introduce DragPoser, a novel deep-learning-based motion reconstruction system that accurately represents hard and dynamic constraints, attaining real-time high end-effectors position accuracy. This is achieved through a pose optimization process within a structured latent space. Our system requires only one-time training on a large human motion dataset, and then constraints can be dynamically defined as losses, while the pose is iteratively refined by computing the gradients of these losses within the latent space.

In VR, misalignment between the user’s body and their virtual avatar can disrupt the sense of embodiment and hinder interaction. This study examines five different interaction modes, manipulating factors like virtual controller visibility and hand positioning. The results reveal that "stretching" the avatar’s arms to align with the user’s hands significantly improves embodiment, proprioception, user preference, and task performance. Interestingly, whether the virtual controllers are rendered or not had no significant impact.

SparsePoser is a novel deep learning-based approach for reconstructing full-body human motion in VR using a reduced set of six tracking devices. By learning the human motion manifold from motion capture data and incorporating a learned IK component, it overcomes the limitations of traditional IK methods and IMU-based approaches, producing high-quality, continuous poses. Our extensive evaluations on motion capture datasets and real-time demos demonstrate that SparsePoser outperforms state-of-the-art techniques and can be applied to users with varying body dimensions.

Self-avatars are essential for enhancing presence and communication in VR. However, ensuring accurate dimensions often requires manual measurements, which can be time-consuming and error-prone. This paper proposes an automatic method using a head-mounted display (HMD), hand controllers, and trackers to accurately estimate user measurements and adjust the virtual skeleton. This method improves the alignment of the avatar with the user’s body, surpassing solutions that rely on uniform scaling or assumptions about joint locations.

This research investigates the impact of avatar animation fidelity on various tasks in VR. Comparing three animation techniques (two using Inverse Kinematics and one using a motion capture system), the study found that the quality of animation influences the user’s sense of embodiment. Surprisingly, IK-based solutions, despite using fewer sensors, sometimes outperformed MoCap in tasks requiring precise positioning due to lower latency and less positional drift.

Current VR avatar animation methods suffer from inaccurate lower body movement due to the limited tracking data available from consumer-grade devices. This paper presents a novel approach that utilizes a neural network to estimate user body orientation from HMD and controller inputs. By combining this orientation data with HMD velocity and rotation, a feature vector is generated to drive a motion matching algorithm. Tested on a MoCap database of VR users, this system produces a wider range of lower body animations that precisely match user orientation, enabling more realistic and diverse movement representation in VR.

Self-avatars in VR can greatly enhance user experience, especially in collaborative settings. Current methods for animating avatars can lead to mismatches between the user’s movement and the avatar’s. This paper introduces a simple but effective technique for calculating precise offsets for each user, resulting in significantly improved avatar movement and a stronger sense of embodiment.