Ingenuity Labs: Characterizing VR Headset Performance with Cobots

I interned at Ingenuity Labs at Queen’s University in the summer of 2024 under the supervision of Dr. Matt Pan (MITHRIL), working with PhD candidate Eric Godden on human-robot interaction and virtual reality. We evaluated the hand-tracking accuracy of commercially available headsets using a seven-DOF KUKA LBR iiwa R7 and an open-source, anthropomorphic 3D-printed hand. We tested the Meta Quest Pro and Quest 3 by moving the arm through repeatable trajectories. Our work contributed to a paper published at IEEE VR 2025 in France, available here.

Our ground-truth system used multiple motion-capture cameras to track reflective markers with sub-millimetre accuracy. We embedded markers in the 3D-printed hand and added another set to the VR headset. Knowing both poses allowed us to calculate the hand position relative to the headset in the motion-capture frame. We compared that transformation with the headset’s internal estimate to measure hand-tracking error.

We tested hand tracking at many positions throughout the workspace. The robot repeated the same trajectory for each headset, eliminating variation between trials and reducing uncertainty in the test conditions.

The Quest Pro produced slightly better hand-tracking performance. In the heat map below, blue cells indicate lower Euclidean error and red cells indicate higher error across the test volume. The paper provides a more complete interpretation of these results.

The results also indicate that Meta uses predictive algorithms to estimate where the hand is moving. The plot on the right shows initial lag at the start of a motion, followed by the estimate catching up as the headset gathers more trajectory data.