Colin Cassidy
Introduction
The latest research from Meta, the parent company of Oculus, has introduced a groundbreaking development in virtual reality (VR) and augmented reality (AR) technology. Mark Zuckerberg, the CEO of Meta, recently shared a video on Instagram demonstrating a surface-locked virtual keyboard. This innovative advancement could revolutionize text input in VR and AR devices, addressing the current limitations and significantly improving the typing experience for users.
Improving Typing Speed in VR and AR
Typing in virtual environments can be arduous, with slower input speeds compared to traditional keyboards on PCs and smartphones. The existing floating virtual keyboards require users to hold their hands up and press one key at a time, providing no haptic feedback and preventing natural wrist resting. On average, people type at around 40 words per minute on a regular keyboard, while professional typists reach speeds between 70 and 120 words per minute.
Meta’s research aims to enhance text input capabilities in VR and AR by turning any flat surface into a virtual keyboard. This development would offer partial haptic feedback and allow users to rest their wrists, similar to physical keyboards, without requiring an additional physical device.
Challenges and Solutions
Prior to Meta’s research, developers could technically create surface-locked virtual keyboards on the Oculus Quest by utilizing hand tracking and calibrating the surface’s position through taps. However, even slight deviations in the virtual surface height from the real surface often resulted in false key presses.
While Meta has not revealed specific details about how they addressed this issue, the video shared by Mark Zuckerberg indicates the presence of fiducial markers on a desk. Potential solutions to the calibration challenge involve preprogramming the system with accurate dimensions of these markers, providing a robust dynamic calibration system.
Quest 3 and Future Possibilities
The upcoming Quest 3 headset by Meta will feature a depth sensor for the first time. Leaked setup clips already show the headset generating a 3D mesh of its environment. If this mesh proves to be precise enough, it could potentially support the virtual keyboard functionality demonstrated in Meta’s research. However, it is important to note that Meta currently characterizes this development as ongoing research rather than a ready-to-market product.
Future Reveals at Meta Connect
Meta is scheduled to unveil more of its VR and AR research at Meta Connect, an event commencing on September 27th, 2022. This event is expected to showcase further advancements in immersive technologies, potentially including additional updates on the virtual keyboard research.
Conclusion
Meta’s research on transforming any flat surface into a virtual keyboard opens up new possibilities for text input in VR and AR. By overcoming current limitations, this development could significantly enhance user experiences, allowing for faster and more comfortable typing. While the specifics of how Meta solved the calibration challenge remain undisclosed, the use of fiducial markers indicates a potential solution. With the upcoming Quest 3 headset and its depth sensor capabilities, Meta may pave the way for the mainstream adoption of virtual keyboards in the near future.
GIPHY App Key not set. Please check settings