Colin Cassidy
Mark Zuckerberg Interviewed in Virtual Reality Using Meta’s Photorealistic Codec Avatars
In a groundbreaking interview conducted by Lex Fridman, Mark Zuckerberg, the CEO of Meta, sat down to discuss the future of remote communication using Meta’s prototype photorealistic Codec Avatars. This technology aims to revolutionize the way we interact with others by creating convincing real-time avatars that are driven by headsets with face tracking sensors.
Meta’s Codec Avatars project was first unveiled in 2019 and has since undergone multiple fidelity updates, including the introduction of a full-body version. These avatars are generated using a specialized capture rig with over 100 cameras, but researchers at Meta are developing a smartphone scan technology to replace this setup.
During the interview, both Zuckerberg and Fridman used the Quest Pro headsets, which feature built-in face tracking and eye tracking capabilities. Although it remains unclear whether the avatars were decoded and rendered onboard or on a PC, both headsets were connected via a USB-C cable, possibly to prevent battery drain rather than for a PC VR connection.
Fridman expressed his amazement and disbelief throughout the interview, stating that it genuinely felt as if he was sitting in front of Zuckerberg. At one point, he even admitted to forgetting that the avatar he was engaged with wasn’t a real person. This level of realism showcases Meta’s success in surpassing the uncanny valley, as discussed in a previous hands-on experience.
While Meta’s current avatars, known as Meta Avatars, have a basic cartoony art style that is far from photorealistic, the company still has a long way to go before reaching a level of fidelity suitable for shipping products. Interestingly, a leaked Meta roadmap suggests that Codec Avatars could potentially be integrated into the next version of Quest Pro, slated for a 2025 release through a partnership with LG.
The Impact of Codec Avatars on Remote Communication
Meta’s Codec Avatars technology has the potential to revolutionize remote communication by providing users with an immersive and realistic virtual meeting experience. With the integration of photorealistic avatars driven by headsets equipped with advanced sensors, individuals can connect with others as if they were physically present.
By achieving this level of realism, Codec Avatars eliminate the barriers posed by traditional video conferencing tools. Users can communicate and collaborate without feeling the disconnect caused by pixelated or artificial representations of themselves. The implications of this technology extend beyond personal interactions, offering new possibilities in fields such as remote work, healthcare, and education.
The Road Ahead for Meta and Codec Avatars
While Meta has made significant progress with its Codec Avatars research project, it acknowledges that there are still challenges to overcome before this technology is ready for widespread adoption. The present limitations of Meta Avatars highlight the need for further development and refinement.
The leaked Meta roadmap indicates that the next version of Quest Pro, in collaboration with LG, could be the platform for showcasing Codec Avatars. However, it is essential to bear in mind that roadmaps are subject to change, and there may be unforeseen hurdles along the way.
Despite the potential obstacles, Meta’s commitment to pushing the boundaries of remote communication is evident. As technology advances and hardware becomes more advanced and accessible, we may soon find ourselves interacting with photorealistic avatars in our day-to-day lives.
GIPHY App Key not set. Please check settings