No more GPU? This is how Nvidia’s DLSS 10 could look like

Nvidia’s Deep Learning Super Sampling (DLSS) has become a significant selling point for its top-performing graphics cards, and the company has even grander plans in mind. Bryan Catanzaro, Nvidia’s VP of Applied Deep Learning Research, envisions DLSS 10 as a complete neural representation that eliminates the need for graphics cards to render frames physically.

During a panel discussion hosted by Digital Foundry, Catanzaro delved into the potential future evolution of DLSS and the problems that machine learning could address in gaming. While we already have DLSS 3, which can generate complete frames (a great leap forward from DLSS 2, which only generated pixels), Catanzaro confidently stated that the future of gaming lies in neural representation.

“I feel that we will achieve greater realism and, hopefully, make it cheaper to create incredible AAA environments by moving towards a more neural representation,” said Catanzaro. “I believe this will be a gradual process.”

The Potential of Neural Representation

Neural representation may sound like a futuristic concept, but Nvidia has been working on it for quite some time. As early as 2018, the company demonstrated real-time rendering of an open-world game using neural rendering at the NeurIPS Conference. The graphics appeared simplistic even by 2018 standards, let alone now, but the entire scene was generated through neural rendering. This process relied on data and input from the UE4 game engine to determine which objects should be displayed in the scene and their placements.

While Catanzaro believes that hypothetical DLSS 10 will be a “fully neural rendering system,” he also acknowledged the human aspect of game creation. If AI can generate everything, won’t it make game developers and artists obsolete? Catanzaro doesn’t think so.

“What’s so crucial about the traditional 3D pipeline and game engines is that they are controllable. You can have teams of artists building things, creating coherent stories and consistent environments. You can truly build a world with these tools, and we will certainly need them,” said Catanzaro. “I don’t think AI is going to build games in a way where you just write a paragraph like ‘make me a cyberpunk game, and I want lots of neon reflections and tall buildings with occlusions,’ and poof, Cyberpunk comes.”

Jakub Knapik, Vice President of Art and Global Art Director at CD Projekt Red (the studio behind Cyberpunk 2077), reacted to the perspective of DLSS 10 by saying, “It scares the hell out of me.” However, Knapik sees the potential of using machine learning to enhance game interactivity.

Nvidia is also actively working on Nvidia ACE, a technology that will help developers make their games more interactive and responsive by adding a chatbot to in-game NPCs. Other platform updates even enable developers to determine how toxic a character should be towards players.

While DLSS 10, if named as such upon release, is still far away, it appears that the future of gaming may lie in AI, whether for improvement or even generation of a significant portion of the game.