One of the most exciting things about graphics chip design house NVIDIA is that the company has vision; it is always taking us one step closer to the future. While best known for its gaming GPUs, NVIDIA has also used its vision to stake leadership positions in areas such as high performance computing (HPC), desktop virtualization and artificial intelligence. And now with the launch of its new Pascal architecture, the company is taking the market one step closer to realizing the vast potential of virtual reality (VR).
To be clear, the amazing performance of the recently launched GeForce GTX 1080 has traditional gamers excited and lining up to get hold of the fastest gaming GPU in the world. Based on NVIDIA's next-generation Pascal architecture, the GTX 1080 features 2560 CUDA Cores running at speeds over 1600 MHz, while delivering a 70% performance increase over the Maxwell-based GeForce GTX 980.
The GTX 1080 also delivers 3x the power efficiency of the Maxwell architecture, allowing for unheard of clock speeds of over 1700MHz while consuming only 180 watts of power. Pascal-based GPUs also provide 8GB of GDDR5X memory in a 256-bit memory interface running at 10Gb/sec, with this expanded bandwidth driving a 1.7x improvement in effective memory bandwidth over standard GDDR5. NVIDIA has achieved all this through a design that takes advantage of a leading-edge 16nm FinFET manufacturing process that packs 7.2 billion transistors into one chip, delivering a dramatic increase in performance and efficiency.
Virtual reality enhancements
Amazingly, the new GeForce GTX 1080 can double the virtual reality performance of the Maxwell-based GeForce GTX TITAN X. One driver of this new level of performance is pure horsepower. For example, VR headsets render games and applications at a rendering resolution equivalent to 3024x1680, and need to do so at a sustained 90 frames per second (FPS). Failure to maintain a constant 90 FPS results in stuttering and hitching that ruin the experience.
But NVIDIA understands VR performance is not only about horsepower. The company has always been keenly interested in the capabilities of VR and has been working in the field for about a dozen years. Up to this point, NVIDIA has mostly been working in the professional and vertical segments developing high-end immersive environments that could cost millions of dollars. Bringing VR to the mainstream has been a goal of the company over the past several years, especially when a little startup called Oculus came to its GTC Conference in 2013. In line with industry developments, NVIDIA started considering VR rendering as part of its previous-generation Maxwell. And now with Pascal, NVIDIA has tailored the architecture to enable a whole new level of presence in VR.
For example, one major feature of the Pascal architecture implemented to optimize the VR experience is the Simultaneous Multi-Projection (SMP) engine, which basically changes the way a GPU renders to a display. Since the early days of 3D rendering, the graphics pipeline has been designed with a simple assumption that the render target is a single, flat display screen. With the SMP engine, the GPU can generate multiple projections of a single stream, thus mapping a single image onto up to sixteen different projections from the same viewpoint. This capability enables the GeForce GTX 1080 to accurately match the curved projection required for VR displays, the multiple projection angles required for surround display setups, and other emerging display use cases such as augmented reality (AR).
This technology is especially suited for helping with VR display systems, which also put a lens between the viewer and the screen, requiring a new type of projection that is different from the standard flat planar projection that traditional GPUs support. With Pascal, this can now be done efficiently. Traditional GPUs also make the claim that they support these types of displays, but they can only achieve this with significant inefficiencies - either requiring multiple rendering passes, or rendering with overdraw and then warping the image to match the display, or both.
Pascal also can render geometry for the left and right eyes simultaneously in a single pass, meaning an application can instruct the GPU to replicate geometry up to 32 times (16 projections x 2 projection centers) without additional application overhead.
In addition, all the processing is hardware-accelerated, and the stream of data never leaves the chip. Since the multi-projection expansion happens after the geometry pipeline, the application saves all the work that would otherwise need to be performed in upstream shader stages. The savings are particularly important in geometry-heavy scenarios, such as tessellation.
Asynchronous timewarp
Another requirement of some virtual reality applications is the ability to handle multiple independent workloads that work together to contribute to the final rendered image. For example, timewarp is a feature implemented in the Oculus SDK that allows for the rendering of an image and then to perform a postprocess on that rendered image to adjust it for changes in head motion during rendering. To lower latency, a technique called asynchronous timewarp is used to regenerate a final frame based on head position just before display scanout, interrupting the rendering of the next frame to do so.
The challenge for the GPU is that an asynchronous timewarp operation must be completed before scanout starts or a frame will be dropped. In this scenario, the GPU needs to support very fast and low latency preemption to move the less critical workload off of the GPU so that the more critical workload can run as soon as possible.
Unfortunately, a single rendering command from a game engine can potentially contain hundreds of draw calls, with those draw calls containing hundreds of pixels that need to be shaded and rendered. If preemption is implemented at a high level in the graphics pipeline, it would have to complete all of this work before switching tasks, resulting in a potentially very long delay. To address this issue, Pascal is the first GPU architecture to implement Pixel Level Preemption. This is a process where the graphics units keep track of their intermediate progress on rendering work, so that when preemption is requested, they can stop where they are and preempt quickly.
And this can be done at a very fine grain level, at a single pixel level.
VR Works
On top of the changes in the GPU architecture, NVIDIA has instituted a new VR Works SDK to support Pascal. VRWorks is NVIDIA's suite of APIs, libraries, and engines to allow application and headset developers to get the best experience out of NVIDIA GPUs.
In the past, NVIDIA focused its VRWorks development on areas such as reducing latency, improving graphics performance and creating plug-and-play compatibility for headsets that improve the overall graphics quality and multi-projection capabilities. However, with Pascal the company has introduced some new features to improve the user's overall presence in virtual reality.
The first is a feature called VRWorks Audio, which basically calculates sound propagation. Traditional audio is positional, using volume differences to tell you where the source is coming from within an environment. But this doesn't really reflect the real world, where sound spreads in many directions, not just directly toward a listener, meaning we also experience sound in the form of reverberation. But this indirect sound also depends on the size, shape, and material properties of the surrounding area. NVIDIA VRWorks Audio uses ray tracing, a technique used in generating images in computer graphics to trace the path of audio propagation through a virtual scene. VRWorks Audio simulates the propagation of acoustic energy through the surrounding environment using its NVIDIA OptiX ray tracing engine to calculate the movement and propagation of sound around a virtual environment.
NVIDIA has also announced PhysX for VRWorks which enables realistic collision detection. When a user reaches out with a touch controller and grabs an object, the objects will respond realistically using PhysX. This interaction with the virtual object allows the game engine to provide an accurate response, both in terms of what I see and the haptic response. All objects in the environment, whether it be water or fire, will behave realistically and as I would expect them, so when a user steps in and puts on that headset, he or she really steps into and is immersed fully in the VR world.
One of the demos, NVIDIA has prepared to show developers how they can take full advantage of Pascal, and how much fun and immersive it can be, is called VR Funhouse. Players can bounce between ten different mini-games, which let them tackle carnival-inspired challenges such as tossing basketballs, popping balloons, and shoot at targets fired from a cartoon cannon. VR Funhouse is the first full VR experience that NVIDIA is publishing itself and will be distributed through Steam as open source, so developers can get a jump start in creating similarly compelling experiences. Visitors at Computex Taipei will also be able to tap into the power of Pascal by experiencing NVIDIA VR Funhouse.
NVIDIA role in VR market
It is fair to say that NVIDIA delivers one of the core enabling technologies for the VR ecosystem, with its processors powering the headsets while also enhancing the video and now audio for the applications. NVIDIA GPUs are also used for 360 stitching and for most of the 360 videos developed. People are also starting to use GPUs for advanced input and tracking.
As the market moves forward, NVIDIA believes VR is going to be increasingly important as a competing platform that changes the way we enjoy entertainment, the way we interact with friends and family socially and the way we get business done. The company points out the development of VR will be somewhat analogous to that of smartphones, where over the past 10 years, smartphones have brought about a new revolution in video content creating and consumption, delivered new types of mobile gaming and brought about new ways to interact with friends with applications like Facebook and Snapchat, and doing business with applications like Uber.
Moreover, VR applications are expected to develop holistically. Countless verticals will utilize the technology to market their products, such as through virtual home tours; educate the public, such as virtually traveling up Mount Everest; or to train their staff, such as at hospitals.
On the other side, there will be the gaming community to develop the applications and hardware that will make VR more widespread for consumers.
And NVIDIA will be right in the middle every step of the way, or perhaps one step ahead. Visit NVIDIA at Computex to learn more.
With the launch of its new Pascal architecture, NVIDIA is taking the market one step closer to realizing the vast potential of virtual reality (VR).
