A big focus at Nvidia is AI calculations that can be used by tensor cores in graphics cards. Nvidia’s Instant NeRF, a technology that can create a three-dimensional scene from two-dimensional images from different perspectives very efficiently, also works on this basis. Read more about this below.
Just in time for the 75th anniversary of the Polaroid photo, Nvidia introduced new technology that can create a 3D environment from 2D images using AI calculations. The technique is called Neural Radiance Fields (NeRF) and should be able to be trained in a few seconds with several dozen images.
Buy graphics card: Geforce offers from RTX 3050 to RTX 3090 available with discounts of up to 24%
David Luebke, Vice President of Graphics Research, compares NeRFs to bitmap images on the premise that traditional 3D representations, such as TECH GAMING REPORT meshes, are comparable to vector images. He sees the technology’s intended use primarily in creating avatars and scenes for the virtual world, e.g. B. the three-dimensional recording of the participants in a videoconference or to reconstruct digital scenes in 3D. For demonstration purposes, as part of GTC 2022, Nvidia took a mockup of an iconic photograph by artist Andy Warhole and transformed the two-dimensional image into a three-dimensional scene using Instant NeRF.
Neural networks are used to train NeRFs, which are trained using the inputs. In doing so, the neural network must have a few dozen images from different perspectives around the corresponding scene and the exact position of the camera must also be entered. If there are people or other moving objects in the images, they must be taken at extremely short intervals, otherwise the corresponding parts of the 3D display will be blurred due to large differences between the images. Due to the many different angles and perspectives, NeRF can fill in the colors and lighting due to training and thus create the 3D scene.
The technology that NeRF is based on has been known for a long time at Nvidia, but the main issue of long training and corresponding long render times has often made the technology very demanding. Instant NeRF intends to change that by using a so-called multi-resolution Hash Grid encoding, which is meant to achieve decent results very quickly with the help of a small neural network. This technology is compatible with Nvidia GPUs and uses Tensor cores. The manufacturer also sees practical applications outside of computing in the training of autonomous robots or cars, which can use the two-dimensional images with the help of NeRF to determine the sizes and shapes of real obstacles or other objects.
Source: nvidia
Introvert. Beer guru. Communicator. Travel fanatic. Web advocate. Certified alcohol geek. Tv buff. Subtly charming internet aficionado.
