AI makes GTA V more realistic but less beautiful

A team of Intel researchers has taken on the difficult task of making GTA V more realistic thanks to the intervention of an AI. The impressive results still question the usefulness of hyperrealism in video games.

Is a more realistic game necessarily more beautiful? This is the question Intel engineers asked themselves when creating an artificial intelligence capable of transforming GTA V images into photorealistic rendering.

This little technical feat was made possible by to the work of a team from the Intel Intelligent Systems Lab, a cell dedicated to research on artificial intelligence, machine learning and robotics at Intel. As an introduction to your research article., the three authors explain that ” Photorealism has been the goal of video game graphics for half a century. “And that recent advances in the world of artificial intelligence allow us to improve” bridging the appearance gap between synthetic and real images ».

Mercedes everywhere, realism nowhere

To prove this, the researchers modified the images from GTA V, Rockstar’s hugely popular game, to give them a more realistic look and show their results in photos and videos.

To make GTA V more realistic, the Intel team fed a homemade artificial intelligence with the Cityscape database, It contains many photos of the urban environment. The machine was then tasked with dissecting the game images into various “layers.” This allows you to have a complete and accurate view of various aspects of the rendering, such as the lighting of the scene, the back of the camera and distances, even the brightness of 3D objects. By cleverly superimposing images from Cityscape and GTA, the AI ​​has produced more realistic images.

This operation may seem simple on paper, but it actually required a lot of work on the part of the researchers, as the machine learning methods traditionally used in this type of operation produced very random results here. One of these methods, for example, added Mercedes trim to all the cars, simply because the Cityscape images are primarily from Germany, where many Mercedes cars drive on the roads.

Should video games be hyper-realistic?

To address these issues, it was decided not to let the AI ​​analyze the full context of an image (which is traditionally done in these cases, to help the machine understand its surroundings), but rather to divide each image in the game into small parts, to match real pictures. Therefore, small elements of the GTA landscape, such as the view of a cyclist or the dividing line of a road, will be compared to similar photos in Cityscape.

The result obtained by Intel is impressive. In fact, the images resulting from their experiments seem very close to our reality. However, they are often less flattering to look at. The extreme contrast of the 3D rendering, the saturated colors of the weighing, the swarm of details give way to a more realistic vision, but less rich.

Over the years, video games have developed a specific grammar and universe that we have become accustomed to. Bright green trees and fluffy clouds don’t impact a video game like they would in life. Applying a hyper-realistic treatment to GTA causes it to lose some of its gaming identity, just like the “haunting valley” in humans.

The video continuation