An AI appears that plays “Grand Theft Auto V” without a game engine. Reflections and shadows are also reproduced, but there are two police cars | AUTOMATION

A technical demo has appeared that “reproduces” the popular Rockstar Games play “Grand Theft Auto V” with AI. The creators are YouTuber / programmer Harrison Kinsley and also programmer Daniel Kukieła. The program they created and called “GAN Theft Auto” makes it possible to play part of the “Grand Theft Auto V” game using only AI technology, without using the original game engine.

“GAN Theft Auto” is a program that uses the “GameGAN” AI model created by NVIDIA. Looking at how it works, although the image quality is blurry, the car seems to be running, turning by turning the wheel, and the lighting is also implemented as “Grand Theft Auto V”. However, all of these images are “auto-generated images” created by AI. Let’s explain the mechanism of this program step by step.

This program uses AI technology called “GAN (Generative Adversarial Network)”. It is a kind of generative model called “unsupervised learning”, and is characterized by a mechanism in which two networks called “generator” and “discriminator” compete with each other to improve the accuracy of learning … For example, it is a technology used in programs such as “generating an image of the face of a non-existent person” and “generating an image from the name of an object”.

GameGAN was developed by NVIDIA using this GAN. This is an AI model that literally specializes in playing games, and on the occasion of the 40th anniversary of “Pac-Man”, NVIDIA will use this AI model for “Pac-Man” which can actually be played without a game engine.RealizationWas. “No game engine” means that the AI ​​receiving user input will automatically generate a “probable” game screen in response to the operation.

It appears that the “Pac-Man” input signal operated by another AI and a huge data set of screen transitions were used to learn GameGAN. As a result of the learning, the basic rules such as Pac-Man eating dots and the movement of the ghosts are reproduced. In other words, the AI ​​learns the “laws” of the game from “Pac-Man” and reproduces them on its own without the help of the game engine.

This “GAN Theft Auto” reproduces “Grand Theft Auto V” using previous technology. Since this is also automatically generated by AI, all reactions and drawings to the operation are projected on the screen which AI predicted “this will come”. In addition to the simple behavior, the light reflected from the car body, the shadows cast on the road and the movement of the landscape are also reproduced. It is surprising that all these drawings are reproduced by learning AI.

Kinsley rented a workstation called DGX Station A100 from NVIDIA to develop “GAN Theft Auto.” This is a machine that specializes in artificial intelligence computing, equipped with four NVIDIA A100 GPUs that cost more than 1 million yen alone. Kinsley and Kukieła said they were running 12 AIs on the machine at the same time, learning data from the same road in the Grand Theft Auto V game. Regarding the output video, the approximate pixel image is said to have been smoothed out by technology. super sampling AI.

However, it appears that the AI ​​that has shown results also has weaknesses, one of which is collision detection. In Kinsley’s video, it can be seen that contact with the NPC vehicle is making it difficult to drive the vehicle itself. However, the NPC vehicle eventually disappeared from the screen, probably because it was unable to reproduce the complicated behavior. Also, in a head-on collision with a police NPC vehicle, the opponent’s vehicle splits in two. NPC vehicles display much more diverse movements than their own vehicles and landscapes, so it may have been difficult for the AI ​​to learn. This is also an interesting experimental result.

Also, “GAN Theft Auto” was developed in about two months, and the show is from Mr. KinsleyGithubIt is published in. On this, “it should be possible to operate on a normal PC.” However, keep in mind that the Python OE and machine learning library need to be installed to run.

Isn’t it surprising that AI technology can reproduce even advanced games like “Grand Theft Auto V” to some extent? And the progress of AI, which is closely related to game development, will be a delight for gamers. Depending on future developments, the day may come when you can play “AI-made games.”