Researchers are training a deep learning algorithm to beat humans at StarCraft
Computers have already crushed humans at chess, Go and poker. Now, researchers at the IT University of Copenhagen are training an algorithm to beat the best human players in the sci-fi computer game StarCraft. If the project succeeds, it will be another quantum leap forward for artificial intelligence (AI).
Computer Science DepartmentResearchalgorithmscomputer gamesartificial intelligence
Written 4 October, 2017 06:49 by Vibeke Arildsen
The more complex a game is, the more it resembles reality with all its unpredictability and unknowns. This is why AI researchers are now taking on computer games like StarCraft, which is estimated to have 101685 possible configurations in each match.
Niels Justesen is a year into his PhD project at the IT University of Copenhagen, and is working towards developing a self-learning AI algorithm that can beat even professional StarCraft players.
VIDEO: Watch the researchers’ algorithm (blue) beat StarCraft’s built-in training bot (purple).
www.youtube.com/embed/Ps_5P8Xx3_s
"Until now, research on computer games and deep learning algorithms has focused on trying to learn the games end-to-end, but that approach would be very difficult in a game like StarCraft because it is so complex. In StarCraft, you have uncertainty about how the opponent is going to act, just as in the real world. At the same time, the player must control an entire army and delegate tasks to different units of troops. Learning an algorithm to make the right decisions in such a complex system is a very general challenge for AI that, if resolved, can affect many other areas," he says.
AI trained on 2,000 games
So far, Niels Justesen and supervisor Sebastian Risi have trained the algorithm on 2,000 games involving some of the world's best StarCraft players. Based on this data set, the algorithm has gradually learned how human players have reacted in nearly 800,000 different game situations. To a certain extent, the algorithm can generalize this knowledge and use it make decisions in situations it has not experienced before. At this point, the algorithm plays at the same level as intermediate human StarCraft players.
The next step will be to get the algorithm to make better strategic decisions than even the professional StarCraft players using a method called deep reinforcement learning.
"A deep reinforcement learning algorithm learns to develop good strategies by experimenting in an environment. It tries different things, and thereby learns to do more of what it gets rewards from, and less of what it is punished for. Much the same way as a child learns to navigate in the world by trying things out,” Niels Justesen explains.
A milestone for AI
The ITU researcher is not the only one who works to develop a StarCraft algorithm. Facebook and the Google-owned company DeepMind are among those who are working with other aspects of the StarCraft problem. Finding an algorithm that can beat the best humans at the game will probably be the result of a collaboration between researchers and companies, says Niels Justesen.
"Our approach is to split the game into smaller parts and solve the problems one at a time instead of trying to solve the big complex problem at once. In this way, we are gradually trying to solve the overall strategy of the game, while Facebook, for example, is working on getting the algorithm to control a lot of troops at the same time," he says.
Ultimately, the aim is not to humiliate man in yet another game, but to drive AI research a major step forward. A number of the world's leading AI researchers recently estimated that the StarCraft game will be resolved within the next five years. This will be a major step forward that can will have implications far outside the world of gaming," says Niels Justesen.
A deep reinforcement learning algorithm learns to develop good strategies by experimenting in an environment. It tries different things, and thereby learns to do more of what it gets rewards from, and less of what it is punished for.
Niels Justesen, PhD student
"Google has already used algorithms originally designed to play games to reduce power consumption by 40 percent in their huge data centers. Researchers are also working on using deep learning algorithms to optimize the management of the large power networks known as smart grids. In the medical world, progress in deep learning could help doctors determine diagnoses based on data about the history of thousands of patients," he says and adds:
"Solving StarCraft would be a milestone a bit like the moon landing. You may not get that much out of being on the moon, but the technology you have developed in the process can be used in many other arenas."
If you want to challenge the algorithm at StarCraft, you can do it on Culture Night at ITU on Friday, October 13, 2017.
Vibeke Arildsen, Press Officer, phone 2555 0447, email viar@itu.dk