June 28, 2018
OpenAI, an artificial intelligence research group backed by Elon Musk, stated that its software can beat “teams of five skilled human players” in Valve’s video game “Dota 2.” If verified, the achievement would be a milestone in computer science and a leap beyond other AI researchers working on mastering complex games. IBM’s software mastered chess in the late 1990s, and Alphabet’s DeepMind created software that dominated “Go” in 2016. “Dota 2” is a multiplayer sci-fi fantasy game where teams advance through exploration.
“Our team of five neural networks, OpenAI Five, has started to defeat amateur human teams at ‘Dota 2’,” explains OpenAI in a detailed blog post. “While today we play with restrictions, we aim to beat a team of top professionals at The International in August subject only to a limited set of heroes.” (Below: OpenAI Five takes on the OpenAI employee team.)
Bloomberg reports that, “OpenAI said its software in mid-June beat a semi-professional team that is ranked among the top 1 percent of ‘Dota 2’ players and an amateur team ranked in the top 10 percent — both times winning two games to one in a best-of-three series.” It plans to “take on the world’s highest-ranked pros at The International,” the game’s premier pro tournament, which will take place from August 20-25 in Vancouver.
“Dota 2,” which is “many times more complicated than chess or ‘Go’,” has more than 1 million active players and “also is one of the most popular and lucrative in professional eSports.” Last year’s International had a prize pool of more than $24 million.
OpenAI’s software learned via “a relatively simple reinforcement learning algorithm it released last year,” that encourages the AI to “try new things without drifting too far from whatever it is currently doing that seems to be working.” Researchers gradually space out the rewards once the bot learns the basics, to encourage it to “think more about longer-term strategy and ultimate victory as opposed to shorter-term payoffs.”
To train for “Dota 2,” “OpenAI used 128,000 computing cores … as well as 256 graphics processing units,” and “the software played the equivalent of 180 years-worth of games against itself every day throughout a 19-day training cycle.”
“What ‘Dota’ does is show that today’s algorithms can go a lot further toward being able to solve those real-world challenges than people realized,” said OpenAI co-founder/chief technology officer Greg Brockman.
At the University of Alberta, AI/game expert Jonathan Schaeffer said that OpenAI’s reinforcement learning “could be promising for solving real-world situations, particularly those that could be couched as games — whether that is military war games or those meant to simulate politics or business,” although “the amount of data and computing power required to use the technique effectively limited its applications.”
“Humans have the ability to learn with very few examples,” he said. “Humans also have the ability to generalize and learn at a higher level of abstraction than what we currently see being done by computer programs.”