OPENAI’S DOTA 2 DEFEAT IS STILL A WIN FOR ARTIFICIAL INTELLIGENCE

Image Credit: Getty Images

By  James Vincent

Lnast week, humanity struck back against the machines — sort of.

Actually, we beat them at a video game. In a best-of-three match, two teams of pro gamers overcame a squad of AI bots that were created by the Elon Musk-founded research lab OpenAI. The competitors were playing Dota 2, a phenomenally popular and complex battle arena game. But the match was also something of a litmus test for artificial intelligence: the latest high-profile measure of our ambition to create machines that can out-think us.

In the human-AI scorecard, artificial intelligence has racked up some big wins recently. Most notable was the defeat of the world’s best Go players by DeepMind’s AlphaGo, an achievement that experts thought out of reach for at least a decade. Recently, researchers have turned their attention to video games as the next challenge. Although video games lack the intellectual reputation of Go and chess, they’re actually much harder for computers to play. They withhold information from players; take place in complex, ever-changing environments; and require the sort of strategic thinking that can’t be easily simulated. In other words, they’re closer to the sorts of problems we want AI to tackle in real life.

OPENAI’S DEFEAT IS JUST A “BUMP IN THE ROAD” FOR AI PROGRESS

Dota 2 is a particularly popular testing ground, and OpenAI is thought to have the best Dota 2 bots around. But last week, they lost. So what happened? Have we reached some sort of ceiling in AI’s ability? Is this proof that some skills are just too complex for computers?

The short answers are no and no. This was just a “bump in the road,” says Stephen Merity, a machine learning researcher and Dota 2 fan. Machines will conquer the game eventually, and it’ll likely be OpenAI that cracks the case. But unpacking why humans won last week and what OpenAI managed to achieve — even in defeat — is still useful. It tells us what AI can and can’t do and what’s to come.

A screenshot of Dota 2, a fantasy arena battle game where two teams of five heroes fight to destroy one another’s base. Gameplay is complex, and matches typically last more than 30 minutes. 

Image: Valve

LEARNING LIKE A BOT: IF AT FIRST YOU DON’T SUCCEED

First, let’s put last week’s matches in context. The bots were created by OpenAI as part of its broad research remit to develop AI that “benefits all of humanity.” It’s a directive that justifies a lot of different research and has attracted some of the field’s best scientists. By training its team of Dota 2 bots (dubbed the OpenAI Five), the lab says it wants to develop systems that can “handle the complexity and uncertainty of the real world.”

The five bots (which operate independently but were trained using the same algorithms) were taught to play Dota 2 using a technique called reinforcement learning. This is a common training method that’s essentially trial-and-error at a huge scale. (It has its weaknesses, but it also produces incredible results, including AlphaGo.) Instead of coding the bots with the rules of Dota 2, they’re thrown into the game and left to figure things out for themselves. OpenAI’s engineers help this process along by rewarding them for completing certain tasks (like killing an opponent or winning a match) but nothing more than that.

“100 HUMAN LIFETIMES OF EXPERIENCE EVERY SINGLE DAY”

This means the bots start out playing completely randomly, and over time, they learn to connect certain behaviors to rewards. As you might guess, this is an extremely inefficient way to learn. As a result, the bots have to play Dota 2 at an accelerated rate, cramming 180 years of training time into each day. As OpenAI’s CTO and co-founder Greg Brockman told The Verge earlier this year, if it takes a human between 12,000 and 20,000 hours of practice to master a certain skill, then the bots burn through “100 human lifetimes of experience every single day.”

Part of the reason it takes so long is that Dota 2 is hugely complex, much more so than a board game. Two teams of five face off against one another on a map that’s filled with non-playable characters, obstacles, and destructible buildings, all of which have an effect on the tide of battle. Heroes have to fight their way to their opponent’s base and destroy it while juggling various mechanics. There are hundreds of items they can pick up or purchase to boost their ability, and each hero (of which there are more than 100) has its own unique moves and attributes. Each game of Dota 2 is like a battle of…Continue reading…

Article Source: https://www.theverge.com/2018/8/28/17787610/openai-dota-2-bots-ai-lost-international-reinforcement-learning