OpenAI Five crushes Dota2 world champs, and soon you can lose to it too
|TechCrunch 15 Apr 2019 at 12:56|
In a blog post , OpenAI detailed how its game-playing agent has progressed from its younger self â it seems wrong to say previous version, since it really is the same extensive neural network as many months ago, but with much more training.
The version that played at Dota2âs premiere tournament, The International, gets schooled by the new version 99 percent of the time. And itâs all down to more practice:
In total, the current version of OpenAI Five has consumed 800 petaflop/s-days and experienced about 45,000 years of Dota self-play over 10 realtime months (up from about 10,000 years over 1.5 realtime months as of The International), for an average of 250 years of simulated experience per day.
To the best of our knowledge, this is the first time an RL [reinforcement learning] agent has been trained using such a long-lived training run.
One is tempted to cry foul at a datacenter-spanning intelligence being allowed to train for 600 human lifespans. But really itâs more of a compliment to human cognition that we can accomplish the same thing with a handful of months or years, while still finding time to eat, sleep, socialize (well, some of us) and so on.
Dota2 is an intense and complex game with some rigid rules but a huge amount of fluidity, and representing it in a way that makes sense to a computer isnât easy (which likely accounts partly for the volume of training required). Controlling five âheroesâ at once on a large map with so much going on at any given time is enough to tax a team of five human brains. But teams work best when theyâre acting as a single unit, which is more or less what Five was doing from the start. Rather than five heroes, it was more like five fingers of a hand to the AI.
Interestingly, OpenAI also discovered lately that Five is capable of playing cooperatively with humans as well as in competition. This was far from a sure thing â the whole system might have frozen up or misbehaved if it had a person in there gumming up the gears. But in fact it works pretty well.
Weâre launching OpenAI Five Arena, a public experiment where weâll let anyone play OpenAI Five in both competitive and cooperative modes. Weâd known that our 1v1 bot would be exploitable through clever strategies; we donât know to what extent the same is true of OpenAI Five, but weâre excited to invite the community to help us findÂ out!
Although a match against pros would mean all-out war using traditional tactics, low-stakes matches against curious players might reveal interesting patterns or exploits that the AIâs creators arenât aware of. Results will be posted publicly, so be ready for that.
Youâll need to sign up ahead of time, though: The system will only be available to play from Thursday night at 6 PM to the very end of Sunday, Pacific time. They need to reserve the requisite amount of computing resources to run the thing, so sign up now if you want to be sure to get a spot.
OpenAIâs team writes that this is the last weâll hear of this particular iteration of the system; itâs done competing (at least in tournaments) and will be described more thoroughly in a paper soon. Theyâll continue to work in the Dota2 environment because itâs interesting, but what exactly the goals, means, or limitations will be are yet to be announced.