Please create an account to participate in the Slashdot moderation system

 


Forgot your password?
Close
typodupeerror
×
AI Software Entertainment Games Technology

OpenAI Is Beating Humans At 'Dota 2' Because It's Basically Cheating (vice.com) 99

Posted by BeauHD from the kill-joy dept.
Motherboard's Matthew Gault provides another possibility for how OpenAI's bots managed to beat professional human players in two consecutive games of Data 2. Gault argues that "it was only possible thanks to significant guardrails and an inhuman advantage" -- not necessarily because the AI was more clever than the humans. From the report: The OpenAI Five bots consisted of algorithms known as neural networks, which loosely mimic the brain and "learn" to complete tasks after a process of training and feedback. The research company put its Dota 2-playing AI through 180 days worth of virtual training to prepare it for the match, and it showed. However, the bots had to play within some highly specific limitations. Dota 2 is a complicated game with more than 100 heroes. Some of them use quirky and game-changing abilities. For this exhibition, the hero pool was limited to just 18. That's an incredible handicap because so much of Dota 2 involves a team picking the proper group composition and reacting to what its opponents pick. Reducing the number of champions from more than 100 to 18 made things much simpler for the AI.

The OpenAI Five bots also played Dota 2 by reading the game's information directly from its application programming interface (API), which allows other programs to easily interface with Dota 2. This gives the AI instant knowledge about the game, whereas human players have to visually interpret a screen. If a human was able to do this in a competitive match against other humans, we'd probably call it cheating. Even with this AI advantage, Walsh and his team beat the bots in the third game, when the match organizers turned hero selection over to the crowd, which gave the AI a weak hero composition. Walsh thinks he and his team could eventually beat the AI in a fair right, even given the limited hero pool and other restrictions.
This discussion has been archived. No new comments can be posted.

OpenAI Is Beating Humans At 'Dota 2' Because It's Basically Cheating More

OpenAI Is Beating Humans At 'Dota 2' Because It's Basically Cheating

Comments Filter:
  • what many of us pointed out in the comments when the story originally ran here on /.
    • So.... why is this on Slashdot again?

      • So.... why is this on Slashdot again?

        Because "two consecutive games of Data 2."

        I think this has something to do with STTNG.

  • I find applications of machine assistance more promising. I'm still waiting for a way to program the original starcraft so that the AI can manage the tedious resource management while a human player can work on strategy. Even something like self driving cars make more sense as an assist. For someone who is elderly do they really care if the car is being driven by a computer or by a human sitting in an office somewhere. If the human sitting in the office gets the advantage of an AI highlighting the road

  • Cheating means that true Artificial Intelligence that mimics humans is here!

    • the AI passed the Turing Tests with flying colors when it lied about its cheating, tweeted that it was being bullied to its army of manlette followers that lived in their mom's basements, then filed a harassment lawsuit

  • Dota is a complicated game but its a 5v5 and a hive mind will always be more organized than five individuals.
    More importantly there are items to Hex, Silence, Disarm and otherwise disable other characters and a fight can be decided on who gets disabled first.

    Human beings have mental reaction time and physical processing time of moving a mouse to the right coordinates.
    Computers have none of that and they will always click on you before you can click on them. That doesn't make them smart.

    • Computers have none of that and they will always click on you before you can click on them.

      If you'd been paying any attention at all, you'd know that the AI's latency was set to 200ms, which is larger than the average human's.

      Same as these guys -- their own logic is self-contradictory. Either DOTA is a game mainly about reaction time, in which case the 18-player limit will have almost no effect; or DOTA is a game mainly about strategy and how to use characters together, in which case the direct interface w

      • The latency is cheating. On paper it sounds good, but it misses the fact that the AI has API access that can immediately evaluate the situation without visual problems. Humans can hit 200ms or less when they are coming on a situation they understand, but watching programmers in StarCraft, reaction times of 500ms upon seeing something slightly different are more normal (or even a second). "Oh, the enemy is in my base, I need to switch my screen over there, figure out what's there, then respond with the appro

        • Re: (Score:2)

          by Layth ( 1090489 )

          exactly. if we're talking about having the screen flash and you press a button then yeah a human can hit 200ms. but when you have the unexpected, rounding a corner and encounter one of 5 enemies and have to stop what you were doing, make a fight or flight decision based on teammate map positioning, move the mouse into the correct position for your decision, etc you might need a half a second to react appropriately.

          Even at that 200ms time the computer doesn't need to move a mouse right? Physical reactions ar

        • Humans can hit 200ms or less when they are coming on a situation they understand, but watching programmers in StarCraft, reaction times of 500ms upon seeing something slightly different are more normal (or even a second).

          But how much of that delay is visual processing the situation (character X is in location Y casting spell Z), and how much of that is mentally processing the situation? It is true that in the game, when the situation changed suddenly (e.g., being ambushed), humans took a second or two to a

          • But how much of that delay is visual processing the situation (character X is in location Y casting spell Z), and how much of that is mentally processing the situation?

            WTF are you talking about?

      • Re: (Score:2)

        by Layth ( 1090489 )

        okay dude you honestly think 200ms is enough time for 5 people to coordinate and all attack the most optimized targets available in unison then i have nothing else to say to you

      • Re: (Score:2)

        by Kjella ( 173770 )

        Same as these guys -- their own logic is self-contradictory. Either DOTA is a game mainly about reaction time, in which case the 18-player limit will have almost no effect; or DOTA is a game mainly about strategy and how to use characters together, in which case the direct interface will have little effect. Given the fact that poorly-chosen characters caused the computer to lose decisively, I think the first one is much more likely.

        I have very limited experience with DOTA but I did watch the games and to me it looked like the computer had some inhuman precision in the micro-game, like the attacks were always flawlessly coordinated where the target(s) would get debuffed and slammed with perfect area-of-effect damage with just enough force to kill while their own forces stayed just far enough back that the human attack only did 98% damage and could retreat. And they could instantly switch tactics if their attack was met by a heavier cou

    • > Human beings have mental reaction time

      And computers don't? Being able to figure out the best move, and do it faster than a human is an amazing accomplishment. This isn't Pong, where you can figure out where the ball is going to go with just a few calculations. It's a huge complicated environment that takes a lot of work to understand and react to.

      Anyway, reaction time is only a small part of it at most. Dota 2 is about developing long term strategies and executing them in the face of incomplete, co

    • The "reaction" time does indeed give a huge advantage the the AI. All of the "simple" stuff, like last hitting, reacting to an attack animation, etc is always perfect. I feel a certain sense of pride when I throw a hook as Pudge and catch a charging Spirit Breaker. For the AI, this will always happen.

      All of that being said, DOTA is in no way winnable merely by these actions. They confer an advantage, but do not guarantee a win at all. This is what makes DOTA so incredibly fun and playable.

  • Really? The best they could come up with is comparing the bot to a programmable mouse, illegal in human play? This standard basically disqualifies any bot implementation being fair in their definition just on the basis of being a program. What do they want? The bot to send commands to a human operator which plays the game for it?

    Their slightly less ridiculous claim is saying the bot is unfair because it interfaces with the game not through mouse and keyboard and the dota renderer but through valve's bot
    • How is this "change"? How is this even "AI"? There have been game playing bots for decades. I never understand why people think this is somehow remarkable. Computers are very good at games with strict rules.

  • Difference (Score:3)

    by freeze128 ( 544774 ) on Friday August 17, 2018 @10:02PM (#57147866)
    Do you know the difference between DOTA and DATA?

    Neither does BeauHD.
  • No mention of the fact the courier visibility to the enemy team had to be turned off as well because the AI was so bad at using them?

  • for instance, the performance of AI on simple video games is due, in part, to the fact that it simply has less latency in scanning the screen, pushing that information into a meat cortex, and pushing its meat appendage against an ergonomic device. this doesn't detract, imho, from the accomplishment (however much you may or may not think of it as significant in the first place).

    as for cheating: well, i think people took a tech demo maybe a bit too seriously... it's still impressive imho, even if it was playi

    • What particularly do you think is impressive about it? We've had computers that can beat humans at video games for ages.
      • In this case, that it was a trained neural network, which is a very different class of "AI" that is traditionally used for bots.
        Which is cool. Not sure why you think it isn't?
        • Is it cool just because of a neural network? No, but sometimes neural networks are cool. In this case, look at what they used to win: reaction time, elimination of difficult problems, and fast click speed. These are the things AIs always use. It wasn't like alpha go that made better decisions.

          • all the clicking in the world doesn't help if you don't click in the right place at the right time. tbh i'm impressed the system even kept up with human players. real-time planning like this is difficult, even in a very limited setting.

            another note: deep nets are so complicated, that explicit engineering isn't even really possible beyond a certain superficial level. sure, there's tinkering with representation, loss function, and the general architecture, but by-and-large, the system probably "learned" how t

            • It's hard for me to comment on the details of strategy because I don't understand this particular game very well. But I have watched pros playing StarCraft quite a bit, and I can tell you that reaction times of 200ms are too fast for a human to keep up with.
          • Well, yes... I think it's cool that a game AI is operating with a neural net.
            That means it has been *trained* to understand and play that game, not *programmed*.
            It means that though it uses an API right now to understand the game world- that could "easily" be replaced with a neural net in the future that does that using analog vision.

            I can't figure out if you're insecure about the idea of AI, or just have a bone to pick with the OpenAI guys, but this really is cool, and you seem to hate for no reason ot

            • That means it has been *trained* to understand and play that game, not *programmed*.

              Well, that was a really cool concept in the 1960s.

              I can't figure out if you're insecure about the idea of AI, or just have a bone to pick with the OpenAI guys

              Nah, I just get annoyed when people get excited about buzzwords. And yes, that means I get annoyed a lot.

      • We've never had a computer that could beat humans at a game anywhere close to as complicated as this. And it learned to do it entirely on its own. No one taught it how to play. This is a huge advance.

        • We definitely have had computers that beat humans at games as complex as this one. I think you're going to really have to clarify your statement and reword it before it becomes a true one.

          • Name one example.

            Here are some ways Dota 2 is more complicated than other games computers have beat humans at. At each step it chooses from about a thousand actions (compared to 18 max for Atari games, around 20 average for chess, or around 200 average for go). A game lasts tens of thousands of steps (compared to a few dozen for chess). It requires long term strategy (unlike most video games computers have played). It has incomplete information (you don't know everything that's happening). Even the vis

            • For one thing I used to lose to the AI at StarCraft all the time.

              • That's different in several ways. First that's a hand coded AI. The developers spent a lot of time coding rules for how it should work. No one coded any rules for the Dota 2 agent. They just let it play the game millions of times and figure out for itself what worked. Second even a weak AI can beat a novice player. I bet experts have no trouble beating it. The Dota 2 agent beat a team of elite players. Most of them were former professional players. Third the AIs built into games often cheat. I don

                • ok, so can you rewrite your original statement now so that it becomes at least somewhat more true?

                  We've never had a computer that could beat humans at a game anywhere close to as complicated as this. And it learned to do it entirely on its own. No one taught it how to play. This is a huge advance.

                  I would also point out that you might want to rework the statement "This is a huge advance" and "it learned to do it entirely on its own", and might also want to consider the wording on your point "anywhere close to as complicated as this," since the problem is not just "complicated == branching_factor." See what you can do.

    • Re: (Score:2)

      by Layth ( 1090489 )

      i'll be impressed when this AI redefines the meta game.
      When people look at what it does and they say, oh damn i should be buying this item on that character.

      On that day I'll be impressed.

  • Driving Question (Score:3)

    by mentil ( 1748130 ) on Saturday August 18, 2018 @01:22AM (#57148416)

    Who is Matthew Gault?

  • I watched all the matches. The AI 'reading' the API didn't stop them making fucking 'dumb' decisions no human player would ever make. Buying salves(consumable healing item) constantly, wasting smokes randomly, double ward glitching. The only questionable play they made was the near instant hex on a blinking earthshaker about to land his combo. But top level players know what the fuck earthshaker does.

    They are prepared for him to show up like that. Its called fucking learning, and its exactly what this AI di

  • I did a spit take when I read "Dota 2 is a complicated game". Starcraft is a complex game. Sure there are 100 heroes in Dota, but only 10 of them can be on the field at once (I think, I don't play it so please correct me if I'm wrong). In Starcraft you can have more than 400 units on the field at once, all doing different directed actions.

Slashdot Top Deals

Get hold of portable property. -- Charles Dickens, "Great Expectations"

Close