Sony Built an AI That Can Beat Users at Video Games, With Honor (fastcompany.com) 37
Japanese tech giant Sony revealed this week that it has trained the toughest-ever opponent for the race-car simulator Gran Turismo -- a champion that can beat top-class e-sports drivers at their own games. From a report: Forged on the battlegrounds of over 1,000 PlayStation 4 consoles, the AI racer-bot has grown smart enough to identify optimal course routes and can execute skilled tactical maneuvers to pass or block competitors, even in vehicular scrum. It does so with ruthless effectiveness -- while still respecting the human etiquette of the game, Sony claims. The company published research on its brainchild -- dubbed Gran Turismo Sophy -- in Nature journal this week. The development process paired "state-of-the-art, model-free, deep reinforcement learning algorithms with mixed-scenario training to learn an integrated control policy that combines exceptional speed with impressive tactics," it said. "In addition, we construct a reward function that enables the agent to be competitive while adhering to racing's important, but under-specified, sportsmanship rules."
In a media-broadcast demonstration, Sophy bested four of the world's top Gran Turismo drivers in head-to-head contests, proving the tech's superiority to mere mortals. But Sophy's aspiration was never to crush humanity's spirits or to leave it feeling defeated. On the contrary, it was meant to spark fresh excitement in e-sports, especially among elite players who felt they had no challenge left to answer. "I feel frustrated, that never happened before battling with an AI," Tomoaki Yamanaka, one of the four racers, said after the loss. "I drove like I would drive against a human. That's a really amazing thing."
In a media-broadcast demonstration, Sophy bested four of the world's top Gran Turismo drivers in head-to-head contests, proving the tech's superiority to mere mortals. But Sophy's aspiration was never to crush humanity's spirits or to leave it feeling defeated. On the contrary, it was meant to spark fresh excitement in e-sports, especially among elite players who felt they had no challenge left to answer. "I feel frustrated, that never happened before battling with an AI," Tomoaki Yamanaka, one of the four racers, said after the loss. "I drove like I would drive against a human. That's a really amazing thing."
Very clever, but Sony is in the gaming business (Score:2)
This is like if a company that sold mountain climbing gear constructed human looking androids that were better than everyone else at climbing mountains. They shouldn't be surprised if some of their customers then thought mountain climbing had lost some of its je ne sais quoi.
Re: (Score:1)
And anonymous cowards are not correct. It's how people post when they know they are just trolling.
Re: (Score:2)
> Video games are not sports.
And every chess player along with [chess.com] The International Olympic Committee disagrees with you.
If only we had a different word to designate between physical activity and mental activity. /s Wait I know, we could call them Esports. Nah, that would never work. [wikipedia.org]
I'll leave the pedantics to argue if chess is a sports, or esports, or both.
Actually Sony is in the insurance business (Score:2)
Mega corporations get involved in everything and a handful of them basically own everything. Koch media made the new King of fighters gam
Re: (Score:2)
The point, for me as someone who plays racing games, is not that this is better than a human. You can easily detune it to give humans a competitive edge, by limiting what racing lines it chooses. Just throw out the lines that are faster than the player's average lap times.
It's the fact that it races like a human and knows the unwritten sportsmanship rules of racing. I will happily ram a dumb AI and use other tactics I don't use against human drivers, because the AI will not remember, and will not try dirty
But Sophy's aspiration was never to crush humanity (Score:2)
And east germany never intended to erect a wall.
technical question (Score:3)
so I understand how ML works generally speaking, I use it in my day-to-day job and know how to set it up, run it, etc; I would consider myself more of a technician, however, not an expert in the hardcore maths behind it
I do have a huge gap in understanding how one takes to results of the modelling/training and plug them in to a realtime system, like the driving game
how do all the jillions of inputs/decisions/outputs that entail such a game get wired up to get a realtime system to beat a human which up to now, was the ultimate realtime computer
not even sure how to ask the question correctly but am hoping that's close enough
Re: (Score:3)
I have a better question. How does this AI "view" this game? And by that I mean, is it like a human who watches and reacts/anticipates what to do next, or is it hooked into the game's software stream and can "see" ahead?
A better example might be online poker. How do we know the software can't see how the deck(s) of cards is/are shuffled ahead of time and deals accordingly to prevent players from winning and thus making them continue to pour money into the game to try and win back what they lost? A type
Re: (Score:2)
So, how do you "simulate" honor? How do you turn the steering wheel with honor? How do you accelerate with honor? Is there some sort of Bayesian feedback where a human tells the AI if the moves are honorable?
Re: (Score:2)
Interesting. Do you remember if it was text or graphics based? I wonder what Apple 2 game that was? There couldn't have been that many poker games. I wonder if that was Strip Poker (1982) ... for research purposes. /s
Re: (Score:2)
"How does this AI "view" this game?"
that's close to what I was asking, I replied to another post down below with a refinement of my question
put another way, if the human brain is analogus to the AI model, what are the model's equivalent to a person's eyes, ears, fingers, etc? all those I/O costs in meatspace would have to be accounted for to make things equivalent, I would think
Re: (Score:2)
Mod parent up, it's not a fair fight if the computer is getting anything other than the video and audio feed. If the AI computer is getting track data and other car positional data fed in as xyz positional data then it's cheating and this doesn't count as a true AI win in my book since the computer isn't playing the same game.
This also goes for all of the other game AI bots, if they're winning by getting data directly in a feed from the game instead of getting a feed like a human gets the game then they're
Re: (Score:1)
I'm ignorant and I'm not gonna go read the stinking article either.
My rough understanding is that this is more like a neural network where they just give the machine lots of inputs and outputs and criteria to let the machine figure out the right ML points to pay attention to. e.g. automating your portion as a ML technician.
No surprise that it resembles human racing: The criteria was, "race kinda like these people do, but win"
At least, that's my guess.
Re: (Score:2)
You may find these videos helpful:
* MarI/O - Machine Learning for Video Games [youtube.com]
* But what is a neural network? | Chapter 1, Deep learning [youtube.com]
Re: (Score:2)
great links, thanks!
Re: (Score:1)
Amazon Deepracer may give you some insight into how this may be done: https://aws.amazon.com/deeprac... [amazon.com]
As I understand it, Deepracer has two models, one which interprets the visual scene and one which choose the output actions (turn, accelerate, etc.).
The training data is processed in batches, one batch per lap. As the car goes around, at each tick the car assess where it is and what input, then makes a prediction, with some randomness put into the prediction to make the training better. At the beginning o
Re: (Score:2)
I don't know how they did it in this particular example, but in the past, they've just taken the pixels from the screen and piped that directly into the neural network.
Re: (Score:3)
I think you've answered a bit more in the direction I was inquiring... after reading some of the other comments, let me refine:
let's start with a human, who would take visual info and a bit of audio as inputs, not forgetting that it has a time cost to make it ready for the brain's use; these will then rattle around in the head with neurons a-firing, with decisions and actions being taking based on whatever the human has learned; then they send signals to control motor functions in the fingers which then m
Re: (Score:2)
if not, then is it a fair comparison? nitpicking maybe, but there are time costs in the form a latency between the human brain's I/O so is that accounted for in the AI version? if the AI version never has go back and forth to the digital realm, that would seem to be a big advantage, no?
This has been a big issue with AI versions of Starcraft. Since there is no latency, AIs can easily win (in Starcraft) just by clicking faster. How to simulate that latency without actually building a robot hand has been an ongoing problem.
This just in! Computers outperform humans... (Score:3)
The game is already nothing more than a set of algorithms which collectively simulate the real-world task of auto racing. Telling me there's a computer program which can react to these algorithms' outputs better than a human doesn't seem like that monumental a task.
Re: (Score:3, Interesting)
Reminds me of the old Star Trek book: The Kobayashi Maru. In it, a few members of star fleet (kirk, scotty, sulu, iirc) share their experience of trying to defeat the unwinnable game. Its a very fun read!
SPOILERS!
Scotty describes that he WON the game with a very special arrangement of torpedo spread and detonation that destroyed all enemy ships. He then told the test administrators that its too bad that it wouldn't work in real life and they all asked what he meant. They did their research and found a paper
Re: (Score:2)
Really, if you think about it, Scotty and Kirk both won by cheating. Scotty just used a cheat in the physics he already knew were in the game, rather than programming (or having programmed, depending on which version you're following) his cheat into the computer himself. :)
Re: (Score:2)
Absolutely. The other stories besides Kirk and Scotty are also good.
SPOLIER:
Winning on their terms, imo.
Re: (Score:2)
Kirk cheated- he changed the conditions of the test (hacked the game). Scotty played the game as written after telling them it was broken and they didn't fix it.
Re: (Score:2)
This isn't quite chess. (Score:2)
Ever had a bot score repeated direct hits on you at range with a railgun, when all you can see of the bot is a sixteen pixel block? There's a bunch of hand-eye junk going on here that a bot (or AI) will never have to deal with - including the delay between sensing an event and cognitively registering that event. Unlike chess (where the AI has to meet or exceed human cognitive skill), here the AI has the tremendous advantage of sub-millisecond recognition and r
Re: (Score:2)
"Now, in chess, all the computer has is to play better than its opponent. It doesn't get that tremendous advantage of our inability to match machine precision and speed. Chess is a far more demanding problem than a video game (at least, for a computer)."
No, that's not true. Even if we consider pure-playing actors (that is, without an included library of matches) speed is also to its advantage. Both speed of learning -how much time takes a human player to analyse, say, the 100 most famous matches in chess
Re: (Score:1)
"users" (Score:2)
"Sony Built an AI That Can Beat Users at Video Games, With Honor"
So... someone who plays a game is no longer a player, they are all "users".
Looking forward to see helpdesk tickets titled "player can't access server XYZ".
"All your race are now belong to us..." ? (Score:2)
Now load it into a Tesla and we might seo some real self-driving... :-)
When you know the code (Score:2)
Re: (Score:1)
This already is started:
Minecraft: https://minerl.io/ [minerl.io]
Starcraft: https://deepmind.com/blog/arti... [deepmind.com]
With roots back in 2013 with ATARI games: https://towardsdatascience.com... [towardsdatascience.com]
Really just a glimpse into the future (Score:1)
AI. will be better at literally everything involving thought before I die (I and I don't have that long to live).
If the thought can be implemented in a physical model (like a self driving truck), it will be better (lower accident rate) plus have superhuman traits (work 24 hours a day... extremely low unscheduled downtime... no unions (tho the truck manufacturer may pull a John Deere on you)... no harassment suits... wont' get tired... won't find a way to skim valuable cargos... etc. etc.
It's estimated to el
Not new for Sony (Score:2)
AI that can beat humans is nothing new for Sony. Way way back on the PlayStation 1, my local crowd of friends used to play a LOT of the game Armored Core. At times, you had to fight AI adversaries in a sort of death match, or you could play against another human if you had the link cable. We did a LOT of that. Fun by the ton.
But it became apparent the AI was cheating to win. Or at least had an advantage no human could match. The human players were limited by the effectiveness of the radar system eq