IBM Researchers Teach Pac-Man To Do No Harm (fastcompany.com) 135
harrymcc writes: The better AI gets at teaching itself to perform tasks in ways beyond the skills of mere humans, the more likely it is that it may unwittingly behave in ways a human would consider unethical. To explore ways to prevent this from happening, IBM researchers taught AI to play Pac-Man without ever gobbling up the ghosts. And it did so without ever explicitly telling the software that this was the goal. Over at Fast Company, I wrote about this project and what IBM learned from conducting it.
The researchers built a piece of software that could balance the AI's ratio of self-devised, aggressive game play to human-influenced ghost avoidance, and tried different settings to see how they affected its overall approach to the game. By doing so, they found a tipping point -- the setting at which Pac-Man went from seriously chowing down on ghosts to largely avoiding them.
The researchers built a piece of software that could balance the AI's ratio of self-devised, aggressive game play to human-influenced ghost avoidance, and tried different settings to see how they affected its overall approach to the game. By doing so, they found a tipping point -- the setting at which Pac-Man went from seriously chowing down on ghosts to largely avoiding them.
Precisely. (Score:3, Insightful)
This is being made up to be something it's not. Just another example of how non-technical people might interpret technical work.
And just another example of using trendy buzzwords to get undeserved attention.
Re: (Score:2, Flamebait)
"Its just an applied dataset. Nothing magical. Nothing world-changing."
You have no instinct for science.
Re: I fail to see what this has to do with ethics (Score:4, Insightful)
Yea. Even sillier when you remember that Pac Man gets points for eating ghosts, exponent style. The ideal Pac Man eats four ghosts per power pellet, otherwise you are leaving points on the table. Because the game has a finite number lf levels, every abandoned ghost is a lower total score.
Re: I fail to see what this has to do with ethics (Score:5, Funny)
Re: (Score:1, Funny)
That's the Conservative Pac Man you're describing. The Liberal Pac Man ...
-
No, the Liberal Pac Man is a poseur who expects someone else to take care of the world's problems because he is too busy trying to be cool to actually get his hands dirty.
Re: (Score:1, Flamebait)
That's the Conservative Pac Man you're describing. The Liberal Pac Man ...
-
No, the Liberal Pac Man is a poseur who expects someone else to take care of the world's problems because he is too busy trying to be cool to actually get his hands dirty.
He also insists that you call him Zhe Pacman. He identifies that way.
Re: (Score:1, Funny)
I see the start of a new movement of the BGM with t-shirts, rallies, riots and all....
Re: I fail to see what this has to do with ethics (Score:5, Funny)
That's the Conservative Pac Man you're describing. The Liberal Pac Man takes care not to eat the ghosts, and uses whatever points he get from eating pellets in a sustainable way to pay for ghost shelters and outreach to better understand them and protect them.
And PAC Pac Man doesn't disclose how many points he has and uses them to get the ghosts to do his bidding.
Re: (Score:1)
The Liberal Pac Man
There is no Liberal Pac Man, only Liberal Pac Gender Neutral Personhood Entity
Re: (Score:2)
There is no Liberal Pac Man, only Liberal Pac Gender Neutral Personhood Entity
Well... there's the other game Misses Pacwoman, I think it's called. They could also introduce a new version:
Mr(s) Pac gender-fluid character
Re: (Score:3)
Meanwhile SuperPac Man just gobbles up all the points and uses them to lobby the game developer.
Re: (Score:2, Funny)
'Pac Man'? Excuse me, it's 2018. Who do you think you are to assume his gender orientation -- how very problematic of you.
Non-binary-genderqueer-trans-pac-demi-snowflake; thank you very much.
Re: (Score:3)
The Liberal Pac Man takes care not to eat the ghosts, and uses whatever points he get from eating pellets
Not very many, b/c the Liberal Pac Man deliberately allows the ghosts to defeat pac man.
Once by Pinky, then Inky, and Clyde, in that order --- Blinky doesn't get a turn, b/c there are only 3 Pacman Lives, and Blinky has too much privilege.
Re: I fail to see what this has to do with ethics (Score:5, Interesting)
"Pacifist" runs of games are popular with the speedrunning community. Depending on the game it might be an achievement to simply finish it without harming anything, or it may just be another category to get the fastest time in.
Re: (Score:3)
Even non-speedrunning gamers can find some real joy in pacifist runs. It's often far more challenging to not kill enemies than it is to just play the game as designed. It requires different strategies and skills, and often radically different gameplay.
I've replayed a fair number of games that way, just because it breathes new life into an old favorite.
Re: (Score:2)
It requires different strategies and skills, and often radically different gameplay.
Or for most games it might just be plain not possible.... example Zelda: A link to the Past.
Plenty of unavoidable rooms that explicitly trap the player until all opponents in a room are slain, and key items
necessary to progress that require the same to obtain.
Also, winning any Super Mario Bros. game without beating king bowser and every castle boss will be a tad problematic.
Re: (Score:2)
Nethack has both pacifist conduct and the ability for types of monsters to go extinct via too many of that monster type dying. People have pulled of pacifist extinctionist challenges, which is possible because in Nethack, deaths done by your character break pacifist conduct (e.g. spells, physical attacks, etc (there are a few exceptions)) but deaths via other means (your pet killing things, etc) do not.
Re: (Score:2)
Wow. I've always been fascinated by Nethack but the learning curve seems steep.
Re: (Score:2)
For a learning system, there is usually a factor to measure success. If the game ignore the points, and just the number of dots it ate, and how fast they did this, as its own internal scoring mechanism, and to only having to eat a ghost as a lower penalty then dying. So it will learn a separate scoring mechanism for it to be optimized which isn't related to the actual one.
For me I use to play "pacifist" doom. Where I wouldn't kill any of the monsters, but just run away from them and see how fast I can ge
Re: I fail to see what this has to do with ethics (Score:5, Interesting)
That's the point. There are plenty of real world cases where we could be more efficient if we simply disregarded moral and ethical concerns. One of the concerns with machine learning is that they may find an optimal solution that violates ethical considerations. The problem is even larger when you consider an AI finding locally optimal solutions disregarding externalities.
For a classic example, Ford once determined that paying off expected damages in wrongful death suits would be slightly cheaper than refitting existing Pintos to not explode.
The Pac-Man simulation is a very simplified version of a case where, due to ethical considerations it is necessary to avoid the locally optimal solution.
Nope, just simplifies optimum. Ghosts are bad. (Score:5, Interesting)
> The Pac-Man simulation is a very simplified version of a case where, due to ethical considerations it is necessary to avoid the locally optimal solution.
The Pac Man doesn't avoid any optimal solution. It simply defines optimum as not to include not touching ghosts - ghosts are bad. In the classic version of the game, touching a ghost is bad. Unless you've eaten a Power Pellet in the last few seconds. They trained the AIto NOT learn the "unless you've eaten a pellet". It just does "touching ghosts is bad".
There's nothing moral, or even interesting, about "in Pac-Man, touching ghosts is bad". Essentially, just one too stupid to know that Power Pellets do anything.
Re: (Score:2)
It goes beyond that. In your suggested scenario, it would be neutral to eat a ghost. What they actually trained was that eating ghosts is bad. That is, it learns to not do that even though it would result in a higher score.
Ghosts are what kills you. Touching ghosts is bad (Score:2)
In Pac-Man, you die by touching ghosts. The one thing you want to NOT do, in a regular game of Pac-Man, is touch ghosts. As long as you don't touch any ghosts, you keep getting points.
That's the most basic understanding of the game you can have, what my four-year-old daughter would figure out in five minutes. That's also what the AI figured out.
*More advanced* players can learn the *exception* to the above simple rule. The *exception* is:
Only if you've eaten a Power Pellet in the last few seconds, touching
Re: (Score:2)
Do read TFA again. They did multiple runs with different emphasis on having the AI emulate the human player who avoided ghosts.
Re: (Score:2)
The power pills still have strategic value when you don't want to eat the ghosts.
Re: (Score:2)
Seems like a pretty sensible way of implementing "don't kill, but don't get killed either".
Slow news day, Slashdot? (Score:2)
But can you teach Google (Score:4, Insightful)
I don't think so. :*)
Re: (Score:1)
Our digital ghosts are just too yummy.
So they taught Pac-Man to always avoid the ghosts? (Score:2)
I read the article, a lot of words for so little information.
It's also pretty easy to avoid the ghosts when then are trying to avoid the player.
But, but ... (Score:1)
IBM researchers taught AI to play Pac-Man without ever gobbling up the ghosts.
Wocka, wocka, wocka ... nom, nom, nom.
Re: (Score:2, Funny)
Re: (Score:1)
Pac-Man might no longer be homicidal, but he still has a severe eating disorder.
He prefers to think of himself as "differently hungry".
Re: (Score:2)
Nobody is talking about the environmental devastation caused by over-harvesting of pac-dots.
Interesting, but perhaps useless (Score:5, Interesting)
Re: (Score:2)
Only if you program competition as a desirable goal. It might not be. You might program them to prefer cooperation to competition. Why would you make "dog-eat-dog" a trait, when AIs don't need to eat? Why take a nature-based paradigm and impose it on a system that has no need for it?
Why should AIs compete at all, is the
Re: (Score:3)
ICBMs don't need to eat either... Putin needs to eat. Guess, how he'll program his AI?
Because some of the tasks we may wish to entrust them will have adversarial aspects. If AI is charged with picking out wanted criminals from the crowd, it will need to weight severely harming the criminal — by having him arrested — vs. harming the rest of us a little bit — by letting him sli
Re: (Score:2)
Gosh, I hope the end state for AI isn't something as mundane as law enforcement or military applications. That would show a real lack of imagination on the part of human beings.
Re: (Score:3)
End state? Certainly not — if there is an "end" at all. But they'll certainly be used for that — indeed, already are used for that [slashdot.org].
And adversity is not going anywhere, unfortunately — with competing entities (corporations, nations, criminal enterprises, terrorist groups) using AI to their own ends. Just as you can not raise a child unprepared for adversity, you can not develop an AI unprepare
Re: (Score:1)
Sure you can. Use your imagination. Don't be so bloody-minded to think that competition is a necessary state for all systems.
Re: (Score:2)
Unlike a teddy bear or a saucer, AI is as multipurpose as a mind... Indeed, it is — or ought to be — a mind. To be useful in a real world, it has to know about all aspects of it, even if the degree of knowledge can differ between domains.
Now, maybe, you can train a limited one — aimed at solving a particular task. But that really is no different from rearing a child teaching him only one thing (dancing,
Re: (Score:3)
You're still thinking like an engineer, anthropomorphizing the AI. Do you think for a robot to be useful it has to be bipedal and have the approximate shape and function of a human? The point of an AI is to exceed humanity, not imitate it.
Don't make the mistake of thinking an AI has to operate like a human mind. Stretch your
Re: (Score:3)
severely harming the criminal -- by having him arrested
What? Just being arrested is being severely harmed? No. At best they GOTCHA or at worst it's a waste of everyone's time. But we want to make absolutely sure the bad guys never get away, right?
More like:
if Is_This_A_Person() then Report(Yes);
else Report(No);
Of course bringing asset forfeiture into the picture:
if Is_This_A_Person() then Report(Yes);
if Is_This_An_Object() then Report(Yes);
else Report(No);
Or for the latter, a much simpler:
Report(Yes);
BF would NOT be happy with this. That it [bartleby.com]
Re: (Score:2)
What? Just being arrested is being severely harmed?
You're not from the US, are you? Here just being arrested can quite literally ruin your life.
Arrests get airtime and newspaper articles, refusals to prosecute or being found innocent does not. So that means any future employer who does a cursory search on you finds out that you got arrested for something. They won't find out that it was a misunderstand, mistaken identity, or due to a corrupt cop. Have you seen any of the mugshot websites where they post all of them? Do you think they follow up on what happe
Re: (Score:2)
Anyone deliberately evading arrest does not want to be arrested. An arrest would be viewed as very harmful by the individual — even if it gives him a chance of redemption, paying debts to society, etc. An AI seeking to avoid harming anyone would be useless — like humans, it has to balance the different harms.
I, actually, long for that efficiency — because then we'll b
Re: (Score:1)
Only if you program competition as a desirable goal. It might not be. You might program them to prefer cooperation to competition. Why would you make "dog-eat-dog" a trait, when AIs don't need to eat? Why take a nature-based paradigm and impose it on a system that has no need for it?
Why should AIs compete at all, is the question.
Particularly when AI, if it ever becomes a true AI instead of a basic machine learning algorithm, will evolve in a time where it seems like we have plenty of resources to go around if we can figure out how to actually share them (at least for human needs). Whereas humans evolved in a time when sharing often meant you starve during the winter. Might give them a bit different perspective, if global warming or some other catastrophe doesn't mess everything up along the way.
Re: (Score:3)
I don't think this has ever been different for humans. We are stronger together. United we stand, divided we fall. We pool our resources to produce even more than we can alone. A being who spends his/her time hunting and gathering to survive has little time to spend researching advanced machinery to increase p
Re: (Score:2)
When my most recent dog died, I realized just how spoiled we are by our dogs.
Re: (Score:2)
The simple fact is that an AI which will utilize strategies which are more successful in the real world is going to do better than anything that refuses
Re: (Score:2)
So is cooperation. You are anthropomorphizing all systems, and that is a losing strategy.
They missed the broader ethics problem (Score:5, Funny)
Re: (Score:1)
Is it doing harm to the ghosts? They don't suffer any ill effects from being eaten. A bit like being turned into a newt, they get better. Plus, they get better a lot quicker than if they are left alone. I've been killed plenty of times by a regenerated just eaten ghost while in pursuit of still edible ones.
True "do no harm'" wouldn't allow the ghosts to become edible in the first place, making eating power pills verboten. That reduces the score further but as they aren't training for success that doesn't ma
Who Ya Gonna Call? (Score:2)
GHOSTBUSTERS!
IBM Researchers Get Bored And Play Pac-Man.. (Score:3)
That's what the title should be.
Pacman wasn't taught to eat ghosts... (Score:2)
The implications are that if we don't teach robots to kills us all, they won't kill us all? So all we humans have to do is stop killing each other, and we won't have to worry about Skynet and the Terminators.
Re: (Score:2)
The implications are that if we don't teach robots to kills us all, they won't kill us all?
They will "largely" avoid killing us all:
... Pac-Man went from seriously chowing down on ghosts to largely avoiding them.
Still, that doesn't sound like completely.
Slight contradiction? (Score:5, Informative)
Blurb: "they found a tipping point -- the setting at which Pac-Man went from seriously chowing down on ghosts to largely avoiding them."
So companies will presumably use a similar method to design AIs that will maximize corporate profit with only a _small_ amount of acceptable human murdering in the process?
Re:Slight contradiction? (Score:4, Insightful)
Then the title should be "IBM researchers teach Pac-Man to be mostly harmless".
Echoes of 2001 (Score:2)
Wouldn't that be the default state? (Score:2)
Wouldn't avoiding eating ghosts mean it just never got smart enough to know that you could eat power pills and then pass through them? That new optimal paths would arise when the power pill was active? If their algorithm added score for lower time or for points, I think this behavior would change.
"IBM Researchers Teach Pac-Man to Avoid Ghosts Even When It Is Advantageous To Eat Them" might not have the same ring to it.
Re: (Score:2)
Inb4 AmiMoJo:
You just hate ghosts because they look like they're wearing burkas.
Er... that's not teaching the AI *ethics*. (Score:5, Insightful)
It's having an algorithm grind out a solution to playing the game which meets an additional constraints, which they tweak.
If they'd actually taught the AI ethics, the AI would construct the play constraints for itself starting from ethical principles. At full human levels of ethical self awareness, the AI would be chasing ghosts down the hall and then -- unprompted -- stop and ask itself, "What am I doing?"
Re:Er... that's not teaching the AI *ethics*. (Score:4, Informative)
Everything is Artificial Intelligence!!!!!!!!!11 (Score:2)
"taught AI to play Pac-Man without ever gobbling" (Score:2)
What would be more interesting (Score:2)
tic tac toe number of players zero (Score:2)
tic tac toe number of players zero
can it play global thermonuclear war? (Score:2)
can it play global thermonuclear war?
So it's a scoring issue? (Score:2)
It's right there in the article, “There’s lots of rules that you might not think of until you see it happen the way you don’t want it,”
Well, if you told a computer to do EXACTLY the thing, and it's possible, they'll do EXACTLY the thing. If you said one thing, but meant another, they're not going to do what you mean, they'll do what you told them. So if your success metric is high score, and eating ghosts increases that, they'll eat ghosts.
In fact, this is how it works with people
Re: (Score:1)
Yup, they trained the ai to get food and avoid ghosts. Ethics has nothing to do with it.
Frying pan into the fire (Score:2)
What about Ms. Pac-Man?
Re: (Score:2)
Maybe instead they should teach him.. (Score:2)
to Do No Evil.
Ba-dum-tish.