Poker Pros Win Against AI, But Experts Peg Match As Statistical Draw 65
hypnosec writes with some positive news for Skynet watchers, in that humans still have at least a slight lead against the AIs who might one day imprison us in energy-harvesting goo tanks, or at least beating us in Las Vegas. The two-day poker showdown involving four of the world's top (human) players and a Carnegie Mellon University AI program called Claudico saw the professionals win, after several days of heads-up no-limit Texas Hold'em. "Despite the win, the poker players' $732,713 collective lead over Claudico wasn't quite large enough to attain statistical significance, experts have said. This means that the results can't be accepted as scientifically reliable thereby indicating that the "Brains Vs. Artificial Intelligence" competition effectively ended in a statistical tie." On the other hand, the computers sure got over what looked like a rout by the humans.
So which is it? (Score:4, Interesting)
Halfway through the competition, the four human pros had a cumulative lead of 626,892 chips. Though much could change in the week remaining, a lead of around 600,000 chips is considered statistically significant.
or
"Despite the win, the poker players' $732,713 collective lead over Claudico wasn't quite large enough to attain statistical significance"
Re: (Score:3)
Probably something to do with average winnings/hand, playing more hands with the total winnings not increasing much makes the significance weaker.
"Statistical Significance" (Score:3)
And then when all's said and done, if indeed the level of significance was too low (e.g. p too high), maybe we could get
Re: (Score:2)
This is why we need to teach students that theory drives stats and not the other way around. There are tons of bullshit results out there that are perfectly statistically valid but logically bullshit.
Re: (Score:2)
Maybe the mainstream media can't understand anything more complicated than that woefully misleading phrase, but why can't "techie news" and "news for nerds" give us the details? As-is, we can't have any sort of meaningful discussion here about the results of this experiment; we can only speculate.
Re: (Score:2)
We can't have that for the same reason we're constantly flooded with "Ew nerds are gross!" stories. They're pandering to a different crowd that can't understand the details.
Re: (Score:2)
Probably something to do with average winnings/hand, playing more hands with the total winnings not increasing much makes the significance weaker.
Is there a difference between what a gambler calls a "useful advantage" and what statistician calls one?
I know a couple of professional poker players - they reckon the "house" has as 5.5% advantage, and are pretty pleased with any system that gives them a 2% advantage.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:1)
Per Player? (Score:2)
Maybe it means a 600k chip lead per individual head-to-head. Or it could mean FUCK THS COMPUTARZ ROOL
It was the beginning of the rout of civilization, (Score:1)
Re:Why non-conclusive? (Score:5, Funny)
Personally, when I gamble and end up about 3/4 of a million dollars in the hole, I assume that I lost.
Re: (Score:1)
Mod up.
Re: (Score:2)
Personally, when I gamble and end up about 3/4 of a million dollars in the hole, I assume that I lost.
That sounds more like a conclusion than an assumption.
But the question isn't "Who won?" It is: "On the basis of this result what can we say about who will win next time?"
I don't know what kind of measures they used, and there are a couple of links in this discussion to papers pointing out how problematic p-values are, but it is perfectly possible for the weaker competitor to win any given competition. All it requires is that the width of the performance distributions be large enough to give significant over
Re: (Score:1)
There is a slightly better description here:
http://www.smh.com.au/world/man-bests-machine-in-twoweek-poker-tournament-20150509-ggxwuz.html
Re: (Score:1)
Obviously you lost, but the question is why. Was your expectation negative or did you get unlucky?
Re: (Score:3)
Welcome to most poker. 75-85% of hands you fold at a full table without putting any money in. Another big chunk you fold to a bet either pre or post flop. Most of your money in any session is won or lost in a few big hands.
Optimal outcome (Score:2)
I think this is the optimal outcome for the scientists. They can show: 1) We have done something (i.e. the poker bot is not too far from the best human players so our time has not been wasted). 2) There is more to be done (i.e. give us more money to look at this).
Also, I do think it is a quite impressive outcome.
Re: (Score:2)
reading the details this looks more like a complete route and they are just trying to put a nice face on things. A person that relies heavily on maths and is a tight player would have similar results, these are the type of players that professionals make their living off, players that slowly lose money too you, not fast enough to scare them away and make them realize they suck though.
Is playing a game Artificial Intelligence? (Score:1)
Re:Is playing a game Artificial Intelligence? (Score:4, Informative)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Any AI problem that is solved, is no longer AI.
Re: (Score:3, Interesting)
In this case, yes, it is. All they gave it as inputs were the rules to the game. The AI had to make its own determinations after that for what the optimal strategy was, how it would go about achieving it, and how it should respond to the individual characteristics of each of its opponents.
Alternatively, ask yourself if playing a game takes intelligence at all. We could argue that all that the pros are doing is making their best estimation of the statistical likelihood that they'll win a hand, then betting a
Re: (Score:2)
All they gave it as inputs were the rules to the game. The AI had to make its own determinations after that for what the optimal strategy was,...
Yes, that was where the original question was going (despite someone thinking it was trolling and modding me down).
When the domain is constrained by a set of rules and the programmers (not really "the AI") have built a decision tree on how to act based on current conditions, it makes me wonder what is meant by intelligence. Obviously it takes some intelligence for a person to be good at the game; but it also takes some intelligence to multiply two numbers together. I don't think anyone claims multiplicatio
Re: (Score:2)
Alternatively, ask yourself if playing a game takes intelligence at all. We could argue that all that the pros are doing is making their best estimation of the statistical likelihood that they'll win a hand, then betting accordingly along an optimal path that they've cultivated through experience.
Perhaps that's true.
It's really no different than what this AI was doing either, it would seem.
Except that human players somehow manage to make their estimates through an advanced "higher-level" intuition, while TFA says: "Claudico [the AI] sets its own strategy, Brown noted, and that strategy occupies about two terabytes of data -- far more than the CMU team could analyze."
And here comes the problem with a single definition of "intelligence." If you believe (as I do) that "intelligence" fundamentally requires a level of adaptability, as well as abiilty to alter one's own behav
Re: (Score:2)
Yet somehow humans manage to do well without all of that perfect access and recall to huge quantities of information.
Humans also have access and recall to huge quantities of information. It may not be perfect, but in the case of professional poker players, it's close enough.
Re: (Score:2)
I completely agree with everything you said in response to me. The only thing I feel a need to address is this:
Meanwhile, you ask whether playing a game takes "any intelligence at all." If it does not [...]
I was trying to call attention to a double-standard he had set up. I took it as a given that we would all agree that games take intelligence to be played, and that as a result we would recognize the invalidity in his assertion that this is nothing more than applied statistics, since that assertion could be applied equally as well to natural intelligence.
As you, I think that what we've seen on displ
Re: (Score:2)
Statistical (Score:2)
The losing team of every competition likes to say things like 'statistical draw'.
Re: (Score:1)
20,000 hands against each player? (Score:1)
Re: (Score:3)
the reason people can multi-table so much is because you rarely have to do anything. mostly you are just folding until you find hands or positions you want to play from, multi tabling reduces the boredom and stops you from playing in hands you shouldn't. It is not uncommon to go 10-20 hands in a row on a table where all you are doing is folding. multi tabling is not hard.
Re: (Score:2)
the reason people can multi-table so much is because you rarely have to do anything. mostly you are just folding until you find hands or positions you want to play from, multi tabling reduces the boredom and stops you from playing in hands you shouldn't. It is not uncommon to go 10-20 hands in a row on a table where all you are doing is folding. multi tabling is not hard.
Yeah sure, you are right and multi-tabling works great if you are a level above your competition. Where multi-tabling falls down though is studying your opponents style and play. Necessary to give you an edge against players who are of a similar skill to you.
Re: (Score:2)
in online poker that isn't hard. There are a heap of programs that track the betting behaviours and stats of your opponents so you can know as much as possible without having to monitor the table constantly. online poker by its very nature is far more limited in the information you have and as such is not to difficult to track.
If statistical result is important in poker (Score:2)
$732,713 is a statistical tie? (Score:2)
Awesome. I want to play poker with these guys... we can just draw right at the beginning, and they'll give me 700 grand, right?
Give it a try yourself... (Score:1)