How Do You Detect Cheating In Chess? Watch the Computer 328
First time accepted submitter Shaterri writes "Which is more likely: that a low-ranked player could play through a high-level tournament at grandmaster level, or that they were getting undetected assistance from a computer? How about when that player is nearly strip-searched with no devices found? How about when their moves correlate too well with independent computer calculations? Ken Regan has a fascinating article on one of the most complex (potential) cheating cases to come along in recent memory."
Some possibilities.... (Score:5, Interesting)
1 -- they learned chess mostly/exclusively by playing against a machine rather than against human opponents. Then their strategy would mostly be informed by or similar to the type of gameplay which they have observed kicking their own ass as they learned to play. Thus they might "play like a computer" because they have internalized the computer's algorithms as they learned to play chess.
2 -- they randomly play chess in manners that appear like a computer's algorithms. In fact, hey, when they say that the person's moves closely mirror the moves a computer would make, shouldn't they specify which computer program/algorithm they mean for making chess moves? If you're running gnu/linux, you can play Xboard ( http://en.wikipedia.org/wiki/Xboard [wikipedia.org] ) as the front-end (visual GUI) with multiple possible engines [wikipedia.org] driving it underneath (such as Gnu chess [wikipedia.org]). You can even run Xboard to provide a running analysis of a game being played by others as you enter the moves played (see the man pages for analysis options). Different engines would probably come up with different moves/styles of play, right? So saying that a person's moves and play style mirror a computer is an insufficiently detailed accusation. The chess engine being suspected ought to be specified and indicated, in my opinion.
3 -- yes it is strange that someone with a normally low rating would suddenly get so far against a grand-master, and yes it is less suspicious when that happens with a yougner player, but why couldn't it occur with an adult player? Suspicion is just suspicion, not evidence.
4 -- there is a comment in the article about using Faraday cages at the match in order to decrease the risk for cheating. Remember that these days computers are very small, smaller than a deck of cards (yes, fancy phone in your pocket, I'm talking about you being as powerful as a supercomputer from the 1970s or 1980s). They could rig a fancy interface for their toes and have a shoe computer for all that you know.
5 -- is this all fallout from the pete rose type stuff, or because of lance armstrong from yesterday?
A cheating scandal in chess. Wowza.
Re:Simply put.. (Score:5, Interesting)
If you win against a computer you are cheating
I thought it was more if you win playing the same moves that a computer would make you are cheating.
This presupposes that computers play chess differently to humans. My understanding with chess is that there are certain 'stock' moves, openings and such like, that players memorize and use to their advantage. What if someone has set up positions and studied a computer response to those positions or play, would repeating the learned computer moves be the equivalent of cheating? What impact does an eidetic memory have on this where a person is able recall those positions and moves exactly?
The idea that there was some undetected cheating mechanism at play in the case in the article seems to go against the principle of Occam's Razor. The simplest solution to the issue is that either Ivanov just had a great tournament, or that his opponents played into situations for which he'd prepared with the aid of a computer, or a combination of the two. Such appears to be the level of mistrust in chess though that this simple solution is dismissed in search of something more fantastical.
Re:Some possibilities.... (Score:5, Interesting)
I play chess at the tournament level, and have played computer chess since the early 80's when the things were little more than jokes.
You simply cannot internalize the chess computer's algorithms. Believe it or not computers suck at chess and positional understanding. I did an experiment where I played a series of games against Fritz. I gave myself infinite time, sometimes taking 30-40 minutes per moves. I am not a titled player, but am above average for a tournament player. I did very well against Fritz when I had time to make sure my calculations were solid and found many times that Fritz really misevaluated the position. In one case, it insisted that it was up by 1.5 pawns but after 6 or 7 normal humans moves that a "C" player would have found, Fritz realized it was actually slightly worse.
Put a computer in a closed position and it flounders. The computer does not understand a position, it simply has a fairly decent evaluation engine combined with the ability to see every stinking possibility. It does not get tired. It does not have the emotional baggage that sometimes makes chess mistakes.
The computers understanding (evaluation) of a position is perhaps FIDE (ELO) 2000. It's calculation ability is perhaps FIDE 4000. Combine the two, and you get a "person" capable of FIDE 3000 chess. Give a grandmaster more time, and you tip the balance to the positional understanding rather than the raw calculation speed.
So now you get to the point about "internalizing" the chess moves is simply not possible. Put a computer in a complex Queen vs Rook ending, and you will see the computer play moves that a human just would never do. It isn't based on a few principles and understanding them. It is based on a 12 eyed monster seeing every stinking move possible 12-14 plies deep. Computers revolutionized our understanding of this endgame and many more.
Beyond the endgame, there are many points in a chess game where you can tell a computer made a move. First, the move objectively works, but does not fit any type of theme, or normal principle of the game. It isn't simply a good or even great move, it isn't that it just doesn't make sense immediately but rather it doesn't fit any framework of human understanding.
So, yes, I am convinced that you can pick up on cheating based upon a series of moves given the right circumstances.
And no, this is nothing new. Cheating has gone on in chess for decades. Computers have just made it easier for the non-elite to cheat.
Re:Simply put.. (Score:5, Interesting)
If you win against a computer you are cheating
I thought it was more if you win playing the same moves that a computer would make you are cheating.
This presupposes that computers play chess differently to humans. My understanding with chess is that there are certain 'stock' moves, openings and such like, that players memorize and use to their advantage. What if someone has set up positions and studied a computer response to those positions or play, would repeating the learned computer moves be the equivalent of cheating? What impact does an eidetic memory have on this where a person is able recall those positions and moves exactly?
I can comment a bit at this. I used to play in chess tournaments in my state some years ago. I was at a very low level in most of them. To put it in simple terms, I was about as far away in talent from the best players in my state (not my country or the world, but just my state) as I could be. I gave up playing chess because bluntly put, computers ruined it. You are right that players memorize openings. The list of known openings and known variations of those openings is staggering. Honestly, it's more than most people can memorize. Back in the 1990s when I played, it was unusual for a known opening to go beyond maybe 7 or so moves before you "got out of book" as they put it and responses started to deviate from known ones. Keep in mind that while you could always deviate very early from known responses, the odds of such being successful were quite low as if the move was really any good, it would already be known. Now add to this the knowledge that since white moves first, he controls the game. So if I as a player think "I'm really hoping white opens with e4 as I've been dying to try out the black side of this variation of the Ruy Lopez", white may open with d4, destroying my chance to defend an e4 opening. Even if white opens with e4 as I hope, on his 2nd move he may prevent the Ruy Lopez variation that I wanted to play. So you can see that what you have to learn is quite enormous because when you play black,you have to be prepared for all kinds of openings that you may not ever play when you have the white pieces.
Computer analysis took to openings to deeper levels of known good responses. So an opening that used to be maybe 7 moves long before you got out of book was now 13-14 moves long. At some point it just becomes impossible to keep up. To be honest with you, I put a lot of time into trying to improve and I really didn't make much progress. It was already tough enough for me to keep up before computers got involved and I just gave up as I felt like I was getting left further and further behind. To be honest with you, a lot of the tournaments weren't much fun. A lot of the guys who showed up to them were really weird. It made me question whether I really wanted to spend a lot of time getting better at something that attracted defective people to it. It's not unheard of for guys to be exceptionally good at chess and be homeless because they can't keep a job. Fischer himself was a genius player but if there was ever a crazier World Champion than him, I don't know who that would be.
Re:No way! (Score:4, Interesting)
Not off-topic. Dead on.
Lance Armstrong was initially judged by the USADA to have used PED based not on testing results, but on the testimony of former teammates, some of whom failed their own tests, and may have had an ax to grind. First, because they feel they may have been singled out because of their assocation with Armstrong,second because they may have been pressured by Armstrong or the relationship to use PED, third because they may actually have witnessed Armstrong either taking PEDs or encouraging it, and fourth ALL of the above. The end result is that no one in cycling at the international level will be able to withstand the mere accusations. Non-analytical positives will become the norm. Every champion will be suspect, unless 100% testing is done, and then, as in Armstrong's case, new tests will be conducted on previosuly collected samples, in effect finding athletes guilty in arrears for using PEDs not yet known. Eventually coffee and Gatorade will be banned. And this will stain cycling to the point that fans like myself will turn away.
Chess will go this route. No Master of any rank will be allowed to exceed their 'reasonable' ability. Analysis will be conducted, perhaps electronic surviellance will be used to both check for transmissions and as forensics to be subjected to detailed analysis, suspects will be accused, strip-searched, imaged, run through the metal detectors, scrutinized, and judged guilty based on non-analytical positives. Chess will devolve into the meanest of states, blood sport not for the winners, but for the losers. I expect past upsets to be scrutinized for problems and winners discredited, even posthumously.
A pox on all of it. I'm watching the America's Cup. Less cheating, more suspense, and people could drown.
Re:Some possibilities.... (Score:4, Interesting)
1 -- they learned chess mostly/exclusively by playing against a machine rather than against human opponents. Then their strategy would mostly be informed by or similar to the type of gameplay which they have observed kicking their own ass as they learned to play. Thus they might "play like a computer" because they have internalized the computer's algorithms as they learned to play chess.
Not possible. The cheater made exactly the same moves as the chess program Houdini 2.0c. Not "play like a computer", but play identical to a specific version of an actual computer program. With the exception of the opening moves, which of course are just a random choice from the usual standard openings.
Statistical analysis used in online tournaments (Score:5, Interesting)
Over the board (OTB) is one thing, but online (c)heating becomes incredibly hard to detect in situ, for pretty obvious reasons. The online chess community has taken a couple of approaches to detect this. For PlayChess Online (a server that hosts online games), they try to detect if your computer is running another process that is a known Chess Engine while you are playing your game. Easily subverted by having two computers, or even a Virtual Box setup.
The most successful way to detect cheating is in postmortem review. I worked with the ICC/FICS Slow Time Control league team (one guy usually) who would run move correlation statistics off suspicious games. There were lots of parameters in his analysis to tweak: ignore book (pre-planned) openings, use endgame tables, tolerance threshold, plys deep to look, how many branches to examine, etc. I was part one of the peer reviewers of the system and an occasional game. The basic idea was to run the moves through a few engines and find out how high the move correlation was for both players. In certain points of the game, the move correlation is very high because good candidate moves are obvious. However, over a single 35move game (avg), GM correlation with any of the popular chess engines (even HIARCS, which supposedly plays more like a human) was around 23%. 1800 level players (club level) were even less. Magnus Carlsen wasn't on the scene yet; he apparently learned more from the computer than any human. Perhaps he'd be higher. The typical cheater scored around 98%.
This of course is not to say that there couldn't be a player who "thought like a computer". But this would put in question the main criticism of game specific AI, and general AI, that they do not actually model how the human brain thinks. Finding a human who thought like a computer would actually be incredibly interesting to the whole field of AI. That being said, the burden at that point is on the cheater to prove because he is well beyond a reasonable doubt.
Re:Simply put.. (Score:4, Interesting)
Going to the FA,
Either
or
The level of mistrust in chess is not that high but this case was exceptional.
There was one thing in the article which was pretty much garbage - Although Magnus Carlsen recently broke Garry Kasparovâ(TM)s all-time rating record to reach 2861, my program for "Intrinsic Ratings" clocked Ivanov's performance in the range 3089-3258 depending on which games and moves are counted according to supplementary information in the case . . ..
Magnus Carlsen's rating is based on his results over the last 12 months. He has played tournaments to a standard of over 3000, just not over a whole year. If you play a tournament containing strong players, win most of your games and draw the rest, you will have a stratospheric rating from that tournament. Ivanov actually lost a game or two and the author is clearly cherry-picking, only counting games where he won and ignoring those where he did not.
ELO ratings are based on wins, draws and defeats, along with the opponent's rating. The quality of the moves made is totally irrelevant.
How we detect cheats on Yahoo Chess 1 minute games (Score:3, Interesting)
Setup a Fool's mate [wikipedia.org] intentionally. A cheater using an auto-play program will fall for it at no time. A human cannot spot a fool's mate that fast. As long as the game is finished (checkmate or not) within 30 seconds since it started, the game will not count as rated.
Re:Simply put.. (Score:4, Interesting)
I also played (back in the 80s...never got beyond a 1500 rating myself). Your comment that most openings didn't go beyond 7 or so moves is incorrect. Look up "The Encyclopedia of Chess Openings", and you'll see what I mean. It was a five volume set of books back in my time. As you pointed out, you had to learn to play a whole set of openings. As black, learning the Pirc (e4 opening), Grunfield (d4 opening), and English (c4 opening) covered 99% of all the games I ever played. Beyond that, learning strategic positioning, and basic endgames will allow you to win against nearly anyone who doesn't play regularly. There's a lot of pattern recognition necessary to play well, which requires a lot of practice. Also, learning to pick out "candidate moves" (see "Think Like a Grandmaster" by Sammy Reshevsky), and how to analyze them helped me a lot.
Re:Some possibilities.... (Score:4, Interesting)
The computer is there to be abused :) I am very interested in the process of improving at chess and what causes people to plateau. So, I've taken to a number of strategies to evaluate effectiveness. Ultimately, to improve, I found that one must truly understand what one does not understand. This sounds superficial or even tautological, but it isn't. Too often players chalk losses up to a "random blunder" or not having memorized an opening enough. The reality is that our minds have a very small set of "rules" we use to select moves.
During these sessions I actually wrote down my candidate moves for each move, and then wrote a rationale for why I chose the move. Often, one can make the right move for the wrong reasons and the other way around as well. By understanding thinking patterns, i can later identify mistakes and enlist stronger players in reviewing my games. It is effective, but very very very time consuming and energy consuming.
Fortunately, the computer is a patient partner. The downside is it cannot offer truly insightful commentary to help a human player. For that you need a mentor, or at the minimum a peer to assist.
Re:Simply put.. (Score:5, Interesting)
It might play at "grandmaster level," but it will not play "like a grandmaster." Humans and computers approach chess very differently, which is why this case is so interesting. From the statistical analysis, the guy's either cheating, or he's the not only best player in the world (3000+ level gameplay) but also the first human who thinks like a computer. Well, a human who thinks like a computer so long as a live internet feed of the game is being broadcast, but suddenly plays like a 2200-level human when the feed is cut.