Chess Ratings — Move Over Elo 133
databuff writes "Less than 24 hours ago, Jeff Sonas, the creator of the Chessmetrics rating system, launched a competition to find a chess rating algorithm that performs better than the official Elo rating system. The competition requires entrants to build their rating systems based on the results of more than 65,000 historical chess games. Entrants then test their algorithms by predicting the results of another 7,809 games. Already three teams have managed create systems that make more accurate predictions than the official Elo approach. It's not a surprise that Elo has been outdone — after all, the system was invented half a century ago before we could easily crunch large amounts of historical data. However, it is a big surprise that Elo has been bettered so quickly!"
umm (Score:5, Informative)
Not really. Jeff Sagarin has had two systems of rating sports teams for a while now. One, ELO_CHESS, is based purely on win-loss, while the other, PURE POINTS, takes into account margin of victory. According to him, the latter is better at predicting future results. From his analysis:
Submission error (Score:3, Informative)
Already three teams [kaggle.com] have managed create systems that make more accurate predictions than the official Elo approach.
1 EdR* 0.729125
2 whiteknight* 0.731656
3 Elo Benchmark* 0.738107 {-- The "official Elo approach"
Maybe we're counting from zero and they forgot to put it on the leaderboard?
More like commenter error (Score:3, Informative)
That number is "Root Mean Square Error", so lower is better
Re:Apples and oranges? (Score:3, Informative)
Since the Elo system is not designed to predict future performance (it's designed to capture current relative rankings), then is it really surprising that programs designed to predict future performance are better at it?
And if my current relative rank is higher than yours, doesn't that imply that if we play each other I should win? If not, what purpose does the rank serve?
Re:how are victory margins relevant to chess? (Score:2, Informative)
Re:Submission error (Score:4, Informative)
Re:how are victory margins relevant to chess? (Score:4, Informative)
If you ever find yourself in a game where you can sacrifice all your pieces to get to that position, DO IT!
Re:Microsoft's TrueSkill beat Elo before this comp (Score:3, Informative)
Here’s the problem with Battle.net 2.0: 2002s Warcraft III: Reign of Chaos is one of the most underrated video games ever created. And that’s before you learn its online apparatus is the foundation for modern matchmaking, where Blizzard Entertainment should get royalties every time you brag about your X-Box Live Trueskill rating. (Then again, I shouldn’t be giving Blizzard ideas right now.)
Here’s how Warcraft III matchmaking worked: Everyone starts at level one. The maximum level is fifty. You play players within six levels of your own. Win five games, gain a level. Lose five games, lose a level. The penalty for losing is reduced during levels one to nine. Thus, players who win half their games will become level ten.
It was simple and transparent. That was the hook, and people choked on it. It turned Warcraft III ladder play into what ICCUP serves for Starcraft players, a stomping ground so competitive that climbing the food chain gave you a shot at the guys who played for a living. That’s what a good online gaming system does.
The quote comes from Battle.net 2.0: The Antithesis of Consumer Confidence [the-ghetto.org]. I would encourage you to read the entire thing, but for reasons completely unrelated to this thread.
Re:differences are minute (Score:2, Informative)
Elo Anecdote (Score:5, Informative)
Not relevant specifically to this story, but I always laugh at the story of how a prisoner manpiulated the Elo system via closed pool ratings inflation [wikipedia.org].
Short summary: said prisoner only played against other prisoners, who he'd trained. Due to careful scheduling of the games, he rose from his true strength (probably sub-master) to being the second-highest rated played in the U.S. in 1996.
Re:More like commenter error (Score:1, Informative)
The leaderboard changes over time, and also consider this:
Re:Submission error (Score:3, Informative)
1 Elo BenchmarkOpen 0.723834
2 EdROpen 0.729125
3 whiteknightOpen 0.731656
so at this moment elo is back on top.
Could it be that people have been done some quickly jumpening to conclusions?
I guess george [nanc.com] is working at /. now.
Re:Apples and oranges? (Score:3, Informative)
No we don't. This is not the crawler you're looking for.
OG.