Gamers Outdo Computers At DNA Sequence Alignments 61
ananyo writes "In another victory for crowdsourcing, gamers playing Phylo have beaten a state-of-the-art program at aligning regions of 521 disease-associated genes form different species. The 'multiple sequence alignment problem' refers to the difficulty of aligning roughly similar sequences of DNA in genes common to many species. DNA sequences that are conserved across species may play an important role in the ultimate function of that particular gene. But with thousands of genomes likely to be sequenced in the next few years, sequence alignment will only become more difficult in future. Researchers now report that players of Phylo have produced roughly 350,000 solutions to various multiple sequence alignment problems, beating the accuracy of alignments from a program in roughly 70% of the sequences they manipulated."
would be interesting to mine their data (Score:5, Interesting)
I'm highly skeptical that these gamers are really using some un-automatable human-only deep skills, especially since they aren't exactly extensively trained in this game, not to the level of, say, good Go players. So the interesting question to me is not that they beat current algorithms, but whether data mining these hundreds of thousands of alignments can tell us something about how they're doing it. My guess is that there are some heuristics that can be mined from this data that would massively speed up search.
That's a more general point about how these stories are always pushed, though, sometimes by media, sometimes by the researchers themselves. Imo the most exciting thing about successful uses of "human computation" isn't that we can harness people to do things, but that we can gain some large data sets that will make it so we don't have to get people to do them anymore. Or at least, that should be the baseline, imo: that humans can beat some hand-crafted algorithm is one thing, but can they beat machine-learned algorithms trained on those humans' own gameplay logs?
Re:would be interesting to mine their data (Score:5, Funny)
Re:would be interesting to mine their data (Score:4, Interesting)
This is not as silly as you might think. If it weren't for generally fucked up academic politics, this would work wonders. Get a bunch of popular porn sites to accept phylo points as payment. My bet is that there'd be plenty teenagers and basement dwellers who can trade plenty of time for the money they don't have to pay for porn :)
Re:would be interesting to mine their data (Score:5, Funny)
Worst case scenario is that the crackers write a really good DNA Sequencing programs to beat the captchas.
Re: (Score:2)
Wank and contribute to science in TDMA fashion, FTW!
Re: (Score:1)
This would have worked 10 years ago but now its easier to find porn about anything you want than it is to find pics of celebrities babies.
Re: (Score:2)
Unless you're trying to find porn of celebrities' babies, that is.
Re: (Score:1)
Re: (Score:1)
Re: (Score:3)
Re:would be interesting to mine their data (Score:4, Interesting)
wouldn't the problem at hand be NP-hard? maybe that's why gamers are beating the algos?
could this be a new way to "monetize" the internet? outsourcing hard problems for cash. with a cloud paradigm, it doesn't matter whether it's a cluster of computers or a crowd of aspies when the end result is the same.
Re: (Score:1)
Yes, the problem is NP-hard. Computers can solve NP-hard problems, just the algorithms to do so are often too slow to be useful so approximation algorithms are used instead. The humans are competing against results generated by an approximation algorithm. Having humans do the computation is more or less a different approximation algorithm. Given enough time, a computer could simply work out the full solution, but the amount of computation would be way too high.
Paying people for better results would be an in
Re: (Score:2)
Re: (Score:2)
Though they could have a separate large payment for an algorithm. Sure, it wouldn't be as much as paying for the work over time forever, but the algorithm inventor/discoverer is betting that someone else doesn't come up wit
Study the moves (Score:2)
Re: (Score:2)
The basic visual sense of a large animal includes an insane amount of brainpower for pattern recognition, interpretation and such things. There's a reason even very dumb animals can maneuver through the world much faster than our smartest robots. The heuristics used by humans in tasks like that are likely backed by the enormous processing power of the brain when it comes to analyzing pictures and patterns so they may not be terribly useful for computers.
Re: (Score:2)
Re: (Score:1)
I don't have the data to look through, but the general process of a human learnign new rules can be described as a sort of 'brute cunning algorithm.' It starts as a brute force, but recognizes certain trends, assumes consistency, and then makes jumps, narrowing back until they find a peak. Each person will display a different balance of brute force and portion skipping, so with a large enough gamerbase, you will get a collection of results that includes local maximums and a good chance of the true maximum
Re:would be interesting to mine their data (Score:4, Insightful)
That's true; a legitimate hypothesis is that this task involves very difficult skills that humans are naturally adept at, like object recognition in images does. My guess is that aligning DNA sequences is not as strong an example of one of those kinds of problems as object recognition, in particular because it doesn't involve the large amount of general knowledge about the world that we bring to bear when interpreting scenes; aligning sequences is more of a "formal" problem, than recognizing what constitutes a "chair". But I'll admit I could be wrong. One way to find out would be to try to see how much can be mined from the data. ;-)
Re: (Score:2)
brains don't recognise "chair", so much as they recognise $object and we are trained by our environment to be good at spotting whatever $object is our specialty. this doesn't take long, and i'd reckon that a problem like this would take minutes of training to get equal or better than a computer, and a bit longer to far surpass it.
brains are very plastic, changable things.
Re: (Score:2)
I don't think that's universally true; for example, computers are much better at characterizing integer sequences than (almost?) any human is, because humans are just not that good at integer sequences, especially those with any sort of non-trivial mathematical relationship underpinning them. Humans are good at a fairly specific set of pattern-recognition tasks, like object recognition in images. Even there, they vary surprisingly strongly by the specific nature of the task; for example, humans are much bet
Re: (Score:1)
Re: (Score:2)
Games only learn what they are programmed to learn. They aren't remotely as flexible as large animals because that kind of learning is so damn complex that it'd take way too much processing power. To learn from an action you first have to understand what the action was, games only store simple things like "player likes to use item X" but if a situation arises that the designer of the algorithm has not anticipated the game cannot adapt. Clicking on a button does not tell you why clicking the button was the r
Re: (Score:3)
Makes the original premise of The Matrix that much better than the "lol we're batteries!"
Re: (Score:2)
Diff? (Score:2)
Imagine... (Score:2)
Re: (Score:3)
call me when you can make a Beowulf culster of human brains, i bet porting C will be a real bitch though
Re: (Score:2)
Hooray for 'Many eyes' (Score:2)
A fantastic example of why the building blocks of human life should not be patentable and hidden away by pharmaceutical companies.
Re: (Score:2)
Not only that, they also seem to be illiterate. Having to watch a video tutorial, narrated by a girl who couldn't read a book for kids lest her live be saved, just to learn how the damn thing is scored? I thought they know how to write, being in the academia and all? Their results are obviously secret, because if you just happened to educate yourself on a puzzle that you can't solve, the par results are inaccessible. Big stinkin' sikret, I tell ya. But no, they must have included the stupid car game countdo
Re:Time limit (Score:4, Interesting)
I haven't played the "game", but I suspect that there are a lot of things like time limits that can serve as a motivation factor that actually increase user output in the aggregate. Having a time limit can give you a sense of urgency that will force you to work faster. The error rate may increase, but overall productivity could still be higher given that higher number of "answers" given per unit time.
Imagine two Magic The Gathering players. One assembles decks painstakingly, spending hours crafting card ratios just right, and researching combos to get the perfect balance of # cards to power of combo. Then he play tests it, goes back and makes adjustments, etc. The other throws decks together quickly and play tests them very quickly. He adjusts the deck without as much deliberate thought, but rather more quickly (perhaps intuitively). He is able to iterate much faster, and it's easy to imagine that if each player were given 1 month to pursue these strategies, the latter could easily come out with more decks that met some minimum standard of success (that was suitably high).
(Obviously, it's easy to see how inane, useless rewards can spur gamers to expend more time and "contribute" more to the game... just look at badges, trophies, etc. But I think it's just as possible that "negative" reinforcement ideas, such as a time limit, can have the same effect.)
Re: (Score:1)
WHAT?!?! AMAZING! (Score:1)
Achievement/Trophy Unlocked! (Score:2)
Read the fine print... (Score:5, Informative)
Re: (Score:2)
I'm not surprised. Their UI is disgusting, their scoring rules hidden behind a most amateurishly done video (they must expect you to write down fucking notes), and the whole project just seems in-your-face obnoxious. What a let-down :(
Re: (Score:2)
If you were to ever try to solve a real MSA problem by hand, you would quickly understand how completely hopeless it is.
Nope nope nope [u-strasbg.fr]. From scratch, perhaps it looks daunting. But the big parts are actually pretty easy. I should stress that BAliBASE is used as a benchmark for new alignment programs, including MultiZ (which, btw, is actually a little old now.)
Re: (Score:2)
Re: (Score:2)
Wholly agreed—but it should be emphasized that the mere existence of BAliBASE asserts that the trickiest part still requires direct intervention. There are precious few things in the universe that a computer can do that a human can't do more slowly or in smaller chunks, after all—and most of those are comparatively silly things like set voltages. I could, for example, implement ClustalW by hand, no sweat—just give me your favourite BLOSUM table, a few other parameters for gap size, a stack
Working link (Score:1)
http://phylo.cs.mcgill.ca/eng/ [mcgill.ca]
Failed. (Score:1)
Let's see... (Score:2)
Humans: Millions of years
Computers: Tens of years.
Not sure there is a story, here...