Crowdsourcing Game Helps Diagnose Infectious Diseases

Lucas123 writes "Researchers at UCLA have created an online crowdsourcing game designed to let players help doctors in key areas of the world speed the lengthy process of distinguishing malaria-infected red blood cells from healthy ones. So far, those playing the game have collectively been able to accurately diagnose malaria-infected blood cells within 1.25% of the accuracy of a pathologist performing the same task (PDF). The researchers hope that users of the game can help eliminate the high cost and sometimes poor accuracy of diagnosis in areas like sub-Saharan Africa, where malaria accounts for some 20% of all childhood deaths."

Didn't know a DDX could have the word balls in it.

[RyuuzakiTetsuya]

on one hand, gamers have been known to be gracious and kind enough to donate and keep CHild's play running.

On the other hand, gamers are dicks.

I hope this ends well.

Re:

[Anonymous Coward]

Usually it's not an issue with crowdsourcing. You rely on maybe 100 or more people classifying the same thing correctly, the key point is collective classification. See Citizen Sky for more examples.

Re:Didn't know a DDX could have the word balls in

[Anonymous Coward]

Once the troll population reaches sufficent levels:

"I have a sore throat and a slight fever."
*evaluating*
*You have AIDS, leprosy, and lupus.*

applying machine learning?

[tomlue]

Whenever I see these games I wonder if there is some effort to solve the task with ml as well. It seems like if you are getting a large number of players then you should have data on how those players are classifying the red blood cells. Given that data we can at least attempt a couple different techniques at a classifier.

The benefit could be two fold. On one hand a decent classifier could 'help eliminate the high cost and sometimes poor accuracy of diagnosis.' On the other hand if a diagnostician is looking at a blood sample having a classifier give some probability of classification (ie. saying 'this sample is 90% probable to be malaria infected') could help the doctor in their diagnosis.

I'd love to work on that problem. A very quick search of google scholar and citeseer doesn't pop up anything on the subject though, and I don't see any api on the linked site.

Re:applying machine learning?

[fuzzyfuzzyfungus]

I suspect that you could dodge some of the liability issues in this case.

Yeah, if Joe American goes in to MGH for a CT, pays north of 5k(after insurance), and learns that some unfeeling robot, rather than a tired radiologist, misdiagnosed him, it'll be malpractice lawyer time.

However, in areas where the current standard of care often doesn't include pathologist inspection of cells; because there aren't any qualified pathologists, or they are too expensive for the majority of patients, I suspect you'll find a much greater willingness to embrace the idea that you can perform the diagnosis with a glorified webcam(wasn't there some story on slashdot a little while back about some research group hacking microscope optics onto cellphone cameras?) and a nickel worth of CPU time...

It sounds crass when you say it in so many words; but what you can get away with in medicine is very much a product of what the alternative would be. If the current standard is sufficiently dire, even mediocrity counts as lifesaving. If it just so happens that machines are actually really good at this classification problem, all the better.

Re:

[account_deleted]

Comment removed based on user account deletion

Re:

[Iniamyen]

What's interesting though is that since you are effectively taking an average (not a mathematical mean necessarily, but broadly speaking an average), any individual answer (even the smart people's answers) could still be worse than the crowdsourced answer. I heard a story on NPR about a crowdsourcing project where this exact thing happened. I am sorry, I can't remember the details. In this case though, you can't necessarily build an algorithm, because it will still give an inferior answer to the crowdsourc

Re:

[Defenestrar]

A step up from that would be to embed a well trained machine in a simple lab on a chip. Print a few million and package them with a finger lance. If the LED turns red, then you foot your kid to a WHO clinic with proof in your pocket that you need to be at the front of the line. Not to mention the usefulness something like this would have at the same WHO clinic.

Re:applying machine learning?

[chooks]

From the referenced PDF:

we also developed an automated machine learning algorithm to detect the presence of malaria parasites,

Slashdotted-ish.

[game kid]

Signed up, website still works but game is stuck at "Retrieving Authentication" on Firefox 12 and IE8 (my machine with IE9 is getting an overhaul), so guess I'll just have to wait a while to find those buggy blood cells.

Re:

[Anonymous Coward]

Likewise.
 
Way to go, /. You just let tens of thousands of people with malaria die, you insensitive clods.

Machine learning

[kbg]

Would it not be simpler to have the pathologist train a machine learning algorithm and then use that instead of the game?

Re:

[degeneratemonkey]

The point of this approach is that machines can make pretty good decisions ~90% of the time, but when a more refined judgment call needs to be made, a human can be provided with an abstract rendition of the data and any relevant context, and can inadvertently perform a useful classification for the solver.

Its purpose is to solve precisely the kinds of problems that arise when you simply train an ML algorithm with known inputs and outputs, and then encounter an input that appears at least partially ambigu

Use as CAPTCHA-like system?

[Guppy]

I wonder if these sort of games could be implemented as a sort of Captcha challenge for niche situations. A simple yes/no diagnosis has insufficient complexity to serve as Captcha of course; you'd have to increase the difficulty by presenting a panel of several samples, multiple diseases, or require more detailed responses ("click on the abnormal Red Blood Cells").

This wouldn't be suitable for use as general purpose Captchas of course; any user seeing it for the first time won't have a clue what they're looking at. Rather, it could be used in settings where you have a body of dedicated users who repeatedly use a service over a long period of time, yet require anonymity without permanent user profiles. Forcing the users to go through initial training could be seen as a bonus, creating an additional obstacle for Mechanical-Turk workers. As an example of such a situation that pairs a persistent user pool with anonymity, consider something like 4chan.

On the downside, because it's 4chan, half your diagnosis results will be "You have AIDS (Pool's Closed)".