The researchers trained the system on multiple tasks, including looking at chest X-rays to diagnose conditions like a collapsed lung. When asked to diagnose cardiomegaly (an enlarged heart), the human-AI hybrid model performed eight percent better than either the AI or medical professionals could on their own.

“There are many obstacles that understandably prohibit full automation in clinical settings, including issues of trust and accountability,” says David Sontag, lead author of a paper that the CSAIL team presented at the International Conference on Machine Learning. “We hope that our method will inspire machine learning practitioners to get more creative in integrating real-time human expertise into their algorithms.”

Next, the researchers will test a system that works with and defers to several experts at once. For instance, the AI might collaborate with different radiologists who are more experienced with different patient populations. 

The team also believes their system could have implications for content moderation because it’s able to detect offensive text and images. As social media companies struggle to remove misinformation and hate, a tool like this could help alleviate some of the burden on content moderators without resorting to full automation.