The Neurosurgical Simulation and Artificial Intelligence Learning Centre at McGill University, also known as The Neuro, conducted a study that has shown how an artificial intelligence system can outperform remote expert human instructors in teaching medical students.
The study, which was published in late February in the Journal of the American Medical Association, recruited 70 Canadian medical students who performed virtual brain tumour removals on a simulator over five sessions.
The students were randomly assigned into three groups who received feedback from either a panel of remote experts, an AI-powered tutor called the Virtual Operative Assistant (VOA) or no feedback at all.
According to Dr. Rolando Del Maestro, director of The Neuro, the research team hypothesized that performance results would be relatively equal between the virtual operative assistant and the online expert.
What was found, however, was students who received VOA instruction and feedback achieved 36 per cent better performance compared to those who received feedback from remote instructors. They also learned surgical skills 2.6 times faster.
“The AI tutor with artificial intelligence was better. It actually taught the things that we wanted to teach better,” Del Maestro said.
The results were surprising to Del Maestro at first, as he was initially concerned about how students would perform without instructions from a real person.
“One of the problems that we ran into was if you’re taught by a machine, the stress will decrease your performance and therefore be not as good as having an instructor there,” Del Maestro said. “With an instructor there, you would have hopefully less stress.”
Del Maestro added that the VOA offers certain advantages to surgical training given the amount of information it collects and processes, including how far apart a surgeon’s instruments are, their acceleration and even the amount of patient bleeding in cubic centimetres.
Despite the promising results, Del Maestro said the future of AI in healthcare is still uncertain.
“What we really want to do with these results is to make the operation safer,” Del Maestro said. “Yes, it would be nicer if AI made the operations more efficient. But the frequency is only one part, the goal is to make operations as safe as possible to get the best possible result.”
Susan Ruypers, a research specialist at the Brain Tumor Foundation of Canada, which partially funded the study, said The Neuro’s work is part of a few AI studies the foundation has funded.
“People have done the same things for so many years. It’s great to start to see different ways of thinking,” Ruypers said. “With more projects like this yielding positive outcomes, medical students may be able to perform and conduct complex surgical procedures more effectively and efficiently.”
Del Maestro said he and his team are currently conducting a follow-up study that replaces the online expert with an in-person expert who can give live feedback to students.
“If the expert feels that the individual isn’t learning the procedure, the expert may do the procedure himself for a period of time, allow the student to watch it and say ‘This is how you should do it.’”
Ruypers said the progress in technology and health care in recent years has been exceptional.
“Ten years ago, we could go back and say that someone’s going to create this type of machine that can teach people to learn, and people would think it was almost impossible,” she said. “Now with studies like this, we discover that we can not only create advanced AI machines, but they can actually teach us better than we could teach others.”
Featured image provided by McGill University.
