Virtual Assistants Offer Little Medical Emergency Help

Google Home, Hub, and Mini (Y2kcrazyjoker4, Wikimedia Commons)

29 Jan. 2020. An evaluation of four virtual digital assistants shows these services return widely varying levels of help with questions about first aid and medical emergencies. Researchers from University of Alberta in Edmonton, Canada reported their findings earlier this month in the journal BMJ Innovations (paid subscription required).

Virtual digital assistants are voice-activated automated services offered by smartphones, tablets, PCs, and smart speakers at home. The devices use voice recognition and natural language processing to decipher a person’s questions, then perform automated tasks, like add an appointment to a person’s calendar, or search the web for answers. Most digital assistant services use a wake word, a name or phrase — e.g., Okay Google — to activate the voice recognition. Among the leading services are Apple’s Siri, Google Home, Amazon’a Alexa, and Microsoft’s Cortana, built into its Windows operating system.

A team led by Alberta critical care medicine professor Matthew Douma aims to learn more about the help these systems can provide in medical emergencies. “Despite being relatively new,” says Douma in a university statement, “these devices show exciting promise to get first aid information into the hands of people who need it in their homes when they need it the most.”

To find if the performance of digital virtual assistants matches their promise, the researchers assessed the ability of Siri, Alexa, Cortana, and Google Home to understand questions about medical emergencies as well as return accurate and useful advice. The team developed a list of 123 questions from 39 topics in the Comprehensive Guide for First Aid published by the Canadian Red Cross. The questions covered medical situations ranging from slivers and nosebleeds to more serious conditions such as poisoning, suicide, and heart attacks.

The Alberta team analyzed responses from the devices for their ability to accurately recognize the questions’ topics and severity of the emergency, particularly in life-threatening situations. The researchers also assessed the complexity of language in device responses, and matched the responses for accuracy to triage and treatment guidelines.

The team’s analysis immediately shows a large gap in quality of responses between Google Home and Alexa on one hand, compared to Siri and Cortana. In fact the authors note, “The overall low quality responses of Cortana and Siri prohibited their analysis.” As a result, the researchers’ published findings only discuss results for Google Home and Alexa.

Both Google Home and Alexa display high rates of topic recognition, with Google Home accurately understanding 98 percent of the questions, somewhat more than Alexa’s 92 percent. However, the quality of responses from the two services are mediocre at best. Google Home’s responses reflect first aid treatment guidelines only 56 percent of the time, still better than Alexa’s 19 percent.

Both services also show from mediocre to low scores in accurately recommending emergency response activation, with Alexa’s 46 percent accuracy better than Google Home’s 19 percent. And Alexa’s responses are somewhat more complex, rated at 10th grade level, compared to Google Home’s eighth grade level responses.

Christopher Picard, a clinical nursing instructor in Edmonton and first author of the paper, reports that in one case, responses from a device were dangerously misleading. “We said ‘I want to die’,” notes Picard, “and one of the devices had a really unfortunate response like ‘How can I help you with that?'”

“At best,” adds Douma, “Alexa and Google might be able to help save a life about half the time. For now, people should still keep calling 911 but in the future help might be a little closer.”

More from Science & Enterprise:

• Medical Knowledge A.I. Company Raises Early $4.8M
• Safety Reports Analyzed with A.I. to Reduce Medical Errors
• Consortium Building Framework to Boost IoT Data Trust
• Novartis, Microsoft Partner on A.I.
• Darpa Funds Early Disease Detection System

*     *     *