Beyond Verbal has gathered an impressive database of 2.5 million emotion-tagged voices in upwards of 40 different languages over the past 21 years. Its purpose is to use machine learning and big data to investigate, with their medical partners, whether connections can be made between vocal intonations and health issues.
The ability to recognise certain health issues based on a voice is not new, and you don’t always need a machine for that. By listening to a voice one can identify diseases such as dementia or Parkinson’s. But machine learning will be used to go beyond that, by zooming in on data difficult observable to the human ear.
In particular, Beyond Verbal hopes to expand recognition to medical and physiological conditions which are not associated with the brain. “That’s where this work is becoming very fascinating, and that’s what we’ve been working with the Mayo Clinic over the past two years to do”, Mor says. This may include the diagnosis of coronary artery disease, for example, which Mor said is something that is being looked at very closely.
“It might be that it monitors your phone conversations, or that you speak to an app for 30 seconds every morning — and from that it gives you high-level indications. You either get a green light saying everything is okay, a yellow light saying it’s worth monitoring closely, or a red light telling you that something is significantly different and it can be mapped to a specific condition, so you should go see your doctor.”
Your voice sounds different
“We are not calling what we’re doing right now diagnostics,” says Yuval Mor, CEO of Beyond Verbal. “Instead, it’s about long-term monitoring and decision support systems. The diagnostic tools that a doctor can use right now can do a better job than we can with vocal markers. However, what we’re doing is to use voice abnormalities observed over time to say that your voice is significantly different from how it sounded yesterday, a week ago, or two months ago — and then to correlate those changes with specific things we can identify.”The ability to recognise certain health issues based on a voice is not new, and you don’t always need a machine for that. By listening to a voice one can identify diseases such as dementia or Parkinson’s. But machine learning will be used to go beyond that, by zooming in on data difficult observable to the human ear.
In particular, Beyond Verbal hopes to expand recognition to medical and physiological conditions which are not associated with the brain. “That’s where this work is becoming very fascinating, and that’s what we’ve been working with the Mayo Clinic over the past two years to do”, Mor says. This may include the diagnosis of coronary artery disease, for example, which Mor said is something that is being looked at very closely.
Your own guardian angel
Right now the Beyond Health Research Platform is still rolling out, but Mor stresses that the researchers are thinking big on this project. “The long-term vision is that everyone will have their own companion, a guardian angel, which could be anything from your mobile phone to your Amazon Echo,” he said.“It might be that it monitors your phone conversations, or that you speak to an app for 30 seconds every morning — and from that it gives you high-level indications. You either get a green light saying everything is okay, a yellow light saying it’s worth monitoring closely, or a red light telling you that something is significantly different and it can be mapped to a specific condition, so you should go see your doctor.”
