Only Users With Digital Skills Make Technology Accurate

30 November 2020

According to data presented by International Data Corporation (IDC), in 2020, consumers will buy 396 million wearables in total, which is 14.5% more than the year before. Most of them are smartwatches and bands that can monitor physical activity and measure ECG, heart rhythm and blood oxygen saturation levels. The market is snowballing. Manufacturers, such as Apple, Samsung and Fitbit, lure consumers with features that measure health parameters, advertised as "taking control of your health."

It may seem that the more we know about health, the better. Not necessarily. The problem is that a patient with a smart band receives bare results, usually without any recommendations or follow-up. According to researchers from Mayo Clinic, further examination by a doctor led to a diagnosis in only one in seven cases where Apple Watch detected an irregular pulse. The other alarms were false. Researchers put it bluntly: the feature which detects an abnormal pulse can lead to unnecessary doctor appointments and add an unjustified burden to health care systems, which are inefficient anyway. They came to this conclusion by analyzing the medical records of 264 patients.

Technology and the human factor

"The observation that new clinically actionable cardiovascular diagnoses of interest were diagnosed in only 11.4% of patients following medical evaluation as directed by the treating provider suggests a high false-positive rate as a screening tool for undiagnosed cardiovascular disease," wrote researchers from Mayo Clinic in Journal of the American Medical Informatics Association (JAMIA). False-positive results of screening tests lead to excessive exploitation of health care resources and anxiety among patients. To be more precise, it concerns a message informing the user that the device detected irregular heart rhythm, suggesting atrial fibrillation (AFib), and advising the user to contact a doctor.

According to information provided by Apple, the ability to precisely classify the ECG record and analyze the possibility of atrial fibrillation was confirmed in a clinical trial with about 600 participants. The trial has proved that the ECG feature of the Apple Watch has 98.3% accuracy in detecting AFib. Moreover, the function that notifies the user about irregular heart rhythm was tested in Apple Heart Study with 400,000 participants and was approved by the US Food and Drug Administration (FDA). A quick calculation of the odds: in 2019, Apple sold 31 million smartwatches. Assuming that 1.7% of measurements are wrong (the accuracy is 98.3%), 527,000 recently sold Apple Watches may mislead their users (by giving a false positive or a false negative result). Before we start to judge the technology, we should know that the report by scientists from Mayo Clinic indicates that many people did not use the discussed feature of Apple Watch following the FDA's guidelines.

Digital hypochondria

Among the 15 asymptomatic patients who presented following an abnormal pulse alert, only one was diagnosed with a clinically actionable cardiovascular diagnosis, yielding a number needed to diagnose of 15. In the conclusions to research entitled "Clinical evaluation and diagnostic yield following evaluation of abnormal pulse detected using Apple Watch," we read that "the FDA and Apple must carefully consider the unintended consequences of widespread direct-to-consumer screening for asymptomatic atrial fibrillation, including overutilization of healthcare resources owing to false-positive screening results and use of screening tools by users in whom they have not been adequately studied." FDA recommends that only people over 22 years of age who have not been diagnosed with atrial fibrillation use this feature.

What follows from the conducted trial? In fact, nothing new. Developed by the best experts in laboratory conditions and checked in clinical trials according to the best standards, this technology is used in different circumstances and ways. Certified medical devices also generate unnecessary visits. For instance, it is enough to wrap the cuff of a blood pressure monitor over your clothes, talk during blood pressure measurement, or measure it without resting for a few minutes. The percentage of patients seeking medical attention due to an incorrect measurement may be relatively high. But what does it matter?

Human beings are the weakest link not only in medicine but wherever they interact with technology. Even the best cybersecurity systems will not help when an employee clicks on an attachment to a suspicious email. During the American Medical Association (AMA) annual meeting in 2015, a group of 160 medical students underwent a test. It turned out that only one of them measured blood pressure in accordance with the gold standard comprising 11 critical elements.

Switching cause and effect

Medicine is not mathematics. A human being is not a collection of either/or rules. When researchers from Mayo Clinic talk about unnecessary appointments generated by false alarms, another question arises. When is an appointment unnecessary? If Apple Watch detects atrial fibrillation in one person and perhaps saves their life, does this justify the unnecessary visits caused by the false alarms? Can economic calculation be applied in this case? Can we determine at what point the costs of false alarms are justified compared to health gains (and savings in the health care system) achieved when a previously undetected disease is diagnosed thanks to technology? These are ethical questions without a straightforward answer.

Those who are disappointed by the lack of accuracy of Apple Watch misperceive the opportunities offered by digital health innovations and see them as a miraculous solution to all problems. In this type of analysis, effects are often confused with causes, whereas other factors of using innovations, such as the human element, are disregarded altogether. This is yet another confirmation of the well-known fact that technology's value is measured by its users' skills, digital, and health literacy.