We are constantly told healthcare is “ripe” for disruptive innovation, a process of creative destruction enabled by the exciting new technologies of digital health. Yet, the robust demonstration of almost any substantive benefit (beyond perhaps transient delight and toxic self- absorption) has been slow in coming.
The challenges of moving a health-related technology from promise to impact are illustrated nicely by two recent attempts to carefully evaluate the benefits of intriguing new devices.
The Propeller Health Inhaler Monitor
The first study (NCT01509183) was sponsored by Asthmapolis (now Propeller Health), and sought to examine whether use of the company’s inhaler monitoring device resulted in improved asthma control, the study’s primary endpoint. Control, measured by the Asthma Control Test (ACT) would be assessed at 4, 8, and 12 months.
The device senses inhaler use – in this study, the use of fast-acting “rescue” inhaler medications, specifically — and communicates data to patients (via an app) and to providers (via reports and change of status alerts), according to MedPage Today. In this study, asthmatic patients who had experienced a “healthcare utilization event” in the preceding year were randomized into either an active group , receiving the device and associated data, or a group receiving a deactivated device that provided no supplemental information.
As study co-author Bob Quade explains in an email, “the fundamental intervention is information.
At a national meeting last November, interim results were presented. These data revealed slight improvements in the ACT in the active arm compared to the control arm in both adults and children, differences not statistically significant.
However, additional analysis found subjects in the actively monitored group used rescue inhalers significantly less often than the control group. Perhaps most strikingly, inpatient days were reduced from 0.225 per person per year in control group to 0.087 in active group; emergency room visits decreased marginally (from 0.141 per person per year in control group to 0.103 in active group) as well.
This reduction in utilization is claimed to be associated with a savings of $ 688.05 per patient (compared to baseline); subjects in the control group, notably, were found to save $ 281.95 per patient compared to baseline, suggesting a regression to the mean effect, not surprising given that subjects were selected based on experiencing a healthcare event in the previous year (most asthma patients don’t experience events in any given year). It’s also possible that simply participating in the study may have contributed to a change in utilization (the Hawthorne effect at work). Both possibilities were acknowledged by Quade.
However, the active group still exhibited a dramatic savings (~$ 400 per patient) associated with reduced utilization compared to the control group, despite the lack of significant changes in asthma control as measured by ACT scores. This presents a real puzzle.
Some, such as wellness guru Al Lewis (author of Why Nobody Believes The Numbers, and more recently, Surviving Workplace Wellness) look at these results incredulously.
“Their own numbers don’t add up,” he asserts in an email. “First, they say it’s not statistically a significant difference.”
“Even if it were significant,” Lewis continues, the difference reported on the ACT “is something like 2.2% improvement in the score. Then, they claim an inpatient reduction of 62%. This violates several of the rules in Why Nobody Believes. First, costs can’t decline that much and second, the cost reduction has to tie to the quality improvement.”
An alternative explanation, however, is that there are improvements in care not reflected in the ACT measure; for example, if the intervention doesn’t prevent exacerbation but catches them early enough to enable them to be managed as an outpatient rather than an inpatient, this could theoretically save significant dollars, yet be difficult to pick up on the ACT.
This study highlights the challenges of rigorously demonstrating the benefit of a digital health intervention that appears to make a world of sense. It seems logical, even obvious, that improved monitoring of rescue inhaler use, coupled with improved, immediate communication with patients and providers would improve disease management. As study principal investigator Rajan Merchant points out, it enables the care of asthma patients to transition from episodic to continuous – one of the key ambitions of digital health.
I look forward to examining a peer-reviewed publication once the study is completed later this year, and all data are read out. My suspicion is that there may ultimately be a cost-savings achieved (vs the control group), and perhaps eventually a small improvement in asthma control. Whether this economically justifies the use of the intervention will need to be determined, given the cost of the devices and software, and the extra provider time potentially required.