Triggering Hypertrophic Cardiomyopathy (HCM): A QI initiative to decrease incidents of missed opportunities in the diagnosis of HCM
Project Lead: Moein Enayati, PhD
Project AIM(s): To design an AI-based clinical decision support tool in the practice of echo lab, that can communicate with providers in real-time about the chance of HCM diagnosis and the existence of missing measurements. As a result of proper implementation, more complete echo reports will be generated with enough important measurements and indicative impressions to reduce the chance of missed or delayed diagnosis of HCM.
Narrative Description:
The present study aimed to develop and evaluate an AI/ML-based Clinical Decision Support System (CDSS) for reducing diagnostic errors and enhancing the diagnosis of Hypertrophic Cardiomyopathy (HCM) in an echocardiography lab. The study was conducted in three cycles following the Plan-Do-Study-Act (PDSA) model.
In the first cycle, an ensemble learning model was trained on 37,528 echocardiograms, with a median age of 66 years and 56% of the patients being male. The model demonstrated an accuracy of 96%, a sensitivity of 86%, a specificity of 99%, and a detection rate of 75%.
In the second cycle, the ML model was integrated into the Echocardiography Lab's Information Management System (EIMS) in the form of a CDSS, designed with a minimal graphical user interface. The system was made available to a select group of collaborating providers in the lab for testing purposes, and their feedback and suggestions were incorporated into the final version of the tool.
In the third cycle, a larger group of providers was involved in evaluating the impact of the CDSS on diagnostic errors. The results showed that the CDSS reduced the number of missed or delayed HCM diagnoses by predicting the chance of HCM and providing suggestions for additional measurements and impressions.
The study emphasized the significance of involving clinical experts and adopting a user-centered design approach in the development and evaluation of such systems. The integration of the model into the EIMS was deemed crucial to the success of the CDSS. The feedback from the providers, who found the CDSS useful in their daily practice, supported this conclusion. The study showed a reduction of approximately 30% in the rate of missing measurements in the echo reports, with several cases where providers found the AI tool helpful in identifying potential errors or missing measurements.
The project received positive feedback and generated interest among providers, who provided suggestions for its future expansion to other diseases. The HCM AI tool is planned to be integrated with other AI models and shared between Mayo Clinic sites, and eventually made available to other institutions.
While the HCM AI tool showed promising results, there is still room for improvement. Steps can be taken to streamline the automation process and reduce the burden on providers even further. Our vision is to achieve this aim through a following fourth PDSA cycle.
Diagnostic quality problem type, failure, or category (symptoms, observed problems, gaps in performance) addressed by the intervention
- Information gathering
- Information integration
- Information interpretation
- Establishing an explanation (diagnosis)
Root causes/causative factors addressed by the intervention
- Workflow (includes testing, follow-up, and referrals)
- Health information sharing and accessibility via health IT
- Knowledge gaps/inexperience
Setting of the diagnostic quality improvement intervention
- Ambulatory medical care setting (e.g., clinic, office, urgent care)