Applying Supervised Learning Technologies to Predict Misdiagnosis and Overtreatment Using Data from Medical Examinations

June 6, 2017 8:00 AM – 9:15 AM

Isaac Li, National Board of Osteopathic Medical Examiners
Qiongqiong Liu, National Board of Osteopathic Medical Examiners
Edward Tsai, National Board of Osteopathic Medical Examiners

The National Board of Osteopathic Medical Examiners introduced clinical scenario-based Decision-Making (CDM)/Key Feature (KF) questions to the Comprehensive Osteopathic Medical Licensing Examination Level 3. Key features refer to the critical steps to correctly diagnose and treat a clinical case. The CDM/KF questions are designed to evaluate how well post-graduate Doctors of Osteopathic Medicine (DOs) in their early residency make such decisions.  

Questions in a CDM/KF case allow for multiple responses to simulate real practice settings. If a decision made in the response may harm the patient (i.e., “killer” responses), it is considered a proxy for misdiagnosis or mistreatment; additionally, when more options were selected than allowed, it is an instance of over-testing or over-treatment. Research suggested that candidates who chose killers or exceeded option limits did not show competence on the domains as assessed by CDM/KF questions. It is imperative to enable knowledge-driven, proactive interventions in anticipation of such an outcome.

Data mining extracts latent information from large databases to predict future behaviors. Supervised learning, a technique in data mining for regression and classification, often provides better prediction than traditional methods like linear regression modeling.

This study proposes to apply different supervised learning procedures such as classification tree and random forests to examination and demographics data and build a predictive model for misdiagnosis and overtreatment as evidenced in performance on the CDM/KF cases. Identified predictors and their effects will connect a DO’s school learning with patient outcome.