Case Study: Chest X-rays

In this case study, we’ll be diving into a particular case of model bias in the medical sector. In December 2021, a paperSeyyed-Kalantari, L., et al. “Underdiagnosis Bias of Artificial Intelligence Algorithms Applied to Chest Radiographs in Under-served Patient Populations.” 2021. Pacific Symposium on Biocomputing, vol. 26, pp. 232-243. PMID: 33691020. published in Nature revealed that algorithms meant to diagnose patients based on their chest radiographs were often biased in a way that underdiagnosed underserved populations.

Background

Chest x-rays are one particularly useful area for ML and AI models to speed up diagnoses. Chest x-rays are easily obtainable data sources that have clear targets (diagnosis or no diagnosis) for medical practitioners to provide labels to. Models trained on this data become “experts” at identifying abnormalities in these images.

In the United States, patients from minority populations are often disadvantaged and disenfranchised from a medical perspective—they often don’t have access to strong primary healthcare and auxiliary services like physical therapy. Biased models, particularly those that underdiagnose (i.e., declare a patient as healthy when there’s in fact an abnormality, also called false negatives), can perpetuate this issue.

In this study, researchers systematically tested CXR (chest x-ray) prediction models on three large, public radiology datasets (MIMIC-CXR, CheXpert, ChestX-ray14). They focused their testing on an intersectional analysis across race, socioeconomic status (proxied by insurance type), sex, and age.

The standard practice in the medical industry is to report the model’s performance on the overall population, but as we know, good performance overall can still mask unfairness at the subpopulation level.

Metrics

This study measured fairness by the false decision rate (FDR). Given an intersectional group (i.e., Black female patients), the FDR is calculated as follows: